**Logistic Regression**, despite its name, is a linear model for classification rather than regression. Logistic regression is also known in the literature as logit regression, maximum-entropy classification (MaxEnt) or the log-linear classifier. In this model, the probabilities describing the possible outcomes of a single trial are modeled using a logistic function.

Source: [scikit-learn](https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression)

Image: [Michaelg2015](https://commons.wikimedia.org/wiki/User:Michaelg2015)

**Teacher-Tutor-Student Knowledge Distillation** is a method for image virtual try-on models. It treats fake images produced by the parser-based method as "tutor knowledge", where the artifacts can be corrected by real "teacher knowledge", which is extracted from the real person images in a self-supervised way. Other than using real images as supervisions, knowledge distillation is formulated in the try-on problem as distilling the appearance flows between the person image and the garment image, enabling the finding of dense correspondences between them to produce high-quality results.

Teacher-Tutor-Student Knowledge Distillation

Parser-Free Virtual Try-on via Distilling Appearance Flows

Logistic Regression

**SRU**, or **Simple Recurrent Unit**, is a recurrent neural unit with a light form of recurrence. SRU exhibits the same level of parallelism as [convolution](https://paperswithcode.com/method/convolution) and [feed-forward nets](https://paperswithcode.com/methods/category/feedforward-networks). This is achieved by balancing sequential dependence and independence: while the state computation of SRU is time-dependent, each state dimension is independent. This simplification enables CUDA-level optimizations that parallelize the computation across hidden dimensions and time steps, effectively using the full capacity of modern GPUs. 

SRU also replaces the use of convolutions (i.e., ngram filters), as in [QRNN](https://paperswithcode.com/method/qrnn) and KNN, with more recurrent connections. This retains modeling capacity, while using less computation (and hyper-parameters). Additionally, SRU improves the training of deep recurrent models by employing [highway connections](https://paperswithcode.com/method/highway-layer) and a parameter initialization scheme tailored for gradient propagation in deep architectures.

A single layer of SRU involves the following computation:

$$
\mathbf{f}\_{t} =\sigma\left(\mathbf{W}\_{f} \mathbf{x}\_{t}+\mathbf{v}\_{f} \odot \mathbf{c}\_{t-1}+\mathbf{b}\_{f}\right) 
$$

$$
\mathbf{c}\_{t} =\mathbf{f}\_{t} \odot \mathbf{c}\_{t-1}+\left(1-\mathbf{f}\_{t}\right) \odot\left(\mathbf{W} \mathbf{x}\_{t}\right) \\
$$

$$
\mathbf{r}\_{t} =\sigma\left(\mathbf{W}\_{r} \mathbf{x}\_{t}+\mathbf{v}\_{r} \odot \mathbf{c}\_{t-1}+\mathbf{b}\_{r}\right) \\
$$

$$
\mathbf{h}\_{t} =\mathbf{r}\_{t} \odot \mathbf{c}\_{t}+\left(1-\mathbf{r}\_{t}\right) \odot \mathbf{x}\_{t}
$$

where $\mathbf{W}, \mathbf{W}\_{f}$ and $\mathbf{W}\_{r}$ are parameter matrices and $\mathbf{v}\_{f}, \mathbf{v}\_{r}, \mathbf{b}\_{f}$ and $\mathbf{b}_{v}$ are parameter vectors to be learnt during training. The complete architecture decomposes to two sub-components: a light recurrence and a highway network,

The light recurrence component successively reads the input vectors $\mathbf{x}\_{t}$ and computes the sequence of states $\mathbf{c}\_{t}$ capturing sequential information. The computation resembles other recurrent networks such as [LSTM](https://paperswithcode.com/method/lstm), [GRU](https://paperswithcode.com/method/gru) and RAN. Specifically, a forget gate $\mathbf{f}\_{t}$ controls the information flow and the state vector $\mathbf{c}\_{t}$ is determined by adaptively averaging the previous state $\mathbf{c}\_{t-1}$ and the current observation $\mathbf{W} \mathbf{x}_{+}$according to $\mathbf{f}\_{t}$.

Year	2000
Data Source	CC BY-SA - https://paperswithcode.com