**Adaptive Input Embeddings** extend the [adaptive softmax](https://paperswithcode.com/method/adaptive-softmax) to input word representations. The factorization assigns more capacity to frequent words and reduces the capacity for less frequent words with the benefit of reducing overfitting to rare words.

**ENet Dilated Bottleneck** is an image model block used in the [ENet](https://paperswithcode.com/method/enet) semantic segmentation architecture. It is the same as a regular [ENet Bottleneck](https://paperswithcode.com/method/enet-bottleneck) but employs dilated convolutions instead.

ENet Dilated Bottleneck

ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation

Adaptive Input Representations

Adaptive Input Representations for Neural Language Modeling

**STraTA**, or **Self-Training with Task Augmentation**, is a self-training approach that builds on two key ideas for effective leverage of unlabeled data. First, STraTA uses task augmentation, a technique that synthesizes a large amount of data for auxiliary-task fine-tuning from target-task unlabeling texts. Second, STRATA performs self-training by further fine-tuning the strong base model created by task augmentation on a broad distribution of pseudo-labeled data.

In task augmentation, we train an NLI data generation model and use it to synthesize a large amount of in-domain NLI training data for each given target task, which is then used for auxiliary (intermediate) fine-tuning. The self-training algorithm iteratively learns a better model using a concatenation of labeled and pseudo-labeled examples. At each iteration, we always start with the auxiliary-task model produced by task augmentation and train on a broad distribution of pseudo-labeled data.

Source	Adaptive Input Representations for Neural Language Modeling
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com