**Neural Tangent Transfer**, or **NTT**, is a method for finding trainable sparse networks in a label-free manner. Specifically, NTT finds sparse networks whose training dynamics, as characterized by the neural tangent kernel, mimic those of dense networks in function space.

**SRGAN Residual Block** is a residual block used in the [SRGAN](https://paperswithcode.com/method/srgan) generator for image super-resolution. It is similar to standard [residual blocks](https://paperswithcode.com/method/residual-block), although it uses a [PReLU](https://paperswithcode.com/method/prelu) activation function to help training (preventing sparse gradients during [GAN](https://paperswithcode.com/method/gan) training).

SRGAN Residual Block

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Neural Tangent Transfer

Finding trainable sparse networks through Neural Tangent Transfer

**Slanted Triangular Learning Rates (STLR)** is a learning rate schedule which first linearly increases the learning rate and then linearly decays it, which can be seen in Figure to the right. It is a modification of Triangular Learning Rates, with a short increase and a long decay period.

Source	Finding trainable sparse networks through Neural Tangent Transfer
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com