**Activation Normalization** is a type of normalization used for flow-based generative models; specifically it was introduced in the [GLOW](https://paperswithcode.com/method/glow) architecture. An ActNorm layer performs an affine transformation of the activations using a scale and bias parameter per channel, similar to [batch normalization](https://paperswithcode.com/method/batch-normalization). These parameters are initialized such that the post-actnorm activations per-channel have zero mean and unit variance given an initial minibatch of data. This is a form of data dependent initilization. After initialization, the scale and bias are treated as regular trainable parameters that are independent of the data.

**PonderNet** is an adaptive computation method that learns to adapt the amount of computation based on the complexity of the problem at hand. PonderNet learns end-to-end the number of computational steps to achieve an effective compromise between training prediction accuracy, computational cost and generalization.

PonderNet

PonderNet: Learning to Ponder

Activation Normalization

Glow: Generative Flow with Invertible 1x1 Convolutions

A **Local Relation Layer** is an image feature extractor that is an alternative to a [convolution](https://paperswithcode.com/method/convolution) operator. The intuition is that aggregation in convolution is basically a pattern matching process that applies fixed filters, which can be inefficient at modeling visual elements with varying spatial distributions. The local relation layer adaptively determines aggregation weights based on the compositional relationship of local pixel pairs. It is argued that, with this relational approach, it can composite visual elements into higher-level entities in a more efficient manner that benefits semantic inference.

Source	Glow: Generative Flow with Invertible 1x1 Convolutions
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com