Gumbel activation function, is defined using the cumulative Gumbel distribution and it can be used to perform Gumbel regression. Gumbel activation is an alternative activation function to the sigmoid or softmax activation functions and can be used to transform the unormalised output of a model to probability. Gumbel activation  $\eta_{Gumbel}$ is defined as follows:

$\eta_{Gumbel}(q_i) = exp(-exp(-q_i))$

It can be combined with Cross Entropy loss function to solve long-tailed classification problems. Gumbel Cross Entropy (GCE) is defined as follows:

$GCE(\eta_{Gumbel}(q_i),y_i) = -y_i \log(\eta_{Gumbel}(q_i))+ (1-y_i) \log(1-\eta_{Gumbel}(q_i))$

A wavelet **scattering transform** computes a translation invariant representation, which is stable to deformation, using a deep [convolution](https://paperswithcode.com/method/convolution) network architecture. It computes non-linear invariants with modulus and averaging pooling functions. It helps to eliminate the image variability due to translation and is stable to deformations. 

Image source: [Bruna and Mallat](https://arxiv.org/pdf/1203.1513v2.pdf)

ScatNet

Invariant Scattering Convolution Networks

Gumbel Activation

Long-tailed Instance Segmentation using Gumbel Optimized Loss

The Fast-YOLOv4-SmallObj model is a modified version of Fast-[YOLOv4](https://paperswithcode.com/method/yolov4) to improve the detection of small objects. Seven layers were added so that it predicts bounding boxes at 3 different scales instead of 2.

Source	Long-tailed Instance Segmentation using Gumbel Optimized Loss
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com