A **Selective Kernel Convolution** is a [convolution](https://paperswithcode.com/method/convolution) that enables neurons to adaptively adjust their RF sizes among multiple kernels with different kernel sizes. Specifically, the SK convolution has three operators – Split, Fuse and Select. Multiple branches with different kernel sizes are fused using
[softmax](https://paperswithcode.com/method/softmax) attention that is guided by the information in these branches. Different attentions on these branches yield different sizes of the effective receptive fields of neurons in the fusion layer

**Feature Fusion Module v2** is a feature fusion module from the [M2Det](https://paperswithcode.com/method/m2det) object detection model, and is crucial for constructing the final multi-level feature pyramid. They use [1x1 convolution](https://paperswithcode.com/method/1x1-convolution) layers to compress the channels of the input features and use a concatenation operation to aggregate these feature map. FFMv2 takes the base feature and the largest output feature map of the previous [Thinned U-Shape Module](https://paperswithcode.com/method/tum) (TUM) – these two are of the same scale – as input, and produces the fused feature for the next TUM.

FFMv2

M2Det: A Single-Shot Object Detector based on Multi-Level Feature Pyramid Network

Selective Kernel Convolution

Selective Kernel Networks

**CaiT**, or **Class-Attention in Image Transformers**, is a type of [vision transformer](https://paperswithcode.com/methods/category/vision-transformer) with several design alterations upon the original [ViT](https://paperswithcode.com/method/vision-transformer). First a new layer scaling approach called [LayerScale](https://paperswithcode.com/method/layerscale) is used, adding a learnable diagonal matrix on output of each residual block, initialized close to (but not at) 0, which improves the training dynamics. Secondly, [class-attention layers](https://paperswithcode.com/method/ca) are introduced to the architecture. This creates an architecture where the transformer layers involving [self-attention](https://paperswithcode.com/method/scaled) between patches are explicitly separated from class-attention layers -- that are devoted to extract the content of the processed patches into a single vector so that it can be fed to a linear classifier.

Source	Selective Kernel Networks
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com