A **Deformable Kernels** is a type of convolutional operator for deformation modeling. DKs learn free-form offsets on kernel coordinates to deform the original kernel space towards specific data modality, rather than recomposing data. This can directly adapt the effective receptive field (ERF) while leaving the receptive field untouched. They can be used as a drop-in replacement of rigid kernels. 

As shown in the Figure, for each input patch, a local DK first generates a group of kernel offsets $\{\Delta \mathcal{k}\}$ from input feature patch using the light-weight generator $\mathcal{G}$ (a 3$\times$3 [convolution](https://paperswithcode.com/method/convolution) of rigid kernel). Given the original kernel weights $\mathcal{W}$ and the offset group $\{\Delta \mathcal{k}\}$, DK samples a new set of kernel $\mathcal{W}'$ using a bilinear sampler $\mathcal{B}$. Finally, DK convolves the input feature map and the sampled kernels to complete the whole computation.

A log-time alternative to feedforward layers outperforming both the vanilla feedforward and mixture-of-experts approaches.

Fast Feedforward Networks

Deformable Kernel

Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation

Gated Attention Networks (GaAN) is a new architecture for learning on graphs. Unlike the traditional multi-head attention mechanism, which equally consumes all attention heads, GaAN uses a convolutional sub-network to control each attention head’s importance.

Image credit: [GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs](https://paperswithcode.com/paper/gaan-gated-attention-networks-for-learning-on)

Source	Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com