**Inverse Q-Learning (IQ-Learn)** is a a simple, stable & data-efficient framework for Imitation Learning (IL), that directly learns *soft Q-functions* from expert data. IQ-Learn enables **non-adverserial** imitation learning, working on both offline and online IL settings. It is performant even with very sparse expert data, and scales to complex image-based environments, surpassing prior methods by more than **3x**. 

It is very simple to implement requiring ~15 lines of code on top of existing RL methods.

<span class="description-source">Source: [IQ-Learn: Inverse soft Q-Learning for Imitation](https://arxiv.org/abs/2106.12142)</span>

**AMSBound** is a variant of the [AMSGrad](https://paperswithcode.com/method/amsgrad) stochastic optimizer which is designed to be more robust to extreme learning rates. Dynamic bounds are employed on learning rates, where the lower and upper bound are initialized as zero and infinity respectively, and they both smoothly converge to a constant final step size. AMSBound can be regarded as an adaptive method at the beginning of training, and it gradually and smoothly transforms to [SGD](https://paperswithcode.com/method/sgd) (or with momentum) as time step increases. 

$$ g\_{t} = \nabla{f}\_{t}\left(x\_{t}\right) $$

$$ m\_{t} = \beta\_{1t}m\_{t-1} + \left(1-\beta\_{1t}\right)g\_{t} $$

$$ v\_{t} = \beta\_{2}v\_{t-1} + \left(1-\beta\_{2}\right)g\_{t}^{2}$$

$$ \hat{v}\_{t} = \max\left(\hat{v}\_{t-1}, v\_{t}\right) \text{ and } V\_{t} = \text{diag}\left(\hat{v}\_{t}\right) $$

$$ \eta = \text{Clip}\left(\alpha/\sqrt{V\_{t}}, \eta\_{l}\left(t\right), \eta\_{u}\left(t\right)\right) \text{ and } \eta\_{t} = \eta/\sqrt{t} $$

$$ x\_{t+1} = \Pi\_{\mathcal{F}, \text{diag}\left(\eta\_{t}^{-1}\right)}\left(x\_{t} - \eta\_{t} \odot m\_{t} \right) $$

Where $\alpha$ is the initial step size, and $\eta_{l}$ and $\eta_{u}$ are the lower and upper bound functions respectively.

AMSBound

Adaptive Gradient Methods with Dynamic Bound of Learning Rate

IQ-Learn

A variant of [CutMix](https://paperswithcode.com/method/cutmix) which randomly samples masks from Fourier space.

Year	2000
Data Source	CC BY-SA - https://paperswithcode.com