**Capsule Network** is a machine learning system that is a type of artificial neural network that can be used to better model hierarchical relationships. The approach is an attempt to more closely mimic biological neural organization.

**Parameterized Exponential Linear Units**, or **PELU**, is an activation function for neural networks. It involves learning a parameterization of [ELU](https://paperswithcode.com/method/elu) in order to learn the proper activation shape at each layer in a CNN. 

The PELU has two additional parameters over the ELU:

$$ f\left(x\right) = cx \text{ if } x > 0 $$
$$ f\left(x\right) = \alpha\exp^{\frac{x}{b}} - 1 \text{ if } x \leq 0 $$

Where $a$, $b$, and $c > 0$. Here $c$ causes a change in the slope in the positive quadrant, $b$ controls the scale of the [exponential decay](https://paperswithcode.com/method/exponential-decay), and $\alpha$ controls the saturation in the negative quadrant.

Source: [Activation Functions](https://arxiv.org/pdf/1811.03378.pdf)

PELU

Parametric Exponential Linear Unit for Deep Convolutional Neural Networks

CapsNet

Dynamic Routing Between Capsules

Trainable activation function as a sigmoid-based generalization of ReLU, Swish and SiLU.

Source	Dynamic Routing Between Capsules
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com