AICurious Logo

What is: Hard Sigmoid?

SourceBinaryConnect: Training Deep Neural Networks with binary weights during propagations
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

The Hard Sigmoid is an activation function used for neural networks of the form:

f(x)=max(0,min(1,(x+1)2))f\left(x\right) = \max\left(0, \min\left(1,\frac{\left(x+1\right)}{2}\right)\right)

Image Source: Rinat Maksutov