AICurious Logo

What is: Singular Value Clipping?

SourceTemporal Generative Adversarial Nets with Singular Value Clipping
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Singular Value Clipping (SVC) is an adversarial training technique used by TGAN to enforce the 1-Lipschitz constraint of the WGAN objective. It is a constraint to all linear layers in the discriminator that satisfies the spectral norm of weight parameter WW is equal or less than one. This means that the singular values of weight matrix are all one or less. Therefore singular value decomposition (SVD) is performed after a parameter update, replacing all the singular values larger than one with one, and the parameters are reconstructed with them. The same operation is applied to convolutional layers by interpreting a higher order tensor in weight parameter as a matrix W^\hat{W}.