AICurious Logo

What is: Hamburger?

SourceIs Attention Better Than Matrix Decomposition?
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Hamburger is a global context module that employs matrix decomposition to factorize the learned representation into sub-matrices so as to recover the clean low-rank signal subspace. The key idea is, if we formulate the inductive bias like the global context into an objective function, the optimization algorithm to minimize the objective function can construct a computational graph, i.e., the architecture we need in the networks.