What is: Hamburger?
Source | Is Attention Better Than Matrix Decomposition? |
Year | 2000 |
Data Source | CC BY-SA - https://paperswithcode.com |
Hamburger is a global context module that employs matrix decomposition to factorize the learned representation into sub-matrices so as to recover the clean low-rank signal subspace. The key idea is, if we formulate the inductive bias like the global context into an objective function, the optimization algorithm to minimize the objective function can construct a computational graph, i.e., the architecture we need in the networks.