**LAMB** is a a layerwise adaptive large batch optimization technique. It provides a strategy for adapting the learning rate in large batch settings. LAMB uses [Adam](https://paperswithcode.com/method/adam) as the base algorithm and then forms an update as:

$$r\_{t} = \frac{m\_{t}}{\sqrt{v\_{t}} + \epsilon}$$
$$x\_{t+1}^{\left(i\right)} = x\_{t}^{\left(i\right)}  - \eta\_{t}\frac{\phi\left(|| x\_{t}^{\left(i\right)} ||\right)}{|| m\_{t}^{\left(i\right)} || }\left(r\_{t}^{\left(i\right)}+\lambda{x\_{t}^{\left(i\right)}}\right) $$

Unlike [LARS](https://paperswithcode.com/method/lars), the adaptivity of LAMB is two-fold: (i) per dimension normalization with respect to the square root of the second moment used in Adam and (ii) layerwise normalization obtained due to layerwise adaptivity.

A **3D Convolution** is a type of [convolution](https://paperswithcode.com/method/convolution) where the kernel slides in 3 dimensions as opposed to 2 dimensions with 2D convolutions. One example use case is medical imaging where a model is constructed using 3D image slices. Additionally video based data has an additional temporal dimension over images making it suitable for this module. 

Image: Lung nodule detection based on 3D convolutional neural networks, Fan et al

3D Convolution

LAMB

Large Batch Optimization for Deep Learning: Training BERT in 76 minutes

A dual graph convolutional neural network jointly considers the two essential assumptions of semi-supervised learning: (1) local consistency and (2) global consistency. Accordingly, two convolutional neural networks are devised to embed the local-consistency-based and global-consistency-based knowledge, respectively.

Description  and image from: [Dual Graph Convolutional Networks for Graph-Based Semi-Supervised Classification](https://persagen.com/files/misc/zhuang2018dual.pdf)

Source	Large Batch Optimization for Deep Learning: Training BERT in 76 minutes
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com