**TAPAS** is a weakly supervised question answering model that reasons over tables without generating logical forms. TAPAS predicts a minimal program by selecting a subset of the table cells and a possible aggregation operation to be executed on top of them. Consequently, TAPAS can learn operations from natural language, without the need to specify them in some formalism. This is implemented by extending [BERT](https://paperswithcode.com/method/bert)’s architecture with additional embeddings that capture tabular structure, and with two classification layers for selecting cells and predicting a corresponding aggregation operator.

**GShard** is a intra-layer parallel distributed method. It consists of set of simple APIs for annotations, and a compiler extension in XLA for automatic parallelization.

GShard

GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding

TAPAS

TAPAS: Weakly Supervised Table Parsing via Pre-training

The **Feature-wise linear modulation** (**FiLM**) module combines information from both noisy waveform and input mel-spectrogram. It is used in the [WaveGrad](https://paperswithcode.com/method/wavegrad) model. The authors also added iteration index $n$ which indicates the noise level of the input waveform by using the [Transformer](https://paperswithcode.com/method/transformer) sinusoidal positional embedding. To condition on the noise level directly, $n$ is replaced by $\sqrt{\bar{\alpha}}$ and a linear scale $C = 5000$ is applied. The FiLM module produces both scale and bias vectors given inputs, which are used in a UBlock for feature-wise affine transformation as:

$$ \gamma\left(D, \sqrt{\bar{\alpha}}\right) \odot U + \zeta\left(D, \sqrt{\bar{\alpha}}\right) $$

where $\gamma$ and $\zeta$ correspond to the scaling and shift vectors from the FiLM module, $D$ is the output from corresponding [DBlock](https://paperswithcode.com/method/dblock), $U$ is an intermediate output in the UBlock.

Source	TAPAS: Weakly Supervised Table Parsing via Pre-training
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com