**GShard** is a intra-layer parallel distributed method. It consists of set of simple APIs for annotations, and a compiler extension in XLA for automatic parallelization.

**NoisyNet-Dueling** is a modification of a [Dueling Network](https://paperswithcode.com/method/dueling-network) that utilises noisy linear layers for exploration instead of $\epsilon$-greedy exploration as in the original Dueling formulation.

NoisyNet-Dueling

Noisy Networks for Exploration

GShard

GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding

**TAPAS** is a weakly supervised question answering model that reasons over tables without generating logical forms. TAPAS predicts a minimal program by selecting a subset of the table cells and a possible aggregation operation to be executed on top of them. Consequently, TAPAS can learn operations from natural language, without the need to specify them in some formalism. This is implemented by extending [BERT](https://paperswithcode.com/method/bert)’s architecture with additional embeddings that capture tabular structure, and with two classification layers for selecting cells and predicting a corresponding aggregation operator.

Source	GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com