**Two-Way Dense Layer** is an image model block used in the [PeleeNet](https://paperswithcode.com/method/peleenet) architectures. Motivated by [GoogLeNet](https://paperswithcode.com/method/googlenet), the 2-way dense layer is used to get different scales of receptive fields. One way of the layer uses a 3x3 kernel size. The other way of the layer uses two stacked 3x3 [convolution](https://paperswithcode.com/method/convolution) to learn visual patterns for large objects.

Galactica is a language model which uses a Transformer architecture in a decoder-only setup with the following modifications:

- It uses GeLU activations on all model sizes
- It uses a 2048 length context window for all model sizes
- It does not use biases in any of the dense kernels or layer norms
- It uses learned positional embeddings for the model
- A vocabulary of 50k tokens was constructed using BPE. The vocabulary was generated from a randomly selected 2% subset of the training data

Galactica

Galactica: A Large Language Model for Science

Two-Way Dense Layer

Pelee: A Real-Time Object Detection System on Mobile Devices

**Ape-X DQN** is a variant of a [DQN](https://paperswithcode.com/method/dqn) with some components of [Rainbow-DQN](https://paperswithcode.com/method/rainbow-dqn) that utilizes distributed [prioritized experience replay](https://paperswithcode.com/method/prioritized-experience-replay) through the [Ape-X](https://paperswithcode.com/method/ape-x) architecture.

Source	Pelee: A Real-Time Object Detection System on Mobile Devices
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com