**RandWire** is a type of convolutional neural network that arise from randomly
wired neural networks that are sampled from stochastic network generators, in which a human-designed random
process defines generation.

A **Laplacian Pyramid** is a linear invertible image representation consisting of a set of band-pass
images spaced an octave apart, plus a low-frequency residual. Formally, let $d\left(.\right)$ be a downsampling operation that blurs and decimates a $j \times j$ image $I$ so that $d\left(I\right)$ is a new image of size $\frac{j}{2} \times \frac{j}{2}$. Also, let $u\left(.\right)$ be an upsampling operator which smooths and expands $I$ to be twice the size, so $u\left(I\right)$ is a new image of size $2j \times 2j$. We first build a Gaussian pyramid $G\left(I\right) = \left[I\_{0}, I\_{1}, \dots, I\_{K}\right]$, where
$I\_{0} = I$ and $I\_{k}$ is $k$ repeated application of $d\left(.\right)$ to $I$. $K$ is the number of levels in the pyramid selected so that the final level has a minimal spatial extent ($\leq 8 \times 8$ pixels).

The coefficients $h\_{k}$ at each level $k$ of the Laplacian pyramid $L\left(I\right)$ are constructed by taking the difference between adjacent levels in the Gaussian pyramid, upsampling the smaller one with $u\left(.\right)$ so that the sizes are compatible:

$$ h\_{k} = \mathcal{L}\_{k}\left(I\right) = G\_{k}\left(I\right) − u\left(G\_{k+1}\left(I\right)\right) = I\_{k} − u\left(I\_{k+1}\right) $$

Intuitively, each level captures the image structure present at a particular scale. The final level of the
Laplacian pyramid $h\_{K}$ is not a difference image, but a low-frequency residual equal to the final
Gaussian pyramid level, i.e. $h\_{K} = I\_{K}$. Reconstruction from a Laplacian pyramid coefficients
$\left[h\_{1}, \dots, h\_{K}\right]$ is performed using the backward recurrence:

$$ I\_{k} = u\left(I\_{k+1}\right) + h\_{k} $$

which is started with $I\_{K} = h\_{K}$ and the reconstructed image being $I = I\_{o}$. In other words, starting at the coarsest level, we repeatedly upsample and add the difference image h at the next finer level until we return to the full-resolution image.
Source: [LAPGAN](https://paperswithcode.com/method/lapgan)

Image : [Design of FIR Filters for Fast Multiscale Directional Filter Banks](https://www.researchgate.net/figure/Relationship-between-Gaussian-and-Laplacian-Pyramids_fig2_275038450)

Laplacian Pyramid

RandWire

Exploring Randomly Wired Neural Networks for Image Recognition

**Contrastive BERT** is a reinforcement learning agent that combines a new contrastive loss and a hybrid [LSTM](https://paperswithcode.com/method/lstm)-[transformer](https://paperswithcode.com/method/transformer) architecture to tackle the challenge of improving data efficiency for RL. It uses bidirectional masked prediction in combination with a generalization of recent contrastive methods to learn better representations for transformers in RL, without the need of hand engineered data augmentations.

For the architecture, a residual network is used to encode observations into embeddings $Y\_{t}$. $Y_{t}$  is fed through a causally masked [GTrXL transformer](https://www.paperswithcode.com/method/gtrxl), which computes the predicted masked inputs $X\_{t}$ and passes those together with $Y\_{t}$ to a learnt gate. The output of the gate is passed through a single [LSTM](https://www.paperswithcode.com/method/lstm) layer to produce the values that we use for computing the RL loss. A contrastive loss is computed using predicted masked inputs $X_{t}$ and $Y_{t}$ as targets. For this, we do not use the causal mask of the Transformer.

Source	Exploring Randomly Wired Neural Networks for Image Recognition
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com