**Graph Contrastive Coding** is a self-supervised graph neural network pre-training framework to capture the universal network topological properties across multiple networks. GCC's pre-training task is designed as subgraph instance discrimination in and across networks and leverages contrastive learning to empower graph neural networks to learn the intrinsic and transferable structural representations.

The Stacked Denoising Autoencoder (SdA) is an extension of the stacked autoencoder [Bengio07] and it was introduced in [Vincent08].

Denoising autoencoders can be stacked to form a deep network by feeding the latent representation (output code) of the [denoising autoencoder](https://paperswithcode.com/method/denoising-autoencoder) found on the layer below as input to the current layer. The unsupervised pre-training of such an architecture is done one layer at a time. Each layer is trained as a denoising autoencoder by minimizing the error in reconstructing its input (which is the output code of the previous layer). Once the first k layers are trained, we can train the k+1-th layer because we can now compute the code or latent representation from the layer below.

Once all layers are pre-trained, the network goes through a second stage of training called fine-tuning. Here we consider supervised fine-tuning where we want to minimize prediction error on a supervised task. For this, we first add a [logistic regression](https://paperswithcode.com/method/logistic-regression) layer on top of the network (more precisely on the output code of the output layer). We then train the entire network as we would train a multilayer perceptron. At this point, we only consider the encoding parts of each auto-encoder. This stage is supervised, since now we use the target class during training. (See the Multilayer Perceptron for details on the multilayer perceptron.)

This can be easily implemented in Theano, using the class defined previously for a denoising autoencoder. We can see the stacked denoising autoencoder as having two facades: a list of autoencoders, and an MLP. During pre-training we use the first facade, i.e., we treat our model as a list of autoencoders, and train each autoencoder seperately. In the second stage of training, we use the second facade. These two facades are linked because:
* the autoencoders and the sigmoid layers of the MLP share parameters, and
* the latent representations computed by intermediate layers of the MLP are fed as input to the autoencoders.

Extracted from [webpage](http://deeplearning.net/tutorial/SdA.html)

Image: [Jigar Bandaria](https://miro.medium.com/max/701/1*wbaL5CvUkVkZxlSUsRS5IQ.png)

**Source**:

Image: [Jigar Bandaria](https://blog.insightdatascience.com/brain-mri-image-segmentation-using-stacked-denoising-autoencoders-4e91417688f6)

Webpage: [deeplearning.net](http://deeplearning.net/tutorial/SdA.html)

Webpage: [www.iro.umontreal.ca](http://www.iro.umontreal.ca/~pift6266/H10/notes/SdA.html)

Paper:

[Vincent, H. Larochelle Y. Bengio and P.A. Manzagol, Extracting and Composing Robust Features with Denoising Autoencoders](https://doi.org/10.1145/1390156.1390294)

[Vincent, H. Larochelle Y. Bengio and P.A. Manzagol, Extracting and Composing Robust Features with Denoising Autoencoders](http://www.iro.umontreal.ca/~lisa/publications2/index.php/publications/show/217)

SDAE

Graph Contrastive Coding

GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training

**Discrete Cosine Transform (DCT)** is an orthogonal transformation method that decomposes an
image to its spatial frequency spectrum. It expresses a finite sequence of data points in terms of a sum of cosine functions oscillating at different frequencies. It is used a lot in compression tasks, e..g image compression where for example high-frequency components can be discarded. It is a type of Fourier-related Transform, similar to discrete fourier transforms (DFTs), but only using real numbers.

Image Credit: [Wikipedia](https://en.wikipedia.org/wiki/Discrete_cosine_transform#/media/File:Example_dft_dct.svg)

Source	GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com