**Structurally Regularized Deep Clustering**, or **SRDC**, is a deep network based discriminative clustering method for domain adaptation that minimizes the KL divergence between predictive label distribution of the network and an introduced auxiliary one. Replacing the auxiliary distribution with that formed by ground-truth labels of source data implements the structural source regularization via a simple strategy of joint network training.

**ReGLU** is an activation function which is a variant of [GLU](https://paperswithcode.com/method/glu). The definition is as follows:

$$ \text{ReGLU}\left(x, W, V, b, c\right) = \max\left(0, xW + b\right) \otimes \left(xV + c\right) $$

ReGLU

GLU Variants Improve Transformer

SRDC

Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering

**Twins-SVT** is a type of [vision transformer](https://paperswithcode.com/methods/category/vision-transformer) which utilizes a [spatially separable attention mechanism](https://paperswithcode.com/method/spatially-separable-self-attention) (SSAM) which is composed of two types of attention operations—(i) locally-grouped self-attention (LSA), and (ii) global sub-sampled attention (GSA), where LSA captures the fine-grained and short-distance information and GSA deals with the long-distance and global information. On top of this, it utilizes [conditional position encodings](https://paperswithcode.com/method/conditional-positional-encoding) as well as the architectural design of the [Pyramid Vision Transformer](https://paperswithcode.com/method/pvt).

Source	Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com