**TransferQA** is a transferable generative QA model, built upon [T5](https://paperswithcode.com/method/t5) that combines extractive QA and multi-choice QA via a text-to-text [transformer](https://paperswithcode.com/method/transformer) framework, and tracks both categorical slots and non-categorical slots in DST. In addition, it introduces two effective ways to construct unanswerable questions, namely, negative question sampling and context truncation, which enable the model to handle “none” value slots in the zero-shot DST setting.

**SimCLRv2** is a semi-supervised learning method for learning from few labeled examples while making best use of a large amount of unlabeled data. It is a modification of a recently proposed contrastive learning framework, [SimCLR](https://www.paperswithcode.com/method/simclr). It improves upon it in three major ways:

1. To fully leverage the power of general pre-training, larger [ResNet](https://paperswithcode.com/method/resnet) models are explored. Unlike SimCLR and other previous work, whose largest model is ResNet-50 (4×), SimCLRv2 trains models that are deeper but less wide. The largest model trained is a 152 layer ResNet with 3× wider channels and [selective kernels](https://paperswithcode.com/method/selective-kernel-convolution) (SK), a channel-wise attention mechanism that improves the parameter efficiency of the network. By scaling up the model from ResNet-50 to ResNet-152 (3×+SK), a 29% relative improvement is obtained in top-1 accuracy when fine-tuned on 1% of labeled examples.

2. The capacity of the non-linear network $g(·)$ (a.k.a. projection head) is increased, by making it deeper. Furthermore, instead of throwing away $g(·)$ entirely after pre-training as in SimCLR, fine-tuning occurs from a middle layer. This small change yields a significant improvement for both linear evaluation and fine-tuning with only a few labeled examples. Compared to SimCLR with 2-layer projection head, by using a 3-layer projection head and fine-tuning from the 1st layer of projection head, it results in as much as 14% relative improvement in top-1 accuracy when fine-tuned on 1% of labeled examples.

3. The memory mechanism of [MoCo v2](https://paperswithcode.com/method/moco-v2) is incorporated, which designates a memory network (with a moving average of weights for stabilization) whose output will be buffered as negative examples. Since training is based on large mini-batch which already supplies many contrasting negative examples, this change yields an improvement of ∼1% for linear evaluation as well as when fine-tuning on 1% of labeled examples.

SimCLRv2

Big Self-Supervised Models are Strong Semi-Supervised Learners

TransferQA

Zero-Shot Dialogue State Tracking via Cross-Task Transfer

**BIDeN**, or **Blind Image Decomposition Network**, is a model for blind image decomposition, which requires separating a superimposed image into constituent underlying images in a blind setting, that is, both the source components involved in mixing as well as the mixing mechanism are unknown.  For example, rain may consist of multiple components, such as rain streaks, raindrops, snow, and haze. 

The Figure shows an example where $N = 4, L = 2, x = {a, b, c, d}$, and $I = {1, 3}$. $a, c$ are selected then passed to the mixing function $f$, and outputs the mixed input image $z$, which is $f\left(a, c\right)$ here. The generator consists of an encoder $E$ with three branches and multiple heads $H$. $\bigotimes$ denotes the concatenation operation. Depth and receptive field of each branch is different to capture multiple scales of features. Each specified head points to the corresponding source component, and the number of heads varies with the maximum number of source components N. All reconstructed images $\left(a', c'\right)$ and their corresponding real images $\left(a, c\right)$ are sent to an unconditional discriminator. The discriminator also predicts the source components of the input image $z$. The outputs from other heads $\left(b', d'\right)$ do not contribute to the optimization.

Source	Zero-Shot Dialogue State Tracking via Cross-Task Transfer
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com