The Auditory Cortex ResNet, briefly AUCO ResNet, is proposed and tested. It is a deep neural network architecture especially designed for audio classification trained end-to-end. It is inspired by the architectural organization of rat's auditory cortex, containing also innovations 2 and 3. The network outperforms the state-of-the-art accuracies on a reference audio benchmark dataset without any kind of preprocessing, imbalanced data handling and, most importantly, any kind of data augmentation.

**OMGD**, or **Online Multi-Granularity Distillation** is a framework for learning efficient [GANs](https://paperswithcode.com/methods/category/generative-adversarial-networks). The student generator is optimized in a discriminator-free and ground-truth-free setting. The scheme trains the teacher and student alternatively, promoting these two generators iteratively and progressively. The progressively optimized teacher generator helps to warm up the student and guide the optimization direction step by step.

Specifically, the student generator $G\_{S}$ only leverages the complementary teacher generators $G^{W}\_{T}$ and $G^{D}\_{T}$ for optimization and can be trained in the discriminator-free and ground-truth-free setting. This framework transfers different levels concepts from the intermediate layers and output layer to perform the knowledge distillation. The whole optimization is conducted on an online distillation scheme. Namely,  $G^{W}\_{T}$, $G^{D}\_{T}$ and $G\_{S}$ are optimized simultaneously and progressively.

OMGD

Online Multi-Granularity Distillation for GAN Compression

AUCO ResNet

AUCO ResNet: an end-to-end network for Covid-19 pre-screening from cough and breath

Gradual self-training is a method for semi-supervised domain adaptation. The goal is to adapt an initial classifier trained on a source domain given only unlabeled data that shifts gradually in distribution towards a target domain. 

This comes up for example in applications ranging from sensor networks and self-driving car perception modules to brain-machine interfaces, where machine learning systems must adapt to data distributions that evolve over time.

The gradual self-training algorithm begins with a classifier $w_0$ trained on labeled examples from the source domain (Figure a). For each successive domain $P_t$, the algorithm generates pseudolabels for unlabeled examples from that domain, and then trains a regularized supervised classifier on the pseudolabeled examples. The intuition, visualized in the Figure, is that after a single gradual shift, most examples are pseudolabeled correctly so self-training learns a good classifier on the shifted data, but the shift from the source to the target can be too large for self-training to correct.

Source	AUCO ResNet: an end-to-end network for Covid-19 pre-screening from cough and breath
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com