A resnet-like architecture with deconvolution feature normalization (Ye et al. 2020, ICLR) layers in the first few layers for sparse low-level feature identification, and batch normalization layers in the later layers.

**SGDW** is a stochastic optimization technique that decouples [weight decay](https://paperswithcode.com/method/weight-decay) from the gradient update:

$$ g\_{t} =  \nabla{f\_{t}}\left(\theta\_{t-1}\right) + \lambda\theta\_{t-1}$$

$$ m\_{t} =  \beta\_{1}m\_{t-1} + \eta\_{t}\alpha{g}\_{t}$$

$$ \theta\_{t} = \theta\_{t-1} - m\_{t} - \eta\_{t}\lambda\theta\_{t-1}$$

SGDW

Decoupled Weight Decay Regularization

hdxresnet

Predicting galaxy spectra from images with hybrid convolutional neural networks

A **Connectionist Temporal Classification Loss**, or **CTC Loss**, is designed for tasks where we need alignment between sequences, but where that alignment is difficult - e.g. aligning each character to its location in an audio file. It calculates a loss between a continuous (unsegmented) time series and a target sequence. It does this by summing over the probability of possible alignments of input to target, producing a loss value which is differentiable with respect to each input node. The alignment of input to target is assumed to be “many-to-one”, which limits the length of the target sequence such that it must be $\leq$ the input length.

Source	Predicting galaxy spectra from images with hybrid convolutional neural networks
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com