**HardELiSH** is an activation function for neural networks.  The HardELiSH is a multiplication of the [HardSigmoid](https://paperswithcode.com/method/hard-sigmoid) and [ELU](https://paperswithcode.com/method/elu) in the negative part and a multiplication of the Linear and the HardSigmoid in the positive
part:

$$f\left(x\right) = x\max\left(0, \min\left(1, \left(\frac{x+1}{2}\right)\right) \right) \text{ if } x \geq 1$$
$$f\left(x\right) = \left(e^{x}-1\right)\max\left(0, \min\left(1, \left(\frac{x+1}{2}\right)\right)\right) \text{ if } x < 0 $$

Source: [Activation Functions](https://arxiv.org/pdf/1811.03378.pdf)

using transfer learning to transfer knowledge from one building to predict the energy consumption of another building with scarce data

b2b transfer learning

Transfer Learning in Deep Learning Models for Building Load Forecasting: Case of Limited Data

HardELiSH

The Quest for the Golden Activation Function

**SegFormer** is a [Transformer](https://paperswithcode.com/methods/category/transformers)-based framework for semantic segmentation that unifies Transformers with lightweight [multilayer perceptron](https://paperswithcode.com/method/feedforward-network) (MLP) decoders. SegFormer has two appealing features: 1) SegFormer comprises a novel hierarchically structured Transformer encoder which outputs multiscale features. It does not need positional encoding, thereby avoiding the interpolation of positional codes which leads to decreased performance when the testing resolution differs from training. 2) SegFormer avoids complex decoders. The proposed MLP decoder aggregates information from different layers, and thus combining both local attention and global attention to render powerful representations.

Source	The Quest for the Golden Activation Function
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com