**UNETR**, or **UNet Transformer**, is a [Transformer](https://paperswithcode.com/methods/category/transformers)-based architecture for [medical image segmentation](https://paperswithcode.com/task/medical-image-segmentation) that utilizes a pure [transformer](https://paperswithcode.com/method/transformer) as the encoder to learn sequence representations of the input volume -- effectively capturing the global multi-scale information. The transformer encoder is directly connected to a decoder via [skip connections](https://paperswithcode.com/methods/category/skip-connections) at different resolutions like a [U-Net](https://paperswithcode.com/method/u-net) to compute the final semantic segmentation output.

**Implicit PointRend** is a modification to the [PointRend](https://paperswithcode.com/method/pointrend) module for instance segmentation. Instead of a coarse mask prediction used in [PointRend](https://paperswithcode.com/method/pointrend) to provide region-level context to distinguish objects, for each object Implicit PointRend generates different parameters for a function that makes the final pointwise mask prediction. The new model is more straightforward than PointRend: (1) it does not require an importance point sampling during training and (2) it uses a single point-level mask loss instead of two mask losses. Implicit PointRend can be trained directly with point supervision without any intermediate prediction interpolation steps.

Implicit PointRend

Pointly-Supervised Instance Segmentation

UNETR

UNETR: Transformers for 3D Medical Image Segmentation

**Layer-wise Adaptive Rate Scaling**, or **LARS**, is a large batch optimization technique.  There are two notable differences between LARS and other adaptive algorithms such as [Adam](https://paperswithcode.com/method/adam) or [RMSProp](https://paperswithcode.com/method/rmsprop): first, LARS uses a separate learning rate for each layer and not for each weight. And second, the magnitude of the update is controlled with respect to the weight norm for better control of training speed.

$$m\_{t} = \beta\_{1}m\_{t-1} + \left(1-\beta\_{1}\right)\left(g\_{t} + \lambda{x\_{t}}\right)$$
$$x\_{t+1}^{\left(i\right)} = x\_{t}^{\left(i\right)}  - \eta\_{t}\frac{\phi\left(|| x\_{t}^{\left(i\right)} ||\right)}{|| m\_{t}^{\left(i\right)} || }m\_{t}^{\left(i\right)} $$

Source	UNETR: Transformers for 3D Medical Image Segmentation
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com