**LLaMA** is a collection of foundation language models ranging from 7B to 65B parameters. It is based on the transformer architecture with various improvements that were subsequently proposed. The main difference with the original architecture are listed below.

- RMSNorm normalizing function is used to improve the training stability, by normalizing the input of each transformer sub-layer, instead of normalizing the output.
- The ReLU non-linearity is replaced by the SwiGLU activation function to improve performance.
- Absolute positional embeddings are removed and instead rotary positional embeddings (RoPE) are added at each layer of the network.

The Contour Proposal Network (CPN) detects possibly overlapping objects in an image while simultaneously fitting pixel-precise closed object contours. The CPN can incorporate state of the art object detection architectures as backbone networks into a fast single-stage instance segmentation model that can be trained end-to-end.

Contour Proposal Networks for Biomedical Instance Segmentation

LLaMA

LLaMA: Open and Efficient Foundation Language Models

**Spatial Group-wise Enhance** is a module for convolutional neural networks that can adjust the
importance of each sub-feature by generating an attention factor for each spatial location in each semantic group, so that every individual group can autonomously enhance its learnt expression and suppress possible noise

Inside each feature group, we model a spatial enhance mechanism inside each feature group, by scaling the feature vectors over all the locations with an attention mask. This attention mask is designed to suppress the possible noise and highlight the correct semantic feature regions. Different from other popular attention methods, it utilises the similarity between the global statistical feature and the local ones of each location as the source of generation for the attention masks.

Source	LLaMA: Open and Efficient Foundation Language Models
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com