**EfficientNetV2** is a type convolutional neural network that has faster training speed and better parameter efficiency than [previous models](https://paperswithcode.com/method/efficientnet). To develop these models, the authors use a combination of training-aware [neural architecture search](https://paperswithcode.com/method/neural-architecture-search) and scaling, to jointly optimize training speed. The models were searched from the search space enriched with new ops such as [Fused-MBConv](https://ai.googleblog.com/2019/08/efficientnet-edgetpu-creating.html).

Architecturally the main differences are:

- EfficientNetV2 extensively uses both [MBConv](https://paperswithcode.com/method/inverted-residual-block)  and the newly added fused-MBConv in the early layers.
- EfficientNetV2 prefers smaller expansion ratio for [MBConv](https://paperswithcode.com/method/inverted-residual-block) since smaller expansion ratios tend to have less memory access overhead.
- EfficientNetV2 prefers smaller 3x3 kernel sizes, but it adds more layers to compensate the reduced receptive field resulted from the smaller kernel size. 
- EfficientNetV2 completely removes the last stride-1 stage in the original EfficientNet, wperhaps due to its large parameter size and memory access overhead.

**XGrad-CAM**, or **Axiom-based Grad-CAM**, is a class-discriminative visualization method and able to highlight the regions belonging to the objects of interest. Two axiomatic properties are introduced in the derivation of XGrad-CAM: Sensitivity and Conservation. In particular, the proposed XGrad-CAM is still a linear combination of feature maps, but able to meet the constraints of those two axioms.

XGrad-CAM

Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNs

EfficientNetV2

EfficientNetV2: Smaller Models and Faster Training

**A Spatial Attention-Guided Mask** is a module for [instance segmentation](https://paperswithcode.com/task/instance-segmentation) that predicts a segmentation mask on each detected box with a spatial attention map that helps to focus on informative pixels and suppress noise. The goal is to guide the mask head for spotlighting meaningful pixels and repressing uninformative ones. 

Once features inside the predicted RoIs are extracted by [RoIAlign](https://paperswithcode.com/method/roi-align) with 14×14 resolution, those features are fed into four conv layers and the [spatial attention module](https://paperswithcode.com/method/spatial-attention-module) (SAM) sequentially. To exploit the spatial attention map $A\_{sag}\left(X\_{i}\right) \in \mathcal{R}^{1\times{W}\times{H}}$ as a feature descriptor given input feature map $X\_{i} \in \mathcal{R}^{C×W×H}$, the SAM first generates pooled features $P\_{avg}, P\_{max} \in \mathcal{R}^{1\times{W}\times{H}}$ by both average and [max pooling](https://paperswithcode.com/method/max-pooling) operations respectively along the channel axis and aggregates them via concatenation. Then it is followed by a 3 × 3 conv layer and normalized by the sigmoid function. The computation process
is summarized as follow:

$$
A\_{sag}\left(X\_{i}\right) = \sigma\left(F\_{3\times{3}}(P\_{max} \cdot P\_{avg})\right)
$$

where $\sigma$ denotes the sigmoid function, $F\_{3\times{3}}$ is 3 × 3 conv layer and $\cdot$ represents the concatenate operation. Finally, the attention guided feature map $X\_{sag} ∈ \mathcal{R}^{C\times{W}\times{H}}$ is computed as:

$$
X\_{sag} = A\_{sag}\left(X\_{i}\right) \otimes X\_{i}
$$

where ⊗ denotes element-wise multiplication. After then, a 2 × 2 deconv upsamples the spatially attended feature map to 28 × 28 resolution. Lastly, a 1 × 1 conv is applied for predicting class-specific masks.

Source	EfficientNetV2: Smaller Models and Faster Training
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com