**FastMoE ** is a distributed MoE training system based on PyTorch with common accelerators. The system provides a hierarchical interface for both flexible model design and adaption to different applications, such as [Transformer-XL](https://paperswithcode.com/method/transformer-xl) and Megatron-LM.

**NAS-FPN** is a Feature Pyramid Network that is discovered via [Neural Architecture Search](https://paperswithcode.com/method/neural-architecture-search) in a novel scalable search space covering all cross-scale connections. The discovered architecture consists of a combination of top-down and bottom-up connections to fuse features across scales

NAS-FPN

NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection

FastMoE

FastMoE: A Fast Mixture-of-Expert Training System

**TD3** builds on the [DDPG](https://paperswithcode.com/method/ddpg) algorithm for reinforcement learning, with a couple of modifications aimed at tackling overestimation bias with the value function. In particular, it utilises [clipped double Q-learning](https://paperswithcode.com/method/clipped-double-q-learning), delayed update of target and policy networks, and [target policy smoothing](https://paperswithcode.com/method/target-policy-smoothing) (which is similar to a [SARSA](https://paperswithcode.com/method/sarsa) based update; a safer update, as they provide higher value to actions resistant to perturbations).

Source	FastMoE: A Fast Mixture-of-Expert Training System
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com

Viet-Anh on Software

What is: FastMoE?

Viet-Anh on Software