Viet-Anh on Software Logo

What is: PipeDream?

Year2019
Data SourceCC BY-SA - https://paperswithcode.com

PipeDream is an asynchronous pipeline parallel strategy for training large neural networks. It adds inter-batch pipelining to intra-batch parallelism to further improve parallel training throughput, helping to better overlap computation with communication and reduce the amount of communication when possible.