Viet-Anh on Software Logo

What is: Tofu?

SourceSupporting Very Large Models using Automatic Dataflow Graph Partitioning
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Tofu is an intra-layer model parallel system that partitions very large DNN models across multiple GPU devices to reduce per-GPU memory footprint. Tofu is designed to partition a dataflow graph of fine-grained tensor operators used by platforms like MXNet and TensorFlow. To optimally partition different operators in a dataflow graph, Tofu uses a recursive search algorithm that minimizes the total communication cost.