Accelerate Large Model Training using PyTorch Fully Sharded Data ParallelBy Hugging Face - Blog / May 2, 2022