Mixed Precision Training: The Art of Teaching AI to Think in Multiple Languages at Once

In 2019, a research team at NVIDIA discovered they could train a model twice as fast by simply changing how numbers were stored in memory.