Fine-Tuning a 7B LLM Required 4 A100s. LoRA Did It on One GPU. Here Is the Math Behind Why.

A complete technical deep-dive into Low-Rank Adaptation, the mathematics of matrix decomposition, QLoRA with 4-bit NF4 quantisation, the…