How Do Modern LLMs Cheat the Scaling Laws? (In a Good Way).

If you’ve been following LLMs closely, you’ve probably noticed a pattern: parameter counts explode, GPU bills explode, but inference still…