The Power of Power Law: Asymmetry Enables Compositional Reasoning
arXiv:2604.22951v1 Announce Type: new
Abstract: Natural language data follows a power-law distribution, with most knowledge and skills appearing at very low frequency. While a common intuition suggests that reweighting or curating data towards a unifo…