Forget BIT, It is All about TOKEN: Towards Semantic Information Theory for LLMs
arXiv:2511.01202v5 Announce Type: replace-cross
Abstract: Despite the empirical successes of Large Language Models (LLMs), the prevailing paradigm is heuristic and experiment-driven, tethered to massive compute and data, while a first-principles theor…