Goose: Anisotropic Speculation Trees for Training-Free Speculative Decoding
arXiv:2604.02047v1 Announce Type: new
Abstract: Speculative decoding accelerates large language model inference by drafting multiple candidate tokens and verifying them in a single forward pass. Candidates are organized as a tree: deeper trees accept …