To Memorize or to Retrieve: Scaling Laws for RAG-Considerate Pretraining
arXiv:2604.00715v1 Announce Type: new
Abstract: Retrieval-augmented generation (RAG) improves language model (LM) performance by providing relevant context at test time for knowledge-intensive situations. However, the relationship between parametric k…