On Semantic Loss Fine-Tuning Approach for Preventing Model Collapse in Causal Reasoning
arXiv:2605.05438v1 Announce Type: new
Abstract: Standard fine-tuning of transformer models on causal reasoning tasks leads to catastrophic model collapse, where models learn trivial solutions such as always predicting “Yes” or “No” regardless of input…