LoRA-FA: Efficient and Effective Low Rank Representation Fine-tuning
arXiv:2308.03303v2 Announce Type: replace
Abstract: Fine-tuning large language models (LLMs) is crucial for improving their performance on downstream tasks, but full-parameter fine-tuning (Full-FT) is computationally expensive and memory-intensive. Pa…