Preference-Aligned LoRA Merging: Preserving Subspace Coverage and Addressing Directional Anisotropy
arXiv:2603.26299v1 Announce Type: new
Abstract: Merging multiple Low-Rank Adaptation (LoRA) modules is promising for constructing general-purpose systems, yet challenging because LoRA update directions span different subspaces and contribute unevenly….