Stevessr 在 LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation [转] (LoRA with Reduced Interference ，是LoRA的一个变体) 中发帖Low-Rank Adaptation (LoRA) has emerged as a popular parameter-efficient fine-tuning (PEFT) method for Large Language Models (LLMs), yet it still incurs notable overhead and suffers from parameter interference in multi-task scenarios. We propose LoRA with Reduced Interference (LoRI), a simple yet effective approach that freezes the projection matrices A as random projections and sparsifies the...

Stevessr 在 LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation [转] (LoRA with Reduced Interference ，是LoRA的一个变体) 中发帖

Low-Rank Adaptation (LoRA) has emerged as a popular parameter-efficient fine-tuning (PEFT) method for Large Language Models (LLMs), yet it still incurs notable overhead and suffers from parameter interference in multi-task scenarios. We propose LoRA with Reduced Interference (LoRI), a simple yet effective approach that freezes the projection matrices A as random projections and sparsifies the...