Experts Weights Averaging: A New General Training Scheme for Vision Transformers
Paper
• 2308.06093 • Published
• 2
Merging
-
-
Platypus: Quick, Cheap, and Powerful Refinement of LLMs
Paper • 2308.07317 • Published • 25 -
Beyond Attentive Tokens: Incorporating Token Importance and Diversity for Efficient Vision Transformers
Paper • 2211.11315 • Published • 1 -
LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition
Paper • 2307.13269 • Published • 34 -
Resolving Interference When Merging Models
Paper • 2306.01708 • Published • 17 -
ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization
Paper • 2311.13171 • Published • 1 -
LM-Cocktail: Resilient Tuning of Language Models via Model Merging
Paper • 2311.13534 • Published • 3 -
Merging Experts into One: Improving Computational Efficiency of Mixture of Experts
Paper • 2310.09832 • Published • 1 -
ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs
Paper • 2311.13600 • Published • 47 -
Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy
Paper • 2310.01334 • Published • 3 -
Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging
Paper • 2406.16330 • Published • 1