[Paper] Meta Expert: Cross-Layer Expert Sharing for Storage- and Retrieval-Efficient MoE
As model parameters scale up, sparsely activated Mixture-of-Experts (MoE) approaches have become essential for model scaling.
As model parameters scale up, sparsely activated Mixture-of-Experts (MoE) approaches have become essential for model scaling.