Qwen1.5-MoE-A2.7B

Qwen1.5-MoE-A2.7B is Alibaba's first open mixture-of-experts model, released in March 2024. With 14.3B total and 2.7B active parameters, upcycled from Qwen-1.8B, it matched 7B dense models while cutting inference cost substantially.