Moonlight-16B-A3B

Moonlight-16B-A3B is the pre-trained base version of Moonshot AI's open mixture-of-experts model released on February 23, 2025, with 16B total and 3B active parameters. It was trained on 5.7 trillion tokens with the Muon optimizer to demonstrate Muon's efficiency at scale, and is released under the MIT License.