MosaicML Pretrained Transformer - 30B Instruct
MosaicML Pretrained Transformer - 30B Instruct is an AI model published by MosaicML, released on 2023-06-22, for 基础大模型, with 300.0B parameters, and 2K tokens context length, requiring about 60 storage, under the CC-BY-NC-SA-3.0 license.
Data sourced primarily from official releases (GitHub, Hugging Face, papers), then benchmark leaderboards, then third-party evaluators. Learn about our data methodology
MPT-30B-Instruct是MosaicML开源的MPT-30B系列中针对指令进行微调的大模型。该模型代码依然采用Apache 2.0开源协议开源，但是其预训练结果却是CC-BY-NC-SA-3.0协议，不允许商用。
MPT-30B是MosaicML训练的300亿参数规模的大模型，其基础模型相比较7B版本有了很大的改进，在效果和训练推理效率上都有很大的提升。
MPT-30B-Instruct通过在Dolly HHRLHF上对MPT-30B进行微调而建立的，Dolly HHRLHF来源于Databricks Dolly-15k和Anthropic Helpful and Harmless（HH-RLHF）数据集。它还在Competition Math、Duorc、CoT GSM8k、Qasper、Quality、Summ Screen FD和Spider上训练。
欢迎关注 DataLearner 官方微信，获得最新 AI 技术推送
