Data sourced primarily from official releases (GitHub, Hugging Face, papers), then benchmark leaderboards, then third-party evaluators. Learn about our data methodology

LongChat-7B-16K

Model basics

Reasoning traces

Not supported

Context length

16K tokens

Max output length

No data

Model type

基础大模型

Release date

2023-06-29

Model file size

13.5GB

MoE architecture

Total params / Active params

70.0B / N/A

Knowledge cutoff

No data

Inference modes

No mode data

LongChat-7B-16K

Open source & experience

Code license

Apache 2.0

Weights license

开源不可商用- 不可以商用

GitHub repo

https://github.com/DachengLi1/LongChat

Hugging Face

https://huggingface.co/lmsys/longchat-7b-16k

Live demo

No live demo

LongChat-7B-16K

Official resources

Paper

How Long Can Open-Source LLMs Truly Promise on Context Length?

DataLearnerAI blog

No blog post yet

LongChat-7B-16K

API details

API speed

No data

No public API pricing yet.

LongChat-7B-16K

Benchmark Results

No benchmark data to show.

LongChat-7B-16K

Publisher

LM-SYS

View publisher details

LongChat-7B-16K

Model Overview

LongChat-7B-16K是由LM-SYS开源的一个支持超长上下文输入的聊天大语言模型。也是目前开源领域对超长上下文支持最好的模型之一。

LongChat-7B-16K是基于MetaAI开眼的LLaMA-7B进行微调得到的。使用ShareGPT中提供的用户与GPT的对话数据微调得到。原始的LLaMA模型只支持2048长度的输入。而LongChat-7B则通过将原始输入长度进行重新编码后得到支持16K长度的上下文输入。具体方法参考： https://www.datalearner.com/blog/1051688257255268

Foundation model

LLaMA

View details

DataLearner 官方微信

欢迎关注 DataLearner 官方微信，获得最新 AI 技术推送