ERNIE 2.0

Name: Enhanced Representation through kNowledge IntEgration 2.0
Author: 百度

基础大模型

Enhanced Representation through kNowledge IntEgration 2.0

Release date: 2019-07-00更新于: 2022-05-10 20:13:45.519534

Live demo

Enhanced Representation through kNowledge IntEgration 2.0 is an AI model published by 百度, released on 2019-07-00, for 基础大模型, with 3.4B parameters, and 2K tokens context length.

Data sourced primarily from official releases (GitHub, Hugging Face, papers), then benchmark leaderboards, then third-party evaluators. Learn about our data methodology

ERNIE 2.0

Model basics

Reasoning traces

Not supported

Thinking modes

Thinking modes not supported

Context length

2K tokens

Max output length

No data

Model type

基础大模型

Release date

2019-07-00

Model file size

No data

MoE architecture

Total params / Active params

3.4B / N/A

Knowledge cutoff

No data

ERNIE 2.0

Open source & experience

Code license

No data

Weights license

No data

GitHub repo

GitHub link unavailable

Hugging Face

Hugging Face link unavailable

Live demo

No live demo

Enhanced Representation through kNowledge IntEgration 2.0

Model Overview

最近，预训练模型在各种语言理解任务中取得了最先进的成果，这表明在大规模语料库上的预训练可能在自然语言处理中发挥了关键作用。目前的预训练程序通常侧重于用几个简单的任务来训练模型，以掌握单词或句子的共现性。然而，除了共现之外，训练语料中还存在其他有价值的词汇、句法和语义信息，如命名实体、语义接近度和话语关系。为了最大限度地提取训练语料中的词汇、句法和语义信息，我们提出了一个名为ERNIE 2.0的持续预训练框架，该框架通过不断的多任务学习，逐步建立和学习预训练任务。实验结果表明，ERNIE 2.0在16个任务上的表现优于BERT和XLNet，包括GLUE基准上的英语任务和中文中的几个常见任务。

ERNIE 2.0

Model basics

Open source & experience

Official resources

API details

Benchmark Results

Publisher

Model Overview

DataLearner 官方微信