openwebtext Skylion007

Dataset Card for "openwebtext" Dataset Summary An open-source replication of the WebText dataset from OpenAI, that was used to train GPT-2. This distribution was created by Aaron Gokaslan and Vanya Cohen of Brown University. Supported Tasks and Leaderboards More Information Needed Languages More Information Needed Dataset Structure Data Instances plain_text Size of downloaded dataset… See the full description on the dataset page: https://huggingface.co/datasets/Skylion007/openwebtext.

类型
dataset
许可
cc0-1.0
语言
en
下载量
58,643
点赞
530
访问
public
文件
0

标签

  • 预训练语料
  • 文本数据集
  • 文本生成
  • 语言建模
  • 多语言语料
  • NLP

摘要

OpenWebText是OpenAI用于训练GPT-2的WebText数据集的开源复刻，由布朗大学的研究者构建，包含约800万个从Reddit热帖链接爬取的网页文档，共约40GB纯英文文本。该数据集经过多语言过滤、去重和清洗处理，广泛应用于大语言模型的预训练与语言建模任务，是NLP领域最具影响力的公开文本预训练语料之一。

README

--- annotations_creators: - no-annotation language_creators: - found language: - en license: - cc0-1.0 multilinguality: - monolingual pretty_name: OpenWebText size_categories: - 1M<n<10M source_datasets: - original task_categories: - text-generation - fill-mask task_ids: - language-modeling - maske…

查看完整页面 · 查看原文