MINT-1T-HTML mlfoundations
🍃 MINT-1T:Scaling Open-Source Multimodal Data by 10x: A Multimodal Dataset with One Trillion Tokens 🍃 MINT-1T is an open-source Multimodal INTerleaved dataset with 1 trillion text tokens and 3.4 billion images, a 10x scale-up from existing open-source datasets. Additionally, we include previously untapped sources such as PDFs and ArXiv papers. 🍃 MINT-1T is designed to facilitate research in multimodal pretraining. 🍃 MINT-1T is created by a team from the University of Washington in… See the full description on the dataset page: https://huggingface.co/datasets/mlfoundations/MINT-1T-HTML.
- 类型
- dataset
- 许可
- cc-by-4.0
- 语言
- en
- 下载量
- 186,938
- 点赞
- 97
- 访问
- public
- 文件
- 0
标签
- 多模态数据集
- 预训练语料
- 多模态
- 大语言模型
- 视觉模型
- 预训练
- 文本数据集
- 图像数据集
摘要
MINT-1T 是目前最大的开源多模态交错数据集，包含1万亿文本Token和34亿张图像，规模是现有开源数据的10倍，并纳入PDF和ArXiv论文等此前未被利用的来源。该数据集解决了大规模多模态预训练数据稀缺的问题，通过严格的内容、图像安全和去重过滤保障质量，适合用于预训练Idefics2、XGen-MM、Chameleon等多模态大模型。
README
--- license: cc-by-4.0 task_categories: - image-to-text - text-generation language: - en tags: - multimodal pretty_name: MINT-1T size_categories: - 100B<n<1T configs: - config_name: data-v1.1 data_files: - split: train path: data_v1_1/*.parquet --- <h1 align="center"> 🍃 MINT-1T:<br>将开源多模态数据规模扩大 10…