dolphin QuixiAI
Dolphin 🐬 https://erichartford.com/dolphin Dataset details This dataset is an attempt to replicate the results of Microsoft's Orca Our dataset consists of: ~1 million of FLANv2 augmented with GPT-4 completions (flan1m-alpaca-uncensored.jsonl) ~3.5 million of FLANv2 augmented with GPT-3.5 completions (flan5m-alpaca-uncensored.jsonl) We followed the submix and system prompt distribution outlined in the Orca paper. With a few exceptions. We included all 75k of CoT in the FLAN-1m… See the full description on the dataset page: https://huggingface.co/datasets/QuixiAI/dolphin.
- 类型
- dataset
- 许可
- apache-2.0
- 语言
- en
- 下载量
- 1,541
- 点赞
- 434
- 访问
- public
- 文件
- 0
标签
- 指令微调
- 文本生成
- 大语言模型
- 预训练语料
- 合成数据
- 文本数据集
摘要
Dolphin 是一个旨在复现微软 Orca 论文结果的大规模指令微调数据集，包含约100万条利用GPT-4补全、350万条利用GPT-3.5补全的FLANv2增强样本。数据遵循Orca论文的submix分布与系统提示配置，并过滤了对齐、拒绝、回避和偏见样本，以产生可用于训练无审查模型的语料。该数据集适用于指令微调大语言模型，并允许在此基础上叠加个性化的对齐LoRA。
README
--- license: apache-2.0 task_categories: - text-generation language: - en configs: - config_name: flan1m-alpaca-uncensored data_files: flan1m-alpaca-uncensored.jsonl - config_name: flan5m-alpaca-uncensored data_files: flan5m-alpaca-uncensored.jsonl --- Dolphin 🐬 https://erichartford.com/dolphin ##…