MMMLU openai
Multilingual Massive Multitask Language Understanding (MMMLU) The MMLU is a widely recognized benchmark of general knowledge attained by AI models. It covers a broad range of topics from 57 different categories, covering elementary-level knowledge up to advanced professional subjects like law, physics, history, and computer science. We translated the MMLU’s test set into 14 languages using professional human translators. Relying on human translators for this evaluation increases… See the full description on the dataset page: https://huggingface.co/datasets/openai/MMMLU.
- 类型
- dataset
- 许可
- mit
- 语言
- ar
- 下载量
- 11,123
- 点赞
- 523
- 访问
- public
- 文件
- 0
标签
- 评测基准
- 多语言语料
- 问答
- 多语言
- 大语言模型
- NLP
- 测试数据
摘要
MMMLU是原版MMLU评测基准的多语言翻译版本，由OpenAI聘请专业人工译员将测试集翻译成14种语言（含阿拉伯语、孟加拉语、约鲁巴语等低资源语言）。它覆盖57个学科类别，从基础常识到法律、物理等专业领域，用于评估AI模型的多语言常识理解和问答能力。该数据集适用于多语言大模型的评测基准研究，特别关注低资源语言的评测覆盖。
README
--- task_categories: - question-answering configs: - config_name: default data_files: - split: test path: test/*.csv - config_name: AR_XY data_files: - split: test path: test/mmlu_AR-XY.csv - config_name: BN_BD data_files: - split: test path: test/mmlu_BN-BD.csv - config_name: DE_DE data_files: - s…