gpqa Idavidrein
Dataset Card for GPQA GPQA is a multiple-choice, Q&A dataset of very hard questions written and validated by experts in biology, physics, and chemistry. When attempting questions out of their own domain (e.g., a physicist answers a chemistry question), these experts get only 34% accuracy, despite spending >30m with full access to Google. We request that you do not reveal examples from this dataset in plain text or images online, to reduce the risk of leakage into foundation… See the full description on the dataset page: https://huggingface.co/datasets/Idavidrein/gpqa.
- 类型
- dataset
- 许可
- cc-by-4.0
- 语言
- en
- 下载量
- 113,141
- 点赞
- 503
- 访问
- gated
- 文件
- 0
标签
- 评测基准
- 问答
- 科学数据
- 文本数据集
- 数学推理
摘要
GPQA是一个由生物学、物理学、化学领域专家编写并验证的多选题问答数据集，题目难度极高。专家在本领域外作答时准确率仅为34%（即使上网搜索30分钟以上），用于评测前沿大语言模型的科学推理能力。该数据集主要适用于大语言模型评测基准和科学知识推理能力的衡量。