GPQA

A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.

id gpqa · max 1 · 250 models reported

#	Model	Score
1	GPT-6 Astra OpenAI	0.96
2	Claude Mythos Preview Anthropic	0.95
3	GPT-5.6 Sol OpenAI	0.95
4	Gemini 3.1 Pro Google	0.94
5	Claude Opus 4.7 Anthropic	0.94
6	Claude Opus 4.8 Anthropic	0.94
7	GPT-5.5 OpenAI	0.94
8	Kimi K3 Moonshot AI · open	0.94
9	GPT-5.2 Pro OpenAI	0.93
10	Grok 4.5 xAI	0.93
11	GPT-5.6 Terra OpenAI	0.93
12	GPT-5.4 OpenAI	0.93
13	Qwen3.8 Max Alibaba Cloud / Qwen Team · open	0.93
14	GPT-5.2 OpenAI	0.92
15	Qwen3.7 Max Alibaba Cloud / Qwen Team	0.92
16	GPT-5.6 Luna OpenAI	0.92
17	Hy4 preview Tencent · open	0.92
18	Gemini 3 Pro Google	0.92
19	Qwen3.8 Flash Alibaba Cloud / Qwen Team	0.92
20	Qwen3.8-Flash-Next Alibaba Cloud / Qwen Team · open	0.92
21	Claude Opus 4.6 Anthropic	0.91
22	GLM-5.2 Zhipu AI · open	0.91
23	DeepSeek-V4.1-Flash DeepSeek · open	0.91
24	Kimi K2.6 Moonshot AI · open	0.91
25	Gemini 3 Flash Google	0.90
26	Hy3 Tencent · open	0.90
27	Qwen3.6 Plus Alibaba Cloud / Qwen Team	0.90
28	Qwen3.7-Plus Alibaba Cloud / Qwen Team	0.90
29	DeepSeek-V4-Pro-Max DeepSeek · open	0.90
30	Claude Sonnet 4.6 Anthropic	0.90
31	Inkling-Small Thinking Machines Lab · open	0.90
32	Muse Spark Meta	0.90
33	Qwen3.8-27B Alibaba Cloud / Qwen Team · open	0.89
34	Solar Pro 4 Upstage	0.89
35	Seed 2.0 Pro ByteDance	0.89
36	Grok-4 Heavy xAI	0.88
37	Qwen3.5-397B-A17B Alibaba Cloud / Qwen Team · open	0.88
38	DeepSeek-V4-Flash-Max DeepSeek · open	0.88
39	GPT-5 Medium OpenAI	0.88
40	GPT-5.1 OpenAI	0.88
41	GPT-5.1 High OpenAI	0.88
42	GPT-5.1 Instant OpenAI	0.88
43	GPT-5.1 Thinking OpenAI	0.88
44	GPT-5.4 mini OpenAI	0.88
45	Qwen3.6-27B Alibaba Cloud / Qwen Team · open	0.88
46	Kimi K2.5 Moonshot AI · open	0.88
47	Grok-4 xAI	0.88
48	DeepSeek-V4-Flash-0423 DeepSeek · open	0.87
49	Qwen3 Max Thinking Alibaba Cloud / Qwen Team	0.87
50	GPT-5 High OpenAI	0.87
51	Claude Opus 4.5 Anthropic	0.87
52	Nemotron 3 Ultra (550B A55B) NVIDIA · open	0.87
53	Gemini 3.1 Flash-Lite Google	0.87
54	Qwen3.5-122B-A10B Alibaba Cloud / Qwen Team · open	0.87
55	Gemini 2.5 Pro Preview 06-05 Google	0.86
56	GLM-5.1 Zhipu AI · open	0.86
57	Qwen3.6-35B-A3B Alibaba Cloud / Qwen Team · open	0.86
58	GLM-4.7 Zhipu AI · open	0.86
59	GPT-5 OpenAI	0.86
60	Grok 4 Fast xAI	0.86
61	GPT-5.5 Instant OpenAI	0.86
62	Qwen3.5-27B Alibaba Cloud / Qwen Team · open	0.85
63	Seed 2.0 Lite ByteDance	0.85
64	ERNIE 5.0 Baidu	0.85
65	Claude 3.7 Sonnet Anthropic	0.85
66	Grok-3 xAI	0.85
67	MAI-Code-1-Flash Microsoft	0.85
68	Kimi K2-Thinking-0905 Moonshot AI · open	0.84
69	Gemma 4 31B Google · open	0.84
70	MAI-Thinking-1 Microsoft	0.84
71	Qwen3.5-35B-A3B Alibaba Cloud / Qwen Team · open	0.84
72	ChatGPT-4o Latest OpenAI	0.84
73	Grok-3 Mini xAI	0.84
74	MiMo-V2-Flash Xiaomi · open	0.84
75	Muse Glimmer-30B Meta · open	0.83
76	Claude Sonnet 4.5 Anthropic	0.83
77	o3 OpenAI	0.83
78	Gemini 2.5 Pro Google	0.83
79	Gemini 2.5 Flash Google	0.83
80	GPT-5.4 nano OpenAI	0.83
81	Nemotron 3 Super (120B A12B) NVIDIA · open	0.83
82	DeepSeek-V3.2 DeepSeek · open	0.82
83	DeepSeek-V3.2 (Thinking) DeepSeek · open	0.82
84	Gemma 4 26B-A4B Google · open	0.82
85	GPT-5 mini OpenAI	0.82
86	Qwen3.5-9B Alibaba Cloud / Qwen Team · open	0.82
87	LongCat-Flash-Thinking Meituan · open	0.81
88	Nova 2 Pro Amazon	0.81
89	o4-mini OpenAI	0.81
90	Qwen3-235B-A22B-Thinking-2507 Alibaba Cloud / Qwen Team · open	0.81
91	DeepSeek-R1-0528 DeepSeek · open	0.81
92	GLM-4.6 Zhipu AI · open	0.81
93	MiniMax M2.1 MiniMax · open	0.81
94	Claude Opus 4.1 Anthropic	0.81
95	GPT OSS 120B High OpenAI · open	0.81
96	EXAONE 4.5 33B LG AI Research	0.81
97	LongCat-Flash-Thinking-2601 Meituan · open	0.81
98	GPT OSS 120B OpenAI · open	0.80
99	DeepSeek-V3.2-Exp DeepSeek · open	0.80
100	Claude Opus 4 Anthropic	0.80