MMMU-Pro

A more robust multi-discipline multimodal understanding benchmark that enhances MMMU through a three-step process: filtering text-only answerable questions, augmenting candidate options, and introducing vision-only input settings. Achieves significantly lower model performance (16.8-26.9%) compared to original MMMU, providing more rigorous evaluation that closely mimics real-world scenarios.

id mmmu-pro · max 1 · 72 models reported

#	Model	Score
1	Gemini 3.5 Flash Google	0.84
2	GPT-5.5 OpenAI	0.83
3	GPT-5.6 Sol OpenAI	0.83
4	Seed 2.1 Pro ByteDance	0.83
5	Qwen3.8 Max Alibaba Cloud / Qwen Team · open	0.82
6	Seed 2.1 Turbo ByteDance	0.82
7	Kimi K3 Moonshot AI · open	0.82
8	Gemini 3 Flash Google	0.81
9	GPT-5.4 OpenAI	0.81
10	Gemini 3 Pro Google	0.81
11	GPT-5.6 Terra OpenAI	0.81
12	Gemini 3.1 Pro Google	0.81
13	Muse Spark Meta	0.80
14	Kimi K2.6 Moonshot AI · open	0.80
15	GPT-5.2 OpenAI	0.80
16	Qwen3.7-Plus Alibaba Cloud / Qwen Team	0.79
17	Qwen3.6 Plus Alibaba Cloud / Qwen Team	0.79
18	Kimi K2.5 Moonshot AI · open	0.79
19	GPT-5 OpenAI	0.78
20	GPT-5.6 Luna OpenAI	0.78
21	MiniMax M3 MiniMax · open	0.78
22	MiMo-V2.5 Xiaomi · open	0.78
23	Claude Opus 4.6 Anthropic	0.77
24	Gemma 4 31B Google · open	0.77
25	Qwen3.5-122B-A10B Alibaba Cloud / Qwen Team · open	0.77
26	Gemini 3.1 Flash-Lite Google	0.77
27	GPT-5.4 mini OpenAI	0.77
28	o3 OpenAI	0.76
29	GPT-5.5 Instant OpenAI	0.76
30	Qwen3.6-27B Alibaba Cloud / Qwen Team · open	0.76
31	Claude Sonnet 4.6 Anthropic	0.76
32	Qwen3.6-35B-A3B Alibaba Cloud / Qwen Team · open	0.75
33	Qwen3.5-35B-A3B Alibaba Cloud / Qwen Team · open	0.75
34	Qwen3.5-27B Alibaba Cloud / Qwen Team · open	0.75
35	Inkling-Small Thinking Machines Lab · open	0.74
36	Muse Glimmer-30B Meta · open	0.74
37	Gemma 4 26B-A4B Google · open	0.74
38	Inkling Thinking Machines Lab · open	0.73
39	Seed 1.8 ByteDance	0.73
40	Seed 2.0 Mini ByteDance	0.71
41	Qwen3 VL 235B A22B Thinking Alibaba Cloud / Qwen Team · open	0.69
42	Gemma 4 12B Google · open	0.69
43	EXAONE 4.5 33B LG AI Research	0.69
44	Qwen3 VL 235B A22B Instruct Alibaba Cloud / Qwen Team · open	0.68
45	Qwen3 VL 32B Thinking Alibaba Cloud / Qwen Team · open	0.68
46	GPT-5.4 nano OpenAI	0.66
47	Qwen3 VL 32B Instruct Alibaba Cloud / Qwen Team · open	0.65
48	Nova 2 Pro Amazon	0.64
49	Command A+ Cohere · open	0.63
50	Qwen3 VL 30B A3B Thinking Alibaba Cloud / Qwen Team · open	0.63
51	Nova 2 Lite Amazon	0.62
52	Nova 2 Omni Amazon	0.61
53	Qwen3 VL 30B A3B Instruct Alibaba Cloud / Qwen Team · open	0.60
54	Qwen3 VL 8B Thinking Alibaba Cloud / Qwen Team · open	0.60
55	Mistral Small 4 Mistral AI · open	0.60
56	GPT-4o OpenAI	0.60
57	Llama 4 Maverick Meta · open	0.60
58	Qwen3 VL 4B Thinking Alibaba Cloud / Qwen Team · open	0.57
59	Qwen3 VL 8B Instruct Alibaba Cloud / Qwen Team · open	0.56
60	DiffusionGemma 26B-A4B Google · open	0.54
61	Qwen3 VL 4B Instruct Alibaba Cloud / Qwen Team · open	0.53
62	Gemma 4 E4B Google · open	0.53
63	Qwen2.5 VL 72B Instruct Alibaba Cloud / Qwen Team · open	0.51
64	Qwen2.5 VL 32B Instruct Alibaba Cloud / Qwen Team · open	0.49
65	Qwen2-VL-72B-Instruct Alibaba Cloud / Qwen Team · open	0.46
66	Llama 3.2 90B Instruct Meta · open	0.45
67	Gemma 4 E2B Google · open	0.44
68	Phi-4-multimodal-instruct Microsoft · open	0.39
69	Qwen2.5 VL 7B Instruct Alibaba Cloud / Qwen Team · open	0.38
70	Qwen2.5-Omni-7B Alibaba Cloud / Qwen Team · open	0.37
71	Llama 3.2 11B Instruct Meta · open	0.33
72	LFM2.5-VL-3B Liquid AI · open	0.30