MMLU

Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains

id mmlu · max 1 · 103 models reported

#	Model	Score
1	GPT-5 OpenAI	0.93
2	Seed 1.8 ByteDance	0.92
3	o1 OpenAI	0.92
4	GPT-4.5 OpenAI	0.91
5	o1-preview OpenAI	0.91
6	Qwen3 VL 235B A22B Thinking Alibaba Cloud / Qwen Team · open	0.91
7	Sarvam-105B Sarvam AI · open	0.91
8	Claude 3.5 Sonnet Anthropic	0.90
9	Claude 3.5 Sonnet Anthropic	0.90
10	GPT-4.1 OpenAI	0.90
11	Kimi K2 0905 Moonshot AI	0.90
12	GPT OSS 120B OpenAI · open	0.90
13	LongCat-Flash-Chat Meituan · open	0.90
14	Kimi K2 Instruct Moonshot AI · open	0.90
15	Kimi K2-Instruct-0905 Moonshot AI · open	0.90
16	MiMo-V2.5-Pro Xiaomi · open	0.89
17	Qwen3 VL 235B A22B Instruct Alibaba Cloud / Qwen Team · open	0.89
18	GPT-4o OpenAI	0.89
19	Qwen3 VL 32B Thinking Alibaba Cloud / Qwen Team · open	0.89
20	DeepSeek-V3 DeepSeek · open	0.89
21	Qwen3 235B A22B Alibaba Cloud / Qwen Team · open	0.88
22	Kimi K2 Base Moonshot AI · open	0.88
23	Qwen3 VL 30B A3B Thinking Alibaba Cloud / Qwen Team · open	0.88
24	GPT-4.1 mini OpenAI	0.88
25	Grok-2 xAI	0.88
26	Kimi-k1.5 Moonshot AI	0.87
27	Llama 3.1 405B Instruct Meta · open	0.87
28	o3-mini OpenAI	0.87
29	Claude 3 Opus Anthropic	0.87
30	GPT-4 Turbo OpenAI	0.86
31	GPT-4 OpenAI	0.86
32	Qwen3 VL 32B Instruct Alibaba Cloud / Qwen Team · open	0.86
33	Grok-2 mini xAI	0.86
34	Llama 3.2 90B Instruct Meta · open	0.86
35	Llama 3.3 70B Instruct Meta · open	0.86
36	Gemini 1.5 Pro Google	0.86
37	Nova Pro Amazon	0.86
38	GPT-4o OpenAI	0.86
39	LongCat-Flash-Lite Meituan · open	0.86
40	Llama 4 Maverick Meta · open	0.85
41	GPT OSS 20B OpenAI · open	0.85
42	o1-mini OpenAI	0.85
43	Qwen3 VL 8B Thinking Alibaba Cloud / Qwen Team · open	0.85
44	Sarvam-30B Sarvam AI · open	0.85
45	Hermes 3 405B Nous Research · open	0.85
46	Qwen3 VL 30B A3B Instruct Alibaba Cloud / Qwen Team · open	0.85
47	Phi 4 Microsoft · open	0.85
48	Mistral Large 2 Mistral AI · open	0.84
49	Llama 3.1 70B Instruct Meta · open	0.84
50	Qwen2.5 32B Instruct Alibaba Cloud / Qwen Team · open	0.83
51	Qwen2 72B Instruct Alibaba Cloud / Qwen Team · open	0.82
52	GPT-4o mini OpenAI	0.82
53	Qwen3 VL 4B Thinking Alibaba Cloud / Qwen Team · open	0.81
54	Grok-1.5 xAI	0.81
55	Jamba 1.5 Large AI21 Labs · open	0.81
56	Mistral Small 3.1 24B Base Mistral AI · open	0.81
57	Mistral Small 3 24B Base Mistral AI · open	0.81
58	Qwen3 VL 8B Instruct Alibaba Cloud / Qwen Team · open	0.81
59	Mistral Small 3.1 24B Instruct Mistral AI · open	0.81
60	Mistral Small 3.2 24B Instruct Mistral AI · open	0.81
61	Nova Lite Amazon	0.81
62	DeepSeek-V2.5 DeepSeek · open	0.80
63	Llama 3.1 Nemotron 70B Instruct NVIDIA · open	0.80
64	GPT-4.1 nano OpenAI	0.80
65	Qwen2.5 14B Instruct Alibaba Cloud / Qwen Team · open	0.80
66	Llama 4 Scout Meta · open	0.80
67	Ministral 3 (14B Base 2512) Mistral AI · open	0.79
68	Hermes 3 70B Nous Research · open	0.79
69	Claude 3 Sonnet Anthropic	0.79
70	Gemini 1.5 Flash Google	0.79
71	Phi-3.5-MoE-instruct Microsoft · open	0.79
72	Qwen2.5 VL 32B Instruct Alibaba Cloud / Qwen Team · open	0.78
73	Nova Micro Amazon	0.78
74	Qwen3 VL 4B Instruct Alibaba Cloud / Qwen Team · open	0.77
75	Ministral 3 (8B Base 2512) Mistral AI · open	0.76
76	Command R+ Cohere · open	0.76
77	Claude 3 Haiku Anthropic	0.75
78	Gemma 2 27B Google · open	0.75
79	Qwen2.5-Coder 32B Instruct Alibaba Cloud / Qwen Team · open	0.75
80	Llama 3.2 11B Instruct Meta · open	0.73
81	Gemini 1.0 Pro Google	0.72
82	Gemma 2 9B Google · open	0.71
83	Ministral 3 (3B Base 2512) Mistral AI · open	0.71
84	Qwen2 7B Instruct Alibaba Cloud / Qwen Team · open	0.70
85	GPT-3.5 Turbo OpenAI	0.70
86	Jamba 1.5 Mini AI21 Labs · open	0.70
87	Llama 3.1 8B Instruct Meta · open	0.69
88	Pixtral-12B Mistral AI · open	0.69
89	Phi-3.5-mini-instruct Microsoft · open	0.69
90	Mistral NeMo Instruct Mistral AI · open	0.68
91	Qwen2.5-Coder 7B Instruct Alibaba Cloud / Qwen Team · open	0.68
92	Phi 4 Mini Microsoft · open	0.67
93	Granite 3.3 8B Instruct IBM · open	0.66
94	Ministral 8B Instruct Mistral AI · open	0.65
95	Gemma 3n E4B Instructed Google	0.65
96	Gemma 3n E4B Instructed LiteRT Preview Google · open	0.65
97	Granite 3.3 8B Base IBM · open	0.64
98	Llama 3.2 3B Instruct Meta · open	0.63
99	IBM Granite 4.0 Tiny Preview IBM · open	0.60
100	Gemma 3n E2B Instructed Google	0.60