LifeSci Diamond (2026-11)

LifeSci Diamond is Arca's benchmark of 55 regulatory-affairs tasks (FDA/EMA submissions, labeling, compliance), 18 of them written by outside specialists. Scores are the strict pass rate: a task counts only when every requirement is met and no fail line fires, as judged by a jury of LLM graders. One answer per task per model on the plain API with a web_search tool (Exa), provider default settings. Version frozen 2026-11.

id lifesci-diamond-2026-11 · max 1 · 18 models reported