September 24, 2026

ClinReg benchmark leaderboard

Twenty-five models scored across four agentic clinical and regulatory tasks — IND module 3 drafting, TLF for CSRs, literature screening, and systematic literature review — with cost measured per completed run.

All organizations
Category25 models · 9 open-weight

Overall score

mean of the four task scores · 0–100 · higher is better
0 20 40 60 80 100 GPT-6 Astra · 87.987.9 GPT-6 Astra GPT-5.6 Sol · 86.586.5 GPT-5.6 Sol Gemini 3.8 Flash · 86.386.3 Gemini 3.8 Flash GLM-5.3 · 86.186.1 GLM-5.3 Opus 5 · 85.385.3 Opus 5 GLM-5.3 Flash · 85.185.1 GLM-5.3 Flash Kimi K3 · 84.784.7 Kimi K3 GLM-5.2 · 84.284.2 GLM-5.2 Gemini 3.7 Flash · 84.084.0 Gemini 3.7 Flash GPT-5.5 · 83.583.5 GPT-5.5 GPT-5.6 Terra · 82.982.9 GPT-5.6 Terra Muse Spark 1.3 · 82.782.7 Muse Spark 1.3 Grok 4.5 · 82.082.0 Grok 4.5 Opus 4.8 · 81.181.1 Opus 4.8 Opus 4.7 · 80.780.7 Opus 4.7 Gemini 3.1 Pro · 80.080.0 Gemini 3.1 Pro Opus 4.6 · 78.678.6 Opus 4.6 GPT-5.6 Luna · 77.977.9 GPT-5.6 Luna Sonnet 5 · 76.376.3 Sonnet 5 DeepSeek V4 · 75.775.7 DeepSeek V4 Sonnet 4.6 · 75.575.5 Sonnet 4.6 Kimi K2.6 · 74.674.6 Kimi K2.6 MiniMax M3 · 73.573.5 MiniMax M3 Gemma 4 31B · 71.571.5 Gemma 4 31B Nemotron 3 Ultra · 69.969.9 Nemotron 3 Ultra
Table25 models · sortable by any column
ModelOverallINDTLFLit screenSLRCost per run
avg of tasks		INDTLFLitSLR
GPT-6 Astranew87.990.086.293.681.8$13.39$12.27$15.19$4.95$21.13
GPT-5.6 Sol86.586.385.293.680.9$7.12$3.69$12.82$2.92$9.05
Gemini 3.8 Flashnew86.388.084.193.679.5$3.29$2.56$7.48$0.40$2.72
GLM-5.3opennew86.186.088.293.676.5$5.31$3.31$6.77$0.71$10.43
Opus 585.386.778.593.682.2$21.06$26.03$36.65$4.20$17.36
GLM-5.3 Flashopennew85.185.085.392.577.5$0.44$0.15$0.62$0.12$0.88
Kimi K3open84.786.782.391.578.1$6.10$3.08$6.81$1.70$12.81
GLM-5.2open84.282.786.093.674.7$3.33$2.86$3.27$0.44$6.73
Gemini 3.7 Flashnew84.084.078.793.679.5$1.49$1.37$2.15$0.47$1.99
GPT-5.583.582.381.091.579.1$5.65$3.64$2.19$2.64$14.12
GPT-5.6 Terra82.991.269.593.677.4$2.58$3.37$1.06$1.39$4.51
Muse Spark 1.3new82.790.073.391.575.9$2.53$1.13$3.81$0.55$4.65
Grok 4.582.082.379.291.575.0$5.04$10.82$2.90$1.12$5.31
Opus 4.881.176.777.593.676.8$18.62$30.27$25.91$3.75$14.55
Opus 4.780.780.073.491.577.8$17.74$22.21$31.93$3.81$13.01
Gemini 3.1 Pro80.073.577.591.577.6$4.95$4.33$3.51$1.60$10.36
Opus 4.678.676.771.489.476.9$12.25$22.05$10.23$3.15$13.58
GPT-5.6 Luna77.987.054.993.676.3$1.30$3.52$0.61$0.61$0.47
Sonnet 576.376.767.285.176.2$9.85$6.77$23.70$2.27$6.67
DeepSeek V4open75.775.063.489.475.1$2.09$0.94$1.23$1.03$5.17
Sonnet 4.675.565.072.591.572.9$7.90$13.43$7.35$1.75$9.07
Kimi K2.6open74.671.763.591.571.6$2.84$1.71$1.52$0.67$7.47
MiniMax M3open73.570.063.587.273.5$0.97$1.82$0.91$0.21$0.96
Gemma 4 31Bopen71.572.758.589.465.6$0.79$0.43$2.24$0.25$0.23
Nemotron 3 Ultraopen69.954.365.091.569.0$2.55$4.70$1.55$0.52$3.43
openopen-weightnewadded in this releasescore shadinglow → high within the columnbold = best in the columnoverall = mean of the four task scores · cost = list price per completed run

Overall vs cost

average cost per taskopen-weightclosed
65 70 75 80 85 90 $0.5 $1 $2 $5 $10 $20 average cost per task (USD, log scale) Overall score GPT-6 Astra · 87.9 · $13.39 per run · closed GPT-5.6 Sol · 86.5 · $7.12 per run · closed Gemini 3.8 Flash · 86.3 · $3.29 per run · closed GLM-5.3 · 86.1 · $5.31 per run · open-weight Opus 5 · 85.3 · $21.06 per run · closed GLM-5.3 Flash · 85.1 · $0.44 per run · open-weight Kimi K3 · 84.7 · $6.10 per run · open-weight GLM-5.2 · 84.2 · $3.33 per run · open-weight Gemini 3.7 Flash · 84.0 · $1.49 per run · closed GPT-5.5 · 83.5 · $5.65 per run · closed GPT-5.6 Terra · 82.9 · $2.58 per run · closed Muse Spark 1.3 · 82.7 · $2.53 per run · closed Grok 4.5 · 82.0 · $5.04 per run · closed Opus 4.8 · 81.1 · $18.62 per run · closed Opus 4.7 · 80.7 · $17.74 per run · closed Gemini 3.1 Pro · 80.0 · $4.95 per run · closed Opus 4.6 · 78.6 · $12.25 per run · closed GPT-5.6 Luna · 77.9 · $1.30 per run · closed Sonnet 5 · 76.3 · $9.85 per run · closed DeepSeek V4 · 75.7 · $2.09 per run · open-weight Sonnet 4.6 · 75.5 · $7.90 per run · closed Kimi K2.6 · 74.6 · $2.84 per run · open-weight MiniMax M3 · 73.5 · $0.97 per run · open-weight Gemma 4 31B · 71.5 · $0.79 per run · open-weight Nemotron 3 Ultra · 69.9 · $2.55 per run · open-weight GPT-6 Astra 🏆 GPT-5.6 Sol 🏆 Gemini 3.8 Flash 🏆 GLM-5.3 Opus 5 GLM-5.3 Flash Kimi K3 GLM-5.2 Gemini 3.7 Flash GPT-5.5 GPT-5.6 Terra Muse Spark 1.3 Grok 4.5 Opus 4.8 Opus 4.7 Gemini 3.1 Pro Opus 4.6 GPT-5.6 Luna Sonnet 5 DeepSeek V4 Sonnet 4.6 Kimi K2.6 MiniMax M3 Gemma 4 31B Nemotron 3 Ultra
dotted lines join successive releases within one model line (GPT-5.6, Opus, Sonnet) · 🏆 top 3 on this score
02

What the board says.

LeadersGPT-6 Astra leads at 87.9; GPT-5.6 Sol (86.5) and Gemini 3.8 Flash (86.3) follow. The top ten sit within 4.4 points.
Open weightsOpen-weight models hold 4 of the top 10. Best open-weight: GLM-5.3 at 86.1 ($5.31 per run). Best proprietary: GPT-6 Astra at 87.9 ($13.39).
CostInside the top ten, average cost per run goes from $0.44 (GLM-5.3 Flash, 85.1) to $21.06 (Opus 5, 85.3): 48× the price for 0.2 points more.
Task dependenceGPT-5.6 Luna scores 93.6 on literature screening but 54.9 on TLF programming, the widest spread on the board. Read the task columns, not only the overall.
03

How it is scored.

IND module 3 drafting

Draft the chemistry, manufacturing and controls sections of an IND from a public regulator's assessment report. A judge panel rates the extracted facts for faithfulness and coverage, 0–10; score = that rating × 10.

TLF for CSRs

From raw CRF data and specs, write and run the SDTM → ADaM → TLF pipeline for the CDISC pilot study in one session. Score = (share of output cells matching the published reference × 100 + reviewer panel × 10) / 2.

Literature screening

Full-text eligibility decisions on 47 papers from a Cochrane review, scored against the human reviewers. Score = percent of decisions correct.

Judges

Judged scores use models from three labs, never the model under test alone. IND fabrication and omission calls count only when three of four finders agree; TLF cell matches are exact comparisons to the reference; screening decisions are compared to the Cochrane reviewers' own.

Harness

Claude-family models ran under Claude Code, GPT-family models under Codex, Gemini and open-weight models under OpenCode. A score belongs to a model together with its harness and task skill.

Systematic literature review

Three TrialPanorama tasks on their full published test splits: study search (a PubMed query per review, scored as Recall@500), study screening (include/exclude for 979 candidate studies, scored as F1) and evidence summarization (1,000 multiple-choice questions, scored as accuracy). Score = the mean of the three, each on a 0–100 scale.

Cost

List price at run time: input, output and reasoning tokens, cache reads at the provider's cached rate, averaged over the four tasks. The SLR figure is the average of its three runs; running all three costs about three times that. Prices are verified on the provider's pricing page before a row is posted.
Read the full methods and results on the blog→task construction, judge panels, harness settings and per-model notes for IND, TLF and literature screening; the systematic review task is described above