Model Leaderboard
An independent comparison of how leading AI models perform on Contract Workflows and Data Extraction.
- Substance: Whether the legal work is correct, complete, and responsive. Task pass is the share of tasks that satisfy every applicable criterion in both of the model's independent attempts.
- Form: whether a lawyer would want to use the delivered work as written. Measured as the percentage of applicable binary form checks passed in both of the model's independent attempts.
Contract workflow leaderboard
| Rank | |||||||
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Anthropic | 48.4% | 91.7% | 93.9% | $1.47 | 3m 03s |
| 1 | Claude Fable 5.1 | Anthropic | 48.4% | 90.7% | 89.7% | $1.50 | 3m 51s |
| 1 | Muse Spark 1.3 | Meta | 48.4% | 87.4% | 87.8% | $0.20 | 4m 35s |
| 4 | Claude Opus 5 | Anthropic | 45.2% | 90.1% | 90.8% | $0.90 | 4m 02s |
| 5 | Qwen 3.8 Flash | Qwen | 38.7% | 49.1% | 67.7% | $0.03 | 8m 21s |
| 6 | Claude Sonnet 5.5 | Anthropic | 29.0% | 85.3% | 91.1% | $0.98 | 1m 54s |
| 6 | Qwen 3.8 Max | Qwen | 29.0% | 32.2% | 64.2% | $0.39 | 6m 21s |
| 8 | GPT-6 Astra | OpenAI | 25.8% | 87.1% | 91.9% | $1.13 | 2m 55s |
| 8 | GPT-6.1 Sol | OpenAI | 25.8% | 86.3% | 90.8% | $0.11 | 2m 22s |
| 8 | Gemini 3.8 Flash | 25.8% | 65.9% | 76.4% | $0.22 | 6m 11s | |
| 8 | Grok 4.6 | xAI | 25.8% | 62.2% | 76.8% | $0.28 | 4m 28s |
| 12 | GPT-5.6 Sol | OpenAI | 22.6% | 84.8% | 89.2% | $0.18 | 2m 51s |
| 12 | Kimi K3 | Moonshot AI | 22.6% | 78.8% | 84.7% | $0.26 | 2m 10s |
| 12 | DeepSeek V4 Flash | DeepSeek | 22.6% | 57.1% | 73.4% | $0.01 | 4m 18s |
| 12 | GLM-5.3 Flash | Z.ai | 22.6% | 56.0% | 67.8% | $0.02 | 4m 21s |
| 16 | GPT-5.6 Luna | OpenAI | 19.4% | 80.9% | 86.1% | $0.02 | 2m 54s |
| 16 | DeepSeek V4 Pro | DeepSeek | 19.4% | 78.6% | 88.3% | $0.08 | 3m 58s |
| 16 | GLM-5.3 | Z.ai | 19.4% | 68.1% | 81.9% | $0.09 | 3m 31s |
| 19 | GPT-5.6 Terra | OpenAI | 16.1% | 81.2% | 86.0% | $0.14 | 1m 41s |
| 19 | Claude Sonnet 5 | Anthropic | 16.1% | 80.1% | 86.9% | $0.36 | 2m 52s |
| 21 | MiniMax M3 | MiniMax | 12.9% | 80.1% | 83.4% | $0.03 | 2m 03s |
| 22 | Mistral Medium 3.5 | Mistral | 9.7% | 47.7% | 75.0% | $0.32 | 2m 52s |
Key takeaways
- The drafting lead is shared, at very different prices. Claude Opus 5.5, Claude Fable 5.1, and Muse Spark 1.3 pass the same share of contract tasks, while Muse Spark costs about 86% less than either Claude model. A top score no longer implies a top price.
- Meeting almost every criterion is not the same as finishing the task. GPT-6 Astra meets 87.1% of drafting criteria yet completes about a quarter of tasks end to end. One missed requirement fails the whole task, so review against the full instruction, not overall polish.
- A high task pass rate can rest on narrow coverage. Qwen 3.8 Flash sits near the top on completed drafting tasks while meeting fewer than half of the criteria overall. Its failures cluster in a few tasks, so check the criteria pass rate before trusting a rank.
- Still no model to leave drafting unsupervised. The best contract result leaves more than half of workflows short of at least one requirement. Keep lawyer review in the loop.
How we test
Every model receives the same instructions and native source documents of lawyer-authored tasks. A task passes only when every applicable criterion is satisfied; form is scored separately for clarity, length, and structure. LLM graders are checked through agreement analysis, human spot reviews, and provider-favoritism tests.
Published research
))
))