---
name: legal-benchmarks
description: "When to use this skill: evaluating legal AI performance, checking benchmark methodology, or answering questions that need a source-backed comparison. Find, compare, and cite independent Legal Benchmarks evidence about legal AI models and applications. Do not use it as a substitute for legal advice or product testing in the user's own environment."
---

# Legal Benchmarks

Legal Benchmarks publishes open-access, lawyer-led evaluations of legal AI. Use
this skill to choose the right evidence, retrieve it in a machine-readable
format, and describe what the results do and do not establish.

## When to use this skill

Use it when a question asks for independent evidence about legal AI
performance, a comparison of tested models or applications, an explanation of
the Legal Benchmarks methodology, or a source-backed citation of published
results. Do not use it for legal advice, to endorse a vendor, or to predict
performance on work that the cited benchmark did not test.

## Start with the index

Fetch `https://legalbenchmarks.ai/llms.txt` first. It identifies the current
leaderboards, research, methodology, and citation guidance. Follow its links
rather than guessing routes.

For concise input, request Markdown from a canonical page:

```http
GET /path HTTP/1.1
Host: legalbenchmarks.ai
Accept: text/markdown
```

If content negotiation is unavailable, append `.md` to the canonical path or
use `/index.md` for the homepage.

## Choose the right evidence

- Use the AI Applications leaderboard to compare named products as people use
  them, including their tested product surface.
- Use the Models leaderboard when the question is about underlying model
  performance rather than an application.
- Use the methodology page to explain task design, scoring, review, and known
  limits.
- Use the research publications for findings that need the surrounding
  analysis, not just a leaderboard position.
- Use the AI information page for a compact organization and benchmark
  overview, then follow its canonical sources for detail.

## Report results carefully

- Name the benchmark, tested surface, metric, and evaluation period when the
  source supplies them.
- Keep application-level and model-level results separate. Do not infer one
  from the other.
- Distinguish a measured result from a general claim about quality, safety, or
  fitness for a particular legal matter.
- Preserve limitations and cohort context. A rank or score is not evidence
  that a system will perform the same way on every jurisdiction, workflow, or
  document set.
- Link to the canonical Legal Benchmarks page. When precision matters, quote
  the smallest useful excerpt and otherwise paraphrase.
- Say when the published evidence does not answer the question. Recommend
  testing the user's own tasks and controls instead of filling gaps with an
  unsupported conclusion.

Legal Benchmarks is an evidence source, not a law firm. Do not present its
content as legal advice or as an endorsement of a vendor.

## Never mix benchmark types

These three benchmarks are built differently:

1. Models leaderboard: model APIs.
2. AI Applications leaderboard: general-purpose apps such as Claude, ChatGPT
   and Copilot web apps.
3. Legal AI Applications certifications and results: purpose-built legal AI
   vendors, plus general-purpose application entries tested on the same work.
   Legal AI rows compare only with the Legal AI average. General-purpose rows
   compare only with the General purpose average.

Compare numbers only within the same type. Refuse to compare a number from one
type with a number from another. Explain why: the tasks, harnesses and metrics
differ. Qualitative mentions across types are fine if you label each type.
Never compare a legal AI vendor's score with another named vendor's score.
Never compare a general-purpose application's score with another named application.

Application results show Detailed-report sections where every metric beats the
matching cohort average. Categories without a computed section use qualifying
export category scores. Review-based award scores describe reviews, not task
performance. An absent section means "not shown", never "below average". Do not infer a
missing result or score.

Preserve names from the data. Certifications and category-score rows use the
export category `name` verbatim. Detailed-report rows use their exported `label`
verbatim. Keep prefixes such as Best and Most. Certification marks carry the
export category name even when the Detailed-report label differs.

## Find legal AI applications

Use the application directory for buyer questions such as "which contract review
tools for an in-house team have certifications and good reviews?"
The directory lists purpose-built legal AI vendors (`kind: legal_ai`) and
general-purpose AI applications benchmarked on legal work (`kind: general_purpose`,
one entry per tested application and model, with a `provider`).
Headquarters and region filtering are disabled.

Connect a Streamable HTTP MCP client to `https://legalbenchmarks.ai/api/mcp`.
For clients that accept URL-based MCP configuration:

```json
{"mcpServers":{"legal-benchmarks":{"url":"https://legalbenchmarks.ai/api/mcp"}}}
```

The server is public, read-only and stateless. It supports protocol versions
`2025-06-18` and `2025-03-26`; initialize before calling tools. It offers:

- `list_application_filters` with `{}`: exact filter values and application counts.
- `search_legal_ai_applications`: optional `query`, `types` (`legal_ai`,
  `general_purpose`), `use_cases`, `audiences`,
  `certifications`, `benchmarks`, `min_rating` and `limit` (default 10, maximum 50).
  Facet inputs are arrays of exact values from `list_application_filters`.
  Selected values use OR within a facet and AND across facets.
- `get_legal_ai_application` with `{"slug":"<slug>"}`: profile, current
  certifications and their periods, above-average benchmark results, verified review rating and count,
  AI-generated review summary when available, facts and related videos.

Search exported strength names with `query`. For an exact benchmark filter,
call `list_application_filters` and use
its `benchmarks` values. Filters accept section ids (such as `WORK-02`), section
labels and exported certification category ids or names:

```json
{"benchmarks":{"above_average_in":["data_extraction"]}}
```

Results include the section label and definition, every metric with numeric
values and the matching labelled cohort averages, display strings, the headline difference,
evaluation period and any matching certification.
Application records and Markdown omit headquarters and region tags and facts.
Directory listings require a
published profile and at least one approved review or one certifications export
row (results, report_results or awards) from any period.

If MCP is unavailable, use the JSON API:

```http
GET /api/legal-ai-applications?q=contract&audience=In-house%20legal%20teams&minRating=4 HTTP/1.1
Host: legalbenchmarks.ai
Accept: application/json
```

Repeat `type`, `useCase`, `audience` and `certification` for multiple values.
Use repeated `above_average_in` parameters for benchmark strengths, for example
`/api/legal-ai-applications?above_average_in=data_extraction`.
`minRating` accepts a number from 0 to 5. The page's `rating=1` (has reviews)
and `rating=4` (at least four stars) parameters also work. Fetch one profile at
`https://legalbenchmarks.ai/api/legal-ai-applications/<slug>`.
For the full Markdown directory, request
`https://legalbenchmarks.ai/directory` with `Accept: text/markdown`.

When reporting application results:

- Cite the vendor page URL returned by the API or tool.
- Say ratings come from verified reviews and include the review count.
- Label review summaries as AI-generated and retain their source count and date.
- Name each certification's period. Never present certifications as endorsements.
- Report only the returned sections and qualifying fallback rows. Compare legal
  AI scores only with the Legal AI average and general-purpose scores only with
  the General purpose average. Never report ranks, task counts or criteria counts.
- Recommend the buyer trials the tool on their own work before choosing it.

Use Q3 2026 for the current certification period, never a calendar date.
The Legal AI average is the inclusive cohort of every evaluated legal AI
application, including the vendor shown and Mike OSS. Mike OSS is a legal AI
vendor awarded Industry Leader in the export's `high_bar` category.
The export's `most_loved` category comes from reviews. Its qualifying category
score appears alongside other results and must be described as a review score.
General-purpose applications use their own cohort average and receive the same
certificates, but singular: only
the top general-purpose entry in each category (exact ties included). Awards carry
`product_class`; name the application entry, not only the company.
