Independent Benchmark for In-house Legal Work

Project Snapshot

AI adoption in legal departments is accelerating, but clarity on real-world performance remains limited. This project offers an independent, vendor-neutral benchmark of AI tools based on real in-house legal use cases.

Following our phase 1 report on legal information extraction, phase 2 focuses on evaluating AI tools for contract drafting tasks.

AI Tools

• Purpose-built legal AI tools (early-stage startups to established enterprises)
• General-purpose AI assistants with legal drafting capabilities
• Foundational LLMs (e.g., GPT-4.1, Claude Sonnet, Gemini 2.5)

Phase 2 Goals

• Evaluate contract drafting performance across AI Tools with human lawyers as the reference baseline.
• Identify the strengths and recurring failure patterns in AI-generated drafts
• Uncover best practices for using AI Tools in contract drafting, grounded in real evaluation findings

Phase 2 Timeline

• Phase 2 Launch: Week of June 23 (target)
• Vendor and Contributor Sign-Up: Late June
• Task Collection and Evaluation: Early July
• Data Analysis and Report Drafting: Mid–late July
• Report Publication: Late July to early August

Roles