Independent Benchmark for In-house Legal Work

Project Snapshot

AI adoption in legal departments is accelerating, but clarity on real-world performance, risks, and accuracy remains limited. This project offers an independent, vendor-agnostic benchmark of AI tools based on real in-house legal use cases.

Following our phase 1 report dedicated to information extraction, phase 2 focuses on evaluating AI tools for contract drafting tasks in English, specifically:

1. Basic drafting: generating basic clauses that follow widely accepted language
2. Template-based drafting: adapting an existing contract template based on provided facts
3. Bespoke drafting: drafting custom clauses/agreements from scratch based on unique commercial arrangements

Redlining or markup of existing text is excluded from this phase. Phase 2 focuses on the generative drafting capabilities of AI tools.

Legal AI Solutions in Scope

The following legal AI solutions are included in this phase 2 evaluation

Wordsmith AI

GC AI

InstaSpace

SimpleDocs

Brackets AI

Vecflow

Phase 2 Goals

• Evaluate contract drafting performance across Legal AI solutions, general-purpose LLMs (e.g., OpenAI GPT, Alibaba Qwen, Anthropic Claude, Google Gemini), and Microsoft Copilot, using human lawyers as the reference baseline.
• Identify the strengths and recurring failure patterns in AI-generated drafts.
• Uncover best practices for using AI Solutions in contract drafting, grounded in evaluation findings.

Phase 2 Timeline

• Phase 2 Launch: Week of June 23 (target)
• Vendor and Contributor Sign-Up: Late June
• Task Collection and Evaluation: Early July
• Data Analysis and Report Drafting: Mid–late July
• Phase 2 Report Publication: Late July to early August

Roles