Orrick

Document review is one of the most time-intensive and expensive phases of litigation, often consuming the majority of e-discovery budgets. For Orrick, a leading global law firm with a strong track record in IP litigation, the question was whether generative AI could handle the nuanced, context-dependent judgments that document review demands — not just in theory, but in a live matter. Attorney Cal Yeaman needed to understand how an LLM-based tool would compare to trained human reviewers on accuracy and consistency, and whether it could integrate meaningfully with existing predictive coding workflows without introducing unacceptable risk to clients.

Orrick deployed EverlawAI Assistant's Coding Suggestions feature — built on large language model technology — across approximately 10,000 documents in a live IP litigation matter. Rather than replacing existing workflows outright, the team built a hybrid approach: Coding Suggestions classified documents for relevance using human-crafted prompts, while Predictive Coding ran in parallel to cross-validate the AI's output and flag documents most likely to need human QC. Orrick started with a small sample subset to iteratively refine prompts before scaling to the full corpus. The two technologies operate on distinct mechanisms — Predictive Coding learns from reviewer behavior over time, while Coding Suggestions relies on explicit case-context prompts — making them complementary rather than redundant.

The results exceeded Orrick's expectations on both accuracy and cost. Of all documents flagged as not-relevant by the AI, human reviewers reversed only 1 out of approximately 10,000 documents — a statistically significant margin over human review performance. Orrick estimated the AI-assisted process delivered more than 50% reduction in document review costs. Beyond cost savings, the tool accelerated identification of key documents, allowing the legal team to build case strategy earlier. Key outcomes:

• >50% reduction in document review costs
• 1 reversal out of ~10,000 AI-coded documents
• AI accuracy rated statistically significantly higher than human review
• Faster surfacing of key evidence, improving case readiness

• Pilot on a small sample first. Iterating on prompts with a representative subset before scaling prevents systematic errors from propagating across thousands of documents.
• Combine GenAI with Predictive Coding for mutual validation. The two tools use different underlying mechanisms — using both together improves confidence and helps triage documents that need human QC.
• Match tool to matter type. LLM-based review performs best on linear, complex subject matter where consistent criteria application is critical; not all case types yield the same gains.
• AI performance varies by matter. Accuracy depends on review criteria, data types, and subject complexity — validate results on each new engagement.