Large global law firm eliminates 89% of document review using predictive coding in civil litigation

In a civil litigation matter, a large global law firm collected and processed more than 1 million documents on behalf of the defendant. The firm and opposing counsel agreed to use keyword searches to isolate relevant documents but could not agree on the terms. Opposing counsel insisted on a broader list that would result in a tenfold increase in documents needing review and a dramatic increase in the defendant's cost; the keyword-culled corpus came to nearly 200,000 documents. The firm's attorney team had little experience with predictive coding and called on Consilio for guidance on proposing it as an alternative.

The firm engaged Consilio to implement a predictive coding workflow targeting 90% ±5% recall. Consilio guided a four-round iterative training process: an initial random sample of 2,057 documents, coded by a senior associate, let Consilio set aside 162,000 documents as likely nonresponsive; subsequent rounds refined the model using biased draws, additional random samples, and a disagreement-reversal round in which the associate re-reviewed 108 documents where his coding conflicted most with the model's scores. A parallel workstream was added so junior attorneys could begin second-level review of documents with an over 80% likelihood of being responsive.

Predictive coding set aside 89% of the original document corpus from consideration, reducing the review pool to approximately 21,500 documents, in which 90% ±5% of the expected 15,000 responsive documents in the corpus would be found; the page says the first training round had already achieved the 90% ±5% recall target. The client saved at least $209,000 in variable costs, figured at an average cost of roughly $1.25 per document for first-level review, as well as days or weeks of review time. Consilio says the approach let the client produce documents one day sooner than it otherwise could have.

• Consilio says expert guidance on predictive coding protocols matters: without it, parties risk either failing to reach agreement to use the technology or agreeing to a protocol that turns out to be unworkable and leads to costly disputes and sanctions.
• Human coding inconsistencies (e.g., coding document families without per-document review) can significantly impair model training and must be caught early via QA.
• A disagreement-reversal round—where the model's probability scores conflict most sharply with human coding—can surface and correct reviewer errors that marginal training rounds cannot resolve.