In the first-known judicial opinion in which a court has recognized that the use of computer-assisted review is an acceptable way to search for relevant electronically stored information (ESI) in appropriate cases, Magistrate Judge Andrew J. Peck of the United States District Court for the Southern District of New York issued an opinion on Friday approving the use of “computer-assisted review” to find and produce relevant ESI in an employment discrimination case filed as a class action.
Computer-assisted review (also referred to as “predictive coding” or “technology-assisted review”) generally refers to review software that uses machine learning algorithms to enable a computer to determine what documents are relevant to a case after being trained by a human reviewer. In computer-assisted review, attorneys knowledgeable about the case manually review and analyze a small sample set of potentially relevant ESI to decide which documents are relevant to the case. The relevant documents reviewed by the attorneys are then used as a “seed set” to teach the computer what to search for in the larger data set so that the computer can find additional relevant documents through an automated process. Because computer-assisted review does not require attorneys to review each and every potentially relevant document and can identify fewer non-relevant documents than keyword searching alone, computer-assisted review potentially can result in significant cost savings and a more accurate document production.
Over the past year, the potential of computer-assisted review to reduce the time and money it takes to conduct electronic discovery has received a lot of attention. The New York Times, for example, reported on “armies of expensive lawyers” being replaced by “cheaper software” and explained how advances in artificial intelligence are leading to improvements in e-discovery technology to find and produce relevant documents. Forbes recently commented on the importance of human expertise in guiding computer-assisted review as well as “the complexities of attempting to apply a new technological approach to electronic document review that is transparent, accurate, and fair for all parties.” Judge Peck also wrote an article last year, which he quotes from in his opinion, supporting the use of predictive coding as an alternative to the time and cost of manual review of large document sets. He explained that “the volume of ESI has made full manual review virtually impossible” and how keyword searching alone can have drawbacks.
After holding multiple discovery conferences on the possibility of using predictive coding technology in the employment discrimination case pending before him, Judge Peck ordered the parties earlier this month to submit their “final” protocol relating to the production of ESI using predictive coding. Over the plaintiffs’ objections to the predictive coding methodology proposed by the defendants, Judge Peck entered the protocol as an order last week. He then followed that order up with Friday’s opinion.
In his opinion, Judge Peck recognized that “computer-assisted review is not perfect” but determined that the use of predictive coding in the case before him was appropriate considering the parties’ original assent to using predictive coding, the vast amount of ESI to be reviewed (over three million documents), the superiority of computer-assisted review to available alternatives (such as linear manual review or keyword searches), the need for cost effectiveness and proportionality, and the transparent process proposed by the defendants. Judge Peck further explained the process that the defendants were going to use and rejected the plaintiff’s arguments that the process violated Rule 26(g) of the Federal Rules of Civil Procedure and Evidence Rule 702 of the Federal Rules of Evidence. He also provided guidance for considering the use of computer-assisted review in future cases.
On another issue recently discussed here, Judge Peck also noted The Sedona Conference’s® recent publication of the International Principles on Discovery, Disclosure & Data Protection as support for his denial of discovery of a custodian’s emails stored in France because they likely would be covered by French privacy and blocking laws. He also reiterated his strong endorsement of The Sedona Conference Cooperation Proclamation® and the need for counsel to cooperate on e-discovery issues. Judge Peck believes that an “important aspect of cooperation is transparency in the discovery process,” and he “highly recommends that counsel in future cases be willing to at least discuss, if not agree to, such transparency in the computer-assisted review process.”
Judge Peck concluded his opinion by encouraging counsel to consider using computer-assisted review to search for relevant ESI in appropriate cases: “What the Bar should take away from this Opinion is that computer-assisted review is an available tool and should be seriously considered for use in large-data-volume cases where it may save the producing party (or both parties) significant amounts of legal fees in document review.”