Potential Problems & Perils of eDiscovery Outsourcing

Predictive coding, technology assisted review (TAR), computer assisted document analysis (CADA), predictive analytics, document review, eDiscovery software. Important new tools and concepts for the legal community. As technology, likely unknown and a little disconcerting for many lawyers. A tempting solution may be outsourcing. But while outsourcing seems compelling, outsourcing of eDiscovery predictive coding can pose some potentially serious ethical problems for an attorney.

Understanding Predictive Coding Tools

To understand why ethics problems loom, a lawyer must acquire a basic understanding of how  predictive analytics tools (sometimes called CADA, TAR, or “predictive coding”—all interchangeable) work. That is, this is a case where not-knowing-the-technology as technology could cause serious problems specific to the legal community (something that non-legal vendors might not understand to your peril).

Predictive analytics tools augment a lawyer’s legal analysis capabilities.

In simplest terms, predictive analytics software tools augment a lawyer’s legal analysis capabilities. Read that definition carefully because the augmentation of a lawyer’s legal analysis is what fundamentally distinguishes predictive analytics from other and older technologies;  what gives predictive analytics exceptional power; and what triggers potential ethical issues when performed by non-lawyers.

Using the technology, the lawyer trains the computer (actually training a machine learning algorithm) to recognize that lawyer’s legal analysis of the documents in a specific matter. After training, the computer then predicatively applies the lawyer’s legal analysis to the remaining documents associated with that matter. This predictive step is where the legal augmentation, and power of predictive analytics, occurs.

For example, if the matter has 100,000 documents, a lawyer might analyze a random sampling of 800 documents from the 100,000. For each of the 800 documents, the lawyer applies a classification label to the item such as relevant, non-relevant, privileged, non-privileged, etc. The classified 800 documents are the “seed set” that trains the computer. After training, the computer then applies the lawyer’s analysis to the remaining 99,200 documents—with high accuracy and absolute consistency.

Understanding “training” will become essential for the legal community. Put simply, this is not something that can be summarily “off-loaded” or outsourced to non-lawyers because the activity fundamentally implicates law practice—and because the software augments the analysis, the reviewing attorney must have deep knowledge of the matter and excellent legal analysis skills.

Faulty Analogies to Keyword Search and Forensics

Lawyers are trained to think in parallels, analogies, and similarities. Unfortunately, that training can blind attorneys to important issues. Non-legal vendors promoting one-stop-shop “eDiscovery services” may conflate otherwise critical distinctions between types of services (just because they offer services to lawyers and others might have used them does not necessarily mean they understand the lawyer’s unique obligations and responsibilities).

For example, digital forensics deals with the systematic and reproducible collection of digital data. Predictive analytics deals with legal analysis of documents. A matter might need forensics and predictive analytics services. But forensics and predictive analytics have critically different roles in a matter. Thus, a law firm might outsource forensic data collection to technicians (under attorney oversight) as a basic, technical process. But outsourcing the predictive coding review to a forensics firm is potentially problematic because the predictive coding fundamental implicates issues of law practice.

Similarly, lawyers might understandably become confused when drawing parallels to older technologies such as keyword searches. Vendors assisting with keyword searches provide a different service than predictive coding. Keyword searches attempt to locate specific terms in documents (a search). Drafting the keyword lists and giving them to a keyword-search vendor who generates results for attorney review mirrors 1970s keypunch cards where the keypunch operator simply feeds the words into the system and produces the results (in actuality, there are some additional issues such as decisions regarding indexing but are beyond the scope of this article).

In contrast, lawyer-analyst directly performs the actual legal analysis on a seed-set of documents when training the system during predictive analysis. Then the training is predicatively applied to the remainder of the documents. The distinction might seem subtle (until you learn about these algorithms) but is nevertheless profound. A keypunch-like operator in keyword searches simply carries out the lawyer’s instructions and mechanically runs the searches. The predictive-coding-training requires the  lawyer-analyst to perform a legally significant review to train the system. See the difference?

A Case for Outsourcing eDiscovery

Once understanding the technologies, lawyers can start to distinguish the services that they are contracting for. I am not saying that one cannot outsource some eDiscovery tasks; I am saying that one cannot blindly outsource eDiscovery predictive analytics.

In fact, outsourcing to licensed attorneys might make sense for some matters and for some firms. Dedicated eDiscovery legal professionals might provide more efficiency and familiarity with the tasks. Plus, eDiscovery implicates project management skills (something that many firms lack) and dedicated legal providers should have these work-flows in place.

Nevertheless, remember, the predictive analytics training step is critical in many matters and reflects your legal analysis as the responsible attorney. Thus, when considering outsourcing predictive analytics the lawyer should carefully consider

  • whether the vendor is a law firm/attorney (or just a technical vendor),
  • the knowledge and capabilities of the prospective outsourcing firm,
  • specifically which services are being outsourced and specifically to whom,
  • the distinctions between tasks such as forensic data collection, keyword searches, and predictive, analytics/preditive coding (know specifically what you are outsourcing and why), and
  • how you will actively supervise the outsourcing firm.

Potential Ethical Problems with eDiscovery Outsourcing

So what are the potential ethical problems? Because predictive coding/predictive analytics fundamentally implicates law practice, the potential ethical issues include

  • sharing legal fees with non-lawyers,
  • failure to supervise assistants (5.1/5.3),
  • aiding in the unauthorized practice of law, and
  • aiding a non-professional services company (illegal conduct).

For example, a vendor offering such predictive coding services must, in Pennsylvania, be organized as Professional Services Companies (or be attorneys as sole proprietors or attorney partnerships) if providing legal services. If not, the contracting attorney might be aiding illegal activity. Also, the contracting attorney might be liable for sharing legal fees with non-lawyers for legal services.

Some non-lawyer vendors might simply “hire some lawyers” to provide the legal services to outsiders—i.e., this is not in-house services. This creates even further problems for all involved. Now the hired lawyers might also be facing ethical problems including sharing legal fees and failure to maintain independence.

Because the training step augments the lawyer’s analysis, lawyers delegating this task to staff will need to very carefully supervise the efforts. The supervising lawyer risks garbage-in-garbage-out—but on steroids. (Blindly delegating these tasks without understanding the technologies risks even more serious consequences.)

While the ethical issues might sound far-fetched or persnickety to a lawyer unfamiliar with predictive analytics, the Washington D.C. Bar recently issued the nation’s first ethics opinion precisely on this topic—see the Washington D.C. Bar’s Non–lawyer Ownership of Discovery Service Vendors and District of Columbia Court of Appeals, D.C. UPL Opinion 21-12. That is, these issues are not theoretical  but real.

Avoiding Ethical Problems

Lawyers are reminded that technology as technology is fundamental to today’s law practice.

To maintain the requisite knowledge and skill, a lawyer should keep abreast of changes in the law and its practice, including the benefits and risks associated with relevant technology….– ABA Model Rules of Professional Conduct, Rule 1.1 Cmt.8 (2012).

Lawyers are starting to see the results of a paradigm shift in the legal community. Today, understanding technology goes far beyond being able to simply view email or buy the best law practice management software. Technology as technology has legally material implications—not only for clients, but also for you as an attorney. Importantly, lawyers cannot just rely on vendors (or the young lawyers) to answer many of the emerging and somewhat complex questions arising from the confluence of law and technology—and changes to the professional rules also make ignoring these new technologies a problem.

While eDiscovery outsourcing certainly might make sense in some cases, the lawyer must be very diligent in fully understanding and supervising the following

  • what is being outsourced (forensics, keyword searches, predictive analytics/predictive coding/TAR);
  • what technologies are being used and specifically how are they being used;
  • who is really doing what in the outsourcing arrangement (technicians, lawyers, paralegals); and
  • what is the structure of the outsourcing vendor (must be a bona fide legal entity depending on the services).

Thus, outsourcing might be a viable option but cannot be initiated without very careful study, proper vetting, and keen oversight.


Original Publication Date: 2013-05-27