In 1984, William Harrington, then a lawyer practicing in Connecticut, penned an article in the Law Library Journal titled “A Brief History of Computer-Assisted Legal Research.” It provides a fabulous history of the rise of Westlaw and Lexis. At the end of the article he discusses the benefits of there being not one, but two legal research platforms. Looking to the future, he closes the article with this scenario, which must have seemed radical at the time:

Someday before long the computer in your office may be wakened at 2:00 a.m. by a signal from a satellite. Down from the satellite will come a stream of information, which your computer will receive and file in the appropriate electronic cubbyholes in its memory. When you arrive at your office in the morning, your computer will have prepared a daily digest for you of information selected according to instructions you have left with the computer. When you want to do research, you will use your own computer to scan the information in its own memory, information that is updated daily and perhaps even more often.

Almost 35 years later, you can conduct legal search on your phone using any of a dozen or more platforms! We’d all be bummed if we had to settle for Mr. Harrington’s vision. But, how much better can legal search technology get?

I don’t want to commit the same kind of undershoot that Mr. Harrington did, and anyone who knows me knows I think AI is only just beginning to transform the life of lawyers and of the law. Yet, there are some inherent limits on what more AI can squeeze out of legal search.

In the first place, today’s legal search technology actually is pretty awesome. Close to a dozen platforms offer fast, easy, effective legal search options using some or all of machine learning, natural language processing, and computational topic modeling.  Some, like CaseText’s CARA, and the more recent Eva by ROSS’s, even dispense with the need to enter a “search” by allowing one to drag and drop a draft brief or memo into a portal that identifies more cases like those already cited. And the visualizations Ravel and FastCase provide allow deeper searching based on citation networks. When I enter a traditional Boolean or natural language search I am impatient if it takes more than three seconds to get high quality results, and then I can resort results based on relevance, date, court, etc.  Fastcase also has democratized legal search by teaming up with state and local bar associations that make access free with membership. Bottom line: legal search is already super-fast at producing on target results and available to all. How much faster and on target can it be?

One outer limit is that the dataset is finite. It’s growing, but at any search moment it is finite. So it’s not like one platform can claim to have more federal cases or state statutes than another. In other words, the platforms are not competing based on datasets, they are competing based on how they help us search through the finite dataset.

Another limit is that typically lawyers have fairly specific searches in mind, so most of the dataset is irrelevant to any search. A good search platform will weed out the noise and zero in on cases, statutes, and other materials on target to the specific inquiry. The existing platforms are already quite good at doing both, and doing it fast.

So, what’s left to improve on?  Well, it turns out that the same search entered into the various platforms does not yield the same “top ten” cases in terms of relevance. Law professor Susan Nevelow Mart conducted such a test and reported her results and assessments in a thoughtful article published in the March 2018 ABA Journal. It is well worth the read, showing as the bottom line that, much like the different results one might get for the same music genre across Pandora, Spotify, and Apple Music, the different platforms have unique algorithms that push different cases to the top of the list. She also showed they all basically return the same list of cases—it’s the “top” hits that differ starkly. It also turned out, however, that many of the cases in each platform’s “top ten” were actually not relevant to the search once evaluated by a human.

So, there is room for improvement–still more AI can do to improve legal search if the goal is to have that top ten list contain the most relevant cases. Let the games begin!