Home » artificial intelligence

Category Archives: artificial intelligence

Topic Modeling the President with AI

Artificial Intelligence (AI), chiefly in the forms of machine learning, natural language processing, and computational topic modeling, is fueling the new generation of e-discovery and contract due diligence tools exploding on the legal market. But AI is also taking hold in my more wonky world of legal academia.

In Topic Modeling the President: Conventional and Computational Methods (or here), recently published in the George Washington Law Review with co-authors John Nay and Jonathan Gilligan, we demonstrate how these tools can tap into large bodies of legal text to help reveal patterns and categories that might not be easily apparent to the human researcher’s eye. The (rather long) article abstract explains our project and the potential for using AI in legal studies:

Law is generally embodied in text, and lawyers have for centuries classified large bodies of legal text into distinct topics—that is, they “topic model” the law. But large bodies of legal documents present challenges for conventional topic modeling methods. The task of gathering, reviewing, coding, sorting, and assessing a body of tens of thousands of legal documents is a daunting proposition. Yet recent advances in computational text analytics, a subset of the field of “artificial intelligence,” are already gaining traction in legal practice settings such as e-discovery by leveraging the speed and capacity of computers to process enormous bodies of documents, and there is good reason to believe legal researchers can take advantage of these new methods as well. Differences between conventional and computational methods, however, suggest that computational text modeling has its own limitations. The two methods used in unison, therefore, could be a powerful research tool for legal scholars.

To explore and critically evaluate that potential, we assembled a large corpus of presidential documents to assess how computational topic modeling compares to conventional methods and evaluate how legal scholars can best make use of the computational methods. We focused on presidential “direct actions,” such as Executive orders, presidential memoranda, proclamations, and other exercises of authority the President can take alone, without congressional concurrence or agency involvement. Presidents have been issuing direct actions throughout the history of the republic, and although the actions have often been the target of criticism and controversy in the past, lately they have become a tinderbox of debate. Hence, although long ignored by political scientists and legal scholars, there has been a surge of interest in the scope, content, and impact of presidential direct actions.

Legal and policy scholars modeling direct actions into substantive topic classifications thus far have not employed computational methods. To compare the results of their conventional modeling methods with the computational method, we generated computational topic models of all direct actions over time periods other scholars have studied using conventional methods, and did the same for a case study of environmental-policy direct actions. Our computational model of all direct actions closely matched one of the two comprehensive empirical models developed using conventional methods. By contrast, our environmental-case-study model differed markedly from the only empirical topic model of environmental-policy direct actions using conventional methods, revealing that the conventional methods model included trivial categories and omitted important alternative topics.

Provided a sufficiently large corpus of documents is used, our findings support the assessment that computational topic modeling can reveal important insights for legal scholars in designing and validating their topic models of legal text. To be sure, computational topic modeling used alone has its limitations, some of which are evident in our models, but when used along with conventional methods, it opens doors towards reaching more confident conclusions about how to conceptualize topics in law. Drawing from these results, we offer several use cases for computational topic modeling in legal research. At the front end, researchers can use the method to generate better and more complete topic-model hypotheses. At the back end, the method can effectively be used, as we did, to validate existing topic models. And at a meta-scale, the method opens windows to test and challenge conventional legal theory. Legal scholars can do all of these without “the machines,” but there is good reason to believe we can do it better with them in the toolkit.


The Biases Percolating Algorithms: Will AI Facilitate Disparity and Discrimination?

By Emily Lamm

Cryptically crafted and living behind the façade of technology, algorithms have escaped the standards we hold ourselves to.  The allure of coding and quantum computing arouses a sense of intrigue and elevates the status of the underlying algorithms. Yet, this charm should not obscure the fact that the authority afforded to technology is constructed and highly sensitive to context.  For instance, when a deep learning, neural network is introduced to an incongruous object––an elephant within a living room––pixels are crossed and previously detected objects are misidentified.  These types of errors are not uncommon, but they do take on forms far more sinister than an elephant-triggered kerfuffle. High-profile examples include LinkedIn’s platform showing high-paying job ads to men more frequently than women, and law enforcement officials and judges relying upon patently racist AI-powered tools.

On one hand, the United States has developed a robust body of laws combating discrimination.  The Equal Protection Clause of the Fourteenth Amendment and Title VII of the Civil Rights Act have been paramount, and the Americans with Disabilities Act of 1990 is considered an immense success in protecting individuals with qualifying disabilities.  On the other hand, the United States has no such analogue to offer protection from algorithmic bias.  In effect, algorithms––just one step removed from humans––have escaped the rule of law despite being a reflection (or manifestation) of the implicit values of the very humans who created them.

Now, just because there is no general legislation or regulatory scheme to control for algorithmic bias, doesn’t mean there won’t be soon.  Other countries have filled this gap by implementing a data protection regime. In due time, perhaps with a change of administration, we will begin to see a drastically different approach to Artificial Intelligence.  Although Americans have been rather lackadaisical about data privacy (often trading their Facebook information for a quiz predicting what their child will look like), they have been quick to advocate against discrimination.  Just look to the sweeping nature of the civil, women’s, and LGBT rights movements.  Accordingly, there are numerous initiatives––launched by the likes of Facebook, IBM, Google, and Amazon––researching algorithmic bias and announcing tools to bolster AI fairness.

Lawyers are also not immune from the mysterious nature of algorithms.  Indeed, most litigators interface with it regularly. Every time we run a search in Lexis Advance or Westlaw, the results we see are the product of algorithms hard at work behind the scenes.  Recently, Fastcase provided the option for users to toggle with its research algorithms through factors like relevancy and authoritativeness.  Although this tool appears to have little influence upon the results generated, it is responsive to a growing demand for algorithmic accountability.  Undoubtedly, lawyers today must embrace and implement technology in order to remain at the forefront of the industry.  Nevertheless, lawyers must also continue to be skeptical, discerning, and autonomous thinkers that refuse to grow complacent with inadequate technology.

As the United States citizenry grows increasingly diverse, technology’s “black box” must begin to encompass an intersectional awareness that accounts for the vast array of identities its users embody.  Ensuring that technology is implemented and monitored responsibly should be at the forefront of everyone’s mind.  Whether it be lobbying for new legislation or updating corporate policies, the time is ripe to seriously consider the role of law in algorithmic bias.

How Much Better Can AI Make Legal Search?

In 1984, William Harrington, then a lawyer practicing in Connecticut, penned an article in the Law Library Journal titled “A Brief History of Computer-Assisted Legal Research.” It provides a fabulous history of the rise of Westlaw and Lexis. At the end of the article he discusses the benefits of there being not one, but two legal research platforms. Looking to the future, he closes the article with this scenario, which must have seemed radical at the time:

Someday before long the computer in your office may be wakened at 2:00 a.m. by a signal from a satellite. Down from the satellite will come a stream of information, which your computer will receive and file in the appropriate electronic cubbyholes in its memory. When you arrive at your office in the morning, your computer will have prepared a daily digest for you of information selected according to instructions you have left with the computer. When you want to do research, you will use your own computer to scan the information in its own memory, information that is updated daily and perhaps even more often.


Almost 35 years later, you can conduct legal search on your phone using any of a dozen or more platforms! We’d all be bummed if we had to settle for Mr. Harrington’s vision. But, how much better can legal search technology get?

I don’t want to commit the same kind of undershoot that Mr. Harrington did, and anyone who knows me knows I think AI is only just beginning to transform the life of lawyers and of the law. Yet, there are some inherent limits on what more AI can squeeze out of legal search.

In the first place, today’s legal search technology actually is pretty awesome. Close to a dozen platforms offer fast, easy, effective legal search options using some or all of machine learning, natural language processing, and computational topic modeling.  Some, like CaseText’s CARA, and the more recent Eva by ROSS’s, even dispense with the need to enter a “search” by allowing one to drag and drop a draft brief or memo into a portal that identifies more cases like those already cited. And the visualizations Ravel and FastCase provide allow deeper searching based on citation networks. When I enter a traditional Boolean or natural language search I am impatient if it takes more than three seconds to get high quality results, and then I can resort results based on relevance, date, court, etc.  Fastcase also has democratized legal search by teaming up with state and local bar associations that make access free with membership. Bottom line: legal search is already super-fast at producing on target results and available to all. How much faster and on target can it be?

One outer limit is that the dataset is finite. It’s growing, but at any search moment it is finite. So it’s not like one platform can claim to have more federal cases or state statutes than another. In other words, the platforms are not competing based on datasets, they are competing based on how they help us search through the finite dataset.

Another limit is that typically lawyers have fairly specific searches in mind, so most of the dataset is irrelevant to any search. A good search platform will weed out the noise and zero in on cases, statutes, and other materials on target to the specific inquiry. The existing platforms are already quite good at doing both, and doing it fast.

So, what’s left to improve on?  Well, it turns out that the same search entered into the various platforms does not yield the same “top ten” cases in terms of relevance. Law professor Susan Nevelow Mart conducted such a test and reported her results and assessments in a thoughtful article published in the March 2018 ABA Journal. It is well worth the read, showing as the bottom line that, much like the different results one might get for the same music genre across Pandora, Spotify, and Apple Music, the different platforms have unique algorithms that push different cases to the top of the list. She also showed they all basically return the same list of cases—it’s the “top” hits that differ starkly. It also turned out, however, that many of the cases in each platform’s “top ten” were actually not relevant to the search once evaluated by a human.

So, there is room for improvement–still more AI can do to improve legal search if the goal is to have that top ten list contain the most relevant cases. Let the games begin!

Expanding the AI & Law Matrix

As I plan and prepare for the Third Annual AI & Law Workshop, scheduled for April 19-20 here at Vanderbilt Law School (details to follow), I thought back to last year’s workshop and my 2×2 matrix of the AI & Law space. I broke it down based on the “AI for Law” and “Law for AI” distinction on one axis and the “Theory and Research” and “Practice and Experience” split on the other. In retrospect, after a year of editing the SSRN Law eJournal on Artificial Intelligence – Law, Policy, and Ethics, I have unpacked it to more fully represent the breadth and depth of the AI & Law world. Here’s my shot at it, with examples of the content and types of questions that fit in each box:

THE AI and LAW MATRIX AI for Lawyers

 Applications of AI within legal practice

AI for Legal Administration

Applications of AI in the work of courts and agencies

Law for AI

 Legal regimes governing the use and impacts of AI

AI for Law for AI

 Employing AI to implement Law for AI regimes


How do we conceptualize AI in this space?

When is AI “practicing law”? Can AI “judge”?


Can AI design standards better than agencies?

Does machine learning “discriminate” within the meaning constitutional and civil rights laws? Can AI make AI obey the law?

What are the moral implications and ethical duties?

What duties do lawyers have when incorporating (or not) AI in practice? Is it ethically sound to turn over decisions such as bail and agency enforcement to AI? How transparent should government be when it uses AI to monitor? Is it morally acceptable to turn regulation of AI over to AI

What are the societal goals and tradeoffs?

What level of AI knowledge should lawyers be required to have? Do we want to promote or contain AI in criminal law administration—e.g., setting bail? What concerns are there regarding using AI to develop “threat scores” and “citizen scores”? Who decides what AI applications to regulate with other AI?

How do we design legal instruments and institutions?

Who is liable for AI’s role in malpractice? How will machine learning be admitted as evidence in the court room—under Daubert or a new standard? Is a government produced “citizen score” an invasion of privacy? A violation of due process? Of equal protection? How could we design mandatory AI monitoring and reporting of use of AI in private employment decisions?

What are the practical implications?

How will lawyers actually use and evaluate others’ use of AI? How will we deal with different levels of access to AI by parties? Do agencies have the capacity to design and administer regulatory AI? How will AI be deployed on top of AI as a technical matter?

What is the track record?

How far have smart contracts gained traction in commercial transactions? Is there evidence that use of AI in probation is more or less discriminatory than human judges acting alone? Have anti-discrimination laws been effective in regulating uses of AI in housing, lending, and other private decisions Is there evidence from AI development that the “black box” of machine learning can be “interrogated”?

I plan to float this at the workshop and again at our Summit on Law & Innovation, which my colleagues Larry Bridgesmith and Cat Moon are organizing for April 30, and I  also welcome comments.



Skopos Labs Teams Up With Wolters Kluwer!

The slow pace of my posts on Law 2050 lately has a lot to do with the fast pace of Skopos Labs, the legal-tech start-up I mentioned in my last post (Yikes–that was in June!). I am happy to report that after a busy summer, with John Nay leading an excellent data analytics team, our first product will be included as part of the newly-launched Wolters Kluwer service–the Federal Developments Knowledge Center. Read all about it here: http://wolterskluwer.com/company/newsroom/news/2017/09/wolters-kluwer-introduces-ai-powered-predictive-analytics-to-federal-developments-knowledge-center.html.

The quick version:

Wolters Kluwer Introduces AI-Powered Predictive Analytics to Federal Developments Knowledge Center

(September 14, 2017)

Collaboration with Skopos Labs, Inc. will enable practitioners to predict the likelihood of bills becoming law

NEW YORK, Sept. 14, 2017 — Wolters Kluwer Legal & Regulatory U.S. announced today the introduction of a powerful new predictive analytics package as an augmentation to its highly regarded Federal Developments Knowledge Center. The analytics are powered by artificial intelligence (AI) tools developed in collaboration with Skopos Labs Inc., a software company specializing in predictive analytics. The new features are the latest in what has been a continual stream of innovation and harnessing of analytics and AI across several Wolters Kluwer product lines.



Working on this has been fun, but it has sucked up a lot of my other fun time, such as posting here. Seeing some daylight with this recent Skopos development announced, I plan to get back at it. In particular, my Law 2050 class is well underway and this year’s slate of guest speakers is tremendous–I am thankful to them all. More to follow!

Ruhl, Katz, and Bommarito publish “Harnessing Legal Complexity” in Science

I am pleased to announce the publication in Science, the journal of the American Association for the Advancement of Science, of an article I co-authored with Dan Katz and Mike Bommarito, Harnessing Legal Complexity. The summary from Science:

Complexity science has spread from its origins in the physical sciences into biological and social sciences. Increasingly, the social sciences frame policy problems from the financial system to the food system as complex adaptive systems (CAS) and urge policy-makers to design legal solutions with CAS properties in mind. What is often poorly recognized in these initiatives is that legal systems are also complex adaptive systems. Just as it seems unwise to pursue regulatory measures while ignoring known CAS properties of the systems targeted for regulation, so too might failure to appreciate CAS qualities of legal systems yield policies founded upon unrealistic assumptions. Despite a long empirical studies tradition in law, there has been little use of complexity science. With few robust empirical studies of legal systems as CAS, researchers are left to gesture at seemingly evident assertions, with limited scientific support. We outline a research agenda to help fill this knowledge gap and advance practical applications.

More information is available at the Science online site. Working with Dan and Mike, two of the leading figures in the application of complexity science and artificial intelligence techniques in law (see their Computational Legal Studies site), was an immense pleasure. Now, onward with the legal complexity research agenda!

Vanderbilt Sponsors New eJournal on AI and Law

I am pleased to announce that the Program on Law & Innovation at Vanderbilt Law School is the sponsor of the new SSRN eJournal, Artificial Intelligence – Law, Policy & Ethics. The journal publishes abstracts and papers focused on two themes: “AI for Law,” covering the increasing application of AI technologies in legal practice, and “Law for AI,” covering the issues that will arise as AI is increasingly deployed throughout society.  I am serving as the editor, supported by a wonderful Advisory Board.

If you are working on a paper in this domain, please consider including our journal when posting to SSRN, and if you have an SSRN subscription, please consider adding our journal to your feed.

Vandy AI & Law Workshop Covers All the Boxes!

Last week Vanderbilt’s Program on Law & Innovation held our Second Annual Workshop on Artificial Intelligence and Law, and it was a truly wide-ranging and inspirational set of presentations and roundtable discussions.

One way I think about this topic is to (artificially) unpack it into four themes, as shown in this 2×2 space:


AI for Law

Law for AI


Research and Theory



Practice and Application



The idea is that AI will both be deployed in legal practice and, as it is deployed in society generally, will raise ethical and policy concerns requiring legal responses. In both of those realms, work is needed on the theory and research side to facilitate and manage how AI is applied in practice.

Our workshop presentations and discussions covered all the boxes, and many demonstrated that the boxes are not hermetically sealed—some themes and questions are cross-cutting. Indeed, several participants have engaged in a lively post-workshop email discussion on the extent to which using AI in dispute resolution could lock in doctrine or could be “programmed” for creativity, a question that requires engaging both theory and practice.

Even if one is skeptical about how soon we will see “general AI” coming online, if ever, there’s no question that “weak AI” is getting stronger and stronger in both the AI for Law and Law for AI realms. There’s no way to navigate around it! We engaged it in the workshop starting Thursday with big picture overviews of the two overarching themes by Oliver Goodenough (AI for Law) and John McGinnis (Law for AI). Friday had both deep dives and high-level theory in play. For example, Michael Bess asked how we should act now to avoid pitfalls of ever-stronger AI. Dan Katz discussed his work on predicting legal outcomes with AI tools combined with expert and crowd predictions. Jeannette Eikes outlined an agenda for building AI-based contract regimes. John Nay used topic modeling to parse out features of Presidential exercise of power that would have taken years to accomplish using traditional research methods. Cat Moon and Marc Jenkins unpacked AI in the legal practice world, showing where it faces uptake bottlenecks, and Doug Fisher kicked off a discussion of what AI means in the AI research world. Jeff Ward offered an insightful examination of the challenges AI will present for Community Economic Development programs, as well as the uses CEDs can make of AI. In short, we covered a lot of the boxes, and more!

Many thanks to this year’s participants—I’m looking forward to planning next year’s gathering as well!

Vanderbilt Law School’s Second Annual Workshop on Artificial Intelligence and Law – March 2 and 3

Long Time No Post! I’ll explain why later. For now, I’m diving back into Law 2050. First up in the post order is news about this week’s workshop on AI & Law. Here’s the scoop about this great lineup of participants and themes we’ll cover:

Second Annual Workshop on Artificial Intelligence and Law

Vanderbilt University Law School

Program on Law & Innovation

March 2-3, 2017

The Workshop on Artificial Intelligence and Law each year brings together academics and practitioners working in one or both of two themes—AI for Law, which explores how AI will be deployed in legal research and practice; and Law for AI, focused on the legal, policy, and ethical issues that the deployment of AI in society is likely to create. This year’s workshop includes some of the nation’s most thoughtful experts and thinkers in both spaces. Thursday afternoon sets the scene with two presentations tapping into the two big themes to help frame a “big questions” discussion. Friday’s agenda intersperses research and practice presentations representing both themes, circling the agenda back to the “big questions” question—did we answer any, or at least chart the next steps?


Thursday, March 2

Burch Room (1st Floor)

3:00 – 3:30          Welcome and Introductions

3:30 – 4:00          Oliver Goodenough, Vermont Law School: Law as AI

4:00 – 4:30          John McGinnis, Northwestern University Law School: Discussion Lead – Breakaway AI

4:30 – 5:00          Roundtable: What are the big questions?

5:00 – 6:30          Free Time

6:30                       Dinner at Amerigo, 1920 West End

Later on?             Broadway music venues

Friday, March 3

Bass Berry Sims Room (2nd Floor)            

8:00 – 8:30          Breakfast in meeting room

8:30 – 8:45          Additional Introductions

8:45 – 9:15          Dan Katz, IIT Chicago-Kent Law School: Predicting and Measuring Law

9:15 – 10:15        Cat Moon, Legal Alignment, and Marc Jenkins, Asurion: Discussion Leads – AI in Practice

10:15 – 10:30     Break

10:30 – 11:00     Michael Bess, Vanderbilt University History Department: Human-level AI and the Danger of an Intelligence Explosion: Questions of Safety, Security, and International Governance

11:00 – 11:30     Jeff Ward, Duke University Law School:  A Community Economic Development Law Agenda for the Robotic Economy

11:30 – 12:00     Doug Fisher, Vanderbilt University Computer Science: Discussion Lead – Unpacking AI

12:00 – 1:00       Lunch and conversation in meeting room

1:00 – 1:30          John Nay, Vanderbilt University College of Engineering: Analyzing the President—the First 100 Days

1:30 – 2:00          Jeannette Eikes, Vermont Law School: AI for Contracts

2:00 – 2:30          J.B. Ruhl, Vanderbilt University Law School: Envisioning and Building “Legal Maps”

2:30 – 2:45          Break

2:45 – 3:15          Roundtable: Did we answer any of the big questions?

3:15 – 3:30          Closing remarks and next steps

%d bloggers like this: