ChatGPT Takes Law Exams; Results Suggests a Future Use

Minnesota professors say chatbot got a C-plus, maybe decent enough to be a lawyers' tool
By John Johnson,  Newser Staff
Posted Jan 26, 2023 2:28 PM CST
ChatGPT Takes Law Exams; Results Suggests a Future Use
   (Getty / nirat)

Will the lawyer of tomorrow be a chatbot? Maybe not, but an experiment at the University of Minnesota suggests artificial intelligence could at least become an important tool in the profession. Law professors at the school had the popular ChatGPT take exams alongside real students, reports Reuters. As it turns out, the humans did better, with an average grade of B-plus compared to the AI tool's C-plus. However, that would still be a high enough grade to earn a law degree at the university, though the chatbot would suffer the indignity of being on academic probation.

"Alone, ChatGPT would be (a) pretty mediocre law student," says Jonathan Choi, lead author of the study. "The bigger potential for the profession here is that a lawyer could use ChatGPT to produce a rough first draft and just make their practice that much more effective." The chatbot "struggled with the most classic components of law school exams, such as spotting potential legal issues and deep analysis applying legal rules to the facts of a case,” Choi tells CNN. He predicts, however, that AI assistants will become "standard tools" for lawyers in the not-to-distant future, meaning law schools must start reconciling with that. (One company thinks the age of robot lawyers already has arrived.)

Get the news faster.
Tap to install our app.
Install the Newser News app
in two easy steps:
1. Tap in your navigation bar.
2. Tap to Add to Home Screen.