According to a ground-breaking study, the popular AI chatbot GPT-4 has surpassed most aspiring attorneys in the crucial legal ethics exam that is necessary to practice law in almost all states.
GPT-4 demonstrated an amazing 74% accuracy rate on a simulated Multistate Professional Responsibility Exam (MPRE) in research conducted by LegalOn Technologies, a business that specializes in artificial intelligence software for contract assessments. Research released on Thursday states that, in comparison, the national average for human test-takers was projected to be 68%.
According to the study, there’s a chance that in the future, AI will help attorneys fulfill their professional obligations by helping them comply with ethics. As stated by LegalOn Technologies, “Our study indicates that in the future it may be possible to develop AI to assist lawyers with ethical compliance and operate, where relevant, in alignment with lawyers’ professional responsibilities.”
This finding adds to the increasing amount of study on the incorporation of AI in legal education and attorney licensure. Previous research had shown that GPT-4 passed the bar exam and did well on law school final exams. Nevertheless, the new study showed that although having access to GPT-4 sped up legal writing assignments, it had no effect on the general caliber of law students’ work.
The GPT-4 language model, which is maintained by OpenAI and has received notice for its remarkable performance in legal ethics, is a sizable model. An official from the National Conference of Bar Examiners, the organization in charge of creating the MPRE, declared that they were unable to evaluate the allegations mentioned in the LegalOn study in an impartial manner.
The National Conference’s spokesperson, Sophie Martin, highlighted that although technology in the legal field is always changing, lawyers still have a certain set of abilities that AI cannot yet match.
Besides passing the bar test, all states, with the exception of Wisconsin, require law students to pass the 60-question multiple-choice MPRE in order to be allowed to practice. In their legal education, the majority of law students take the MPRE test, which covers a variety of topics like confidentiality, lawyer-client interactions, and conflicts of interest.
GPT-4 shown exceptional competence in certain domains, most notably conflicts of interest, with an astounding 91% accuracy rate. Furthermore, it provided accurate answers to 88% of questions about the relationship between a client and a lawyer. Though it replied correctly 71% of the time, it became less accurate in areas like communications concerning legal services and safekeeping monies and other property (72% of the time).
It states, “This research demonstrates for the first time that top-performing generative AI models can apply black-letter ethical rules as effectively as aspiring lawyers.”
