- In a new study, AI beats top class lawyers in evaluating legal contracts.
- The AI developed by LawGeex has been trained to identify issues on numerous legal contracts.
- On average, lawyers achieved an accuracy rate of 85%, while the AI achieved 94% accuracy.
LawGeex, a legal Artificial Intelligence (AI) platform, conducted a new study, where top United States lawyers competed against an AI trained to analyze and evaluate legal contracts. The 20 experienced lawyers from Duke University School of Law, Stanford University and University of Southern California, battled technology and they lost.
Both LawGeex AI and human lawyers with decades of experience were assigned a task of spotting issues in 5 NDAs (non-disclosure agreements), which are contractual base for most of the business deals. The AI algorithms have been developed for nearly 3 years and trained on tens of thousands of contracts.
The study was conducted with input from data scientists, academics, legal and machine learning experts, and was overseen by an independent lawyer and consultant. Most of the documents, whether it is operating agreements for businesses, wills, or stuffs like NDAs, they were very similar to what many lawyers do every day.
Human Vs. AI
Both AI and human lawyers were given 4 hours to review 5 NDAs and detect 30 legal issues, including indemnification, arbitration and confidentiality of relationship. The results were based on their performance, i.e. how precisely they identified each issue.
On average, lawyers achieved an accuracy rate of 85%, while the AI achieved 94% accuracy. The AI didn’t only achieve impressive results in ‘preciseness’, but also in ‘document evaluation time’. While lawyers took an average of 92 minutes to evaluate contracts, AI completed its task in 26 seconds.
In fact, in 1 NDA (out of 5), the AI was able to achieve 100% accuracy, whereas the highest human could ever get was 86%. Bottom line, humans were trounced.
The AI developed by LawGeex has been trained to identify issues on numerous legal contracts, including services and software agreements and purchase orders. The study is merely focused on NDAs.
With the help of custom-designed machine learning and deep learning methods, the AI was trained on tens of thousands of NDAs. The training was based on an exclusive corpus of documents that presented the algorithm with a wide range of examples, which allowed it to distinguish between several legal concepts.
Simply put, training an AI is much similar to training an amateur lawyer – exploring different types of documents is very important in developing a deep understanding of the legal practice.
What Does It Mean?
Making technology more intelligence doesn’t mean the end of humanity. Artificial intelligence can actually help lawyers enhance and accelerate their work, and make them focus on other tasks that require human intervention.
For law firms, the AI fills the paralegal spot, freeing up the valuable time for lawyers to focus on client counseling and other critical works. It will never completely replace a human lawyer, but can certainly boost their work by highlighting the crucial factors at the right moment.
These kinds of tools can help junior lawyers and students increase their productivity and shape future legal practice. The existing AI applications are already improving the fundamental role of lawyers as trusted advisors. This helps secure the relevancy of legal professional, enabling them to remain competitive into the next decade.