Prompt Laywering
Unlocking the Full Potential of Legal AI and ChatGPT for Lawyers
A skill that every good lawyer is going to benefit from in the era of Legal AI is prompt engineering. Prompt Lawyering (a term coined here at Legal AI) is the version of that skill which provides the best (and most accurate) legal information when using tools like ChatGPT. In this post, the team here at Legal AI provides not only information about what methods are currently the most effective for prompting ChatGPT for legal information, but tests many of them out to provide you the results. As with any new tool, not every prompt or every attempt to prompt ChatGPT will have the same accuracy or be as comprehensive. As other articles have mentioned, the hallucination effect of these Large Language Models (LLMs) is real, not well understood and an ongoing problem. That being said, ChatGPT can be an effective legal assistant saving you time and providing you stock language you can easily modify to suit your needs. Let’s see how well ChatGPT is doing with a range of legal questions.
Categories Of Legal Information
Case Law Research
ChatGPT can assist lawyers in quickly researching legal cases and statutes providing them with relevant information and insights in a matter of seconds.
Prompt: “Give me information on Mapp v. Ohio”
Mapp v. Ohio (1961) was a landmark case in the United States Supreme Court that significantly impacted the American criminal justice system. The case centered on the question of whether evidence obtained in violation of the Fourth Amendment, which protects citizens against unreasonable searches and seizures, could be used in state courts.
The prompt went on to give the full factual background and ended with the Supreme Court eventually overturning Mapp’s conviction at the state level.
Prompt: “Summarize the case of State v. Estrada, 34 P. 3d 356”
State v. Estrada, 34 P.3d 356 (2001) is a case from the New Mexico Supreme Court that addressed the issue of police officers entering a residence without a warrant and its implications on the subsequent search and seizure.
This result is false. It is a hallucination. Hallucination is arguably a term of art in the world of AI and LLMs. But, it is exactly what you think it is. The LLM model just makes things up and the times in which it will do that and to what degree is unknown. The Estrada case is actually about drug possession and the applicability of a state law to the factual scenario of that case. ChatGPT completely manufactured the content above. The fear of non-lawyers blindly relying on the analysis of these LLMs is well placed. But, there is a fix. Remember, these models were trained on things scraped from the Internet. Which means that they undoubtedly scraped accurate case law and the confabulations of the most deranged persons who were coherent long enough to create a blog post. The fix is what is happening now. Models are being trained on already curated, proofread so to speak, data. Those models will not hallucinate or if they do, it will be much easier to detect and remedy. That day is coming and year will be 2023 when such tools arrive in a form that even non-computer programmer’s can deploy.
Drafting Legal Documents
Legal documents obviously have a series of repeatable clauses used in multiple versions.
Prompt: “Write a demand letter to a grocery store for injuries received by a client falling in the aisle”
[Your Name] [Your Address] [City, State, ZIP Code] [Email Address] [Phone Number] [Date]
[Name of Grocery Store Manager or Owner] [Name of Grocery Store] [Store Address] [City, State, ZIP Code]
Subject: Personal Injury Claim - [Your Client's Full Name]
Dear [Name of Grocery Store Manager or Owner],
I am writing to you as the legal representative of my client, [Your Client's Full Name], who suffered injuries on your premises on [Date of Incident]. On the aforementioned date, my client was shopping at your grocery store located at [Store Address] when they slipped and fell in one of the aisles due to a liquid spill that was not properly cleaned up or marked with warning signs.
As a result of this incident, my client sustained multiple injuries, including [specific injuries, such as fractures, sprains, bruises, etc.]. Consequently, [Your Client's Full Name] has had to undergo medical treatment, including [list any treatments, surgeries, or therapies required]. Additionally, my client has experienced pain and suffering, lost wages due to the inability to work, and a diminished quality of life.
Under premises liability law, property owners and operators have a duty of care to maintain safe conditions for their customers and visitors. It is clear that your grocery store failed to uphold this duty by not promptly addressing the hazardous liquid spill or providing proper warning signage, which directly led to my client's accident and subsequent injuries.
As such, we demand compensation for the following:
Medical expenses: [Specify the amount] - This includes costs for emergency treatment, hospitalization, surgeries, medications, and physical therapy sessions, among other expenses related to the injuries sustained in the accident.
Lost wages: [Specify the amount] - Due to the injuries sustained, my client has been unable to work for [number of days or weeks] and has consequently suffered a loss of income.
Pain and suffering: [Specify the amount] - My client has endured significant physical pain and emotional distress as a result of the accident and the injuries sustained.
In light of the above, we demand a total compensation of [Total Amount] to cover the damages suffered by my client. We kindly request that you or your insurance company respond to this demand letter within [Number of Days, typically 30] days from the date of receipt.
Please note that if we do not receive an adequate response within the stipulated time, we may have no choice but to pursue further legal action to protect our client's rights and interests. However, we hope to reach an amicable resolution without resorting to litigation, as we believe that it is in the best interest of all parties involved.
To discuss this matter further, please feel free to contact me at [Your Phone Number] or [Your Email Address]. We are open to negotiating a fair and reasonable settlement that accurately reflects the injuries and damages suffered by [Your Client's Full Name] due to the negligence of your grocery store.
Please be aware that this demand letter serves as a notice of our intent to pursue legal action if necessary, and that we are prepared to take all necessary steps to ensure our client receives the compensation they deserve for their injuries and losses.
Thank you for your prompt attention to this matter. We look forward to your response and a swift resolution of this claim.
Sincerely,
[Your Name] [Your Law Firm Name, if applicable]
While the writing in the above demand letter could be humanized a bit, it represents a comprehensive set of paragraphs to consider for any such letter. Clients will eventually figure out how to write such letters modifying only the portion about being a lawyer and simply stating they will contact a lawyer if a suitable settlement cannot be reached. These kinds of lawyerly-appearing documents are not fraught with the consequential concerns for non-lawyers trying to do it on their own. These edge cases are going to grow in activity as non-lawyers move closer and closer to the essence of what lawyering is today.
Law Professor wants ChatGPT taught in law school. Maybe this is a good idea to start teaching this now. However, even if it is not being taught, the law students are using it. What for? What would you have used it for if ChatGPT had arrived when you were a first year law student?
Other Forms
ChatGPT is now being used to draft common legal forms and waivers, such as liability waivers and demand letters. For non-lawyers, they are perceiving a savings of money on the front end. For lawyers, they are wary that the output of LLMs are going to sometimes lead clients astray creating problems they will have to fix down the road.
Prompt: “Draft a liability waiver for the sale of fire extinguishers with standard terms and conditions”
Within 5 seconds, a document was produced with the following sections:
Title: Liability Waiver for Sale of Fire Extinguishers
ACKNOWLEDGMENT OF RISK, RELEASE OF LIABILITY, INDEMNIFICATION, ASSUMPTION OF RISK, WARRANTIES, SEVERABILITY and , GOVERNING LAW.1
Contract Review
There are already tools being marketed to lawyers touting their AI capabilities for contract review and drafting. We have written about such tools in other posts. These tools will proliferate driving their subscription costs ever lower and attracting non-lawyers as well. For many small business owners, 5-10 years into the daily running of their business puts them in nearly the same if not equal footing with their lawyers with regard to what their typical contracts should contain. People in such a position are going to think about where they can leverage AI to cut costs. Having an AI tool both draft and review their regular, typical contracts is one such area to consider. Lawyers will have to become increasingly comfortable with their clients arriving to their offices already in possession of a nearly done agreement. Those lawyers in those scenarios are going to be completing those contracts where in the past they would have drafted them from the start. Clients will increasingly think, and perhaps say out loud, why would I pay my lawyer to draft the entire agreement when I can get an AI tool nearly at no cost to draft it 90% of the way?
For lawyers, there is opportunity. An analogy may help. I have experience as a data analyst and now work as a data engineer. Tools like ChatGPT have increased my productivity many times. It has also enabled me to expand into technologies that I have a fundamental understanding of and achieve outcomes as if I was an intermediate or even expert developer in that technology.
For lawyers, the hesitation about expanding into unknown or little known areas of the law can be ameliorated with ChatGPT like tools. Securities law? I have no idea. DUI cases, I passed many of those on when I was in my private practice. But, today, if I were practicing and a client came in with a retainer and asked me to take on the defense of one of those cases, I would think twice about merely referring them out. I have not actually examined this within ChatGPT, but I am quite certain a world of information is easily accessible about the defense of both such cases.
Prompts that do far more than just drafting, question and answer on a document or text extraction are already well underway. Prompts such as this one are yielding detailed analyses with the help of LLM technology:
Prompt: “Please review the following contract and provide me with a summary of its purpose, the three most crucial terms, the duration of the contract, and what the consideration for the contract is: [copy and paste contract]”
Microsoft has an exclusive license with OpenAI, the makers of ChatGPT. They have already launched a service connected to their cloud computing offering, Azure, using OpenAI technology. In one of their recent how-to videos on that service, they featured the analysis of a contract being demonstrated by two obvious non-lawyers. It’s happening already.
By regularly using ChatGPT in this way, an experienced contract lawyer can save time and effort in their daily tasks, allowing them to focus on other important aspects of their work, such as strategy and client relationships, marketing, preparing public presentations and networking.
The Perceived Limits of ChatGPT - Already Gone
OpenAI advises on its website that “GPT-4 generally lacks knowledge of events that have occurred after the vast majority of its data cuts off (September 2021), and does not learn from its experience.”
Why? Because, what it had to do in order to accomplish its ability to produce documents, etc, was train a language model on information culled from the web. But, after the initial ingestion, ending in 2021, the OpenAI folks had to deploy humans to then refine, curate, secure and otherwise modify that content to weed out dangerous, illegal or other unethical materials (by their internal standards). This process took more than six months to complete.
For example, you cannot use ChatGPT to obtain plans to build a nuclear weapon. Of course, you can get that information at thousands of libraries across the country and online, but the OpenAI folks made the call to delete that information. You cannot even use it to produce positive articles or blog posts about criminal conduct. So, curation with regard to these LLMs is a thing that will persist.
You might then breathe a sigh of relief reasoning, “no one can really rely on legal advice or information produced by a machine that has data that is already 2 years out of date.” You would be wrong.
The pace of change in this area, I have to emphasize, is unlike other technology revolutions. This is not merely gas lamps to electricity, or buggy whips to motorized vehicles. Nope. With those Industrial Revolution transformations, the change was obvious, it was coming, but it was not overnight. You can find thousands of images of American cities in that era showing horse drawn vehicles riding alongside combustion engine vehicles rolling past overhead electric trolley lines. All these technologies overlapping as the eventual combustion engine won out. That is not happening with LLMs and AI.
It is now possible, a mere six months after the release of the purportedly date limited ChaptGPT to use the sophistication of that model to query live, current data anywhere online. Not even anticipated by the OpenAI folks, their LLM is now the ultimate search engine. It can be used, with some easy to code add on tools, to scour and search the live internet for answers to natural language posed questions. Google is worried. Released internal Google Memo warns, “we have no moat.”
What this also means is that any lawyer, or non-lawyer can now easily find many services enabling the upload of a 20 page pdf document and the instant creation of a question and answer tool queried by natural language questions. No more frustration for non-lawyers trying to understand legalese. Questions like the date of a contract, the main terms, the governing law and, yes, “what happens if I breach the agreement?” are all within reach of everyone….now. You might consider how you could use a tool to reduce the number of emails or calls from a client when a court issues a decision or a party submits a contract, etc. Simply upload the document to one of these tools, point your client to that tool and they can ask all the questions they want of the document and leave for you only those edge cases the tool cannot adequately address.
Reasons for the Pace of Change Factor
This technology is not available to the elite few (like other new inventions of the past). Think CD players which were $5,000 and as large as a television some of them when they first debuted. This technology is free. ChatGPT is free. A range of other LLMs are free. (e.g. HuggingFace)
Secondly, the number of people worldwide who have the capability to both envision and enact extensions and transformations of what ChatGPT is are essentially limitless. (New people are not only becoming AI developers every day across the globe, but people without such skills are, you guessed it, using ChatGPT to gain those skills).
Third, money. The dotcom boom of the late 90s, early 2000s was a gold rush that ended in gold for some and bankruptcy for others. (Pets.com, Friendster, MySpace). Millions, literally millions, of people are staying up all hours of every day imagining their third house in Hawaii as they look back on these years and what they invented with OpenAI’s LLMs and others and are furiously developing and releasing MVPs (Minimally Viable Products).
LangChain - The Example that is The Rule
In October 2022, just one month after ChatGPT was released, a single person named Harrison Chase developed an add on to OpenAI’s LLM called LangChain.2 It is a tool that removes a lot of the technical challenges for developers accessing and deploying apps using ChatGPT like technologies. In just six months, if you search YouTube right now with “LangChain” there are nearly a thousand how-to videos using this tool and extending it beyond what its originator even thought was possible. Perhaps what might be the final locus of such tools, a LangChain user recently released to the public the very simple steps to enable a user to create the capabilities on their own computer, not connected to the internet, to ingest all their company or personal documents and have them readily accessible via natural language querying.
For lawyers, this means many things. One thing that springs to mind is imagining a lawyer who has been in litigation for the past 10 or 20 years. That lawyer can take all the briefs, pleadings they have filed and those filed against them and put them into this tool. Every lawyer now effectively has the skills of every lawyer they have ever faced and those strategies and arguments are more easily accessible than ever.
The kinds of queries that could yield a 10x improvement in your productivity querying that corpus of documents:
What are the relevant laws or regulations regarding [legal issue]?
What is the history of [legal case/issue]?
What is the legal definition of [legal term or phrase]
What is the legal precedent for [legal case/issue]?
What are the pros and cons of [legal argument/position]? _.
What is the standard for [legal issue] in [jurisdiction]?
What are the key legal arguments in [legal case/issue]?
Provide a summary of [case name].
Summarize the following contract: [copy and paste contract].
But, its use goes beyond just good lawyering. We have all had to hire and consult experts in both criminal and civil cases in the past. Why? Because they know stuff we don’t and failing to rely on their assistance can result in us missing legal issues. But, now, lawyers will do what other non-lawyers are already doing. We will obtain a certain level of expertise about DNA, computer forensics, the intricacies of SEC violation defense, Medicaid fraud, non-compete agreements in Colorado, etc. And, in some cases, you know this is true, lawyers will forego hiring an expert at all and instead rely on what they can learn from using ChatGPT like tools. We are going to save money, time and in some cases, provide superior representation by extending our domain knowledge and ability to learn in new domains using such tools.
The types of prompts that will best serve you interacting with ChatGPT like tools are in some sense a guess and a standard. The standard will be those useful when querying general domain knowledge LLMs like OpenAI. Others will be domain, case and legal field specific and best dealt with as a specialty. Experienced criminal defense lawyers will be better at prompting LLMs or tools focused on criminal case only content. Securities lawyers, the same for securities law LLMs, etc. It is likely that the knowledge base for LLMs will begin to differentiate. There are a few reasons for that. One is that domain specific LLMs are likely to provide the most accurate information and minimize if not eliminate hallucination. Second, they will be more inexpensive to query as they will have a smaller universe of information initially and be less expensive to augment as legal information grows.
Finally, those domain specific LLMs and tools will also learn from users and begin to provide insight to users as to better prompts than the ones they are using. This will have the effect mentioned above of making novice prompt lawyers into intermediate or expert prompt lawyers within a short period of time.
My advice, as often said in this space, do not believe anyone who says they have some certainty about what the future of AI and lawyering will be. They are confabulating. But, do believe that, as we have often written here, that the law is going to change, lawyering is going to change and that pace is going to be faster than lawyers are used to. But, as with any change, there will be those that embrace it and prosper and others who resist and have to play catch up. You are reading this so I know which group you are in. Until next week.
For brevity, I only listed the sections and not the content. However, try the prompt yourself and you will see the section content is perfectly passable as being written by a lawyer.