By: David S. Levine and Ryan M. Hubbard
We recently wrote about one emerging risk of using artificial intelligence (AI) tools in legal practice: the potential loss of attorney-client privilege when clients share information with publicly available AI systems. A related but distinct risk is now emerging. In particular, researchers are developing techniques to manipulate documents so that AI tools misinterpret the contents. For companies increasingly relying on AI to review documents, the implications are significant.
The Risk
We have all heard stories about AI “hallucinations”—the tendency of AI tools to fabricate facts and citations. In the legal context, these errors have been surprisingly frequent and widely publicized.
But a more nefarious risk is now emerging. Researchers and commentators have recently demonstrated several ways to exploit the gap between human and machine review. Researchers have developed custom fonts, for example, that display normally on screen but produce gibberish when processed by an AI tool. More troublingly, researchers have demonstrated techniques that cause AI tools to read entirely different language than what appears on the page to the human reader.
The practical upshot is significant. When a company employee—whether a senior executive or junior assistant, human resources professional or finance analyst—uploads a document to an AI tool, the response will almost certainly appear authoritative. But if, unbeknownst to the individual, the document has been manipulated using the techniques described above, the user will receive a distorted answer.
Implications
The potential implications are wide-ranging. In a corporate transaction, for example, a business executive using AI to summarize a lengthy purchase agreement could be told by the AI tool that the deal contemplates one purchase price when the document actually reflects another. In the employment context, a human resources professional using AI to review a departing employee’s restrictive covenant could be told by the AI tool that the covenant imposes a one-year non-compete when the document on its face imposes a three-year restriction. In litigation, an in-house lawyer using AI to review a large production could be told by the AI tool that a document contains an admission that does not in fact appear in the text.
The risks may be magnified in organizations relying on AI agents or tools to partially automate contract review and approval, or to interact directly with customers. In the purchase price example above, a high-value contract might bypass more intensive review if an AI agent interprets the agreement as having a much lower purchase price and flags it as lower impact.
In each case, the calculation of the bad actor is the same: If it expects its counterparty to rely on AI rather than read the document at issue, manipulating the AI’s output offers a tactical advantage. Indeed, there have been several reports of lawyers in Brazil attempting to use these techniques to manipulate judicial decisions.
To be clear, we do not think that these tools are currently in widespread use, and the AI companies are developing their own measures to combat this manipulation. The ethics and legal rules governing this conduct, moreover, are at best murky. But in significant transactions, litigation, and other matters, companies cannot simply assume that every counterparty will act ethically and that the AI input and output has not been manipulated.
Takeaways
The takeaway here is simpler than in most of our prior updates. AI tools are powerful and often helpful, but they cannot always be trusted to render an accurate account of a document. Anyone using an AI tool to review a document should confirm that the AI’s report matches the actual content of the document. AI “agents”—tools that can take actions on a user’s behalf, such as reading files or sending emails—will especially require safeguards and input controls against manipulation.
On important matters, companies should involve trusted counsel who will read the document themselves and bring their experience to bear on the particular issue. The risks here are only likely to grow as these manipulation techniques become more sophisticated.
Fox Swibel is monitoring these issues closely and can be reached at any time. Please contact David S. Levine, Ryan M. Hubbard, or the Fox Swibel attorney with whom you regularly work to discuss these issues.
David S. Levine
David Levine is a partner in the Firm’s Employment Law and Litigation Groups. David’s practice focuses on representing clients in a wide range of labor and employment matters, as well as other business disputes. As part of his litigation practice, David regularly defends employers in class, collective, and individual actions involving claims under the FLSA, Title VII, ADA, and other federal and state statutes. David also often litigates individual and class actions involving data privacy issues, in particular cases under the Illinois Biometric Information Privacy Act (BIPA). He practices in state and federal courts and before administrative agencies such as the EEOC, DOL, and DOJ.
Ryan M. Hubbard
Ryan Hubbard is a partner in the Litigation and Intellectual Property groups. He concentrates on complex patent and intellectual property litigation in federal court around the country and counselling clients on intellectual property portfolio management, new product development and clearance, and due diligence. He has advised clients regarding cyber-security, privacy, and data breach issues. He represents clients of all sizes, from startups to multinational corporations, and leverages his curiosity and a deep understanding of client needs to deliver practical advice and long-term solutions.
This article contains material of general interest and should not be construed as legal advice or a legal opinion on any specific facts or circumstances. Under applicable rules of professional conduct, this content may be regarded as attorney advertising.