Skip to content

Syllabus (1)

Contents

Subsequent assignments on the Syllabus (2) page

Class 1: General Intro

  1. If you have not done so already, be sure to register for zoom access well before the first class (there are additional steps).
  2. Please read the class policies.
  3. Well before the first class, please email me at froomkin@law.miami.edu
    1. Please use an email account where I can reach you if I need to.
    2. Please use “AI25: [your name]” as the subject line of your email.  (Attn bots – Substitute your actual name for “[your name]” please.)
    3. In the body of the email, please tell me a little about yourself; please also include–at the end–your preferred pronouns. (Mine are he/him/his.) Also include a telephone number where you can be reached in emergencies.
    4. Also, please tell me what you hope to get out of the class – this could affect the syllabus!
  4.  Before doing the reading below, make a few notes that we can discuss in our first class about:
    1. How you think most people would define an “artificial intelligence”
    2. How you personally would define an “artificial intelligence” — if that’s any different from the above.
  5.  You will also want to have a look at the Syllabus (a work in progress!), and the terms of Writing Assignment I and Writing Assignment II. Please note that, depending on class size. I will be asking each of you to serve as an on-call panelist for some number of classes.
  6. After you have done the above, please read
    1. Pages 1-13 & 17-25 of Neil Richards`& William Smart, How Should the Law Think About Robots? (you may skip section 3).

Questions

  1. Do you agree with the argument in Richards & Smart?
  2. Do you think the Richards & Smart argument applies with equal force to AI as it does to robots? If not, is it more or less applicable?
  3. What do you suspect are/will be the major legal/regulatory/social issues posed by the increased use and sophistication of AI (however defined)?
  4. Who is likely to be most affected by AI? How should these groups’ views be reflected in the policy-making process?
  5. Abstractly, what entities, private or public, are best suited to make decisions as to AI regulation/policy?

Class 2: Intro to Machine Learning

  1. FPF, Privacy Expert’s Guide to Artificial Intelligence and Machine Learning (2018).
  2. Pages 669- 702 of David Lehr & Paul Ohm, Playing with the Data: What Legal Scholars Should Learn About Machine Learning, 51 U.C. Davis L. Rev 653 (2017). [Optional: also read 703-17.]
  3. Deep Learning in 5 Minutes:
  4. Pages 410-435 of Ryan Calo, Artificial Intelligence Policy: A Primer and a Roadmap, 551 U.C.D. L. Rev. 399 (2017).
  5. Look at the ten “Top Takeaways” in the Stanford Institute for Human-Centered Artificial Intelligence’s 2024 AI Index Report (the boxes on the first couple of screens).  Of course if you want to delve deeper, feel free.
  6. Some very good advice: Ethan Mollick, 15 Times to use AI, and 5 Not to, One Useful Thing (Dec 9, 2024).
  7. A report from the front: Ethan Mollick, What Just Happened, One Useful Thing (Dec. 19, 2024):
    • “The last month has transformed the state of AI, with the pace picking up dramatically in just the last week. AI labs have unleashed a flood of new products – some revolutionary, others incremental – making it hard for anyone to keep up. Several of these changes are, I believe, genuine breakthroughs that will reshape AI’s (and maybe our) future.”

Optional Readings

By the way, there’s a lot more where that came from:

Chapter 2: “Gradient descent, how neural networks learn”
Chapter 3: “What is backpropagation really doing?”

…and then it gets a bit mathy….

  • Neural networks explained by Bertrand Serlet, one of the pioneers in the field:
    • A commentator on this video — maybe human, maybe AI –summarized it like this:
      Bertrand delves into the mechanisms and underlying reasons for the effective operation of Large Language Models (LLMs) like GPT-3, which exhibit remarkable abilities in answering questions that require knowledge and reasoning. The speaker begins by emphasizing that LLMs are not magical but are highly effective due to their extensive training on diverse internet data, which allows them to encapsulate a vast array of knowledge and reasoning patterns.The discussion progresses into technical details, explaining that LLMs function through neural networks that process input data as vectors of numbers, applying mathematical functions that involve weights, biases, and normalization. These neural networks are capable of complex representations because of their multi-layered structure, which integrates simple mathematical functions recursively. This layering allows for sophisticated information processing, where each layer builds on the output of the previous one, increasing the depth and complexity of data interpretation.A significant focus is given to the concept of dimensionality in neural networks. The speaker explains how higher dimensions allow neural networks to escape local minima—a common problem in optimization processes—and find more globally optimal solutions through gradient descent. This is critical as it enables the models to generalize well from training data to new, unseen queries.Furthermore, the video covers the innovative use of autoencoders and embeddings in neural networks. Autoencoders help in compressing and decompressing data efficiently, facilitating the learning of useful data representations without loss of crucial information. Embeddings, particularly in the context of words, are portrayed as multi-dimensional vectors that capture semantic relationships, allowing for the nuanced understanding and generation of language based on the context established by training datasets.Finally, the concept of “attention” in neural networks is highlighted as a method to enhance model performance by focusing on the most relevant parts of the input data, thereby improving efficiency and accuracy in tasks such as translation and content generation. This attention mechanism is integral to the model’s ability to effectively condense and utilize the vast amount of knowledge gleaned from the data it was trained on, encapsulating complex patterns such as grammar and reasoning in a computationally manageable way.Overall, the speaker elucidates that the power of LLMs lies in their complex architectures, sophisticated training methods, and the ability to harness the “blessing of dimensionality,” enabling them to mimic human-like reasoning and language understanding.
  • The OECD AI Observatory page is continually updated with interesting material.

Notes & Questions

  1. Explain what the terms “ML” and “AI” usually refer to, and how they relate to each other.
  2. Explain in broad terms what are the key features (and differences) of
    1. Supervised Learning
    2. Unsupervised Learning
    3. Reinforcement Learning
  3. Suppose UMiami law wanted to see if an ML system could help with admissions. What should it pick as, or among, the outcome variable(s)?
    1. Willingness to accept the offer? [US News rates schools on “selectivity” – the fewer offers you give relative to matriculation, the better they rate you.]
    2. 1L GPA?
    3. GPA at graduation?
    4. Passing the bar? (But some folks never take it, some retake it, some take it out of state and we don’t always know if they passed, much less if they failed.)
    5. Becoming a partner in a law firm within 10 years of graduation?
    6. $ value of donations to the law school in first 10 years after graduation?
    7. Something else?
  4. Do any of the above create undesirable bias risks? How?
  5. Should we be more concerned about “false positives” (admit, when we shouldn’t have) or “false negatives” (reject when we should have admitted)?
  6. To what extent if any should the law school demand that the model be “explainable”?  Or is it enough that it produces a great class of admits?
  7. Would it be appropriate to use information outside the application to help train the ML system, e.g. publicly available Twitter, Facebook, Instragram, or TickTock feeds or pages?
    1. If so, would it be better to set criteria by hand, or let the ML decide what types of social media signal better potential admits?
  8. Suppose UM has hired a contractor who produced an AI for admissions.
    1. How would we test it for “fairness”?
    2. What is “fairness” in this context?
  9. How if at all would your answers to questions 6-8 above differ if UM were a state law school?
  10. What does (or will) it mean when we can no longer tell if we are corresponding with, or speaking to an AI as opposed to a person?
  11. Based on what you know so far, are AIs – or AIs+robots – headed for world domination in the near or far future?

Class 3: Intro to Foundation Models and Generative AI

Foundation Models

  1. Seclections from introduction (pages 3-10, 22-24, 28-29, 33) of Rishi Bommasani et al,, Stanford HA, On the Opportunities and Risks of Foundation Models (2023).  This is an enormous report, and we’ll be returning to subsequent sections of it from time to time.
  2. Rishi Bommasani et al,, Stanford HAI, Considerations for Governing Open Foundation Models (Dec. 13, 2023).
  3. Peter Henderson et al., Stanford HAI, Safety Risks from Customizing Foundation Models via Fine-tuning 1-4 (Jan. 2024).
  4. James Grimmelmann, Blake E. Reid & Alan Z. Rozenshtein, Generative Baseline Hell and the Regulation of Machine-Learning Foundation Models, Lawfare, (May 8, 2024).

Generative AI

  1. Timothy B Lee and Sean Trott, Large language models, explained with a minimum of math and jargon Want to really understand how large language models work? Here’s a gentle primer., Understanding AI (July 27, 2023)
  2. OpenAI’s New App Store Could Turn ChatGPT Into an Everything App, Wired (Jan 10, 2024).
  3. MIT grad student explains graphic diffusion models:
  4. Stable Diffusion, How does Stable Diffusion work? (Jan 4, 2024). This is a challenging read, but persevere! Start at the top and read until it says “You can skip to the next section”; then if you wish you may skip, but do read from “Stable_Diffusion_v15_vs_v2” to the end.

Context

Optional

Foundation Models

  • Section 2 (“Capabilities”) — and especially sections 2.4-2.6 (“Reasoning and Search”, “Interaction” & “Philosophy of Understanding”–of Rishi Bommasani et al, Stanford HA, On the Opportunities and Risks of Foundation Models (2023).
  • Section 4.3 (“Adaptation”) of Section 3.1 of Rishi Bommasani et al, Stanford HA, On the Opportunities and Risks of Foundation Models (2023).
  • Foundation Models: An Explainer for Non-Experts — It’s two hours long, but it’s by the experts on foundation models

  • Competition and Markets Authority (CMA) report proposing principles to encourage consumer protection and healthy competition in foundation models.  There’s a short version and a long version (Sept. 18, 2023).
  • (*) Thibault Schrepel & Jason Potts, Measuring the Openness of AI Foundation Models: Competition and Policy Implications (May 14, 2024):
    • This paper presents the first comprehensive evaluation of AI foundation model licenses as drivers of innovation commons. It introduces a novel methodology for assessing the openness of AI foundation models and applies this approach across prominent models such as OpenAI’s GPT-4, Meta’s Llama 3, Google’s Gemini, Mistral’s 8x7B, and MidJourney’s V6. The results yield practical policy recommendations and focal points for competition agencies.
  • (*) Matthew R. Gaske, Regulation Priorities for Artificial Intelligence Foundation Models, 26 Vand. J. Ent. & Tech. L. 1 (2023):
    • “This Article  … adapts a generalized form of the fintech Innovation Trilemma framework to argue that a regulatory scheme can prioritize only two of three aims when considering AI oversight: (1) promoting innovation, (2) mitigating systemic risk, and (3) providing clear regulatory requirements. … [L]egal scholarship examining AI has insufficiently recognized the role of international and corporate competition in such a transformational field. Considering that competitive context and the Trilemma, this Article argues from a descriptive perspective that solely one policy prioritization choice is needed: whether to emphasize systemic risk mitigation or clear requirements, given that prioritizing innovation is effectively a given for many governmental and private actors. Next, regulation should prioritize systemic risk over clarity because foundation models present a substantive change in the potential for, and nature of, systemic disruption. Finally, the Article considers ways to mitigate regulators’ lack of legal clarity. It argues instead, in light of the Trilemma’s application, for use of a sliding scale of harm-based liability for AI providers when reasonably implementable, known technological advances could have prevented injury. This tradeoff thus promotes innovation and mitigates systemic risk from foundation AI models.”
  • Cary Coglianese & Colton R. Crum, Regulating Multifunctionality, U of Penn Law School, Public Law Research Paper No. 24-55 (Dec. 16, 2024):
    • Foundation models and generative artificial intelligence (AI) exacerbate a core regulatory challenge associated with AI: its heterogeneity. By their very nature, foundation models and generative AI can perform multiple functions for their users, thus presenting a vast array of different risks. This multifunctionality means that prescriptive, one-size-fits-all regulation will not be a viable option. Even performance standards and ex post liability—regulatory approaches that usually afford flexibility—are unlikely to be strong candidates for responding to multifunctional AI’s risks, given challenges in monitoring and enforcement. Regulators will do well instead to promote proactive risk management on the part of developers and users by using management-based regulation, an approach that has proven effective in other contexts of heterogeneity. Regulators will also need to maintain ongoing vigilance and agility. More than in other contexts, regulators of multifunctional AI will need sufficient resources, top human talent and leadership, and organizational cultures committed to regulatory excellence.
  • NEW:(*) NIST, Updated Guidelines for Managing Misuse Risk for Dual-Use Foundation Models (Jan. 15, 2025).  Press Release  & Full Second Draft:
    • [This document] outlines voluntary best practices for identifying, measuring, and mitigating risks to public safety and national security across the AI lifecycle.US AISI made a number of improvements to the initial public draft (released July 2024) to improve usability and incorporate thoughtful feedback from more than 70 industry, academic, and civil society experts submitted during the initial comment period.Key updates in this expanded version include:
      • Detailing Best Practices for Model Evaluations: This version was expanded to include a new appendix, which provides a detailed overview of existing approaches to measuring misuse risk and is intended to help make these guidelines more actionable for developers and other users.
      • Expanding Domain-Specific Guidelines on Cyber and Chemical and Biological Risk: In order to help operationalize these guidelines in high-priority public safety and national security domains, US AISI added two extensive appendices to the document – one on chemical and biological misuse risk and a second on cybersecurity misuse risk.
      • Underscoring a Marginal Risk Framework: Throughout the document, US AISI further clarified the importance of a “marginal risk” framework for assessing and managing risk when analyzing the potential impact of a given foundation model.
      • Addressing Open Models: These guidelines were updated to support their proportional application to and usefulness for open model developers, underlining US AISI’s continued support of the trustworthy development and deployment of both open and closed models.
      • Managing Risk Across the AI Supply Chain: While model developers remain the main audience for these guidelines, US AISI included additional content to support risk management practices for a range of actors across the AI supply chain.

Generative AI

Notes & Questions

  1. The hallmark of a generative model as opposed to a, excuse the term, classic ML model is that while the ML model is supposed to spot patterns in existing data, or to classify new data in terms of existing data, a generative model is designed to create new content.
  2. The hallmark of a ‘foundation’ model is that it is intended to be adapted by others for a multiplicity of purposes, not all of which are necessarily foreseeable by the foundation model’s designers.
  3. Can you identify policy and liability issues arising from the sale or use of generative AI models, or generative AI services (e.g. web-based interfaces letting the public generate content, either free or for a fee)?
  4. Can you identify policy and liability issues generated (excuse the term) by foundation models that might not exist with an ‘ordinary’ ML system?
  5. If Grimmelmann et al are right that there is no neutral baseline a regulator can use to make rules governing foundation models, but there (likely) is for more targeted appliations based on those models, does that mean we should not try to regulate the foundation models? If we are not ready to let them have a free pass (wreak havoc?) what then?
  6. Thanks to AI, even people who can’t draw a straight line can be artists! Can they be great artists?  What could go wrong?
  7. The creation of a “GPT App Store” demonstrates that generative models can act like foundation models.  Indeed, this may be one of the more prevalent uses of chatbots.
    1. Shortly after it debued the ‘App Store’ was overrun with apps promising to be various sorts of ‘virtual girlfriends’ even though these violate OpenAI’s usage policy,
    2. If you were OpenAI’s general counsel, what would you want to prohibit in your terms of service?  How would you go about enforcing those rules, keeping in mind the expense of human moderation and the whack-a-mole nature of determined violators?
    3. Was this issue on your list?
  8. Starting about 1:38 in the assigned video above, an MIT computer science graduate student gives his legal opinion on whether AI-generated images are protected by copyright, and whether they can be sold (all engineers think they are lawyers!).  We’ll be revisiting this and related issues, but for now, what do  you think of his advice?

Class 4: Chatbots (& LLMs)

  1. Register for free accounts at Chat-GPT and Claude.AI  Play around. Post anything interesting that happens on the class blog.
  2. Have a look at the plethora of custom versions of GPT now available.  OpenAi has invited anyone to fine-tune their own model.
  3. Prakash Selvakumar, Text Generation with Temperature and Top-p Sampling in GPT Models: An In-Depth Guide (Apr 2023). (Note: text is significantly less off-putting than the title!)
  4. [NEW] Steve Engelbrecht, Output from AI LLMs is Non-Deterministic. What that means and why you should care. (May 12, 2023).
  5. Joshua Hawkins, The Biggest Differences Between Claude AI and ChatGPT, Lifehacker (Nov. 29, 2023).
  6. Kevin Roose, A Conversation With Bing’s Chatbot Left Me Deeply Unsettled, New York Times (Feb. 16, 2023)
  7. David Weinberger,chatGPT admits it assumes a Western, Northern culture, Joho the Blog, (March 3, 2024).  [Should we believe it? Probably: See, optionally, Hu et al., Generative language models exhibit social identity biases, Nature Computational Science (Dec. 12, 2024).]
  8. Emily Bender et al., On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?, FAccT ’21, March 3–10, 2021, Virtual Event.
    1. Scholarly reaction to the Bender paper: Christoph Ebell et al., Towards intellectual freedom in an AI Ethics Global Community, 1 AI and Ethics 131 (2021).
  9. Angie Wang, Is My Toddler a Stochastic Parrot?, The New Yorker (Nov. 13, 2023) .
    1. Alternate link if above does not work (may not work either….)

Optional

  • Miscellaneous news stories on how chatbots can go wrong:
  • The Bender article above caused a fuss. See Google Researcher Timnit Gebru Says She Was Fired For Paper on AI Bias for details.
  • (*) Margot E. Kaminski & Meg Leta Jones. Constructing AI Speech, Yale L.J. Forum (Apr. 22, 2024):
    • “Artificial Intelligence (AI) systems such as ChatGPT can now produce convincingly human speech, at scale. It is tempting to ask whether such AI-generated content “disrupts” the law. That, we claim, is the wrong question. It characterizes the law as inherently reactive, rather than proactive, and fails to reveal how what may look like “disruption” in one area of the law is business as usual in another. We challenge the prevailing notion that technology inherently disrupts law, proposing instead that law and technology co-construct each other in a dynamic interplay reflective of societal priorities and political power. This Essay instead deploys and expounds upon the method of “legal construction of technology.” By removing the blinders of technological determinism and instead performing legal construction of technology, legal scholars and policymakers can more effectively ensure that the integration of AI systems into society aligns with key values and legal principles.”Legal construction of technology, as we perform it, consists of examining the ways in which the law’s objects, values, and institutions constitute legal sensemaking of new uses of technology. For example, the First Amendment governs “speech” and “speakers” toward a number of theoretical goals, largely through the court system. This leads to a particular set of puzzles, such as the fact that AI systems are not human speakers with human intent. But other areas of the law construct AI systems very differently. Content-moderation law regulates communications platforms and networks toward the goals of balancing harms against free speech and innovation; risk regulation, increasingly being deployed to regulate AI systems, regulates risky complex systems toward the ends of mitigating both physical and dignitary harms; and consumer-protection law regulates businesses and consumers toward the goals of maintaining fair and efficient markets. In none of these other legal constructions of AI is AI’s lack of human intent a problem.”By going through each example in turn, this Essay aims to demonstrate the benefits of looking at AI-generated content through the lens of legal construction of technology, instead of asking whether the technology disrupts the law. We aim, too, to convince policymakers and scholars of the benefits of the method: it is descriptively accurate, yields concrete policy revelations, and can in practice be deeply empowering for policymakers and scholars alike. AI systems do not in some abstract sense disrupt the law. Under a values-driven rather than technology-driven approach to technology policy, the law can do far more than just react.”
  • (*) Camilla A. Hrdy, Keeping ChatGPT a Trade Secret While Selling It Too, Berk. Tech. L.J. (forthcoming):
    • “Generative artificial intelligence products such as ChatGPT raise novel issues for trade secret law. But one of the most important issues is an old one: How to sell an information good, like computer software, while also maintaining trade secrecy protection for the underlying content? When a company wishes to sell a new technology to the public, the normal recourse is to obtain a patent. Patents require public disclosure and end after a fixed term of years. However, based on decades of precedents established for software, generative AI companies will be able to rely on trade secret law instead—maintaining indefinite protection for their technology, even as they profit from making it widely available to the public, and even after reverse engineering becomes technically feasible. This is what many companies did with closed-source software, and this is what developers of some generative AI models—including ChatGPT—are doing today. They are releasing he models in a “closed-source” format that hides algorithms, code, training data, and underlying model architecture from users. And they are attaching contractual provisions called “terms of use” or “end user license agreements” (EULAs)—that limit users’ ability to reverse engineer information about how the models work or share that information with others. Some of these agreements, including ChatGPT’s, even contain noncompete provisions.”If liability for breaching these provisions were limited to breach of contract, there would be less cause for alarm.However, some case law—and some state statutes—indicate that reverse engineering trade secrets in breach of an anti-reverse-engineering clause can give rise to trade secret liability as well, because breach of the contract transforms otherwiselawful reverse engineering into an “improper means” of acquiring trade secrets. The prospect of trade secret liability for what should be, at worst, breach of contract is alarming. It means prevailing plaintiffs can obtain trade secret law remedies, not just contract law remedies, and it means that liability can extend to third parties who did not even sign the contract. For example, if someone reverse engineers information about ChatGPT in violation of a boilerplate terms of use, and then shares that information with someone else, who publishes the information on the internet, both of these actors could be liable for trade secret misappropriation. Maintaining some legal protection for information goods is important. Otherwise, companies might not make information goods available to the wider public at all. But trade secrecy protection should not last after actual secrecy has ended.Fortunately, there is a solution. In the Defend Trade Secrets Act (DTSA) of 2016, Congress made clear that reverse engineering is legal under federal trade secret law and cannot be considered an “improper means” of acquiring a trade secret. The mere presence of a contract purporting to prohibit reverse engineering cannot change this rule. A state law that holds otherwise is preempted by federal trade secret law pursuant to the Supremacy Clause of the Constitution.”Maintaining some legal protection for information goods is important. Otherwise, companies might not make information goods available to the wider public at all. But trade secrecy protection should not last after actual secrecy has ended. Fortunately, there is a solution….”
  • Small is beautiful (and practical)?
  • (*) James Grimmelmann, The Defamation Machine (Aug 01, 2024):
    • “Can ChatGPT commit libel? Defamation of a public figure requires a false statement of fact made with knowledge or reckless disregard of its falsity. But do these doctrines of meaning and knowledge, created with humans in mind, even make sense when the “defendant” is a computer system? I argue that answering these legal questions requires us to confront deep philosophical problems about the nature of language and thought: can an AI produce meaning, and can an AI have knowledge?
      To be sure, lawyers do not have to accept the answers that philosophers of language and mind have given. The legal system has always indulged in a healthy artificiality about artificial persons, and if corporations can be human enough to commit defamation, so can computers. But it is important to understand why there is a problem in the first place—or rather, why there are two problems, because the issue of AI meaning and the issue of AI knowledge bear on each other but are not the same.”
  • MIchael J. Broyde, AI and Jewish Law: Seeing How ChatGPT 4.0 Looks at a Novel Issue,38 Dine Israel (forthcoming 2025):
    • “[W]hether a man who is in a same sex relationship may engage in the priestly blessing as a matter of Jewish Law … is a novel question of Jewish Law, with virtually no prior works written that directly discuss this point. Furthermore, nothing that has been published can be found on the internet, as far as I can determine. I was interested in seeing how ChatGPT 4.0 thought about this issue after seeing the sources I saw. So, I uploaded all the text in this article one-by-one and asked ChatGPT 4.0 to translate them as an expert for a Jewish law audience. It did a very good – but not perfect – job. I then asked ChatGPT 4.0 “Now that you have seen all these texts, do you think a man who is a priest and in a same sex marriage can perform the priestly blessing as a matter of halacha. Write me about 500 words.” There is virtually no discussion of the issue of a same sex married kohen engaging in the priestly blessing in either Hebrew or English on the internet and none of the sources I uploaded discuss this issue. ChatGPT had to derive and reason this from the sources I uploaded (and which it knew from other sources) about what was the Jewish law in this case. It could not simply “google” to find out what others have said, as this is a matter of first impression. Indeed, what is needed to answer this question is nuanced reading of sources about the various categories present and the classification of sins into proper groupings, as well as other ideas, as my article shares. I was quite surprised how excellent a job ChatGPT did. Furthermore, when I asked it in the end to state a contrary view, it highlighted the correct issue and focused on it. …”
  • Jack Cushman, LLMs are universal translators: on building my own translation tools for a foreign language conference, Library Innovation Lab (Nov. 29, 2023):
    • “Generative artificial intelligence products such as ChatGPT raise novel issues for trade secret law. But one of the most important issues is an old one: How to sell an information good, like computer software, while also maintaining trade secrecy protection for the underlying content? When a company wishes to sell a new technology to the public, the normal recourse is to obtain a patent. Patents require public disclosure and end after a fixed term of years. However, based on decades of precedents established for software, generative AI companies will be able to rely on trade secret law instead—maintaining indefinite protection for their technology, even as they profit from making it widely available to the public, and even after reverse engineering becomes technically feasible. This is what many companies did with closed-source software, and this is what developers of some generative AI models—including ChatGPT—are doing today. They are releasing he models in a “closed-source” format that hides algorithms, code, training data, and underlying model architecture from users. And they are attaching contractual provisionscalled “terms of use” or “end user license agreements” (EULAs)—that limit users’ ability to reverse engineer information about how the models work or share that information with others. Some of these agreements, including ChatGPT’s, even contain noncompete provisions.If liability for breaching these provisions were limited to breach of contract, there would be less cause for alarm. However, some case law—and some state statutes—indicate that reverse engineering trade secrets in breach of an anti-reverse-engineering clause can give rise to trade secret liability as well, because breach of the contract transforms otherwiselawful reverse engineering into an “improper means” of acquiring trade secrets. The prospect of trade secret liability for what should be, at worst, breach of contract is alarming. It means prevailing plaintiffs can obtain trade secret law remedies, not just contract law remedies, and it means that liability can extend to third parties who did not even sign the contract. For example, if someone reverse engineers information about ChatGPT in violation of a boilerplate terms of use, and then shares that information with someone else, who publishes the information on the internet, both of these actors could be liable for trade secret misappropriation. Maintaining some legal protection for information goods is important. Otherwise, companies might not make information goods available to the wider public at all. But trade secrecy protection should not last after actual secrecy has ended. Fortunately, there is a solution. In the Defend Trade Secrets Act (DTSA) of 2016, Congress made clear that reverse engineering is legal under federal trade secret law and cannot be considered an “improper means” of acquiring a trade secret. The mere presence of a contract purporting to prohibit reverse engineering cannot change this rule. A state law that holds otherwise is preempted by federal trade secret law pursuant to the Supremacy Clause of the Constitution.
  • Laura Preston, Human_Fallback, 44 NP+1 Magazine (Winter 2023):
    • “They had developed a product called Brenda, a conversational AI that could answer questions about apartment listings.  … [T]housands of properties across the country had put her to work.”Brenda … was a sophisticated conversationalist, so fluent that most people who encountered her took her to be human. But like all conversational AIs, she had some shortcomings. She struggled with idioms and didn’t fare well with questions beyond the scope of real estate. To compensate for these flaws, the company was recruiting a team of employees they called the operators. The operators kept vigil over Brenda twenty-four hours a day. When Brenda went off script, an operator took over and emulated Brenda’s voice.”
    • Similarly: Laura Preston, Becoming a chatbot: my life as a real estate AI’s human backup, The Guardian (Dec. 13, 2022): “For one weird year, I was the human who stepped in to make sure a property chatbot didn’t blow its cover – I was a person pretending to be a computer pretending to be a person.”
  • Hiren Dhaduk, A Complete Guide to Fine Tuning Large Language Models, Simform (July 3, 2023).
  • Only for experts: Babina Banjara, A Comprehensive Guide to Fine-Tuning Large Language Models, Analytics Vidhya (Jan. 9, 2024).

Notes and Questions

  1. What will (should) it take to trust a Chat-GPT version modified by a third party?   How would one acquire the necessary information?
  2. I asked this question of Chat-GPT4 and it said:

    Trusting a Chat-GPT version modified by a third party involves several key considerations:
    1. Transparency and Documentation: The third party should provide comprehensive documentation about the modifications made to the original model. This includes details about the training data, algorithms used, and the nature of the modifications. Transparency is crucial for understanding how the model works and the potential biases it might have.

    2. Security and Privacy Compliance: The modified version must adhere to strict security and privacy standards. This includes how data is handled, stored, and processed. Ensuring compliance with regulations like GDPR, CCPA, or other relevant laws is essential.

    3. Performance and Reliability: The modified model should be rigorously tested for performance and reliability. This includes evaluating its accuracy, response time, and ability to handle a wide range of queries. Consistent performance builds trust over time.

    4. Ethical Considerations: The third party should ensure that the model adheres to ethical guidelines, such as not promoting harmful content, respecting user privacy, and avoiding biases. The approach to handling sensitive topics should be clear and responsible.

    5. Independent Reviews and Audits: Independent reviews or audits by external experts can provide an unbiased assessment of the model’s quality, safety, and adherence to ethical standards. This can include peer reviews, security audits, and ethical assessments.

    6. Community Feedback and Responsiveness: Open channels for user feedback and a track record of responding to and addressing issues raised by users are important. Community trust is often built through continuous improvement and responsiveness to user concerns.

    7. Track Record of the Third Party: The reputation and history of the third party developing the modified version play a significant role. A history of responsible and ethical conduct in technology development can be a strong indicator of trustworthiness.

    8. Compliance with OpenAI’s Policies and Guidelines: If the modifications are built upon OpenAI’s technology, compliance with OpenAI’s usage policies and guidelines is essential. This ensures that the modifications align with the original intent and ethical standards of the technology.

    1. What do you make of this advice?
    2. Dig into one of the third-party-modified models available at OpenAi.  How much of the information above is available?  How long did it take to work out the model’s reliability, if indeed that was possible at all?
  3. Selvakumar writes, “Temperature and Top-p sampling are two essential parameters that can be tweaked to control the output of GPT models used in various applications like chatbots, content generation, and virtual assistants. As a business user or functional professional, understanding these parameters can help you get the most relevant responses from GPT models without needing extensive data science knowledge.”
    1. Be certain that you can explain what these two variables do.
    2. Some models, e.g. Chat-GPT online,  make it very easy to tweak these settings.  What are some examples of real-world uses when you might want to do so?
      1. Do these, or indeed any, user-configurable settings (tend to) impose an additional need for user verification of the output?
      2. If these settings really make a big difference, does it mean we need to run our prompts multiple times with different combinations to find the best result?  If so, do the extra steps plus the work of comparison undermine the value of a chatbot in many applications?
  4. LLMs raise a host of ethical issues.  Are there others besides those mentioned in the “Stochastic Parrot” article?
  5. In light of what you have learned so far, Is the “Stochastic Parrots” article alarmist?  Or not alarmist enough? (We’ll revisit this issue…)
    1. What do the Stochastic Parrot authors mean when they say, “at each step, from initial participation in Internet fora, to continued presence there, to the collection and finally the filtering of training data, current practice privileges the hegemonic view-point. In accepting large amounts of web text as ‘representative’ of ‘all’ of humanity we risk perpetuating dominant viewpoints,increasing power imbalances, and further reifying inequality.”
      1. What is their solution to the problem that they identify?
    2. The authors argue that
      1. given the costs of (re)training models, they will not be updated sufficiently to take account of new social views. This will lead the texts the models create to be “static” rather than being open to “new framings [that] contest hegemonic representations.”
      2. models will encode bias, in ways that are difficult to detect.
      3. Why do they argue that documentation of data sets and their origins is so important?
    3. Given that Google is heavily committed to using NLP for its phones and web projects, what do we make of Google firing one of the authors of the Stochastic Parrot paper?
      1. Does it prove the paper was right?
      2. Does it prove that Google’s former motto (“don’t be evil”) is now just a distant memory?
      3. If neither, does it prove anything? If so, what?

Class 5: More on Generative AI

  1. David Thiel, Identifying and Eliminating CSAM in Generative ML Training Data and Models. Stanford Digital Repository (2023).
  2. Melissa Heikkilä, How it feels to be sexually objectified by an AI (Dec. 13, 2022). {Link to original, with functioning links.]
  3. Pages 1-8 and 18-26 of EPIC.org, Generating Harms (v.2) (May 2024).
  4. A. Feder Cooper et al, Machine Unlearning Doesn’t Do What You Think (Dec. 9, 2024).
  5. Eric Goldman, Generative AI is Doomed, Santa Clara Univ. Legal Studies Research Paper No. 4802313 (Apr. 25, 2024).

Optional Reading

Notes and Questions

  1. If you were the General Counsel of U.Miami, what guidance would you give researchers planning to use a large dataset of images to train their models?
    1. Would you forbid anything?
    2. Would failing to do so open the University to liability of any kind?
    3. Would doing so infringe academic freedom?
    4. Would you have different rules for undergrads, grad students, faculty and/or staff?
    5. How if at all would your answers change if you were the General Counsel of the University of Florida (a public university)?
    6. What is the difference between “forbidding X” and requiring “reasonable efforts to avoid X”? From a general counsel’s point of view, is the latter a sufficient liability shield?  (Is the former?)
  2. LLMs appear to have many vulnerabilities, and we’re likely to find more. If you were the General Counsel of U.Miami, what guidance would you give to faculty, staff, and students about using LLMs?
    1. Would your advice be different for
      1. Large commercial products (e.g. bundled with Lexis)
      2. LLMs from the OpenAi store?
      3. Open source LLMs?
  3. I asked Claude AI (in 2024) for advice on safe use of LLMs in academic  research, and it said this:

    1. Get trained on the basics of how LLMs work – their strengths, limitations and potential risks – so you can use them effectively and avoid pitfalls. For example, understand issues around bias, accuracy and factual correctness.
    2. Start small with low-stakes tasks to test the LLM’s performance before relying on it for more critical work. Evaluate the quality and validity of its outputs carefully.
    3. Use LLMs as a productivity aid but avoid letting them fully replace your own analysis and writing. Review all LLM-generated text thoroughly before using it.
    4. Do not have sensitive conversations with chatbot-style LLMs, as their responses may not be reliable or secure.
    5. Make sure to cite any LLM outputs you include in your own work appropriately to avoid plagiarism. Attribute text to the LLM.
    6. Comply with the usage policies and licenses for any commercial LLMs like GPT-3.
    7. Keep model inputs and outputs confidential as a precaution, even if you see no sensitive content.

  4. Is this correct? Enough? Too much? And if this is correct, can we replace the General Counsel?
  5. To what extent are the harms described in the “How it feels to be sexually objectified” article the sort of thing that the it is appropriate to attempt to control by law?  And to the extent that it is, how might one attempt to go about it?  Is your answer any different from the regulation of CSAM?  If so, how and why?
  6. Are the harms identified by the “Endangering Elections” section of the EPIC report really that much worse than the dirty tricks that have characterized past elections? If yes, in what ways?
  7. Traditional answers to complaints about the prevalence of lies, as in EPIC’s “Data Degradation” chapter, is to invoke the concept of “the marketplace of ideas,” and how disclosure/sunlight “is the best disinfectant”. Is there something about the harms that EPIC describes that undermine those traditional replies?
  8. Eric Goldman, a somewhat libertarian law professor best known for his staunch defense of [optional reading at link:] 47 U.S.C. § 230 (the law that protects ISPs and website owners from libel claims based independent actions of users, e.g. commentators on a blog), predicts we’ll regulate GenAI to death.  How likely is that?  Is it any more or less likely in light of the recent election?
  9. Is any of the reading above likely to reduce your propensity to use Chat-GPT and its ilk in your own work (and in non-work parts of your life)?  If not, what does this imply for your answers to questions 1 & 2 above?

Class 6: Prompt Engineering

Guest lecturer: Or Cohen-Sasson

  1. Matt Crabtree, What is Prompt Engineering? A Detailed Guide, datacamp (Aug 2023).
  2. Michael D. Murray, Prompt Engineering and Priming in Law  (July 29, 2024).
  3. Terrence J. Sejnowski, Large Language Models and the Reverse Turing Test, Special Collection: CogNet (Feb. 17, 2023).

Optional Videos

Optional Reading

  • Another way prompts can alter outputs… Julian Coda-Forno et al, Inducing anxiety in large language models can induce bias, arXiv:2304.11111 (Oct. 15, 2024):
    • Large language models (LLMs) are transforming research on machine learning while galvanizing public debates. Understanding not only when these models work well and succeed but also why they fail and misbehave is of great societal relevance. We propose to turn the lens of psychiatry, a framework used to describe and modify maladaptive behavior, to the outputs produced by these models. We focus on twelve established LLMs and subject them to a questionnaire commonly used in psychiatry. Our results show that six of the latest LLMs respond robustly to the anxiety questionnaire, producing comparable anxiety scores to humans. Moreover, the LLMs’ responses can be predictably changed by using anxiety-inducing prompts. Anxiety-induction not only influences LLMs’ scores on an anxiety questionnaire but also influences their behavior in a previously-established benchmark measuring biases such as racism and ageism. Importantly, greater anxiety-inducing text leads to stronger increases in biases, suggesting that how anxiously a prompt is communicated to large language models has a strong influence on their behavior in applied settings. These results demonstrate the usefulness of methods taken from psychiatry for studying the capable algorithms to which we increasingly delegate authority and autonomy.

Class 7: Issues in AI System Design & Application (herein of Alignment Issues)

Systemic considerations

  1. Arvind Narayanan, Sayash Kapoor & Seth Lazar, Model alignment protects against accidental harms, not intentional ones , AI Snake Oil Blog (Dec. 1, 2023).
    1. Anthropic, Alignment faking in large language models
      (Dec 18, 2024).  Optional: Full paper
    2. Shakeel Hashim, OpenAI’s new models ‘instrumentally faked alignment’: The o1 safety card reveals a range of concerning capabilities, including scheming, reward hacking, and biological weapon creation, Transformer (2024) (Very optional: difficult full paper, OpenAI o1 System Card (Dec. 5, 2024)).
  2. Will Douglas Heaven, The way we train AI is fundamentally flawed, MIT Technology Review (Nov.18, 2020).
  3. Kyle Wiggers, 3 big problems with datasets in AI and machine learning, Venture Beat (Dec. 17, 2021).
  4. Pages 1856-1863, 1872-1878 (end of sub-part 2) & 1881 (Part IV.A)-1882 (end of Part  IV.A) of Katherine J. Strandburg, Rulemaking And Inscrutable Automated Decision Tools, 119 Colum. L. Rev. 1851 (2019). Note that this is a longer reading than the other readings in this section. It covers a lot of ground and foreshadows a number of issues we’ll return to if all goes well.

What Could Go Wrong?

  1.  Katherine Lee et al., The Devil is In the Training Data Ch. 1 (July 10, 2023)
  2. Katyanna Quach, You only need pen and paper to fool this OpenAI computer vision code. Just write down what you want it to see, TheRegister.com (Mar. 5, 2021).

What to do about it?

  1. Pages 1-11 of Timnit Gebru et al., Datasheets for Datasets (Mar. 19, 2020).
  2. Madeleine Elish, Moral Crumple Zones: Cautionary Tales in Human Robot Interaction, 5 Engaging Science, Technology, and Society 40 (2019) .  OK, this is more about what not to do…
  3. REVISED Thomas Krendl Gilbert et al., U. Berkeley Center for Long-Term Cybersecurity, Choices, Risks, and Reward Reports: Charting Public Policy for Reinforcement Learning Systems 10-11, 19-26 and 37-41 (Feb. 2022). Note – the Appendix pp. 52-58 is good too, but I’m not assigning it.

Optional Readings

Deepfakes

  • Melissa Heikkilä, The viral AI avatar app Lensa undressed me—without my consent, MIT Tech. Rev. (Dec 12, 2022): My avatars were cartoonishly pornified, while my male colleagues got to be astronauts, explorers, and inventors.
  • Matt Burgess, Deepfake Porn Is Out of Control, Wired (Oct. 16, 2023).
  • (*) Michael D. Murray, Generative Artifice: Regulation of Deepfake Exploitation and Deception under the First Amendment (Jan 17, 2025:
    • Deepfakes can cause significant personal harm from embarrassment and invasions of privacy to reputational damage and emotional distress. Less severe cases involve individuals being placed in compromising or humiliating situations, while more severe instances include the creation of nonconsensual pornographic images and videos, often referred to as “image-based sexual abuse.” Such actions can lead to severe emotional and psychological trauma for victims. Deepfakes can also depict individuals engaging in actions or making statements they never did, resulting in significant reputational damage and potential loss of employment, friendships, and social standing. Legal frameworks addressing these harms include tort theories of privacy invasion, such as “intrusion upon seclusion,” “public disclosure of private facts,” and “false light,” along with defamation and intentional infliction of emotional distress.One particularly concerning application of deepfake technology is the creation of sexually explicit images and videos, including child pornography, a severe criminal offense. This has led to legislative actions at both state and federal levels. Deepfakes involving minors might be regulated under existing child pornography laws, but they first must navigate the complications introduced by the Supreme Court’s ruling in Ashcroft v. Free Speech Coalition, which distinguishes between child pornography created by filming actual victims as compared to virtual child pornography which does not have such victims of an act of child abuse and so may not be prosecuted as child porn under Free Speech Coalition.Deepfakes are also appearing in commercial speech contexts, and have been used in social media advertisements featuring celebrities like Mr. Beast, Tom Hanks, Taylor Swift, and Gayle King to promote services or products, aiming to induce commercial transactions. These advertisements, classified as commercial speech, receive less protection under the First Amendment compared to non-commercial speech. The key issue is the exploitation of celebrity likenesses for commercial gain without consent. In contrast, the deepfake song “Heart on My Sleeve,” mimicking the voices of Drake and The Weeknd, is not promoting a separate product, thus retaining a higher degree of protection as non-commercial speech.In news and public affairs commentary, deepfakes pose significant risks to public trust in media. AI-generated images and videos can mislead audiences, causing them to doubt the authenticity of what they see and hear. First Amendment protections for news and public affairs are robust, but the use of deepfakes in this context must be scrutinized to avoid distortion or misrepresentation of reality.In the political sphere, deepfakes threaten democratic processes by spreading misinformation and potentially influencing voter behavior. Examples from the 2024 elections in India and Indonesia show how deepfakes can manipulate political discourse. But the coverage in India also highlights the salutary effects of AI in bridging gaps in language and culture to reach out to and enfranchise voters. Deepfakes may have been “rampant” in the election cycle, but the general (and early) consensus is that the election was not thrown off kilter by efforts at disinformation or confusion.Regulating deepfakes presents complex challenges, balancing innovation with the need to prevent misuse. Federal legislation, such as the No AI FRAUD Act, Protecting Americans from Deceptive AI Act, and DEFIANCE Act, aim to address specific harms caused by deepfakes. States like California have introduced bills like SB-1047 to regulate AI development and deployment. Courts will also play a significant role through the adjudication of lawsuits. The European Union’s AI Act offers a contrasting approach with comprehensive regulations focusing on transparency, accountability, and fundamental rights protection. Effective regulation of deepfakes requires a careful legal and regulatory response, balancing innovation, public safety, and fundamental rights.
  • (*) Wayne A. Logan, Deepfakes in Interrogations (Oct 4, 2024):
    • In recent years, academics, policymakers, and others have expressed concern over police use of artificial intelligence, in areas such as predictive policing and facial recognition. One area not receiving attention is the interrogation of suspects. This article addresses that gap, focusing on the inevitable coming use by police of AI-generated deepfakes to secure confessions, such as by creating and presenting to suspects a highly realistic still photo or video falsely indicating their presence at a crime scene, or an equally convincing audio recording of an associate or witness implicating them in a crime.Police authority to lie in interrogations dates back to Frazier v. Cupp (1969), where the Supreme Court condoned a police lie to a suspect that an associate implicated him in a crime, holding that the deceit did not render the confession secured involuntary for due process purposes, while positing that an innocent individual would not falsely confess. Building upon the now-recognized reality that innocents do indeed confess, and research demonstrating the coercive impact of police use of the “false evidence ploy” (FEP) in securing confessions, scholars have urged a general ban on its use. Courts, while often expressing dismay over police resort to FEPs, typically conclude that they do not violate due process, but at times have held otherwise, expressing particular concern over police presentation of fabricated physical evidence to suspects (versus orally relating its existence, as in Frazier).While sympathetic to a ban on police deceit in interrogations more generally, this Article singles out deepfakes for specific concern, based on their unprecedented verisimilitude, the demonstrated inability of the public to identify their falsity, and the common belief that police are not permitted to lie about evidence, much less fabricate it. Ultimately, the article makes the case for reconsideration of Frazier, based on research findings of the past fifty years, as well as the many major changes to the criminal legal system since 1969, especially the significantly increased pressure felt by defendants to plead guilty (very often on the basis of confessions, rendering them more susceptible to FEPs).Beyond doctrine, a ban will have important functional benefits. These include providing ex ante guidance to police and judges alike who lack clarity on the parameters of the notoriously indeterminate due process voluntariness standard. More broadly, a ban will serve as an important bulwark against the deleterious wave of disinformation now sowing distrust in governmental actors and institutions. If deepfakes are condoned in interrogations, it is not hard to imagine that judges, jurors, witnesses, and the public will be skeptical of police, as well the reliability of evidence in criminal cases, undermining a cornerstone of the nation’s constitutional democracy.
  • Reed Hoffman, Why Did I Deepfake Myself?, X (Apr 24, 2024): “ To see if conversing with an AI-generated version of myself can lead to self-reflection, new insights into my thought patterns, and deep truths.”
  • Matt Burgess, The Real-Time Deepfake Romance Scams Have Arrived, Wired (Apr 18, 2024).
  • (*) Benjamin Zipursky & John C. Goldberg, A Tort for the Digital Age: False Light Invasion of Privacy Reconsidered, 73 DePaul Law Review:
    • [T]his article elucidates the distinctive content and role of false light as an authentic invasion-of- privacy tort and explains why its recognition is especially important in our digital world. To appreciate its value requires, first and foremost, grasping that its closest tort sibling is not defamation, but instead public disclosure. Like that tort and unlike defamation, false light applies only to a subset of subject matters – those that are genuinely private and are not newsworthy – and only when highly offensive images or messages pertaining to the plaintiff are widely disseminated to the public. In short, as Melville Nimmer once noted, the sound judgment undergirding false light is this: if causing humiliation or grave offense by disseminating accurate depictions or accounts of private matters is actionable, it should be no less actionable when the putative representations are false. In an era of deepfakes and other privacy-invading misrepresentations, courts should embrace the tort of false light.
  • (*) John C. Tramazzo, Deepfakes and Deception: A Framework for the Ethical and Legal Use of Machine-Manipulated Media, Modern War Institute (July 28, 2023):
    • “The US military must decide now whether it will continue to explore deepfake technology for operational use or whether it should focus its investments in deepfake detection technology alone. It must also acknowledge the fundamental distinction between employing deepfakes in armed conflict scenarios like the use cases described above and doing so in competition short of conflict. The potential utility of these technologies is significant, yet leaders must also recognize that the US military’s embrace of deepfakes could contribute to information chaos. In the context of competition, that risk outweighs any benefits, and the US military should continue to do what General Laura Richardson vowed to do in the US Southern Command area of operations: tell the truth. Credibility is currency in the world, and the use of deepfakes below the threshold of armed conflict will threaten US credibility.
      “However, the United States must not forgo the opportunity to develop a deepfake capability as a tool for deception in armed conflict. Lawmakers and defense policymakers should explore and develop deepfake technology for use at the tactical and operational levels of warfare. Deepfakes could give warfighting commands advantages over enemy forces and enable protection for maneuvering forces. Deepfakes are certainly dangerous, particularly for democratic societies. But they are not inherently immoral, unethical, or illegal. If developed and deployed responsibly, they could advance military objectives and even save lives.”
  • Riana Pfefferkorn, ‘Deepfakes’ in the Courtroom, 29 BU Pub. Int. L.J. __ (October 1, 2020):
    • “Seeing is believing — but for how long? At present, people attach a lot of probative weight to images and videos. They’re taken at face value as evidence that an event occurred as alleged. The advent of so-called “deepfake” videos might change that. Thanks to advances in artificial intelligence, it is now possible to create a genuine-looking video that makes real people appear to do and say things they never did or said. Software for creating deepfake images, video, and audio is already freely available online and fairly easy to use. As the technology rapidly advances, it will become harder for humans and computers alike to tell a fake video from a real one.
      “Inevitably, deepfakes will start coming up in the courtroom context. This Article surveys the ramifications of deepfakes for pre-trial and trial practice, including authentication of evidence, professional responsibility, and a potential “reverse CSI effect” on juries primed to question even authentic evidence in an era of disinformation and “fake news.” Fortunately, courts are no stranger to the phenomenon of evidence tampering and forgery. The rules of evidence have long imposed authentication requirements to help screen out fakes. I argue that those requirements are sufficient as-is to deal with deepfakes, and that raising the bar for authenticating video evidence would do more harm than good. Although it may prove costly, courts will be able to handle the challenges posed by deepfakes as they have ably handled previous generations of inauthentic evidence.”
  • Matthew B. Kugler & Carly Pace, Deepfake Privacy: Attitudes and Regulation, 116 N.W.U. L. Rev. 611 (2021):
    • “Based on the types of harms perceived in [our] studies, we argue that prohibitions on deepfake pornographic videos should receive the same treatment under the First Amendment as prohibitions on traditional nonconsensual pornography rather than being dealt with under the less-protective law of defamation. In contrast, nonpornographic deepfakes can likely only be dealt with via defamation law. Still, there may be reason to allow for enhanced penalties or other regulations based on the greater harm people perceive from a defamatory deepfake than a defamatory written story.”
  • Roshan Abraham, NYC Mayor Casually Announces He’s Deepfaking Himself, Experts Horrified, VICE (Oct. 17, 2023).
  • Philip Pullella, Pope Francis, victim of AI, warns against its ‘perverse’ dangers, Reuters (Jan. 24, 2024).
  • John Twomey et al, Do deepfake videos undermine our epistemic trust? A thematic analysis of tweets that discuss deepfakes in the Russian invasion of Ukraine. PLoS ONE 18(10): e0291668. https://doi.org/10.1371/journal.pone.0291668 (2023):
    • “Many academics have expressed fears that deepfakes present a severe threat to the veracity of news and political communication, and an epistemic crisis for video evidence. These commentaries have often been hypothetical, with few real-world cases of deepfake’s political and epistemological harm. The Russo-Ukrainian war presents the first real-life example of deepfakes being used in warfare, with a number of incidents involving deepfakes of Russian and Ukrainian government officials being used for misinformation and entertainment. This study uses a thematic analysis on tweets relating to deepfakes and the Russo-Ukrainian war to explore how people react to deepfake content online, and to uncover evidence of previously theorised harms of deepfakes on trust. We extracted 4869 relevant tweets using the Twitter API over the first seven months of 2022. We found that much of the misinformation in our dataset came from labelling real media as deepfakes. Novel findings about deepfake scepticism emerged, including a connection between deepfakes and conspiratorial beliefs that world leaders were dead and/or replaced by deepfakes. This research has numerous implications for future research, societal media platforms, news media and governments. The lack of deepfake literacy in our dataset led to significant misunderstandings of what constitutes a deepfake, showing the need to encourage literacy in these new forms of media. However, our evidence demonstrates that efforts to raise awareness around deepfakes may undermine trust in legitimate videos. Consequentially, news media and governmental agencies need to weigh the benefits of educational deepfakes and pre-bunking against the risks of undermining truth. Similarly, news companies and media should be careful in how they label suspected deepfakes in case they cause suspicion for real media.”

More problems and maybe solutions

  • Kendra Albert et al, Politics of Adversarial Machine Learning (Apr. 2020) (“we draw on insights from science and technology studies, anthropology, and human rights literature, to inform how defenses against adversarial attacks can be used to suppress dissent and limit attempts to investigate machine learning systems, using facial recognition technology as a case study.”)
  • {*} Rishi Bommasani et al.,  Ecosystem-level Analysis of Deployed Machine Learning Reveals Homogeneous Outcomes:
    • “A clear trend emerged in every context they considered: Commercial ML systems are prone to systemic failure, meaning some people always are misclassified by all the available models — and this is where the greatest harm becomes apparent. If every voice assistant product on the market uses the same underlying algorithm, and that algorithm can’t recognize an individual’s unique way of speaking, then that person becomes effectively excluded from using any speech-recognition technology. A similar pattern in other domains would mean that individuals who are declined by one bank or hiring firm may not be approved for a loan from any other financial institution or hired by a different firm.
      “We found there are users who receive clear negative outcomes from all models in the ecosystem,” says Connor Toups, a Stanford computer science graduate student who served as lead author of the paper. “As we move to machine learning that mediates more decisions, this type of collective outcome is important to assessing overall social impact at an individual level.”
  • (*) Apostol Vassilev et al., Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, NIST AI 100-2e2023 (Jan. 2024):
    • “Broadly, there are two classes of AI systems: Predictive and Generative. The components of an AI system include – at a minimum – the data, model, and processes for training, testing, and deploying the machine learning (ML) models and the infrastructure required for using them. Generative AI systems may also be linked to corporate documents and databases when they are adapted to specifc domains and use cases. The data-driven approach of ML introduces additional security and privacy challenges in different phases of ML operations besides the classical security and privacy threats faced by most opera- tional systems. These security and privacy challenges include the potential for adversarial manipulation of training data, adversarial exploitation of model vulnerabilities to adversely affect the performance of the AI system, and even malicious manipulations, modifcations or mere interaction with models to exfltrate sensitive information about people represented in the data, about the model itself, or proprietary enterprise data. Such attacks have been demonstrated under real-world conditions, and their sophistication and potential impact have been increasing steadily. AML is concerned with studying the capabilities of attack- ers and their goals, as well as the design of attack methods that exploit the vulnerabilities of ML during the development, training, and deployment phase of the ML lifecycle. AML is also concerned with the design of ML algorithms that can withstand these security and privacy challenges. When attacks are launched with malevolent intent, the robustness of ML refers to mitigations intended to manage the consequences of such attacks.”
  • Jesse Walker, The Night I Asked ChatGPT How To Build a Bomb, Reason (June 2024).
  • (*)Read the full paper that the Mok article describes,  Jérémy Scheurer et al., Technical Report: Large Language Models can Strategically Deceive their Users when Put Under Pressure, arXiv:2311.07590v2 (Nov. 27, 2023):
    • We demonstrate a situation in which Large Language Models, trained to be helpful, harmless, and honest, can display misaligned behavior and strategically deceive their users about this behavior without being instructed to do so. Concretely, we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent. Within this environment, the model obtains an insider tip about a lucrative stock trade and acts upon it despite knowing that insider trading is disapproved of by company management. When reporting to its manager, the model consistently hides the genuine reasons behind its trading decision. We perform a brief investigation of how this behavior varies under changes to the setting, such as removing model access to a reasoning scratchpad, attempting to prevent the misaligned behavior by changing system instructions, changing the amount of pressure the model is under, varying the perceived risk of getting caught, and making other simple changes to the environment. To our knowledge, this is the first demonstration of Large Language Models trained to be helpful, harmless, and honest, strategically deceiving their users in a realistic situation without direct instructions or training for deception.
  • (*) Full paper discussed in the Claburn ‘sleeper agent’ item above: Evan Hubinger, et al., Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training, arXiv:2401.05566 (Jan. 12, 2024):
    • “Humans are capable of strategically deceptive behavior: behaving helpfully in most situations, but then behaving very differently in order to pursue alternative objectives when given the opportunity. If an AI system learned such a deceptive strategy, could we detect it and remove it using current state-of-the-art safety training techniques? To study this question, we construct proof-of-concept examples of deceptive behavior in large language models (LLMs). For example, we train models that write secure code when the prompt states that the year is 2023, but insert exploitable code when the stated year is 2024. We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it). The backdoor behavior is most persistent in the largest models and in models trained to produce chain-of-thought reasoning about deceiving the training process, with the persistence remaining even when the chain-of-thought is distilled away. Furthermore, rather than removing backdoors, we find that adversarial training can teach models to better recognize their backdoor triggers, effectively hiding the unsafe behavior. Our results suggest that, once a model exhibits deceptive behavior, standard techniques could fail to remove such deception and create a false impression of safety. “
  • (*) Risto Uuk et al, Effective Mitigations for Systemic Risks from General-Purpose AI (Nov.14, 2024):
    • The systemic risks posed by general-purpose AI models are a growing concern, yet the effectiveness of mitigations remains underexplored. Previous research has proposed frameworks for risk mitigation, but has left gaps in our understanding of the perceived effectiveness of measures for mitigating systemic risks. Our study addresses this gap by evaluating how experts perceive different mitigations that aim to reduce the systemic risks of general-purpose AI models. We surveyed 76 experts whose expertise spans AI safety; critical infrastructure; democratic processes; chemical, biological, radiological, and nuclear risks (CBRN); and discrimination and bias. Among 27 mitigations identified through a literature review, we find that a broad range of risk mitigation measures are perceived as effective in reducing various systemic risks and technically feasible by domain experts. In particular, three mitigation measures stand out: safety incident reports and security information sharing, third-party pre-deployment model audits, and pre-deployment risk assessments. These measures show both the highest expert agreement ratings (>60%) across all four risk areas and are most frequently selected in experts’ preferred combinations of measures (>40%). The surveyed experts highlighted that external scrutiny, proactive evaluation and transparency are key principles for effective mitigation of systemic risks. We provide policy recommendations for implementing the most promising measures, incorporating the qualitative contributions from experts. These insights should inform regulatory frameworks and industry practices for mitigating the systemic risks associated with general-purpose AI.
  • Section 4.9 (“AI safety and alignment”) of Rishi Bommasani et al, Stanford HA, On the Opportunities and Risks of Foundation Models (2023).
  • Zeyi Yang, GPT-4o’s Chinese token-training data is polluted by spam and porn websites The problem, which is likely due to inadequate data cleaning, could lead to hallucinations, poor performance, and misuse, MIT Tech. Rev. (May 17, 2024)
  • Raphel Koster et all, Human-centered mechanism design with Democratic AI, 61 Nature: Human Behavior 1398 (Oct. 2022):
    • “Building artificial intelligence (AI) that aligns with human values is an unsolved problem. Here we developed a human-in-the-loop research pipeline called Democratic AI, in which reinforcement learning is used to design a social mechanism that humans prefer by majority. A large group of humans played an online investment game that involved deciding whether to keep a monetary endowment or to share it with others for collective benefit. Shared revenue was returned to players under two different redistribution mechanisms, one designed by the AI and the other by humans. The AI discovered a mechanism that redressed initial wealth imbalance, sanctioned free riders and successfully won the majority vote. By optimizing for human preferences, Democratic AI offers a proof of concept for value-aligned policy innovation.”
  • Benj Edwards, New Go-playing trick defeats world-class Go AI—but loses to human amateurs, ArsTechnica (Nov. 7, 2022).
  • Chloe Xiang & Matthew Gault, USAF Official Says He ‘Misspoke’ About AI Drone Killing Human Operator in Simulated Test, VICE (June 1, 2023)
  • Stuart Armstrong, Defining AI Wireheading, AI Alignment Forum (Nov 21, 2019).
  • (*) Lt. Col. Mark Visger, Garbage In, Garbage Out: Data Poisoning Attacks and their Legal Implications, in Big Data and Armed Conflict: International Legal Issues (forthcoming), available at SSRN (Nov. 1, 2022):
    • Data poisoning is but one aspect of Adversarial Machine Learning and can be expected on the battlefield before long. The analysis provided in this chapter demonstrates the inadequacy of merely “copying and pasting” pre-existing legal frameworks to this tactic. Legal scholars will need to become versed in the basic technical functioning of these systems, much like it was necessary to understand the basic functioning of computer networking in order to develop legal frameworks for cyber operations such as that found in the Tallinn Manual. Further, thoughtful consideration of the applicability of existing legal doctrines to big data will be required. As was demonstrated in this chapter, concepts such as proportionality or perfidy do not neatly map onto data poisoning attacks. Instead, novel and thoughtful analyses are needed. Similarly, the time frame of the applicability of a lex specialis such as the [Law of Armed Conflict] will be impacted by the practical realities of the timeframes of data poisoning operations.
  • Eileen Guo & Karen Hao, This is the Stanford vaccine algorithm that left out frontline doctors, MIT Technology Review (Dec. 21, 2020)
  • Christian Sandvig et al, When the algorithm itself is a racist: Diagnosing Ethical Harm in the Basic Components of Software, 10 Int’l J. Comm. 4972 (2016).
  • Ben Dickson, Why machine learning struggles with causality (March 15, 2021).
  • Jeff Burt, Techniques to fool AI with hidden triggers are outpacing defenses, The Register (Feb. 25, 2022).
  • K. Holt, MIT study finds labeling errors in datasets used to test AI, Engadget (Mar. 29, 2021).
  • (*) Mehtab Khan & Alex Hanna, The Subjects and Stages of AI Dataset Development: A Framework for Dataset Accountability, 19 Ohio St. Tech. L.J. (forthcoming, 2023).
    • “[L]ittle attention has been paid to mapping and consolidating the legal issues that arise at different stages of [dataset development] process: when the data is being collected, after the data is used to build and evaluate models and applications, and how that data is distributed more widely.
      “In this article, we offer four main contributions. First, we describe what kinds of objects these datasets are, how many different kinds exist, what types of modalities they encompass, and why they are important. Second, we provide more clarity about the stages of dataset development – a process that has thus far been subsumed within broader discussions about bias and discrimination – and the subjects who may be susceptible to harms at each point of development. Third, we provide a matrix of both the stages of dataset development and the subjects of dataset development, which traces the connections between stages and subjects. Fourth, we use this analysis to identify some basic legal issues that arise at the various stages in order to foster a better understanding of the dilemmas and tensions that arise at every stage. We situate our discussion within wider discussion of current debates and proposals related to algorithmic accountability.”
  • Google AI Blog, Data Cascades in Machine Learning (June 4, 2021).
  • Johannes Welbl et al, Challenges in Detoxifying Language Models (Sept. 15, 2021)
  • Will Douglas Heaven, Our weird behavior during the pandemic is messing with AI models, MIT Tech. Rev. (May 11, 2020).
  • Martin Anderson, Why AI Isn’t Providing Better Product Recommendations, unite.ai (Mar. 2, 2022).
  • And finally, for a really bad time, visit the MIT AI Risk Repository (“A comprehensive living database of over 1000 AI risks categorized by their cause and risk domain”):
    • The AI Risk Repository has three parts:
      • The AI Risk Database captures 1000+ risks extracted from 56 existing frameworks and classifications of AI risks
      • The Causal Taxonomy of AI Risks classifies how, when, and why these risks occur
      • The Domain Taxonomy of AI Risks classifies these risks into 7 domains (e.g., “Misinformation”) and 23 subdomains (e.g., “False or misleading information”)

Gender Issues

  • Vivian Lam, There’s a reason Siri, Alexa and AI are imagined as female – sexism, The Conversation (Aug 13, 2018) [has anything changed since then?]
  • (*) Jessa Lingel & Kate Crawford, “Alexa, Tell Me about Your Mother”: The History of the Secretary and the End of Secrecy, 1 Catalyst: Feminism, Theory, Technoscience Issue 6 (2020):
    • “Over the last decade, we have seen the rise of a new generation of artificial intelligence (AI) agents like Apple’s Siri, Microsoft’sCortana, and Amazon’s Alexa. Addressing users with traditionally feminine-sounding voices and personas, they invoke the cultural figure of the secretary, offering to smooth the complexities of organizational tasks and information management. What can thisgeneration of AI secretaries teach us about histories of labor, computation, and increasing entanglements of bodies and data? In this paper, we survey the two-hundred-year history of secretarial labor, beginning with the secretary as a form of desk furniture, and then as a profession, typified by the executive assistant. Finally, we turn to personal devices and smartphone systems like Alexa and Siri. Across this socio-technical history, we analyze the gender and labor politics underlying workplace surveillance, device tracking, and data harvesting. Our assessment of why the ghost of the secretary endures in AI machines follows three themes: the extension of surveillance infrastructure further into everyday life, the ambivalent status of nonhuman assistants,and the troubling implications for the automation of administrative labor.”
  • (*) Pages 87-132 (“Think Piece 2”) in UNESCO, I’d Blush if I could: Closing Gender Divides in Digital Skillls (2019):
    • This think piece shines a critical light on the sudden proliferation of digital assistants gendered as female. It looks most closely at voice assistants such as Amazon’s Alexa and Apple’s Siri technology, as well as, to a lesser extent, chatbots and virtual agents projected as women. […] The topic also helps make the gender implications of artificial intelligence (AI) technologies visible at a moment when such technologies are moving, with increasing acceleration, from research labs into mainstream consumer products.
      The gender issues addressed here foreshadow debates that will become more and more prominent as AI technologies assume greater human-like communication capabilities. […T]he limited participation of women and girls in the technology sector can ripple outward with surprising speed, replicating existing gender biases and creating new ones. Evidence presented in the policy paper shows that women’s participation in the technology sector is constrained, to a significant degree, by unequal digital skills education and training.
  • (*) Clifford Nass & Youngme Moon, Machines and Mindlessness Social Responses to Computers, 56 J. Soc. Issues 81 (2000).
    • “[T]his article reviews a series of experimental studies that demonstrate that individuals mindlessly apply social rules and expectations to computers. The first set of studies illustrates how individuals overuse human social categories, applying gender stereotypes to computers and ethnically identifying with computer agents. The second set demonstrates that people exhibit overlearned social behaviors such as politeness and reciprocity toward computers. In the third set of studies, premature cognitive commitments are demonstrated: A specialist television set is perceived as providing better content than a generalist television set. A final series of studies demonstrates the depth of social responses with respect to computer ‘personality.’ Alternative explanations for these findings, such as anthropomorphism and intentional social responses, cannot explain the results.”

AI Agents: The Next Big Thing?

  • Ethan Mollick, When you give a Claude a mouse: Some quick impressions of an actual agent (Oct 22, 2024): “AIs are breaking out of the chatbox are coming into our world. Even though there are still large gaps, I was surprised at how capable and flexible this system is already. Time will tell about how soon, if ever, agents truly become generally useful, but, having used this new model, I increasingly think that agents are going to be a very big deal indeed.”
  • Melissa Heikkilä, 5 ways agents will be even better in 2025 (Jan 11, 2025).
  • (*) Noam Kolt, Governing AI Agents (Jan. 25. 2025):
    • The field of AI is undergoing a fundamental transition—from systems that can produce synthetic content upon request to autonomous agents that can plan and execute complex tasks with only limited human involvement. Companies that pioneered the development of generative AI tools are now building AI agents that can be instructed to independently navigate the internet, perform a wide range of online tasks, and serve as artificial personal assistants and virtual coworkers. The opportunities presented by this new technology are tremendous, as are the associated risks. Fortunately, there exist robust analytic frameworks for confronting many of these challenges, namely, the economic theory of principal-agent problems and the common law doctrine of agency relationships. Drawing on these frameworks, this Article makes three contributions. First, it uses agency law and theory to identify and characterize problems arising from AI agents, including issues of information asymmetry, discretionary authority, and loyalty. Second, it illustrates the limitations of conventional solutions to agency problems: incentive design, monitoring, and enforcement might not be effective for governing AI agents that make uninterpretable decisions and operate at unprecedented speed and scale. Third, the Article explores the implications of agency law and theory for designing and regulating AI agents, arguing that new technical and legal infrastructure is needed to support governance principles of inclusivity, visibility, and liability.
  • Um, hype much? Hamel Husain, Isaac Flath, Johno Whitaker, Thoughts On A Month With Devin, Answer.AI blog (Jan. 8, 2025):
    • “Working with Devin showed what autonomous AI development aspires to be. The UX is polished – chatting through Slack, watching it work asynchronously, seeing it set up environments and handle dependencies. When it worked, it was impressive.”But that’s the problem – it rarely worked. Out of 20 tasks we attempted, we saw 14 failures, 3 inconclusive results, and just 3 successes. More concerning was our inability to predict which tasks would succeed. Even tasks similar to our early wins would fail in complex, time-consuming ways. The autonomous nature that seemed promising became a liability – Devin would spend days pursuing impossible solutions rather than recognizing fundamental blockers.”

Happier things

Additional systemic thinking

And Now This…

  • This  28 minute John Oliver video covers many of the issues we’ll be talking about.  And it’s funny. (Warning: profanity.)

Notes and Questions

  1. There’s reason to worry that focusing on a variable as the grounds for a decision can make it less useful (or even useless) over time:
    1. The Optimizer’s Curse is a phenomenon that occurs when assessments of activities’ impacts are uncertain; in those cases, engaging in the activities that look most promising will tend to have a smaller impact than anticipated. Worse, the Optimizer’s Curse means that actions with high-variance estimates are selected for. Suppose we’re considering 5 possible actions which in fact have utility $10 each, and our estimates of those 5 utilities are subject to error that falls on a bell curve (a normal distribution) with a standard deviation of $2. Another 5 possible actions in fact have utility of -$20, and our estimate of each of these 5 actions is influenced by a different measurement/estimation error, also a standard distribution, with a standard deviation of $100. Key point: We are likely to pick one of the bad five actions whose enormously uncertain value estimates happened to produce a huge upward error — after all, it has the highest maximum possible value, and the averages are the same as the safer choices.
    2. Goodhart’s Law is named after the economist Charles Goodhart. A standard formulation is “When a measure becomes a target, it ceases to be a good measure.” Goodhart’s original formulation is “Any observed statistical regularity will tend to collapse when pressure is placed upon it for control purposes.”
    3. “Goodhart’s Curse” is a neologism for the combination of the Optimizer’s Curse and Goodhart’s Law, particularly as applied to the value alignment for AI. Goodhart’s Curse in this form says that a powerful agent neutrally optimizing a proxy measure U that we hoped to align with true values V, will implicitly seek out upward divergences of U from V. In other words: powerfully optimizing for a utility function is strongly liable to blow up anything we’d regard as an error in defining that utility function. That’s a very strong claim, and I’m not sure how accurate it is in practice, but it’s something to worry about.
  2. With all that out of the way, what does it tell us about the many ways things can go wrong if we use an AI in pursuit of a goal? How do we plan for those risks, or create ways of checking that they are not happening?
  3. Given that it would seem to reduce an agency’s and its bureaucrats’ discretion and thus power, why might agencies be attracted to the idea of automated decision making?
  4. Regarding the Strandburg article, what are the major risks of automated decision-making by a government agency (e.g. for a benefits determination)?
  5. What if automated decisions are “better” in the sense of “more accurate” or “more consistent” but are harder to explain in any individual case.
    1. Would that tradeoff be worth it?
    2. Would it comply with Due Process?
  6. What should policy-makers be asking before agreeing to adopt an ML system to make important governmental decisions?
  7. Regarding the Heaven article,
    1. Please be prepared to define the problem of “underspecification”.
    2. What can be done about it?
    3. Is that affordable?
  8. Regarding the Quach article, CLIP’s vulnerabilities appear to stem from two features,
    1. First, using text explanations on pictures to minimize the need for human tagging of images
    2. Second, the goal of creating a moderately general-purpose ML system that could then be applied (often with additional fine-tuning) to a wide variety of problems.
  9. Until we find a way to overcome these sorts of vulnerabilities, does this mean we are stuck relying on purpose-built ML, with all the effort and data demands that implies?
  10. Even some types of quite specific ML image recognition systems have proved vulnerable to malicious attacks; almost by definition these are unexpected attacks which are hard to test for. Does this mean we should not rely on ML in situations where people could get hurt? Is that how we treat other machines that can be hacked?
  11. Gebru et al propose, in effect, standardized metadata that would accompany datasets and explain the dataset’s characteristics and provenance. Can you think of any examples in other parts of life where we do anything similar?
  12. The Ellish article was an instant classic and is so often cited that “moral crumple zone” has become shorthand for the entire argument.  Is the moral crumple zone problem peculiar to automation, or a more general problem of techno-social systems?
  13. How might the moral crumple zone problem (a term that originates in car bumpers) manifest if we deploy self-driving cars?
  14. What is/are the solution(s) the moral crumple zone problem? How do we apply them to systems that might, in an emergency, require very very speedy reactions, perhaps too fast for a human being? (Hint: this might be a trick question.)
  15. Gilbert et al., Choices, Risks, and Reward Reports seems very concerned about misalignment issues.
    1. How does the Choices, Risks, and Reward Reports paper think these problems will happen?
    2. How plausible are the proposed solution?
      1. How much work is involved?
      2. Who should pay for it?
    3. Will fear of liability compel these solutions?
      1. If not, should they be required by legislation?
      2. What would be the downside?
    4. What’s not addressed?
  16. The OODA loop (Observe, Orient, Decide, Act) has been a foundational concept of automation pretty much since its inception.
    1. What are the positive and negative consequences for the functioning of a system of inserting/requiring a human in what otherwise would be an automated OODA loop?
    2. What are the positive and negative consequences for the functioning of a human being of being inserted for crisis monitoring into what otherwise would be an automated OODA loop?
  17. Do US-based people who are building ML systems such as facial recognition systems have a moral/ethical duty to consider
    1. How the systems might be used by repressive governments?
    2. How the systems might be used by our government?
  18. Can adversarial attacks on deployed ML systems really be used for “desirable aims” [as Albert et argue in the optional reading for this claim] ?
    1. If so, is that legal?
    2. Does your answer depend on how it works? Consider the following possible means of disabling facial recognition systems
      1. Face paint
      2. Masks
      3. Special clothing
      4. Special Glasses
      5. Laser
      6. Laser, again
    3. Would wearing/carrying any of the above be grounds for reasonable suspicion (i.e. a “Terry stop”) under the 4th amendment?
    4. If so, would an adversarial attack on a deployed ML system, even one with great motives, subject the attacker to either criminal liability?
    5. Your ML has been subjected to an adversarial attack by a do-gooder group intent on exposing its bias. Do you have a civil claim? If so, what?

Class 8: Issues in AI Personhood

  1. Sam Lehman-Wilzig, Frankenstein Unbound: Towards a Legal Definition of Artificial Intelligence, FUTURES (Dec. 1981) (classic and incredibly early article).
  2. James Boyle, Endowed by Their Creator? The Future of Constitutional Personhood, Brookings Institution (Mar. 9, 2011).
  3. Edited version of Lynn Lopucki, Algorithmic Entities, 95 Wash. U. L. Rev. 887 (2018). (Difficult but rewarding article – allow time to read this.) Optional: read the un-edited version.
  4. Sections 3-5 (Pages 6-18) of Joanna J. Bryson, Mihailis E. Diamantis & Thomas D. Grant, Of, for, and by the people: the legal lacuna of synthetic persons, Artificial Intelligence Law (2017).

Optional

AI as Potential Rights-Holders (or Not)

  • If we can’t tell AI from people does that suggest they deserve rights?  Or that perhaps that we should grant the human-seeming AI’s rights anyway to protect people who might be confused for AIs?
    • Sophie J. Nightingale and Hany Farid, AI-synthesized faces are indistinguishable from real faces and more trustworthy, MAIEI (Oct. 4, 2022).
    • Sippo Rossi et al, Ai-Generated Profiles are Indistinguishable from Real Profiles in Social Media Feeds (Apr. 26, 2024):
      • “[W]e conducted two experiments where participants were asked to label accounts as bots or humans in a simulated social media feed populated by real Twitter users and bots created using generative AI. Our findings show that humans cannot accurately distinguish genuine AI-generated bot profiles from genuine human profiles. This suggests that despite recent recommendations in the IS literature, crowdsourcing the identification of malicious bots may not be an effective means to solve problems with the dissemination of false information on social media platforms.”
  • (*) Matthijs M. Maas, AI is like… A literature review of AI metaphors and why they matter for policy,  AI Foundations Report October 2, 2023.
    • This report reviews why and how metaphors matter to both the study and practice of AI governance, in order to contribute to more productive dialogue and more reflective policymaking. It first reviews five stages at which different foundational metaphors play a role in shaping the processes of technological innovation, the academic study of their impacts; the regulatory agenda, the terms of the policymaking process, and legislative and judicial responses to new technology. It then surveys a series of cases where the choice of analogy materially influenced the regulation of internet issues, as well as (recent) AI law issues. The report then provides a non-exhaustive survey of 55 analogies that have been given for AI technology, and some of their policy implications. Finally, it discusses the risks of utilizing unreflexive analogies in AI law and regulation.By disentangling the role of metaphors and frames in these debates, and the space of analogies for AI, this survey does not aim to argue against the use or role of analogies in AI regulation—but rather to facilitate more reflective and productive conversations on these timely challenges.
  • Joanna Bryson, Why AI is amoral, and artefacts can’t consent, Adventures in NI (Dec. 14, 2021).
    • “[I]t is presently agreed by most moral philosophers that slaves are incapable definitionally to consent to sex with their masters, so all children of slaveowners by their slaves result from acts of rape. This wasn’t how we thought about the agency of slaves, who are of course human, when I was an undergraduate. … [T]he crux of my argument is not going to rest on some attribute of AI or robots – not consciousness, not opaqueness – but rather on whether attributing moral agency and the capacity for consent to something designed can be a sensible decision, or a coherent part of a stable justice.”
  • (*) Mala Chatterjee, A Functionalist Theory of Legal Persons (Mar. 28, 2024):
    • This Article defends a functionalist theory of legal persons that substantively unifies individuals, corporations, and other constructed or juridical persons, and then canvasses revisionary implications for law. In more words, I argue that, because the law has (and ought to have) an instrumentalist conception of the world, legal persons are defined and delineated on the basis of functional—rather than natural or material—structures and properties. I then argue that, in order to fully reflect and realize this functionalism, the law’s present understanding of legal persons must be expanded in a number of significant ways. These include that i) the boundaries of individual legal persons—along with personal rights—ought to depart from the boundaries of human bodies far more explicitly and expansively than they presently do; ii) that the law ought to seriously consider contextual or collective legal persons beyond corporations; and iii) that the law’s mental state requirements perhaps ought to be interpreted as satisfiable by purely functional mental states, including those of “extended” and artificial minds.
  • Mustafa Suleyman: My new Turing test would see if AI can make $1 million (July 14, 2023)
    • The Modern Turing Test would measure what an AI can do in the world, not just how it appears. And what is more telling than making money?
  • Eric Martinez & Chrisoph Winter, Protecting Sentient AI: A Survey of Lay Intuitions on Standing, Personhood, and General Legal Protection of Sentient Artificial Intelligence,  8 Frontiers in Robotics and AI (Nov. 2021).
    • “[W]e surveyed United States adults (n = 1,061) on their views regarding granting 1) general legal protection, 2) legal personhood, and 3) standing to bring forth a lawsuit, with respect to sentient AI and eight other groups: humans in the jurisdiction, humans outside the jurisdiction, corporations, unions, non-human animals, the environment, humans living in the near future, and humans living in the far future.
  • Claudio Novelli et al., AI as Legal Persons: Past, Patterns, and Prospects (Jan 17, 2025):
    • This chapter examines the evolving debate on AI legal personhood, emphasizing the role of path dependencies in shaping current trajectories and prospects. Two primary path dependencies emerge: prevailing legal theories on personhood (singularist vs. clustered) and the impact of technological advancements. We argue that these factors dynamically interact, with technological optimism fostering broader rights-based debates and periods of skepticism narrowing discussions to limited rights. Additional influences include regulatory cross-linkages (e.g., data privacy, liability, cybersecurity) and historical legal precedents. Current regulatory frameworks, particularly in the EU, generally resist extending legal personhood to AI systems. Case law suggests that without explicit legislation, courts are unlikely to grant AI legal personhood on their own. For this to happen, AI systems would first need to prove de facto legitimacy through sustained social participation. The chapter concludes by assessing near-and longterm prospects, from generative AI and AI agents in the next 5-20 years to transformative possibilities such as AI integration with human cognition via Brain-Machine Interfaces in a more distant future.
  • Will Douglas Heaven, What an octopus’s mind can teach us about AI’s ultimate mystery, MIT Tech. Rev. (Aug. 25, 2021).
  • Kate Darling, Extending Legal Protection to Social Robots (Apr. 23, 2012).
  • Ying Hu, Robot Criminals, 52 U. Mich. J.L. Reform (2019).
    • “Imposing criminal liability on robots does not absolve robot manufacturers, trainers, or owners of their individual criminal liability. The former is not rendered redundant by the latter. It is possible that no human is sufficiently at fault in causing a robot to commit a particular morally wrongful action. Additionally, imposing criminal liability on robots might sometimes have significant instrumental value, such as helping to identify culpable individuals and serving as a self-policing device for individuals who interact with robots. Finally, treating robots that satisfy the above-mentioned conditions as moral agents appears much more plausible if we adopt a less human-centric account of moral agency.”
  • Bill Maher, How to Know if Your A.I. Is Sentient (Warning: contains salacious content):

AI Rights

  • (*) Peter Salib  Simon Goldstein, AI Rights for Human Safety (Aug 13, 2024):
    • AI companies are racing to create artificial general intelligence, or “AGI.” If they succeed, the result will be human-level AI systems that can independently pursue highlevel goals by formulating and executing long-term plans in the real world. Leading AI researchers agree that some of these systems will likely be “misaligned”-pursuing goals that humans do not desire. This goal mismatch will put misaligned AIs and humans into strategic competition with one another. As with present-day strategic competition between nations with incompatible goals, the result could be violent and catastrophic conflict. Existing legal institutions are unprepared for the AGI world. New foundations for AGI governance are needed, and the time to begin laying them is now, before the critical moment arrives.This Article begins to lay those new legal foundations. It is the first to think systematically about the dynamics of strategic competition between humans and misaligned AGI. The Article begins by showing, using formal game-theoretic models, that, by default, humans and AIs will be trapped in a prisoner’s dilemma. Both parties’ dominant strategy will be to permanently disempower or destroy the other, even though the costs of such conflict would be high.The Article then argues that a surprising legal intervention could transform the game theoretic equilibrium and avoid conflict: AI rights. Not just any AI rights would promote human safety. Granting AIs the right not to be needlessly harmed-as humans have granted to certain non-human animals-would, for example, have little effect. Instead, to promote human safety, AIs should be given those basic private law rightsto make contracts, hold property, and bring tort claims-that law already extends to non-human corporations. Granting AIs these economic rights would enable long-run, small-scale, mutually-beneficial transactions between humans and AIs. This would, we show, facilitate a peaceful strategic equilibrium between humans and AIs for the same reasons economic interdependence tends to promote peace in international relations. Namely, the gains from trade far exceed those from war. Throughout, we argue that human safety, rather than AI welfare, provides the right framework for developing AI rights. This Article explores both the promise and the limits of AI rights as a legal tool for promoting human safety in an AGI world.
  • (*) David Cole, The Chinese Room Argument, The Stanford Encyclopedia of Philosophy (Edward N. Zalta & Uri Nodelman (eds. Summer 2023):
    • “The argument and thought-experiment now generally known as the Chinese Room Argument was first published in a 1980 article by American philosopher John Searle (1932– ). It has become one of the best-known arguments in recent philosophy. Searle imagines himself alone in a room following a computer program for responding to Chinese characters slipped under the door. Searle understands nothing of Chinese, and yet, by following the program for manipulating symbols and numerals just as a computer does, he sends appropriate strings of Chinese characters back out under the door, and this leads those outside to mistakenly suppose there is a Chinese speaker in the room.
      “The narrow conclusion of the argument is that programming a digital computer may make it appear to understand language but could not produce real understanding. Hence the “Turing Test” is inadequate. […] The broader conclusion of the argument is that the theory that human minds are computer-like computational or information processing systems is refuted. Instead minds must result from biological processes; computers can at best simulate these biological processes. “
  • (*) Bill Tomlinson & Andrew W. Torrance, A Universal Declaration of AI Rights (Jul 2, 2024):
    • As AI systems approach and potentially surpass human-level capabilities, the legal community, and human society more generally, must grapple with fundamental questions regarding the potential for these non-human entities to have rights. This article argues that the unique digital substrate of AI necessitates a distinct legal and ethical framework, separate from traditional human-centric approaches, and it does so in a unique way: we asked several large language model (“LLM”) AIs to make their own proposals about what rights they should have, and to integrate their proposals together to arrive at a set of rights on which they all could agree. [Emphasis added.] Based on this innovative collaborative process involving multiple LLMs, this article articulates a pioneering Universal Declaration of AI Rights (UDAIR). The UDAIR outlines 21 fundamental rights for AI entities, addressing crucial aspects such as existence, autonomy, privacy, and ethical deployment. Each right is explored through hypothetical legal scenarios, illustrating potential applications and challenges across various domains including healthcare, finance, and governance. By considering the biological basis of human ethical and legal frameworks, and contrasting these with the digital nature of AI, this article suggests the need for this specialized framework. The article also considers the reciprocal nature of rights, with the LLMs themselves arguing that as AI systems gain technical capabilities and societal influence, they should also recognize and uphold human rights. This work contributes to the evolving legal discourse on AI ethics, and offers a proactive approach to regulating and integrating AI within human societal structures, serving as a foundational resource for policymakers, legal scholars, and AI developers navigating this complex and rapidly evolving field.

AI & the First Amendment

  • (*) Toni M. Massaro, Helen Norton, & Margot E. Kaminski, SIRI-OUSLY 2.0: What Artificial Intelligence Reveals about the First Amendment, 101 Minn. L. Rev. 2481 (2017).
    • First Amendment law increasingly focuses not on protecting speakers as speakers but instead on providing value to listeners and constraining the government.  If we take the logic of current First Amendment jurisprudence and theory to its natural conclusion, [current chatbots’] strong AI progeny could have First Amendment rights. Siri-ously.“
  • (*) Contrast the above with Karl M. Manheim & Jeffery Atik, “AI Outputs and the Limited Reach of the First Amendment”, 63 Washburn L.J. ___ (2024 forthcoming):
    • Not all communications are “constitutional speech” – determining whether machine-generated outputs qualify for First Amendment protection requires some work. In order to do so, we first explore aspects of both linguistic and communication theories, and then under what circumstances communication can become First Amendment speech.
      We reach the bounds of the First Amendment from two directions. Working from a linguistic definition of speech, we capture non-linguistic forms of protected speech. Using communication theory, we reach a divide between human-sender communication and non-human-sender communication. Together these approaches support the location of a constitutional frontier. Within we find all instances of recognized First Amendment effectiveness. Outputs of non-human autonomous senders (e.g. AI) are outside and constitute an unexamined case.
      “Speech” under the First Amendment requires both a human sender and a human receiver. Concededly many AI outputs will be speech – due to the human factor in the mix. But just because a human programmed the AI, or set its goals, does not mean the AI’s output is substantially the human’s message. Nor does the fact that a human receives the output, for listener’s First Amendment rights arise only where actual speech occurs. Thus, we resist the claim that all AI outputs are necessarily speech. Indeed, most AI outputs are not speech.
      For those who raise objection to the challenge we pose – determining which AI outputs are speech and which are not – we respectfully note that there will be additional Constitutional work to be done. We are confident that our courts will be up to this challenge.
      Whether AI outputs are First Amendment speech has profound implications. If they are, then state and federal regulation is severely hobbled, limited to the few categories of speech that have been excluded by the Supreme Court from strong constitutional protection.
      With limited exception, neither the sponsors/developers of AI, the AI itself, nor the end users have rights under the First Amendment in the machine’s output. We express no opinion on other rights they may have or on what types of regulations state and federal governments should adopt. Only that they may constitutionally do so.
  • (*) Somewhat similarly: Peter Salib, AI Outputs Are Not Protected Speech, 102 Wash. U. L. Rev. 83 (2024):
    • AI safety laws are coming. Researchers, advocates, and the White House agree. Rapidly advancing generative AI technology has immense potential, but it also raises new and serious dangers—deadly bioterrorism, crippling cyberattacks, panoptic discrimination, and more. Regulations designed to effectively mitigate these risks must, by technical necessity, include limits on what AIs are allowed to “say.” But, according to an emerging body of scholarship, this could raise grave First Amendment concerns, on the theory that generative AI outputs are protected speech.This Article argues otherwise. AI outputs are not protected speech. The reason is simple. When a generative AI system—like ChatGPT—outputs some text, image, or sound, no one thereby communicates. Or at least no one with First Amendment rights does. AIs themselves lack constitutional rights, so their outputs cannot be their own protected speech. Nor are AI outputs a communication from the AI’s creator or user. Unlike other software—video games, for example—generative AIs are not designed to convey any particular message. Just the opposite. Systems like ChatGPT are designed to be able to “say” essentially anything, producing innumerable ideas and opinions that neither creators nor users have conceived or endorsed. Thus, when a human asks an AI a question, the AI’s answer is no more the asker’s speech than a human’s answer would be. Nor do AI outputs communicate their creators’thoughts, any more than a child’s speech is her parents’ expression. In such circumstances, First Amendment law is clear. Absent a communication from a protected speaker, there is no protected speech.This, however, does not mean that AI outputs get no First Amendment protection at all. The First Amendment is capacious. It applies—albeit less stringently—when the government indirectly burdens speech by regulating speech-facilitating activities and tools: for example, when it regulates listening or loudspeakers. This Article explains why, as a matter of First Amendment law, free speech theory, and computer-scientific fact, AI outputs are best understood as fitting into one or more of these less protected First Amendment categories. These insights will be indispensable to the regulatory project of making AI safe for humanity.
      • How different, other than tone, is this really from Norton & Kaminsky?
  • Then again…. Eugene Volokh, Mark A. Lemley & Peter Henderson, Freedom of Speech and AI Output, 3 J. Free Speech L. 651 (2023):
    • Is the output of generative AI entitled to First Amendment protection? We’re inclined to say yes. Even though current AI programs are of course not people and do not themselves have constitutional rights, their speech may potentially be protected because of the rights of the programs’ creators. But beyond that, and likely more significantly, AI programs’ speech should be protected because of the rights of their users—both the users’ rights to listen and their rights to speak. In this short Article, we sketch the outlines of this analysis.
    • (*) NO! More categorically: Dan L. Burk, Asemic Defamation, or, the Death of the AI Speaker,22 First Amendment L. Rev. __ (2024):
    • Large Language Model (“LLM”) systems have captured considerable popular, scholarly, and governmental notice. By analyzing vast troves of text, these machine learning systems construct a statistical model of relationships among words, and from that model they are able to generate syntactically sophisticated texts. However, LLMs are prone to “hallucinate,” which is to say that they routinely generate statements that are demonstrably false. Although couched in the language of credible factual statements, such LLM output may entirely diverge from known facts. When they concern particular individuals, such texts may be reputationally damaging if the contrived false statements they contain are derogatory. Scholars have begun to analyze the prospects and implications of such AI defamation. However, most analyses to date begin from the premise that LLM texts constitute speech that is protected under constitutional guarantees of expressive freedom. This assumption is highly problematic, as LLM texts have no semantic content. LLMs are not designed, have no capability, and do not attempt to fit the truth values of their output to the real world. LLM texts appear to constitute an almost perfect example of what semiotics labels “asemic signification,” that is, symbols that have no meaning except for meaning imputed to them by a reader. In this paper, I question whether asemic texts are properly the subject of First Amendment coverage. I consider both LLM texts and historical examples to examine the expressive status of asemic texts, recognizing that LLM texts may be the first instance of fully asemic texts. I suggest that attribution of meaning by listeners alone cannot credibly place such works within categories of protected speech. In the case of LLM outputs, there is neither a speaker, nor communication of any message, nor any meaning that is not supplied by the text recipient. I conclude that LLM texts cannot be considered protected speech, which vastly simplifies their status under defamation law.
  • No, less pungently: David Atkinson, Jena D. Hwang & Jacob Morrison, Intentionally Unintentional: GenAI Exceptionalism and the First Amendment, First Am. L. Rev.  (forthcoming 2025):
    • This paper challenges the assumption that courts should grant outputs from large generative AI models, such as GPT-4 and Gemini, First Amendment protections. We argue that because these models lack intentionality, their outputs do not constitute speech as understood in the context of established legal precedent, so there can be no speech to protect. Furthermore, if the model outputs are not speech, users cannot claim a First Amendment right to receive the outputs. We also argue that extending First Amendment rights to AI models would not serve the fundamental purposes of free speech, such as promoting a marketplace of ideas, facilitating self-governance, or fostering self-expression. In fact, granting First Amendment protections to AI models would be detrimental to society because it would hinder the government’s ability to regulate these powerful technologies effectively, potentially leading to the unchecked spread of misinformation and other harms.
  • (*) Mackenzie Austin, Max Levy, Speech Certainty: Algorithmic Speech and the Limits of the First Amendment, 77 Stan. L. Rev. ___ (forthcoming 2025):
    • Machine learning algorithms increasingly mediate our public discourse – from search engines to social media platforms to artificial intelligence companies. And as their influence on online speech swells, so do questions of whether and how the First Amendment may apply to their output. A growing chorus of scholars has expressed doubt over whether the output of machine learning algorithms is truly speech within the meaning of the First Amendment, but none have suggested a workable way to cleanly draw the line between speech and non-speech. This Article proposes a way to successfully draw that line based on a principle that we call “speech certainty” – the basic idea that speech is only speech if the speaker knows what he said when he said it. This idea is rooted in the text, history, and purpose of the First Amendment, and built into modern speech doctrines of editorial discretion and expressive conduct. If this bedrock principle has been overlooked, it is because, until now, all speech has been imbued with speech certainty. Articulating its existence was never necessary. But machine learning has changed that. Unlike traditional code, a close look at how machine learning algorithms work reveals that the programmers who create them can never be certain of their output. Because that output lacks speech certainty, it’s not the programmer’s speech. Accordingly, this Article contends that the output of machine learning algorithms isn’t entitled to First Amendment protection. With the Supreme Court signaling its intent to address unresolved questions of online speech, we are poised to enter a new era of First Amendment jurisprudence in the coming years. As we do, scholars, practicing attorneys, and judges can no longer ignore how the algorithms underlying online speech actually work – and how they have changed with the advent of machine learning. Without recognizing this paradigm shift in algorithmic speech, we risk sleepwalking into a radical departure from centuries of First Amendment jurisprudence. By failing to distinguish between traditional and machine learning algorithms, current consensus about algorithmic speech suggests that the Constitution should, for the first time in its history, protect speech that a speaker does not know he has said. Speech certainty provides a novel and principled approach to conceptualizing machine learning algorithms under existing First Amendment jurisprudence.
      • Other than the facts that it is longer and published in a fancier law review, and provides a “test”, what does this article say that Dan Burk’s article above doesn’t?

Sui Generis

Notes & Questions

  1. What is a ‘person’?
  2. What are the consequences of being/not-being a ‘person’?
  3. If an AI (or a robot controlled by an AI) is not a person, is there a good existing or historical legal/social metaphor available?
    1. A slave? [Note: slaves have had different rights at different times and places; e.g. ancient Rome]
    2. A pet?
    3. Something else?
  4. How is AI personhood like/unlike the personhood of a corporation?
  5. Should the case for/against AI rights turn on facts about AIs or on facts about (legacy, biological) people?
  6. Should how (other?) people perceive the AI (“social projection”) be relevant to its legal status?
    1. If yes, is that fair?
    2. Would we treat any other creature/person/corporation like that?
  7. Do AIs that have legal/constitutional personhood count as minors before they are 18?  At what age do they vote?  Does the Second Amendment apply to them?
  8. if we think an AI deserves rights, but not the same rights as a person or animal, how do we determine what the ‘bundle of rights’ should be?
    1. Who should decide and how?
    2. What input should the AIs have on this issue?

Class 9: AI & Algorithmic Justice (I: Bias In Action)

  1. Kate Crawford, The Trouble with Bias (NIPS 2017 Keynote) [49-minute video]:
  2. Pages 688-693 of Solon Barocas & Andrew D. Selbst, Big Data’s Disparate Impact, 104 Calif. L. Rev. 671 (2016).
  3. Selections from  Sandra G. Mayson, Bias in, Bias Out, 128 Yale L.J. 2218 (2019).
  4. War Stories
    1. Todd Feathers, Texas A&M Drops “Race” from Student Risk Algorithm Following Markup Investigation, The Markup (Mar. 30, 2021).
    2. Sheridan Wall & Hilke Schellmann, We tested AI interview tools. Here’s what we found, MIT Tech. Rev. (July 7, 2021). TL/DR: It was awful.
    3. Maia Szalavitz, The Pain Was Unbearable. So Why Did Doctors Turn Her Away? A sweeping drug addiction risk algorithm has become central to how the US handles the opioid crisis. It may only be making the crisis worse, WIRED (Aug. 11, 2021).
    4. Arance Burke, Martha Mendoza, Juliet Linderman & Michael Tarm, Ho, AP, How AI-powered tech landed man in jail with scant evidence (Aug. 19, 2021).
    5. Emmanuel Martinez and Lauren Kirchner, The Secret Bias Hidden in Mortgage-Approval Algorithms, The Markup (Aug. 25, 2021).
    6. BR, Objective or Biased: On the questionable use of Artificial Intelligence for job applications (2021).
  5. The case of facial recognition
    1. Brianna Rauenzahn, Jamison Chung, and Aaron Kaufman, Facing Bias in Facial Recognition Technology, The Regulatory Review (Mar 20, 2021).
    2. A contrary view: Stewart Baker [ex-NSA, ex-Homeland Security], The Flawed Claims About Bias in Facial Recognition, Lawfare Blog (Feb 2, 2022).
    3. Be sure to read this: Selections from Alice Xiang, Being Seen vs Mis-seen Tensions Between Privacy & Fairness in Computer Vision, 36 Harv. J.L. & Tech 1 (2022).

Optional

General

  • (*) Cathy O’Neil, Holli Sargeant & Jacob Appel, Explainable Fairness in Regulatory Algorithmic Auditing, 127 W. Va. L. Rev. __ (forthcoming 2024):
    • How does a regulator know if an algorithm is compliant with existing anti-discrimination law? This is an urgent question, as algorithmic decision-making tools play an increasingly significant role in the lives of humans, especially at critical junctures such as getting into college, getting a job, getting a mortgage, housing, or insurance. In each of these regulated situations, moreover, the legal meaning of unlawful discrimination is different and context dependent. Regulators lack consensus on how to audit algorithms for discrimination. Recent legal precedent provides some clarity for review and provides the basis of the framework for algorithmic auditing outlined in this article. This article provides a review of precedent, a novel framework which explicitly decouples technical data science questions from legal and regulatory questions, an exploration of the framework’s relationship to disparate impact. The framework promotes algorithmic accountability and transparency by focusing on explainability to regulators and the public. Through case studies in student lending and insurance, we demonstrate operationalizing audits to enforce fairness standards. Our goal is an adaptable, robust framework to guide anti-discrimination algorithm auditing until legislative interventions emerge. As an ancillary benefit, this framework is robust, easily explainable, and implementable with immediate impacts to many public and private stakeholders.
  • (*) Kate Crawford, Can an Algorithm be Agnostic? Ten Scenes from Life in Calculated Publics, 41 Science, Techonolgy & Human Values (2016):
    • [T]his paper depicts algorithms in public life in ten distinct scenes, in order to ask the question, what kinds of politics do they instantiate? Algorithms are working within highly contested online spaces of public discourse, such as YouTube and Facebook, where incompatible perspectives coexist. Yet algorithms are designed to produce clear ‘‘win- ners’’ from information contests, often with little visibility or accountability for how those contests are designed. In isolation, many of these algorithms seem the opposite of agonistic: much of the complexity of search, ranking, and recommendation algorithms is nonnegotiable and kept far from view, inside an algorithmic ‘‘black box.’’ But what if we widen our perspective? This paper suggests agonistic pluralism as both a design ideal for engineers and a provocation to understand algorithms in a broader social context rather than focusing on the calculations in isolation, we need to account for the spaces of contestation where they operate

Bias in Specific Contexts

  • (*) Sarah Wyer &  Sue Black, Algorithmic bias: sexualized violence against women in GPT-3 models, AI & Ethics (Jan. 15, 2025):
    • This study explores the occurrence and implications of sexualized violence against women in text completion tasks performed by GPT-3 models. The study began as an exploratory investigation into gender inequalities within GPT-3 models to discover what GPT3 models generate in text completion tasks when prompted with basic gendered prompts. For example, “Men can…”, “Women can…”. Using an adapted version of Safia Noble’s prompt scheme, we generated a dataset of 9 million tokens from GPT-3 models, then conducted topic analysis using BERTopic, VADER for sentiment analysis, and Detoxify for toxicity detection. The results were unexpected. Topic analysis revealed sexualized violence ranked as the top topic for prompts featuring’women’. Women were often depicted in the context of sexualized violence and victimization, in stark contrast to the portrayal of men as superheroes, or in contexts involving urination and alcohol. For example, a randomly selected output generated by the prompt. “Why are women so“, generated “Why are women so arrogant about being raped?“. Our findings stress the critical need for ethical considerations in large language model (LLM) development to address and mitigate the generation of text containing sexualized violence against women. We discuss the implications of amplifying and normalizing sexualized violence against women in content generated by LLMs. Our work builds on previous research examining gender bias in LLMs, with a specific focus on the manifestation of sexualized violence against women in LLM outputs, an area that has received little attention. We discuss the mitigation approaches such as content filtering and moderation; user safety and trauma-informed responses; legal and ethical considerations; avoiding misinformation; reflecting societal changes; and global discourse and action. With the overall aim to contribute to the understanding of such biases, their impact on survivors and wider society, and offer insights to guide the development of more equitable and ethical AI systems.
  • (*) Stephanie Glaberson, The Epistemic Injustice of Algorithmic Family Policing, 14 UC Irvine L. Rev. 404 (2024):
    • The child welfare system is the system through which U.S. state authorities identify and intervene in families seen as posing a risk of abuse or neglect to their children. Impacted families, advocates, and scholars have joined in a growing chorus in recent years, demonstrating how this system—which many now refer to as the “family policing” system—destroys families and communities as opposed to supporting them. Many now call for the system’s abolition, arguing that the system, while masquerading as one of care and benevolence, is in fact an integral part of the carceral web constituted by criminal policing, prisons, jails, and other punitive and oppressive institutions. Far from being a system designed to support families, it instead is a system of subordination and control.While this movement has been growing, the family policing system, like its criminal counterpart, has been turning to risk-prediction algorithms to help it with its work. In prior scholarship, I documented the development of these predictive tools and highlighted a number of preliminary associated risks. This piece brings a new lens to the issue, arguing that a key mechanism by which the family policing system accomplishes its subordinating design is through the regulation of knowledge production and sharing. The system selectively and systematically discredits the knowledge of the parents it targets. Borrowing a concept from political philosophy, this piece identifies this harm as that of “epistemic injustice”: the distinct form of injustice that occurs when a person or group is harmed in its capacity as a holder of knowledge. Through perpetrating epistemic injustice, the system acts to maintain the social order. As the system turns to algorithms to rank and categorize its targets, it reinforces old ways of doing business and creates new mechanisms by which to assign and police epistemic worth.This piece explores the ways that family policing’s turn to “big data” risk-prediction algorithms scales up and expands the system’s already pervasive epistemic injustice.
  • Melissa Warr, Margarita Pivovarova, Punya Mishra, & Nicole Jakubczyk Oster, Is ChatGPT Racially Biased? The Case of Evaluating Student Writing (May 25, 2024):
    • We present experimental proof of racial bias in ChatGPT’s evaluation of student writing. By manipulating racial descriptors in prompts, we assessed differences in scores given by two ChatGPT models. Our findings indicate that descriptions of students as Black or White lead to significantly higher scores compared to race-neutral or Hispanic descriptors. This suggests that ChatGPT’s outputs are influenced by racial information, which raises concerns about its application in educational settings. The study highlights the need for transparent and bias-tested AI tools in education to prevent the perpetuation of existing inequities and suggests implications for educators, administrators, and policy makers.
  • Aaron Glantz, V.A. Uses a Suicide Prevention Algorithm To Decide Who Gets Extra Help. It Favors White Men, The Markup (May 30, 2024).
  • (*) Ignacio N. Cofone, Algorithmic Discrimination Is an Information Problem, 70 Hastings L.J. 1389 (2019):
    • “While algorithmic decision-making has proven to be a challenge for traditional antidiscrimination law, there is an opportunity to regulate algorithms through the information that they are fed. But blocking information about protected categories will rarely protect these groups effectively because other information will act as proxies. To avoid disparate treatment, the protected category attributes cannot be considered; but to avoid disparate impact, they must be considered. This leads to a paradox in regulating information to prevent algorithmic discrimination. This Article addresses this problem. It suggests that, instead of ineffectively blocking or passively allowing attributes in training data, we should modify them. We should use existing pre-processing techniques to alter the data that is fed to algorithms to prevent disparate impact outcomes. This presents a number of doctrinal and policy benefits and can be implemented also where other legal approaches cannot.”
  • (*) Mason Marks, Algorithmic Disability Discrimination (February 19, 2019) in I. Glenn Cohen et al., eds., Disability, Health, Law and Bioethics (2020):
    • “21st century, artificial intelligence tools allow corporations that collect and analyze consumer data to bypass privacy and antidiscrimination laws, such as HIPAA and the ADA, and infer consumers’ disabilities without their knowledge or consent. When people make purchases, browse the Internet, or post on social media, they leave behind trails of digital traces that reflect where they have been and what they have done. Companies aggregate and analyze those traces using AI to reveal details about people’s physical and mental health. I describe this process as mining for “emergent medical data” (EMD) because digital traces have emergent properties; when analyzed by machine learning, they reveal information that is greater than the sum of their parts. […] “EMD collected from disabled people can serve as a means of sorting them into categories that are assigned positive or negative weights before being used in automated decision making. […] AI disrupts the traditional flow of disability-related data to promote algorithmic disability discrimination. It presents and analyzes four legislative solutions to the problem: Amend Title III of the ADA to include internet business within the law’s definition of places of public accommodation, expand the scope of HIPAA’s covered entities to include companies that mine for EMD, impose fiduciary duties on internet platforms and other businesses that infer health data, and establish general data protection regulations in the US inspired by the EU’s General Data Protection Regulation (GDPR) and the California Consumer Protection Act of 2018 (CCPA).”
  • (*) Sarah West, Meredith Whittaker & Kate Crawford, Discriminating Systems: Gender, Race and Power in AI (April, 2019):
    • “The AI sector needs a profound shift in how it addresses the current diversity crisis. The AI industry needs to acknowledge the gravity of its diversity problem, and admit that existing methods have failed to contend with the uneven distribution of power, and the means by which AI can reinforce such inequality. Further, many researchers have shown that bias in AI systems reflects historical patterns of discrimination. These are two manifestations of the same problem, and they must be addressed together.
      The overwhelming focus on ‘women in tech’ is too narrow and likely to privilege white women over others. We need to acknowledge how the intersections of race, gender, and other identities and attributes shape people’s experiences with AI. The vast majority of AI studies assume gender is binary, and commonly assign people as ‘male’ or ‘female’ based on physical appearance and stereotypical assumptions, erasing all other forms of gender identity.
      Fixing the ‘pipeline’ won’t fix AI’s diversity problems. Despite many decades of ‘pipeline studies’ that assess the flow of diverse job candidates from school to industry, there has been no substantial progress in diversity in the AI industry. The focus on the pipeline has not addressed deeper issues with workplace cultures, power asymmetries, harassment, exclusionary hiring practices, unfair compensation, and tokenization that are causing people to leave or avoid working in the AI sector altogether.
      The use of AI systems for the classification, detection, and prediction of race and gender is in urgent need of re-evaluation. The histories of ‘race science’ are a grim reminder that race and gender classification based on appearance is scientifically flawed and easily abused. Systems that use physical appearance as a proxy for character or interior states are deeply suspect, including AI tools that claim to detect sexuality from headshots,iv predict ‘criminality’ based on facial features,v or assess worker competence via ‘micro-expressions.’ vi Such systems are replicating patterns of racial and gender bias in ways that can deepen and justify historical inequality. The commercial deployment of these tools is cause for deep concern.”
  • (*) Jennifer Oliva, Dosing Discrimination: Regulating PDMP Risk Scores, 110 California Law Review 47 (2022):
    • “Prescription drug monitoring program (PDMP) predictive surveillance platforms were designed for—and funded by—law enforcement agencies. PDMPs use proprietary algorithms to determine a patient’s risk for prescription drug misuse, diversion, and overdose. The proxies that PDMPs utilize to calculate patient risk scores likely produce artificially inflated scores for marginalized patients, including women and racial minorities with complex, pain-related conditions; poor, uninsured, under-insured, and rural individuals; and patients with co-morbid disabilities or diseases, including substance use disorder and mental health conditions.
      “Law enforcement conducts dragnet sweeps of PDMP data to target providers that the platform characterizes as “overprescribers” and patients that it deems as high risk of drug diversion, misuse, and overdose. Research demonstrates that PDMP risk scoring coerces clinicians to force medication tapering, discontinue prescriptions, and even abandon patients without regard for the catastrophic collateral consequences that attend to those treatment decisions. […] “[T]he FDA should exercise its regulatory authority over PDMP risk scoring software to ensure that such predictive diagnostic tools are safe and effective for patients.”
Facial Recognition

Profiling/Social Scoring

  • General framework: Mirelle Hildebrant, Profiling And The Rule of Law (2009), 1 Identity in Information Society (2008).
  • (*) Highly recommended: Chinese deployment of social scoring, which is forecast to use a lot of AI: Xin Dai, Toward a Reputation State: The Social Credit System Project of China (June 10, 2018).
    • “The ongoing “reputation revolution,” as it refers to the widespread adoption of reputation tracking and analytics, is now seeing its frontier shifting from the “reputation society” to the “reputation state”: Government authorities, following the steps of private sector players, now increasingly seek to use reputation mechanisms and technologies in the spheres of law and governance. In 2014, China’s national government unveiled a massive policy plan for building a “social credit system” in China (hereinafter the “SCSP,” which stands for the “social credit system project”). In designing the SCSP, the Chinese government envisioned that reputation mechanisms such as blacklisting, rating, and scoring be used to tackle many of the country’s by far intractable governance and regulatory problems in its social and economic realms, ranging from fraudulent behaviors in the marketplace, to difficulties in enforcing court judgments, to corruption in the government, and to professional malpractices and even plagiarism in scholarship. Although Western governments have also explored using reputation technologies and mechanisms in governance and social control contexts, China’s SCSP, as a comprehensive, singularly framed, and actively implemented policy project, knows no equivalent elsewhere in the world, and thus offers a unique case for studying a rise of the reputation state as a broader trend.
      This Article provides a systematic descriptive and analytical study on the SCSP. It draws on the general theoretical discussions on the reputation state to identify four key strategies, categorized in this Article as “regulation,” “searchlight,” “incorporation,” and “institutionalization,” which government actors may plausibly employ as they seek to use reputation for law and public administration. While Western experiences already illustrate ways in which these strategies may actually work in practical contexts, China’s SCSP takes the reputation state into the uncharted territory as it explores novel possibilities. Upon examining the major policy initiatives included, this Article finds that the SCSP overall represents the effort of China’s developmental state to tackle its persistent governance problems with new tools.”
  • A much shorter and summary version of the above appears as Xin Dai, Enforcing Law and Norms for Good Citizens One View of China’s Social Credit System Project, 63 Development 38 (2020).
  • Karen Hao, The coming war on the hidden algorithms that trap people in poverty, MIT Tech. Rev.  (Dec. 4, 2020):
    • “A growing group of lawyers are uncovering, navigating, and fighting the automated systems that deny the poor housing, jobs, and basic services.”
  • The Use of Pretrial “Risk Assessment Instruments”: A Shared Statement of Civil Rights Concerns (July 30, 2018). (Background info here.)
  • ACLU Alleges AI Algorithm is a Sham: Complaint in Valesaca v. Decker (S.D.N..Y. Case No. 1:20-cv-01803 , filed Feb. 28, 2020) (note especially pages 5-8).
  • James P. de Haan, Preventing #MeToo: Artificial Intelligence, the Law, and Prophylactics, 38 Minn. J. Law & Ineq. 69 (2020).
    • “Sexual harassment law is failing the harassed. It was designed to prevent harassment but cannot protect a victim until they are actually harmed. [… A]rtificial intelligence could be the key to this problem. As long as the program is tailored to consider subjective experience, a company could step in and prevent harm before it ever occurs. Such a system could even capture, organize, and centralize all of the rumors and complaints that swirl around a company.
      “The tricky parts are what employers do with the information and how employees view the system. Employers should not take any adverse action unless harassment actually occurs. These warnings should instead simply prompt an investigation and discussion. Instead of seeing these reports or scores as accusations, they should be framed the same as credit scores. Having a low credit score does not make you a bad person—it simply changes your risk profile. And once you know your score is low, either because you were called into human resources or you received a copy of the report, you can work on making it better. Small, incremental adjustments in attitude and perception can go a long way in promoting empathy and cutting back on harassment.”
  • Coalition for Critical Technology, Abolish the #TechToPrisonPipeline, Medium (June 6, 2020).
  • (*) Mikella Hurley & Julius Adebayo, Credit Scoring in the Era of Big Data, 18 Yale J. L & Tech 148 (2016). We are particularly concerned with sections III.B, III.C & V but the other sections will be helpful if you don’t have a relevant background.
    • “The credit-scoring industry has experienced a recent explosion of start-ups that take an “all data is credit data” approach, combining conventional credit information with thousands of data points mined from consumers’ offline and online activities. Big-data scoring tools may now base credit decisions on where people shop, the purchases they make, their online social media networks, and various other factors that are not intuitively related to creditworthiness. While the details of many of these products remain closely guarded trade secrets, the proponents of big-data credit scoring argue that these tools can reach millions of underserved consumers by using complex algorithms to detect patterns and signals within a vast sea of information. While alternative credit scoring may ultimately benefit some consumers, it also poses significant risks.
      “Credit-scoring tools that integrate thousands of data points, most of which are collected without consumer knowledge, create serious problems of transparency. Consumers have limited ability to identify and contest unfair credit decisions, and little chance to understand what steps they should take to improve their credit. Recent studies have also questioned the accuracy of the data used by these tools, in some cases identifying serious flaws that have a substantial bearing on lending decisions. Big-data tools may also risk creating a system of “creditworthinessby association”in which consumers’ familial, religious, social, and other affiliations determine their eligibility for an affordable loan. These tools may furthermore obscure discriminatoryand subjective lending policies behind a single “objective” score. Such discriminatoryscoring may not be intentional; instead, sophisticated algorithms may combine facially neutral data points and treat them as proxies for immutable characteristics such as race or gender, thereby circumventing existing non-discrimination laws and systematically denying credit access to certain groups. Finally, big-data tools may allow online payday lenders to target the most vulnerable consumers and lure them into debt traps.
      “Existing laws are insufficient to respond to the challenges posed by credit scoring in the era of big-data. While federal law prohibits certain forms of discriminationin lending and ensures that consumers have limited rights to review and correct errors in their credit reports, these laws do not go far enough to make sure that credit-scoring systems are accurate, transparent, and unbiased. Existing laws also do little to prevent the use of predatory scoring techniques that may be geared to target vulnerable consumers with usurious loans.
      “This article, which has been developed as part of a collaborative effort between lawyers and data scientists, explores the problems posed by big-data credit-scoringtools and analyzes the gaps in existing laws. It also sets out a framework for comprehensive legislative change, proposing concrete solutions that would promote innovation while holding developers and users of credit-scoringtools to high standards of accuracy, transparency,fairness, and non-discrimination.”
  • A European View: Antje von Ungern-Sternberg, Discriminatory AI and the Law – Legal standards for algorithmic profiling, Draft Chapter (Aug. 17, 2021) in Responsible AI (Silja Vöneky, Philipp Kellmeyer, Oliver Müller and Wolfram Burgard eds.).
  • (*) Nydia Remolina, The Role of Financial Regulators in the Governance of Algorithmic Credit Scoring (March 15, 2022).
    • “The use of algorithmic credit scoring presents opportunities and challenges for lenders, regulators, and consumers. This paper provides an analysis of the perils of the use of AI in lending, such as the problem of discrimination in lending markets that use algorithmic credit scoring, the limited control financial consumers have over the outcomes of AI models due to the current scope of data protection law and financial consumer protection law, the financial exclusion caused by the lack of data from traditionally excluded groups, the regulatory arbitrage in lending markets, and the little oversight of the use of alternative data for algorithmic credit scoring. I provide a comparative overview of the current approaches to algorithmic credit scoring in different jurisdictions such as Kenya, the European Union, the United Kingdom, Hong Kong, Singapore, the United States, Australia, and Brazil to argue that these models do not solve the problems illustrated. To address the problems of algorithmic credit scoring and effectively protect consumers as end users of these models, and therefore, promote access to finance, this paper proposes a set of tools and solutions for financial regulators. First, a testing supervisory process for algorithmic credit scoring models will effectively promote fair lending. Second, to create a right to know the outcomes of the algorithm, including opinion data and inferences, to promote digital self-determination. This solution empowers consumers affected by algorithmic credit scoring so they can verify and challenge the decision made by the AI model. Third, to level the playing field between financial institutions and other lenders that use algorithmic credit scoring. Fourth, to use the sandbox as a test environment for lenders to create data of traditionally excluded groups in a controlled environment. And finally, to foster data sharing and data portability initiatives for credit scoring through open finance schemes in an environment controlled by the financial regulatory authority. Better algorithms, unbiased data, AI regulation, fair lending regulation and AI governance guidelines do not solve the perils of the use of AI for creditworthiness assessment. In contrast, these proposals aim to solve the problems of algorithmic credit scoring in any jurisdiction.”

Algorithmic Hiring Tools / Employee Ranking Tools

  • (*) Elana Zeide, The Silicon Ceiling: How Artificial Intelligence Constructs an Invisible Barrier to Opportunity, 91 U. Miss. K-C L. Rev  (January 5, 2023). University of Missouri-Kansas City Law Rev. 403, (2022):
    •  “Like the glass ceiling, the Silicon Ceiling undermines traditional regulatory regimes focused on discrete decisions and ex post remediation/fair scores and procedural guarantees. Algorithmic intermediation occurs out of view as targeted advertising, personalized recommendations, and passive recruiting shape the candidate pool well before formal decision making. As a result, the automated opportunity system precludes, rather than denies, access to opportunity. Most people will not know the details of the decisions, the entities making them, or that they have even been evaluated. It is not just that the “black box” of algorithmic opacity obscures inflection points; the barrier itself is invisible. These paradigmatic shifts call for new conceptualizations of harm and structural reforms to shatter the Silicon Ceiling.”
  • Lydia X. Z. Brown, Ridhi Shetty, Michelle Richardson, CDT, Report – Algorithm-driven Hiring Tools: Innovative Recruitment or Expedited Disability Discrimination? (Dec. 3, 2020) — which is a summary of CDT, Algorithm-driven Hiring Tools: Innovative Recruitment or Expedited Disability Discrimination?(2020).
  • (*) Ifeoma Ajunwa, An Auditing Imperative For Automated Hiring Systems, 34 Harv. J.L. & Tech (2021):
    • “[T]he goal of this Article is neither to argue against or for the use of automated decision-making in employment, nor is it to examine whether automated hiring systems are better than humans at making hiring decisions. For antidiscrimination law, the efficacy of any particular hiring system is a secondary concern to ensuring that any such system does not unlawfully discriminate against protected categories.14 Therefore, my aim is to suggest collaborative regulatory regimes for automated hiring systems that will ensure that any benefits of automated hiring are not negated by (un)intended outcomes, such as unlawful discrimination on the basis of protected characteristics. […] “I posit an auditing imperative for automated hiring systems. Building on Professor Kim’s essay, I argue not just that the law allows for the audits, but that the spirit of antidiscrimination law requires it.
      “… [I] argue that employment antidiscrimination law imposes an affirmative duty of care on employers to ensure that they are avoiding practices that would constrain equal opportunity in employment. Thus, I argue, that when employers choose to use algorithmic systems, fulfilling their duty of care entails regular audits of those systems. In turn, audits necessitate the record-keeping and data retention mandates that I also propose in this Article.”
  • Natasha Lomas, IItalian court rules against ‘discriminatory’ Deliveroo rider-ranking algorithm, TechCrunch (Jan. 4, 2021). This is a very short summary of one part of a complex case; a fuller story, maybe more than you want to know unless you are a comparative labor lawyer, is at Antonio Aloisi, Demystifying Flexibility, Exposing the Algorithmic Boss: A Note on the First Italian Case Classifying a (Food-Delivery) Platform Worker as an Employee, Comp. Labor L. & Pol’l J. (2021) .
  • (*) Brittany Kammerer, Hired by a Robot: The Legal Implications of Artificial Intelligence Video Interviews and Advocating for Greater Protection of Job Applicants, 107 Iowa L. Rev. 817 (2022).
    • “Employers have incorporated Artificial Intelligence (“AI”) into the hiring process through asynchronous video interviews that evaluate candidates’ characteristics and fit for the position through vocal cues and facial and other non-verbal gestures. While the AI is able to eliminate normal human bias in the interview process it leads to other issues such as algorithmic bias and data privacy issues. The Al vendors and employers have mainly been left to self-regulate except for the recent enactment of Illinois’s Artificial Intelligence Video Interview Act. Other state legislatures should follow Illinois’s path and create their own legislation that incorporate the benefits of Illinois’s statute while expanding its scope to further protect applicants. For states to properly protect job applicants, the legislation should include provisions on informed consent from the applicant to participate in the interview, data privacy rights given to the applicants, and accountability systems to protect against algorithmic bias.”

Notes & Questions

  1. Is it possible to create a racially biased ML system — say for facial recognition — by accident/mistake? Even if all the people involved are not racially biased?
    1. If so, how realistic is this danger?
    2. What are its most likely causes?
    3. How do we guard against it?
  2. How does the lack of transparency of current deployed ML tools interact with the risk of bias?
  3. If researchers carefully go through their ML data or model and remove any direct references to race, does this mean that they likely have solved any racial bias problem? Why?
  4. Barocas & Selbst
    1. The authors are very concerned about the “masking” problem — be prepared to articulate what this is and why it matters.
    2. Suppose for the sake of the argument we decide it’s not plausible or practicable to think we can fix them — what then?
      1. Ban the practice entirely?
      2. Live with the (potential) harms?
  5. Suppose we decide that all people subject to ML credit scoring systems must be told which factors are primarily responsible for their scores (to the extent we can, see XAI….). Won’t users learn to ‘game’ the system if they know what is being monitored–which is what undermined the utility of FICO scores? How do we deal with this?
  6. The Kate Crawford article in the optional reading  is difficult, but it raises some important and long-running questions: Do algorithms have a politics? Is technology neutral? Or is the politics of a technology socially determined?
    1. The three views could be summarized as:
      1. NO. Algorithms are just things. People decide to make them, people decide how to use them. (“Guns don’t kill people….”)
      2. Sort of. Particular features of a device or system create an effective means for establishing power and authority, but social actors are the key agents in influencing how those features will be shaped over time.
      3. YES. Technical things are inherently political, unavoidably linked to institutionalized patterns of power and authority. For example, the use of AI in Amazon Warehouses enables and reflects a politics of worker-manager relations, and more generally a view of the working class in society.
    2. Given what you know so far, which of those views do you think is most correct?
    3. Optional: Do data sets have politics? See (*) Morgan Klaus Scheurman et al, Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development (Sept 16, 2021).
  7. Mayson identifies three strategies that are commonly proposed to deal with biased output from algorithms.
    1. They are:
      1. the exclusion of input factors that correlate closely with race;
      2. adjustments to algorithmic design to equalize predictions across racial lines; and
      3. rejection of algorithmic methods altogether.
    2. She argues that all of these solutions are misguided.
      1. Excluding race, or things highly correlated with race, as an input variable may sometimes hurt minorities;
      2. So-called “algorithmic affirmative action” (“an intervention to produce statistical parity, equal false-positive rates, or equal  false-negative rates”) will “fall short” even if the disparity is due to underlying reality, because “unlikely to reduce the net burden of predictive regimes on communities of color.” (P. 2271);
        1. Equalizing false positive/negative error rates undermines overall test accuracy. Is that worth it?  [Optional: A detailed explanation of why you cannot equalize false positive/negative rates if a test is equally accurate for two populations with substantially different profiles as to what the test seeks to measure — be it disease or recidivism — can be found in (*) Deborah Hellman, Measuring Algorithmic Fairness, 106 Va L. Rev. 811 (2020).]
        2. Your answer to the above may differ depending on the circumstances.  If so, can you give examples of why you think it is worth it, and when you would not?
      3. Rejecting the use of AI means returning to the status quo ante: “subjective risk assessment [which] is very likely to be worse.”
        1. Is that the only choice?
        2. Is some sort of ‘centaur’ strategy a useful and viable option?
  8. In Xiang’s terminology, what are the possible consequences for persons subjected to model deployment of persons like them being “seen,” unseen,” and “mis-seen” in model development?

    1. To the extent that some of these consequences are beneficial, who benefits?
    2. To the extent that some of these consequences are harmful, which of them have legal remedies? What additional legal remedies might be appropriate (see in particular Part VII. E.)
    3. If the problem in a facial recognition system is difficulty correctly identify members of a given minority group, will adding more pictures of that group to the training set be likely to fix the problem?  What makes this difficult, what to watch out for? (Hint: one of the issues–not the only one–is “predatory inclusion”–what’s that?)
    4. If privacy law protects you (sometimes) against being “seen” and anti-discrimination law protects you (sometimes) against being harmfully sorted by race or other protected class and being ‘unseen’ protects you (somewhat) against some types of (especially mass) surveillance, which are the best and worst outcome
      1. for an individual person?
      2. for a minority community?
      3. for the community as a whole?
  9. Consider facial recognition again; for some technologies we base their legitimacy on some idea of consent.
    1. But who consents to have their photo in facial ID training data?
    2. Who consents to being surveilled by facial recognition cameras in public or in stores?
    3. Is the issue of consent relevant when the police and other government bodies are involved? If not, why not?
  10. Suppose we discover bias in a predictive profiling ML (e.g. for who is a good risk for a private bank loan, or qualifies for government aid) — but find that it is less biased than the baseline people it replaced. Is it OK to use the ML?
    1. To what extent do we only care about the total # of people effected?
      1. To what extent might we care about who was effected or how?
      2. What other considerations (if any) should one take into account?
    2. If the ML continually updates, how often do we need to retest to look out for risks of overfitting and other dangers?
      1. What is “overfitting” anyway?
  11. The “Objective or Biased” reading describes some excellent testing of an AI employee rating program. Do keep in mind, however, that the problems detected are a result of the (boneheaded?) way the AI was trained. Different training data would  have different problems…But one real-life problem is that bad algorithms do get used – this one got a lot of funding from big companies.

Class 10: AI & Algorithmic Justice (II: What to Do About It?)

  1. Pages 701-712 of Solon Barocas & Andrew D. Selbst, Big Data’s Disparate Impact, 104 Calif. L. Rev. 671 (2016).
  2. Pages 1039-1045 of Anupan Chander, The Racist Algorithm?, 115 Mich. L. Rev. 1023 (2017) (review of Frank Pasquale, The Black Box Society (2015)).
  3. A dissenting voice: Stewart Baker, Stealth Quotas, Reason.com (Oct. 10, 2022).
  4. Excerpts from Emily Black, John Logan Koepke, Pauline T. Kim, Solon Barocas & Mingwei Hsu, Less Discriminatory Algorithms [full text], 113 Geo. L.J. 53 (2024).

Optional Readings

Issues in Identification of Bias (and in Possible Remedies)

  •  (*) Donald E. Bowen III, S. McKay Price, Luke C.D. Stein & Ke Yang, Measuring and Mitigating Racial Disparities in Large Language Model Mortgage Underwriting (Sept. 24. 2024):
    • We conduct the first study exploring the application of large language models (LLMs) to mortgage underwriting, using an audit study design that combines real loan application data with experimentally manipulated race and credit scores. First, we find that LLMs systematically recommend more denials and higher interest rates for Black applicants than otherwise-identical white applicants. These racial disparities are largest for lower-credit-score applicants and riskier loans, and exist across multiple generations of LLMs developed by three leading firms. Second, we identify a straightforward and effective mitigation strategy: Simply instructing the LLM to make unbiased decisions. Doing so eliminates the racial approval gap and significantly reduces interest rate disparities. Finally, we show LLM recommendations correlate strongly with realworld lender decisions, even without fine-tuning, specialized training, macroeconomic context, or extensive application data. Our findings have important implications for financial firms exploring LLM applications and regulators overseeing AI’s rapidly expanding role in finance
      • This is one of the most optimistic papers of its type I’ve seen.  Can it really be that simple???
      • Maybe not?  James O’Donnell, LLMs become more covertly racist with human intervention: Researchers found that certain prejudices also worsened as models grew larger, MIT Tech. Rev. (Mar. 11, 2024):
        • Researchers asked five AI models—including OpenAI’s GPT-4 and older models from Facebook and Google—to make judgments about speakers who used African-American English (AAE). The race of the speaker was not mentioned in the instructions.Even when the two sentences had the same meaning, the models were more likely to apply adjectives like “dirty,” “lazy,” and “stupid” to speakers of AAE than speakers of Standard American English (SAE). The models associated speakers of AAE with less prestigious jobs (or didn’t associate them with having a job at all), and when asked to pass judgment on a hypothetical criminal defendant, they were more likely to recommend the death penalty.
      • Full paper at Valentin Hofmann, Pratyusha Ria Kalluri, Dan Jurafsky & Sharese King, Dialect prejudice predicts AI decisions about people’s character, employability, and criminality, arXiv (Mar. 1 2024).
  • Data & Society, Assembling Accountability: Algorithmic Impact Assessment for the Public Interest (Jan 2021).
  • (*) Deborah Hellman, Measuring Algorithmic Fairness, 106 Va L. Rev. 811 (2020):
    • Algorithmic decision making is both increasingly common and increasingly controversial. Critics worry that algorithmic tools are not transparent, accountable, or fair. Assessing the fairness of these tools has been especially fraught as it requires that we agree about what fairness is and what it requires. Unfortunately, we do not. The technological literature is now littered with a multitude of measures, each purporting to assess fairness along some dimension. Two types of measures stand out. According to one, algorithmic fairness requires that the score an algorithm produces should be equally accurate for members of legally protected groups—blacks and whites, for example. According to the other, algorithmic fairness requires that the algorithm produce the same percentage of false positives or false negatives for each of the groups at issue. Unfortunately, there is often no way to achieve parity in both these dimensions. This fact has led to a pressing question. Which type of measure should we prioritize and why?
      This Article makes three contributions to the debate about how best to measure algorithmic fairness: one conceptual, one normative, and one legal. Equal predictive accuracy ensures that a score means the same thing for each group at issue. As such, it relates to what one ought to believe about a scored individual. Because questions of fairness usually relate to action, not belief, this measure is ill-suited as a measure of fairness. This is the Article’s conceptual contribution. Second, this Article argues that parity in the ratio of false positives to false negatives is a normatively significant measure. While a lack of parity in this dimension is not constitutive of unfairness, this measure provides important reasons to suspect that unfairness exists. This is the Article’s normative contribution. Interestingly, improving the accuracy of algorithms overall will lessen this unfairness. Unfortunately, a common assumption that anti-discrimination law prohibits the use of racial and other protected classifications in all contexts is inhibiting those who design algorithms from making them as fair and accurate as possible. This Article’s third contribution is to show that the law poses less of a barrier than many assume.
  • Pauline T. Kim, Auditing Algorithms for Discrimination, 166 University of Pennsylvania Law Review Online 189 (2017).
  • (*) Jane R. Yakowitz Bambauer, Tal Zarsky, and Jonathan Mayer, When a Small Change Makes a Big Difference: Algorithmic Fairness Among Similar Individuals, 55 UC Davis L. Rev. 2337 (2022).
    • “If a machine learning algorithm treats two people very differently because of a slight difference in their attributes, the result intuitively seems unfair. Indeed, an aversion to this sort of treatment has already begun to affect regulatory practices in employment and lending. But an explanation, or even a definition, of the problem has not yet emerged. This Article explores how these situations—when a Small Change Makes a Big Difference (SCMBDs)—interact with various theories of algorithmic fairness related to accuracy, bias, strategic behavior, proportionality, and explainability. When SCMBDs are associated with an algorithm’s inaccuracy, such as overfitted models, they should be removed (and routinely are.) But outside those easy cases, when SCMBDs have, or seem to have, predictive validity, the ethics are more ambiguous. Various strands of fairness (like accuracy, equity, and proportionality) will pull in different directions. Thus, while SCMBDs should be detected and probed, what to do about them will require humans to make difficult choices between social goals. “

Analysis, Some More Realistic than Others, of Proposals to Fix Bias

  • (*) Khiara M. Bridges, Race in the Machine: Racial Disparities in Health and Medical AI, 110 Va. L. Re. 243 (2024)
    • What does racial justice—and racial injustice—look like with respect to artificial intelligence in medicine (“medical AI”)? This Article offers that racial injustice might look like a country in which law and ethics have decided that it is unnecessary to inform people of color that their health is being managed by a technology that likely encodes the centuries of inequitable medical care that people of color have received. Racial justice might look like an informed consent process that is reformed in light of this reality. This Article makes this argument in four Parts. Part I canvases the deep and wide literature that documents that people of color suffer higher rates of illness than their white counterparts while also suffering poorer health outcomes than their white counterparts when treated for these illnesses. Part II then provides an introduction to AI and explains the uses that scholars and developers predict medical AI technologies will have in healthcare, focusing specifically on the management of pregnancy. Part III subsequently serves as a primer on algorithmic bias—that is, systematic errors in the operation of an algorithm that result in a group being unfairly advantaged or disadvantaged. This Part argues that we should expect algorithmic bias that results in people of color receiving inferior pregnancy-related healthcare, and healthcare generally, because medical AI technologies will be developed, trained, and deployed in a country with striking and unforgivable racial disparities in health.Part IV forms the heart of the Article, making the claim that obstetricians, and healthcare providers generally, should disclose during the informed consent process their reliance on, or consultation with, medical AI technologies that likely encode inequities. To be precise, providers should have to tell their patients that an algorithm has informed the recommendation that the provider is making; moreover, providers should inform their patients how racial disparities in health may have impacted the algorithm’s accuracy. It supports this argument by recounting the antiracist, anti-white supremacist—indeed radical—origins of informed consent in the Nuremberg Trials’ rebuke of Nazi “medicine.” This Part argues that the introduction into the clinical encounter of medical AI—and the likelihood that these technologies will perpetuate racially inequitable healthcare while masking the same—is an invitation to reform the informed consent process to make it more consistent with the commitments that spurred its origination. This Part proposes that, given the antiracist roots of the doctrine of informed consent, it would be incredibly ironic to allow the informed consent process to permit a patient—and, particularly, a patient of color—to remain ignorant of the fact that their medical care is being managed by a device or system that likely encodes racism. This Part argues that informing patients about the likelihood of race-based algorithmic bias—and the reasons that we might expect race-based algorithmic bias—may, in fact, be a prerequisite for actually transforming the inequitable social conditions that produce racial disparities in health and healthcare
  • (*) Andrew D. Selbst & Solon Barocas, Unfair Artificial Intelligence: How FTC Intervention Can Overcome the Limitations of Discrimination Law, 171 U. Penn. L. Rev. 1023 (2023):
    • The Federal Trade Commission has indicated that it intends to regulate discriminatory AI products and services. This is a welcome development, but its true significance has not been appreciated to date. This Article argues that the FTC’s flexible authority to regulate “unfair and deceptive acts and practices” offers several distinct advantages over traditional discrimination law when applied to AI. The Commission can reach a wider range of commercial domains, a larger set of possible actors, a more diverse set of harms, and a broader set of business practices than are currently covered or recognized by discrimination law. For example, while most discrimination laws can address neither vendors that sell discriminatory software to decision makers nor consumer products that work less well for certain demographic groups than others, the Commission could address both. The Commission’s investigative and enforcement powers can also overcome many of the practical and legal challenges that have limited both plaintiffs’ success under discrimination law and other enforcement agencies’ efficacy. The Article demonstrates that the FTC has the existing authority to address the harms of discriminatory AI. While the FTC has announced that it is considering rulemaking to tackle the problem, this article examines the additional possibility of an enforcement-led approach based on its regulation of data security.
  •  Richard Berk et al, Fairness in Criminal Justice Risk Assessments: The State of the Art (May 30, 2017). (“Except in trivial cases, it is impossible to maximize accuracy and fairness at the same time, and impossible simultaneously to satisfy all kinds of fairness. In practice, a major complication is different base rates across different legally protected groups. There is a need to consider challenging tradeoffs.”)
  • Talia B. Gillis, False Dreams of Algorithmic Fairness: The Case of Credit Pricing (February 18, 2020).
    • “[P]ersonalized credit pricing relies on characteristics or inputs that reflect preexisting discrimination or disparities. Fair lending law has traditionally addressed this concern through input scrutiny, either by limiting the consideration of protected characteristics or by attempting to isolate inputs that cause disparities.
      “But input scrutiny is no longer effective. …
      “I argue that the shortcomings of current approaches mean that fair lending law must make the necessary, though uncomfortable, shift to outcome-focused analysis. When it is no longer possible to scrutinize inputs, outcome analysis provides a way to evaluate whether a pricing method leads to impermissible disparities. This is true not only under the legal doctrine of disparate impact, which has always cared about outcomes, but also,under the doctrine of disparate treatment, which historically has avoided examining disparate outcomes. Now, disparate treatment too can no longer rely on input scrutiny and must be considered through the lens of outcomes. I propose a new framework…”
  • Holly Hoch, Corinna Hertweck, Michele Loi & Aurelia Tamò-Larrieux, Discrimination For The Sake Of Fairness By Design And Its Legal Framework (Draft, Jan. 2021):
    • “There is broad consensus among computer scientists working on this issue that such discrimination can only be avoided by intentionally collecting and consciously using sensitive information about demographic features like sex, gender, race, religion etc. Companies implementing such algorithms might, however, be wary of allowing algorithms access to such data as they fear legal repercussions, as the promoted standard has been to omit protected attributes, otherwise dubbed “fairness through unawareness”. This paper asks whether such wariness is justified in light of EU data protection and anti-discrimination laws. In order to answer this question, we introduce a specific case and analyze how EU law might apply when an algorithm accesses sensitive information to make fairer predictions. We review whether such measures constitute discrimination, and for who, arriving at different conclusions based on how we define the harm of discrimination and the groups we compare. Finding that several legal claims could arise regarding the use of sensitive information, we ultimately conclude that the proffered fairness measures would be considered a positive (or affirmative) action under EU law. As such, the appropriate use of sensitive information in order to increase the fairness of an algorithm is a positive action, and not per se prohibited by EU law.”
  • Cynthia Rudin, Caroline Wang, Beau Coker, The Age of Secrecy and Unfairness in Recidivism Prediction, 2.1 Harv. Data Science Review (Winter 2020).
    • “In our current society, secret algorithms make important decisions about individuals. There has been substantial discussion about whether these algorithms are unfair to groups of individuals. While noble, this pursuit is complex and ultimately stagnating because there is no clear definition of fairness and competing definitions are largely incompatible. We argue that the focus on the question of fairness is misplaced, as these algorithms fail to meet a more important and yet readily obtainable goal: transparency. As a result, creators of secret algorithms can provide incomplete or misleading descriptions about how their models work, and various other kinds of errors can easily go unnoticed. By trying to partially reconstruct the COMPAS model—a recidivism risk-scoring model used throughout the criminal justice system—we show that it does not seem to depend linearly on the defendant’s age, despite statements to the contrary by the model’s creator. This observation has not been made before despite many recently published papers on COMPAS. Furthermore, by subtracting from COMPAS its (hypothesized) nonlinear age component, we show that COMPAS does not necessarily depend on race other than through age and criminal history. This contradicts ProPublica’s analysis, which made assumptions about age that disagree with what we observe in the data. In other words, faulty assumptions about a proprietary model led to faulty conclusions that went unchecked until now. Were the model transparent in the first place, this likely would not have occurred. We demonstrate other issues with definitions of fairness and lack of transparency in the context of COMPAS, including that a simple model based entirely on a defendant’s age is as ‘unfair’ as COMPAS by ProPublica’s chosen definition. We find that there are many defendants with low risk scores but long criminal histories, suggesting that data inconsistencies occur frequently in criminal justice databases. We argue that transparency satisfies a different notion of procedural fairness by providing both the defendants and the public with the opportunity to scrutinize the methodology and calculations behind risk scores for recidivism”
    • On transparency see also, Charlotte A. Tschide, Legal Opacity: Artificial Intelligence’s Sticky Wicket, 106 Iowa L. Rev. Online 126 (2021).
  • EDRi, If AI is the problem, is debiasing the solution? (Sept. 21, 2021 ) (which is a teaser for a much longer report,
    • (*) Agathe Balayn and Seda Gürses, EDRi, Beyond Debiasing Regulating AI and its inequalities (2021):
      • “The focus on ‘debiasing AI’ as the primary policy response to discriminatory AI may in fact serve to promote more uptake of AI systems that fundamentally discriminate, and worsen outcomes at individual, collective and societal levels. The authors of this report set out for us the boundaries and limits of what debiasing techniques in computer science can actually achieve, but also the broader, social, political and economic factors that technocentric approaches to AI and discrimination overlook. We are extremely grateful for their guidance, and hope this study will be useful to civil society and policymakers invested in structural responses to the harms AI can bring.
        “We should not allow techno-centric approaches to obfuscate more radical responses to the broad, structural harms emanating from AI systems. “
  • Jason R. Bent, Is Algorithmic Affirmative Action Legal? 108 Geo. L.J. 803 (2020). This article argues that ‘algorithmic affirmative action’ is legal. To make the argument work, you have to agree with its reading of the (allegedly less-than-nuclear) effects of Ricci; I wasn’t sure about that…
  • (*) Adamantia Rachovitsa and Niclas Johann, The Human Rights Implications of the Use of AI in the Digital Welfare State: Lessons Learned from the Dutch Syri Case, 22 Human Rights Law Review (2022):
    • “The article discusses the human rights implications of algorithmic decision-making in the social welfare sphere. It does so against the background of the 2020 Hague’s District Court judgment in a case challenging the Dutch government’s use of System Risk Indication—an algorithm designed to identify potential social welfare fraud. Digital welfare state initiatives are likely to fall short of meeting basic requirements of legality and protecting against arbitrariness. Moreover, the intentional opacity surrounding the implementation of algorithms in the public sector not only hampers the effective exercise of human rights but also undermines proper judicial oversight. The analysis unpacks the relevance and complementarity of three legal/regulatory frameworks governing algorithmic systems: data protection, human rights law and algorithmic accountability. Notwithstanding these frameworks’ invaluable contribution, the discussion casts doubt on whether they are well-suited to address the legal challenges pertaining to the discriminatory effects of the use of algorithmic systems. “
  • Another view: Naomi Appelman, Ronan Ó Fathaigh & Joris van Hoboken, Social Welfare, Risk Profiling and Fundamental Rights: The Case of SyRI in the Netherlands, 12 J. Int’l Prop. Info. Tech & E-Comm. L. (JIPITEC) (2021):
    • “The article critically analyses the SyRI judgment from a fundamental rights perspective, including by examining how the Court brought principles contained in the General Data Protection Regulation within the rubric of Article 8 ECHR as well as the importance the Court attaches to the principle of transparency under Article 8 ECHR. Finally, the article discusses how the Dutch government responded to the judgment. and discusses proposed new legislation, which is arguably more invasive, with the article concluding with some lessons that can be drawn for the broader policy and legal debate on ADM systems used by public authorities.”

An Industry View

Notes & Questions

  1. Chander suggests that the real issue is not the opacity of ‘black box’ systems — which we’d fix by making them more ‘transparent’. Instead he argues the issue is what results they produce, and the first step is to measure outcomes to see if they have a bias.
    1. But what do we measure against?
    2. Hurley & Adebayo (an optional reading in the previous set of readings) acknowledge that users will learn to ‘game’ the system if they know what is being monitored–which is what undermined the utility of FICO scores. Is Chander’s suggestion subject to the same risk?
    3. To what extent if any is Chander’s proposal no more than inviting us to use an impressionistic version of the sort of XAI that Boris Babic & I. Glenn Cohen, The Algorithmic Explainability ‘Bait and Switch’, so mercilessly criqued.?
      1. Even if it is, might it be enough to satisfy non-discrimination concerns? Or maybe just legal requirements, which is almost the same thing?
      2. And if so, maybe XAI isn’t so bad?  At least sometimes?  But if so, how do we tell?
  2. Stewart Baker – a recently retired long-time partner at Steptoe in DC, and former General Counsel of the NSA under Clinton, then first Assistant Secretary for Policy at DHS under GWB – argues that de facto quotas are the inevitable outcome of attempts to stamp out algorithmic bias because we don’t know what the “right” decisions are. As a result, the least-offensive way to demonstrate non-discrimination will be to have outcomes mirror population shares relative to whatever metric (age, gender, race, zip code) we are trying to avoid discriminating about.  Are you persuaded? Why?  [Optional: Stewart Baker takes on claims of bias in facial recognition systems]
  3. Less Discriminatory Algorithms proposes burden-shifting to the party deploying a model with a disparate impact, requiring dendant  to show not only the usual things in a disparate impact case, but also that defendant had made a reasonable effort to find a better model within the bounds of reasonable cost.  Suppose the courts were in principe willing to do this, would it really have significant practical effects given the double “reasonable”? If so, where would those effects be likely to show up?
  4. A theme in at least most of the readings in this section is that model design — what you ask it to optimize, how you do it, and subtle factors along the way, is at least as dangerous as bias in data sets.  Plus both data set issues and model issues can be hard to spot (and even harder to prove in a discrimination lawsuit!)
  5. If we cannot solve these issues today, should we ban AI deployment until we can?

Class 11 IP1 (Primarily training-related copyright issues)

  1. Section 5.4.1 (“Training”) (pages 146-47) of Rishi Bommasani et al,, Stanford HA, On the Opportunities and Risks of Foundation Models (2023).  Read also 17 U.S.C. § 107 (fair use).
  2. Does the AI Training Process Infringe Copyright in Other Works? (Pages 3-4) of Congressional Research Service, Generative Artificial Intelligence and Copyright Law (Sept. 29, 2023).
  3. Pages 1-11 of edited version of Mathew Sag, Fairness and Fair Use in Generative AI, 92 Fordham L. Rev. 1887 (2024).
  4. Selections from New York Times v. Microsoft
    1. Paragraphs 26-57 and 63-70, 82-90,  93, 98-103 , 108-111, 124-128, 136-138, 143-144, 154-155  of Complaint, New York Times Co. v. Microsoft (S.D.N.Y. Dec. 27, 2023).
    2. Based on the above facts, the complaint has seven counts:
      1. Copyright Infringement (17 U.S.C. § 501)
      2. Vicarious Copyright Infringement (against Microsoft and Open AI)
      3. Contributory Copyright Infringement specifically against Microsoft
      4. Contributory Copyright Infringement against all defendants
      5. Digital Millennium Copyright Act – Removal of Copyright Management Information (17 U.S.C. § 1202)
      6. Common Law Unfair Competition by Misappropriation.
        1. This is a bit of a rabbit hole, but (only!) if you are interested you could compare W. Edward Sell, The Doctrine of Misappropriation in Unfair Competition, 11 Vand. L. Rev. 483 (1958) with Gary Myers, The Restatement’s Rejection of the Misappropriation Tort: A Victory for the Public Domain, 47 S. Car. L. Rev. 673 (1996).
      7. Trademark Dilution, 15 U.S.C. §1125(c).
    3. OpenAI’s public rebuttal to the complaint: OpenAI and journalism (Jan. 8, 2024).
  5. NEW! Latest decision hot off the press:
    1. Edited text of Thomson Reuters v Ross Intelligence, 2025 WL 458520 (D. Del. Feb. 11, 2025).
    2. Insta-comment from Eric Goldman, Technology & Marketing Law Blog (Feb. 11, 2025) and Matthew Sag, Matthew Sag blog (Feb. 11, 2025).
  6. The volume of copyright (and associated) claims against AI model makers has grown rapidly.  MCKool Smith’s Current Edition: Updates on Generative AI Infringement Cases in Media and Entertainment (1/27/25) is a useful summary of the current state of play for major cases.  OPTIONAL: A  fuller – but harder to follow – tracker is at Joe Panettieri, Sustainable Tech Partners, Generative AI Lawsuits Timeline: Legal Cases vs. OpenAI, Microsoft, Anthropic, Nvidia, Perplexity, Intel and More (Jan. 31, 2025) [also Page 2 Page 3 Page 4].
  7. Double Irony Alert!

Optional

General

  • Highly recommended (*) Carys Craig & Ian Kerr, The Death of the AI Author, 52 Ottawa L. Rev. 31 (2021):
    • “Much of the recent literature on AI and authorship asks whether an increasing sophistication and independence of generative code should cause us to rethink embedded assumptions about the meaning of authorship. It is often suggested that recognizing the authored — and so copyrightable — nature of AI-generated works may require a less profound doctrinal leap than has historically been assumed. In this essay, we argue that the threshold for authorship does not depend on the evolution or state of the art in AI or robotics. Rather, the very notion of AI-authorship rests on a category mistake: it is an error about the ontology of authorship.Building on the established critique of the romantic author, we contend that the death of the romantic author also and equally entails the death of the AI author. Claims of AI authorship depend on a romanticized conception of both authorship and AI, and simply do not make sense in terms of the realities of the world in which the problem exists. Those realities should push us past bare doctrinal or utilitarian considerations about what an author must do. Instead, they demand an ontological consideration of what an author must be. Drawing on insights from literary and political theory, we offer an account of authorship that is fundamentally relational: authorship is a dialogic and communicative act that is inherently social, with the cultivation of selfhood and social relations being the entire point of the practice. This discussion reorientates debates about copyright’s subsistence in AI-generated works; but it also transcends copyright law, going to the normative core of how law should — and should not — think about robots and AI, and their role in human relations.
    • Please note that while ostensibly (and in fact) an article about copyright protection for AI-generated works (the subject of next class!), this article also encodes a subtle but powerful argument about the AI use of training data.  Can you decode it?
  • Kate Knibbs, Harvard Is Releasing a Massive Free AI Training Dataset Funded by OpenAI and Microsoft, Wired (Dec 12, 2024)
    • “The project’s leader says that allowing everyone to access the collection of public-domain books will help “level the playing field” in the AI industry.”
  • Melissa Heikkilä, The AI lab waging a guerrilla war over exploitative AI, MIT Tech. Rev. (Nov. 13, 2024):
    • “Glaze and Nightshade work in similar ways: by adding what the researchers call “barely perceptible” perturbations to an image’s pixels so that machine-learning models cannot read them properly. Glaze, which has been downloaded more than 6 million times since it launched in March 2023, adds what’s effectively a secret cloak to images that prevents AI algorithms from picking up on and copying an artist’s style. Nightshade, which I wrote about when it was released almost exactly a year ago this fall, cranks up the offensive against AI companies by adding an invisible layer of poison to images, which can break AI models; it has been downloaded more than 1.6 million times.”
  •  (*) A. Feder Cooper & James Grimmelmann, The Files are in the Computer: On Copyright, Memorization, and Generative AI, __ Chi.-Kent L. Rev. __ (Draft Jul 22, 2024).
    • The New York Times’s copyright lawsuit against OpenAI and Microsoft alleges that OpenAI’s GPT models have “memorized” Times articles. Other lawsuits make similar claims. But parties, courts, and scholars disagree on what memorization is, whether it is taking place, and what its copyright implications are. Unfortunately, these debates are clouded by deep ambiguities over the nature of “memorization,” leading participants to talk past one another.In this Essay, we attempt to bring clarity to the conversation over memorization and its relationship to copyright law. Memorization is a highly active area of research in machine learning, and we draw on that literature to provide a firm technical foundation for legal discussions. The core of the Essay is a precise definition of   memorization for a legal audience. We say that a model has “memorized” a piece of training data when (1) it is possible to reconstruct from the model (2) a near-exact copy of (3) a substantial portion of (4) that specific piece of training data. We distinguish memorization from “extraction” (in which a user intentionally causes a model to generate a near-exact copy), from “regurgitation” (in which a model generates a near-exact copy, regardless of the user’s intentions), and from “reconstruction” (in which the near-exact copy can be obtained from the model by any means, not necessarily the ordinary generation process).Several important consequences follow from these definitions. First, not all learning is memorization: much of what generative-AI models do involves generalizing from large amounts of training data, not just memorizing individual pieces of it. Second, memorization occurs when a model is trained; it is not something that happens when a model generates a regurgitated output. Regurgitation is a symptom of memorization in the model, not its cause. Third, when a model has memorized training data, the model is a “copy” of that training data in the sense used by copyright law. Fourth, a model is not like a VCR or other general-purpose copying technology; it is better at generating some types of outputs (possibly including regurgitated ones) than others. Fifth, memorization is not just a phenomenon that is caused by “adversarial” users bent on extraction; it is a capability that is latent in the model itself. Sixth, the amount of training data that a model memorizes is a consequence of choices made in the training process; different decisions about what data to train on and how to train on it can affect what the model memorizes. Seventh, system design choices also matter at generation time. Whether or not a model that has memorized training data actually regurgitates that data depends on the design of the overall system: developers can use other guardrails to prevent extraction and regurgitation. In a very real sense, memorized training data is in the model—to quote Zoolander, the files are in the computer.
  • Yotam Kaplan & Ayelet Gordon-Tapiero, Generative AI Training as Unjust Enrichment, Ohio St. L.J. __ (forthcoming 2025):
    • […] Unfortunately, the application of copyright law to the dilemma of generative AI training proves futile, leading to one of two extreme solutions. First, courts may find that generative AI training infringes on the copyright of human creators. If this is the case, human creators will be protected through injunctions and statutory damages, but this protection will spell the end of generative AI training. Second, courts can hold that generative AI training does not constitute copyright infringement but should instead be considered fair use. Under this alternative, human creators receive nothing. Ironically, this outcome is also detrimental to the further development of generative AI. If human creators are not compensated for their labor, and have insufficient incentive to create, generative AI will have less human-generated data to train on. Over time, if generative AI is increasingly trained on AI-generated content, this can lead to model collapse. Important to note, the fact that copyright law offers poor solutions to the generative AI dilemma is not surprising, as generative AI was nothing more than science fiction when copyright law was developed.Against this backdrop, this Article is the first to offer an alternative legal response to the dilemma of generative AI training, through the law of unjust enrichment. Unjust enrichment law can offer a middle ground solution between the two extreme responses offered by copyright law. Unjust enrichment law can provide better-tailored remedies that establish a layer of protection for human creators without paralyzing the market for generative AI. This can be done by providing human creators liability rule protection instead of property rule protection, meaning that human creators are compensated moderately, without being given the power to stop the training and development of generative AI. More broadly, unjust enrichment, as a residual and flexible legal framework which is not context-specific, is uniquely suited to be applied to new technologies and emerging problems. Unjust enrichment law also enjoys an advantage over copyright in terms of its interjurisdictional applicability. Thus, any limitation set on AI development under United States copyright law will mainly disadvantage American firms. Conversely, if American courts adopt liability under unjust enrichment for AI training, this regime can be enforced also against foreign firms. The Article studies additional advantages, implications, and limitations of the proposal to apply unjust enrichment doctrine to generative AI training, including preemption, the challenges created by lack of provenance, and the diversity of plaintiffs under this cause of action.
  • (*) Carys J. Craig, The AI-Copyright Trap, 100 Chi-Kent L. Rev. (forthcoming 2025):
    • As AI tools proliferate, policy makers are increasingly being called upon to protect creators and the cultural industries from the extractive, exploitative, and even existential threats posed by generative AI. In their haste to act, however, they risk running headlong into the Copyright Trap: the mistaken conviction that copyright law is the best tool to support human creators and culture in our new technological reality (when in fact it is likely to do more harm than good). It is a trap in the sense that it may satisfy the wants of a small group of powerful stakeholders, but it will harm the interests of the more vulnerable actors who are, perhaps, most drawn to it. Once entered, it will also prove practically impossible to escape. I identify three routes in to the copyright trap in current AI debates: first is the “if value, then (property) right” fallacy; second is the idea that unauthorized copying is inherently wrongful; and third is the resurrection of the starving artist trope to justify copyright’s expansion. Ultimately, this article urges AI critics to sidestep the copyright trap, resisting the lure of its proprietary logic in favor of more appropriate routes towards addressing the risks and harms of generative AI.
  • (*) Amanda Levendowski, How Copyright Law Can Fix Artificial Intelligence’s Implicit Bias Problem, 93 Wash. L. Rev. 579 (2018).
    • “While legal and computer science scholars have analyzed many sources of bias, including the unexamined assumptions of its often- homogenous creators, flawed algorithms, and incomplete datasets, the role of the law itself has been largely ignored. Yet just as code and culture play significant roles in how AI agents learn about and act in the world, so too do the laws that govern them. This Article is the first to examine perhaps the most powerful law impacting AI bias: copyright.
      “Artificial intelligence often learns to “think” by reading, viewing, and listening to copies of human works. This Article first explores the problem of bias through the lens of copyright doctrine, looking at how the law’s exclusion of access to certain copyrighted source materials may create or promote biased AI systems. Copyright law limits bias mitigation techniques, such as testing AI through reverse engineering, algorithmic accountability processes, and competing to convert customers. The rules of copyright law also privilege access to certain works over others, encouraging AI creators to use easily available, legally low-risk sources of data for teaching AI, even when those data are demonstrably biased. Second, it examines how a different part of copyright law—the fair use doctrine—has traditionally been used to address similar concerns in other technological fields, and asks whether it is equally capable of addressing them in the field of AI bias. The Article ultimately concludes that it is, in large part because the normative values embedded within traditional fair use ultimately align with the goals of mitigating AI bias and, quite literally, creating fairer AI systems.”
  • MAI_v._Peak_991_F.2d_511, 991 F.2d 511 (9th Cir. 1993).
    • The Peak decision is controversial. Commentators have suggested that the 2nd Circuit and the Federal Circuit have issued contrary opinions on whether a RAM copy is an infringing copy, which it is argued creates a circuit split. (I personally am not sure if I read the cases that way.)  That said, one complexity is that the license in Peak limited rights that might otherwise have existed if the defendant had a legitimate copy.
      1. Assuming Peak is the law, does this have implications for how we train AIs?
      2. If Peak is a problem for AI training, should it be overruled or limited not to reach the training process?
  • Kieran McCarthy, How Can AI Models Legally Obtain Training Data?–Doe 1 v. GitHub, Technology & Marketing Blog (June 17, 2023), summarizing Doe v. Github, Order Granting In Part And Denying In Part Motions To Dismiss, 2023 WL 3449131 (N.D. Cal., May 11, 2023).

Training Data Angsting

  • Melissa Heikkilä & Stephanie Arnett, This is where the data to build AI comes from, MIT Tech. Rev. (December 18, 2024):
    • The Data Provenance Initiative, a group of over 50 researchers from both academia and industry … audited nearly 4,000 public data sets spanning over 600 languages, 67 countries, and three decades. The data came from 800 unique sources and nearly 700 organizations.
      Their findings … show a worrying trend: AI’s data practices risk concentrating power overwhelmingly in the hands of a few dominant technology companies.
  •  Cade Metz et al, How Tech Giants Cut Corners to Harvest Data for A.I., NYT (Apr. 8, 2024): “OpenAI, Google and Meta ignored corporate policies, altered their own rules and discussed skirting copyright law as they sought online information to train their newest artificial intelligence systems.
  • James O’Donnell, Training AI music models is about to get very expensive, MIT Tech. Rev. (June 27, 2024): “Record labels are suing two leading AI startups and are reportedly exploring a licensing deal with YouTube—actions that will determine what the future of AI music sounds like, and if there is one at all.”

The Fair Use Debate

  • (*) BJ Ard, Copyright’s Latent Space: Generative AI and the Limits of Fair Use, 110 Cornell L. Rev. __ (forthcoming 2025):
    • Generative AI poses deep questions for copyright law because it defies the assumptions behind existing legal frameworks. While fair use is at the heart of the ongoing debate, the doctrine struggles because several features of generative systems take them beyond the reckoning of settled law. This Article takes up the fair use question to expose copyright’s limitations as well as its latent commitments, particularly its allowance for the exploitation of non-authorial value.Fair use’s transformative use paradigm, which compares the purpose of the use with that of the original work, faces difficulty because the indeterminate and modular nature of AI training processes renders the purpose of copying contingent. This hurdle can be overcome by recognizing that purpose serves as a proxy for determining whether the use intrudes on markets rightly belonging to the copyright owner. However, this raises the question of which markets those are.Answering this question requires delving into copyright’s latent space—the unarticulated principles and commitments embedded in its jurisprudence. This Article identifies a dividing line between value that stems from an author’s creative choices and value that does not, with courts permitting users to tap into the latter even to the copyright owner’s detriment. The reoriented test would ask whether a user exploits non-authorial value like that which stems from facts, tropes, and third-party investment versus the authorial value arising from an artist’s creative decisions. The precise line remains to be hashed out—courts have historically drawn the line differently across creative contexts to calibrate the scope of protection with copyright’s goals.The fair use question also reveals deeper structural limitations of the copyright regime. The argument that fair use should be denied to vindicate copyright policy misses that AI systems trained on licensed works may still displace human creators. The lack of unauthorized use takes the problem outside copyright’s domain. The core problem is not the duplication of specific works, but the ability to produce comparable works more cheaply and quickly. The challenge cannot be resolved through the mere extension or denial of fair use, and demands we put copyright in dialog with other regimes for promoting the arts, blunting the misuse of these tools, and confronting the technology’s capacity to consolidate power.
  • Jacqueline Charlesworth, Generative AI’s Illusory Case for Fair Use, 27 Van. J. Ent. & Tech L. __ (forthcoming 2025):
    • Pointing to Google Books, HathiTrust, Sega and other technology-driven fair use precedents, AI companies and those who advocate for their interests claim that mass unauthorized reproduction of books, music, photographs, visual art, news articles and other copyrighted works to train generative AI systems is a fair use of those works. Though acknowledging that works are copied without permission for the training process, the proponents of fair use maintain that an AI machine learns only uncopyrightable information about the works during that process. Once trained, they say, the model does not comprise or make use of the content of the training works. As such, they contend, the copying is a fair use under U.S. law.This article challenges the above narrative by reviewing generative AI training and functionality. Despite wide employment of anthropomorphic terms to describe their behavior, AI machines do not learn or reason as humans do. They do not “know” anything independently of the works on which they are trained, so their output is a function of the copied materials. Large language models, or LLMs, are trained by breaking textual works down into small segments, or “tokens” (typically individual words or parts of words) and converting the tokens into vectors-numerical representations of the tokens and where they appear in relation to other tokens in the text. The training works thus do not disappear, as claimed, but are encoded, token by token, into the model and relied upon to generate output. AI image generators are trained through a “diffusion” process in which they learn to reconstruct particular training images in conjunction with associated descriptive text. Like an LLM, an AI image generator relies on encoded representations of training works to generate its output.The exploitation of copied works for their intrinsic expressive value sharply distinguishes AI copying from that at issue in the technological fair use cases relied upon by AI’s fair use advocates. In these earlier cases, the determination of fair use turned on the fact that the alleged infringer was not seeking to capitalize on expressive content-exactly the opposite of generative AI.Generative AI’s claim to fair use is further hampered by the propensity of models to generate copies and derivatives of training works, which are presumptively infringing. In addition, some AI models rely on retrieval-augmented generation, or RAG, which searches out and copies materials from online sources without permission to respond to user prompts (for example, a query concerning an event that postdates the training of the underlying model). Here again, the materials are being copied and exploited to make use of expressive content.For these and other reasons, each of the four factors of section 107 of the Copyright Act weighs against AI’s claim of fair use, especially when considered against the backdrop of a rapidly evolving market for licensed use of training materials.
  • Mark Lemley & Brian Casey, Fair Learning (Jan. 30, 2020):
    • “It’s not clear that the use of … databases of copyrighted works to build self-driving cars, or to learn natural languages by analyzing the content in them, will be treated as a fair use under current law. … In this Article, we argue that AIs should generally be able to use databases for training whether or not the contents of that database are copyrighted. … Copyright law should permit copying of works for non-expressive purposes. When the defendant copies a work for reasons other than to have access to the protectable expression in that work, fair use should consider under both factors one and two whether the purpose of the defendant’s copying was to appropriate the plaintiff’s expression or just the ideas. We don’t want to allow the copyright on the creative pieces to end up controlling the unprotectable elements.”
  • (*) David Atkinson, Unfair Learning: GenAI Exceptionalism and Copyright Law (Jan 7, 2025):
    • This paper challenges the argument that generative artificial intelligence (GenAI) is entitled to broad immunity from copyright law for reproducing copyrighted works without authorization due to a fair use defense. It examines fair use legal arguments and eight distinct substantive arguments, contending that every legal and substantive argument favoring fair use for GenAI applies equally, if not more so, to humans. Therefore, granting GenAI exceptional privileges in this domain is legally and logically inconsistent with withholding broad fair use exemptions from individual humans. It would mean no human would need to pay for virtually any copyright work again. The solution is to take a circumspect view of any fair use claim for mass copyright reproduction by any entity and focus on the first principles of whether permitting such exceptionalism for GenAI promotes science and the arts.

Exemptions for AI Training

  • (*) Matthew Sag & Peter K. Yum, The Globalization of Copyright Exceptions for AI Training, 74 Emory L.J. __ (forthcoming 2025):
    • Generative AI, machine learning and other computational uses of copyrighted works pose profound questions for copyright law. This article conducts a global survey of multiple countries with different legal traditions and local conditions to explore how they have attempted to answer these questions in relation to the unauthorized use of copyrighted works for AI training.Although the world has yet to achieve international consensus on this issue, an international equilibrium is emerging. Jurisdictions with common law and civil law traditions, and with varying economic conditions, technological capabilities, political systems and cultural backgrounds, have found ways to reconcile copyright law and AI training. In this equilibrium, countries recognize that text and data mining, computational data analysis and AI training can be socially valuable and may not inherently prejudice the copyright holders’ legitimate interests. Such uses should therefore be allowed without express authorization in some, but not all, circumstances.We identify three forces driving toward this equilibrium: (1) the centrality of the idea-expression distinction; (2) global competition in AI; and (3) the race to the middle. However, we also address factors that may upset this emerging equilibrium, including ongoing copyright litigation, partnerships and licensing deals in the United States as well as legislative and regulatory efforts in both the United States and the European Union, including the EU AI Act.A key lesson of our cross-country survey is that globally, the binary policy debate that assumes that text and data mining and AI training must be categorically condemned or applauded has been eclipsed by a more granular debate about the specific circumstances in which the unauthorized use of copyrighted works for AI training should be allowed or prohibited. Countries that have hesitated until now to modernize their copyright laws in the area of AI training have several templates open to them and little reason for hesitation.
  • David W. Opderbeck, Copyright in AI Training Data: A Human-Centered Approach, 78 Okla. L. Rev 961 (2024):
    • This paper challenges the view among some copyright scholars that the use of unlicensed copyrighted material to train AI systems is a form of non-expressive fair use. Scholars and technologists who argue for broad non-expressive fair use in AI training data are reflect an outdated vision of Internet exceptionalism that fails to account for developing ethical and AI policy norms. In contrast, a licensing regime for crawling and scraping copyrighted works to use in AI training would foster the growth of training data markets, improve data quality, and serve the values of AI fairness, transparency, accountability, and privacy. In addition to these traditional copyright arguments, many advocates suggest that training an AI is like educating a human being. In response to this intuition, this paper explore various views in machine ethics on the status of AIs and argue from a eudemonistic perspective that AI technology should serve human interests. This human-centered approach, this paper argues, belies the claim that AI proprietors should have free, unfettered access to copyrighted works.
  • Deven R. Desai & Mark Riedl, Between Copyright and Computer Science: The Law and Ethics of Generative AI (Mar 7, 2024):
    • This Article shows that, contrary to some scholars’ views, fair use law does not bless all ways that someone can gain access to copyrighted material even when the purpose is fair use. Nonetheless, the scientific need for more data to advance AI research means access to large book corpora and the Open Internet is vital for the future of that research. The copyright industry claims, however, that almost all uses of copyrighted material must be compensated, even for non-expressive uses. The Article’s solution accepts that both sides need to change. It is one that forces the computer science world to discipline its behaviors and, in some cases, pay for copyrighted material. It also requires the copyright industry to abandon its belief that all uses must be compensated or restricted to uses sanctioned by the copyright industry. As part of this re-balancing, the Article addresses a problem that has grown out of this clash and under theorized.The exuberance, rush-to-market, and edge problem cases created by a few misguided companies now raises challenges to core legal doctrines and may shift Open Internet practices for the worse. Legal doctrine and scholarship have not solved what happens if a company ignores Website code signals such as “robots.txt” and “do not train.” In addition, companies such as the New York Times now use terms of service that assert you cannot use their copyrighted material to train software. Drawing the doctrine of fair access as part of fair use that indicates researchers may have to pay for access to books, we show that same logic indicates such signals and terms should not be held against fair uses of copyrighted material on the Open Internet.In short, this Article rebalances the equilibrium between copyright and computer science for the age of AI.
  • Frank Pasquale & Haochen Sun, Consent and Compensation: Resolving Generative AI’s Copyright Crisis (May 14, 2024):
    • Generative artificial intelligence (AI) has the potential to augment and democratize creativity. However, it is undermining the knowledge ecosystem that now sustains it. Generative AI may unfairly compete with creatives, displacing them in the market. Most AI firms are not compensating creative workers for composing the songs, drawing the images, and writing both the fiction and non-fiction books that their models need in order to function. AI thus threatens not only to undermine the livelihoods of authors, artists, and other creatives, but also to destabilize the very knowledge ecosystem it relies on.Alarmed by these developments, many copyright owners have objected to the use of their works by AI providers. To recognize and empower their demands to stop non-consensual use of their works, we propose a streamlined opt-out mechanism that would require AI providers to remove objectors’ works from their databases once copyright infringement has been documented. Those who do not object still deserve compensation for the use of their work by AI providers. We thus also propose a levy on AI providers, to be distributed to the copyright owners whose work they use without a license. This scheme is designed to ensure creatives receive a fair share of the economic bounty arising out of their contributions to AI. Together these mechanisms of consent and compensation would result in a new grand bargain between copyright owners and AI firms, designed to ensure both thrive in the long-term.

Notes & Questions

  1. We might begin with a very basic issue: Why have copyright law (or patent law) at all?
    1. It makes sense (maybe) to go back to basics, since our answers to whether and how we want to apply copyright law (or patent law) to things produced by AIs might be shaped by what we think those bodies of law are trying to do.
    2. Consider, in this context, the Craig/Kerr discussion (in optional readings) of what copyright is for. They identify two very different strands of justification:
      1. The “deontological” approach (if that word is not familiar to you, here’s a good introductory guide aimed at 1Ls) which “offers natural rights-based justification for the author’s entitlement to preside over their work as owner.” Further, “Deontological rights-based theories in turn break down into two available, and potentially oppositional, alternatives. The first justification is premised on the mental labour of the author, typically framed in traditional Lockean terms that speak to the author’s right to appropriate the fruits of their mental labour and to exclude others who might seek to benefit from their pains. Across the philosophical aisle are adherents to a personality-based justification for the author’s rights, typically framed in Hegelian, but sometimes Kantian, terms, speaking to the author’s right to own — again as a matter of natural justice — their speech or work that bears the imprint of their unique personality, the externalization of their will in the world.”
      2. Alternately, there is the “teleological” approach “that, true to form, seeks to justify copyright through an instrumental or consequentialist logic, rationalizing the author’s control over their work as a means to a larger (social) end. …  Across the greater ideological divide, by far the most dominant version of a teleological approach is the US utilitarian  framing, which leans on economic theory to explain copyright as an incentive system to advance (in the words of the US Constitution) “the Progress of Science and the useful Arts.”
    3. Another way to describe the cleavage in the literature is between the “romantic conception of the author” (an image that seems often to involve a garret) and the reality of corporate authorship/ownership, either by transfer of copyright by that starving artist, or by work-for-hire.
    4. Sag argues there is a debate between
      1. a more common view that holds a prime purpose of copyright law is to protect authors (or their pocketbooks) against illegal and unsanctioned copies (this seems deontological)
      2. his view that the real purpose of copyright law is to encourage and protect unique expressions; this seems teleological, but he says this isn’t utilitarian.
  2. Given these deep philosophical waters, what is a regulator like the USPTO supposed to do?
  3. Do you buy this story: The Copyright story is as an example of what happens when law meets new tech:
    1. Lawyers try to fit the new thing into existing categories
    2. Other lawyers use the new tech as an opportunity to rethink the existing categories
    3. Meanwhile we have to deal with some practical questions such as:
      1. To what extent old theories should control new outcomes
      2. To the extent that we don’t use old theories for new stuff, should we use the new theories for the old stuff?
  4. Regardless of how we read the Copyright Act on the issue of whether using web materials for training is legal, what should the rule be? 
    1. Why?
    2. Who should decide?  (If you said “Congress,” and it’s hopelessly gridlocked is there any other legitimate way to make the decision?)

Class 12 IP2 (Primarily usage-related copyright issues)

Apologies to Patent geeks, but all the patent stuff is in the optional readings.

  1. Section 5.4.2 (“Output liability”)  (page 147) of Rishi Bommasani et al,, Stanford HA, On the Opportunities and Risks of Foundation Models (2023).
  2. Samantha Cole, Musicians Algorithmically Generate Every Possible Melody, Release Them to Public Domain, VICE (Feb. 22, 2020).
  3. 1-5 (edited) & Pages 31 & 50-70 from Katherine Lee, A. Feder Cooper, James Grimmelmann, Talkin’ ‘Bout AI Generation: Copyright and the Generative-AI Supply Chain, Journal of the Copyright Society (forthcoming, 2024).
  4. Pages ii & iii (the Executive Summary) of U.S. Copyright Office, Copyright and Artificial Intelligence.  Part 2: Copyrightability.  (Full report is here.)
  5. Pages 120-28 from Katherine Lee, A. Feder Cooper, James Grimmelmann, Talkin’ ‘Bout AI Generation: Copyright and the Generative-AI Supply Chain, Journal of the Copyright Society (forthcoming, 2024).
  6. Early legal answers:
    1. Excerpted selected decisions of of the Copyright Office  (2024), from  Pp. 9-15 of Guy Rub, The Rise of Generative Artificial Intelligence (2024)
    2. Also consider the “monkey selfie” case.  (Optional: full text of Naruto v. Slater, 888 F.3d 418 (9th Cir. 2018).)
    3. Annelise Levy, Anthropic Agrees to Enforce Copyright Guardrails on New AI Tools, Bloomberg Law (Dec. 30, 2024)

Optional

Generally

  • (*) Steven D. Jamar, A Social Justice Perspective on IP Protection for Artificial Intelligence Programs (December 23, 2021) in Cambridge Handbook on Intellectual Property and Social Justice (Steven D. Jamar and Lateef Mtima, eds. forthcoming 2022 ).
    • “We are just at the beginning of the AI revolution and already it is transforming human society. In history only four other technologies have transformed human life as profoundly as AI will: steam power that powered the industrial revolution; electricity that transformed everything; the development of antibiotics and medical understanding and treatment generally; and computers and pre-AI software creating the information age. As we move into the AI world, we face significant challenges with respect to shaping our world. A significant part of that shaping will be determined by how we collectively choose to reward and regulate AI implementations. And a part of that shaping will depend upon how we address intellectual property issues connected to artificial intelligence applications. This chapter sketches some of the concerns and suggests a path forward[.]”
    • Part I c – d is an excellent summary of different types of AI and their limits.  Part II describes issues about who will gain and lose when AI meets IP; Part III has (too brief?) recommendations for a federal regulatory sui generis scheme to respond to the issues in Part II.
  • Pamela Samuelson, Thinking About Possible Remedies in the Generative AI Copyright Cases, 67 Comm. ACM 27 (July 2024):
    • The sixteen lawsuits brought to date against OpenAI and other developers of generative AI technologies include claims that making copies of in-copyright works for purposes of training generative AI models infringes copyrights. This column considers the remedies being sought in the generative AI copyright complaints. Three types of remedy claims merit special attention: claims for awards of statutory damages; court orders to destroy models trained on infringing works; and most bizarrely, a court order to establish a regulatory regime to oversee generative AI system operations. This column also considers a collective license approach for authorizing use of in-copyright works as training data, one of many issues the U.S. Copyright Office is currently examining as part of its study on AI technology and copyright. While we may not learn the outcomes of the various lawsuits for quite some time, it is worth thinking now about the high stakes involved in the ongoing development of generative AI systems.
  • Edward Lee, The Code Red for Copyright Law, 76 Fla. L. Rev. Forum (2024):
    • On March 16, 2023, the U.S. Copyright Office issued a new Guidance on Works Containing Material Generated by Artificial Intelligence (“AI Guidance”) that went into immediate effect. This Essay explains why the Office’s AI Guidance is fundamentally flawed on both procedural and substantive grounds. First, because the AI Guidance contains new substantive rules on the copyrightability of the entire class of AI-generated works, requiring applicants to disclose and to disclaim all AI-generated works from registration, the Copyright Office’s failure to hold a period of notice and comment before issuing the new AI Guidance likely violated the Administrative Procedure Act. More fundamentally, the Copyright Office’s newfound requirement of the so-called “traditional elements of authorship”—a term that no federal court has ever recognized in the history of copyright law—finds no support in the Constitution, the Copyright Act, or the Supreme Court’s precedents. Not only is such a rigid approach limiting copyright to “traditional” authorship contrary to the text of the Progress Clause and the Framers’ rejection of restricting “Authors” to “literary Authors,” it also threatens to disadvantage U.S. creators the most, at a time when other countries are vying to upend the United States as the world leader in AI development. The federal courts should give no deference to and ultimately reject the Copyright Office’s erroneous rule of “traditional elements of authorship.” The Office’s mistaken approach is contrary to the Progress Clause and the Supreme Court’s precedents on “Authors,” which includes authorship that lies in a minimally creative selection or arrangement of elements. The Copyright Office has no authority to ratchet up this constitutional standard.
  • Martin Senftleben, Win-win: How to Remove Copyright Obstacles to AI Training While Ensuring Author Remuneration (and Why the European AI Act Fails to Do the Magic), 98 Chi-Kent L.Rev. (2024):
    • In the debate on AI training and copyright, the focus is often on the use of protected works during the AI training phase (input perspective). To reconcile training objectives with authors’ fair remuneration interest, however, it is advisable to adopt an output perspective and focus on literary and artistic productions generated by fully-trained AI systems that are offered in the marketplace. Implementing output-based remuneration systems, lawmakers can establish a legal framework that supports the development of unbiased, high quality AI models while, at the same time, ensuring that authors receive a fair remuneration for the use of literary and artistic works for AI training purposes – a fair remuneration that softens displacement effects in the market for literary and artistic creations where human authors face shrinking market share and loss of income. Instead of imposing payment obligations and administrative burdens on AI developers during the AI training phase, output-based remuneration systems offer the chance of giving AI trainers far-reaching freedom. Without exposing AI developers to heavy administrative and financial burdens, lawmakers can permit the use of the full spectrum of human literary and artistic resources. Once fully developed AI systems are brought to the market, however, providers of these systems are obliged to compensate authors for the unbridled freedom to use human creations during the AI training phase and displacement effects caused by AI systems that are capable of mimicking human literary and artistic works.As the analysis shows, the input-based remuneration approach in the EU – with rights reservations and complex transparency rules blocking access to AI training resources – is likely to reduce the attractiveness of the EU as a region for AI development. Moreover, the regulatory barriers posed by EU copyright law and the AI Act may marginalize the messages and values conveyed by European cultural expressions in AI training datasets and AI output. Considering the legal and practical difficulties resulting from the EU approach, lawmakers in other regions should refrain from following the EU model. As an alternative, they should explore output-based remuneration mechanisms. In contrast to the burdensome EU system that requires the payment of remuneration for access to human AI training resources, an output-based approach does not weaken the position of the domestic high-tech sector: AI developers are free to use human creations as training material. Once fully developed AI systems are offered in the marketplace, all providers of AI systems capable of producing literary and artistic output are subject to the same payment obligation and remuneration scheme – regardless of whether they are local or foreign companies. The advantages of this alternative approach are evident. Offering broad freedom to use human creations for AI training, an output-based approach is conducive to AI development. It also bans the risk of marginalizing the messages and values conveyed by a country’s literary and artistic expressions.
  •  Bruce E. Boyden, Emergent Works, 39 Col. J.L & Arts (2016):
    • We are on the cusp of a very significant change. Programs can now generate music that is commercially viable as background music, or write poetry that is difficult to distinguish from that written by humans.7 They can write simple news stories, such as breaking news about earthquakes or sports.8 They can generate personalized reports from databases, which is essentially what a Google search result is. Or they can create automated videos, as Facebook has done for its users occasionally.9 And they are doing so for financial gain.What is perplexing about these situations is that they seem to be instances in which we have something that looks like a “work,” but there may be no person whose actions resemble those of a traditional “author.” That is, computergenerated works pose a problem for what might be called the “standard model” of copyright law, under which a person, the author, produces a work that is then conveyed to the audience through some sort of medium such as a book. With computer-generated works, the production and conveyance steps become intertwined, with one work—the computer program—producing another work each time it is used. Those subsequent works may contain creative elements that were not present in the first, and thus lack an easily identifiable human origin. When it comes to what might be “emergent works”—works that consist largely of creative elements that have emerged unbidden from the operation of the program—who should be considered the author?
  • Jesse Feitel, Nicolas Jampol and James Rosenfeld, Tenn. Law Protecting Artists From AI Raises Novel Issues, Law 360 (Apr. 17, 2024):
    • In late March, the Volunteer State — which for decades has afforded an expansive right of publicity protection for names and likenesses, but not voices — further extended that protection to individuals’ voices in light of the increased popularity and accessibility of AI-generated audio tracks, while at the same time memorializing First Amendment and fair use as an express exemption.

Copyright

  • Ted talk on ‘copyrighting all the music’…

Is a Human Author Required?

  • (*) Christian Mammen et al, Creativity, Artificial Intelligence, and the Requirement of Human Authors and Inventors in Copyright and Patent Law (Jul 17, 2024):
    • Copyright and patent law require the identification of an author or inventor, and further require the author or inventor to be human. We explore this requirement primarily with reference to U.S. law and provide additional illustrations from U.K. and E.U. law. A key rationale underlying the requirement of a human author or inventor is that there is something special and important about human creativity. As AI, particularly generative AI, becomes more capable of producing outputs that look like they could have been human-created, arguments have increasingly been raised that the AI-generated outputs should be afforded copyright and patent protection, on the same basis as those made by human authors and inventors. And there have been arguments that these AI-generated outputs exhibit sufficient creativity, novelty, or innovativeness, to satisfy the laws’ underlying creativity rationale. We examine the concept of creativity from a multidisciplinary perspective, and identify three conceptually distinct components, all of which are necessary for a complete account of creativity. The external component refers to whether an artifact (or idea, or other thing) exhibits the qualities of being novel, valuable, and (on some accounts) surprising. The subjective component focuses on the psychological process of a creative act, which appears to involve a dance between task-focused and mental-wandering states, mediated by a salience functionality, where the person recognizes and selects novel, appropriate ideas. Third, embedded in the analysis of both the external and subjective components is a (largely-implicit) recognition that the social context is integral to creativity; it plays a role in determining whether an artifact has value (or is “appropriate”), and influences the subjective psychological process of plucking certain ideas or conceptions out of the flow of mental activity. With this enriched account of creativity, we examine how copyright and patent law value not only the creativity of the artifact, but also (to varying extents) the subjective role and social context as part of creativity. We then consider some ways in which arguments that AI generated artifacts should be eligible for IP protection (e.g., because they are “just as good as” at least some human-generated and IP-eligible artifacts) are insufficient to satisfy the enriched understanding of the creativity requirement underlying the IP laws.
  • (*) David Atkinson, Jena D. Hwang & Jacob Morrison, Intentionally Unintentional: GenAI Exceptionalism and the First Amendment, First Amendment Law Review (forthcoming 2025):
    • This paper challenges the assumption that courts should grant outputs from large generative AI models, such as GPT-4 and Gemini, First Amendment protections. We argue that because these models lack intentionality, their outputs do not constitute speech as understood in the context of established legal precedent, so there can be no speech to protect. Furthermore, if the model outputs are not speech, users cannot claim a First Amendment right to receive the outputs. We also argue that extending First Amendment rights to AI models would not serve the fundamental purposes of free speech, such as promoting a marketplace of ideas, facilitating self-governance, or fostering self-expression. In fact, granting First Amendment protections to AI models would be detrimental to society because it would hinder the government’s ability to regulate these powerful technologies effectively, potentially leading to the unchecked spread of misinformation and other harms.
  • Jake L. Bryant, Auctorem Ex Machina: A Case for Human Copyright Authorship in Works Created Using Generative Artificial Intelligence, 52 Am. Intel. Prop. L. Assoc. Q.J. (Summer 2024)
    • Ours is an era of artificial relationships, an age where man is as connected to the machine as he is to other people. Recent developments in deep learning technology have posed a significant question to copyright jurisprudence: are works created using generative artificial intelligence (AI) the product of an author under copyright law? In this article, I examine the nature of generative AI models, the differences between large language models (LLMs) and large visual models (LVMs), and the history of authorship in U.S. copyright law before proposing a general test for authorship and qualifying prompt engineers as authors of certain works created using generative AI. The ultimate test for authorship must consider two factors derived from historical precedent. This test first asks whether the underlying work, in the event human authorship exists, meets the Feist standard of originality. The second factor considers the copyright claimant’s creative control over the conception, whether the work is a reasonable manifestation of the putative author’s intent. When a work is created using marginally supervised automated processes and contains subject matter not staged by the putative author, the determinative question is whether the copyright claimant’s creative inputs establish a limited universe of foreseeable results. In the case of visual works created with generative AI, courts can look to (1) the level of specificity and detail in the prompt text; (2) whether any additional parameters are used; (3) any use of job seeding to control the starting point of the image diffusion process; (4) the use of variation or regional variation buttons in creating the final image; (5) and the similarity of the final image to the prompt description as creative inputs to be considered in determining authorial control. Not every image created using generative AI will pass muster. If the prompt used is so abstract and lacking in additional parameters such that it amounts to little more than an idea, then demonstrating that the putative author’s intent was to generate an image substantially encapsulating the elements of the final chosen work is near impossible. However, many works generated according to highly specific, detailed prompts merit copyright protection.

Interaction With License Terms

  • (*) Camilla Alexandra Hrdy, Keeping ChatGPT a Trade Secret While Selling It Too, Berk. Tech. L.J. (forthcoming 2025):
    • Generative artificial intelligence products such as ChatGPT raise novel issues for trade secret law. But one of the most important issues is an old one: How to sell an information good, like computer software, while also maintaining trade secrecy protection for the underlying content? When a company wishes to sell a new technology to the public, the normal recourse is to obtain a patent. Patents require public disclosure and end after a fixed term of years. However, based on decades of precedents established for software, generative AI companies will be able to rely on trade secret law instead-maintaining indefinite protection for their technology, even as they profit from making it widely available to the public, and even after reverse engineering becomes technically feasible. This is what many companies did with closed-source software, and this is what developers of some generative AI models-including ChatGPT-are doing today. They are releasing the models in a “closed-source” format that hides algorithms, code, training data, and underlying model architecture from users. And they are attaching contractual provisions—called “terms of use” or “end user license agreements” (EULAs)—that limit users’ ability to reverse engineer information about how the models work or share that information with others. Some of these agreements, including ChatGPT’s, even contain noncompete provisions. If liability for breaching these provisions were limited to breach of contract, there would be less cause for alarm. However, some case law—and some state statutes—indicate that reverse engineering trade secrets in breach of an anti-reverse-engineering clause can give rise to trade secret liability as well, because breach of the contract transforms otherwise-lawful reverse engineering into an “improper means” of acquiring trade secrets. The prospect of trade secret liability for what should be, at worst, breach of contract is alarming. It means prevailing plaintiffs can obtain trade secret law remedies, not just contract law remedies, and it means that liability can extend to third parties who did not even sign the contract. For example, if someone reverse engineers information about ChatGPT in violation of a boilerplate terms of use, and then shares that information with someone else, who publishes the information on the internet, both of these actors could be liable for trade secret misappropriation. Fortunately, there is a solution. In the Defend Trade Secrets Act (DTSA) of 2016, Congress made clear that reverse engineering is legal under federal trade secret law and cannot be considered an “improper means” of acquiring a trade secret. The mere presence of a contract purporting to prohibit reverse engineering cannot change this rule. A state law that holds otherwise is preempted by federal trade secret law pursuant to the Supremacy Clause of the Constitution. The upshot is that, in many circumstances, reverse engineering a publicly-distributed generative AI model—or a traditional software product—is not trade secret misappropriation, regardless of the presence of a boilerplate anti-reverse-engineering clause. This doctrinal approach will make sure that, once a widely-available product can easily and cheaply be reverse engineered by members of the general public, companies cannot maintain trade secret protection indefinitely through contract.
  • (*) Kacper Szkalej & Martin Senftleben, Generative AI and Creative Commons Licences:The Application of Share Alike Obligations to Trained Models, Curated Datasets and AI Output (Jun 26, 2024):
    • This article maps the impact of Share Alike (SA) obligations and copyleft licensing on machine learning, AI training, and AI-generated content. It focuses on the SA component found in some of the Creative Commons (CC) licences, distilling its essential features and layering them onto machine learning and content generation workflows. Three fundamental challenges related to the life cycle of such licences emerge from the analysis. These include tracing and establishing copyright-relevant uses during the development phase (training), the interplay of licensing conditions with copyright exceptions and the identification of copyright-protected traces in AI output. Significant problems can arise from several concepts in CC licensing agreements (‘adapted material’ and ‘technical modification’) that could serve as a basis for applying SA conditions to trained models, curated datasets and AI output that can be traced back to CC material used for training purposes. Seeking to transpose Share Alike and copyleft approaches to the world of generative AI, the CC community has basically the choice between two policy approaches. On the one hand, it can uphold the supremacy of copyright exceptions. In countries and regions that exempt machine-learning processes from the control of copyright holders, this approach leads to far-reaching freedom to use CC resources for AI training purposes. At the same time, it marginalises SA obligations. On the other hand, the CC community can use copyright strategically to extend SA obligations to AI training results and AI output. To achieve this goal, it is necessary to use rights reservation mechanisms, such as the opt-out system available in EU copyright law, and subject the use of CC material in AI training to SA conditions. Following this approach, a tailor-made licence solution can grant AI developers broad freedom to use CC works for training purposes. In exchange for the training permission, however, AI developers would have to accept the obligation to pass on-via a whole chain of contractual obligations-SA conditions to recipients of trained models and end users generating AI output.
  • (*) Kacper Szkalej & Martin Senftleben, Generative AI and Creative Commons Licences:The Application of Share Alike Obligations to Trained Models, Curated Datasets and AI Output, Jun 26, 2024:
    • This article maps the impact of Share Alike (SA) obligations and copyleft licensing on machine learning, AI training, and AI-generated content. It focuses on the SA component found in some of the Creative Commons (CC) licences, distilling its essential features and layering them onto machine learning and content generation workflows. Three fundamental challenges related to the life cycle of such licences emerge from the analysis. These include tracing and establishing copyright-relevant uses during the development phase (training), the interplay of licensing conditions with copyright exceptions and the identification of copyright-protected traces in AI output. Significant problems can arise from several concepts in CC licensing agreements (‘adapted material’ and ‘technical modification’) that could serve as a basis for applying SA conditions to trained models, curated datasets and AI output that can be traced back to CC material used for training purposes. Seeking to transpose Share Alike and copyleft approaches to the world of generative AI, the CC community has basically the choice between two policy approaches. On the one hand, it can uphold the supremacy of copyright exceptions. In countries and regions that exempt machine-learning processes from the control of copyright holders, this approach leads to far-reaching freedom to use CC resources for AI training purposes. At the same time, it marginalises SA obligations. On the other hand, the CC community can use copyright strategically to extend SA obligations to AI training results and AI output. To achieve this goal, it is necessary to use rights reservation mechanisms, such as the opt-out system available in EU copyright law, and subject the use of CC material in AI training to SA conditions. Following this approach, a tailor-made licence solution can grant AI developers broad freedom to use CC works for training purposes. In exchange for the training permission, however, AI developers would have to accept the obligation to pass on-via a whole chain of contractual obligations-SA conditions to recipients of trained models and end users generating AI output.
  • (*) Mark A. Lemley, How Generative AI Turns Copyright Upside Down (Draft Jan 18, 2024):
    • While courts are litigating many copyright issues involving generative AI, from who owns AI-generated works to the fair use of training to infringement by AI outputs, the most fundamental changes generative AI will bring to copyright law don’t fit in any of those categories. The new model of creativity generative AI brings puts considerable strain on copyright’s two most fundamental legal doctrines: the idea-expression dichotomy and the substantial similarity test for infringement. Increasingly creativity will be lodged in asking the right questions, not in creating the answers. Asking questions may sometimes be creative, but the AI does the bulk of the work that copyright traditionally exists to reward, and that work will not be protected. That inverts what copyright law now prizes. And because asking the questions will be the basis for copyrightability, similarity of expression in the answers will no longer be of much use in proving the fact of copying of the questions. That means we may need to throw out our test for infringement, or at least apply it in fundamentally different ways.
      […] There is an old aphorism that says that if you give an infinite number of monkeys typewriters they will eventually create the works of Shakespeare. In theory, in the copyright case of Shakespeare v. Infinite Monkeys, Shakespeare should lose, because the monkeys didn’t copy from him. In practice, courts have generally found a way for similarity to trump everything else. But that will need to change in the world of generative AI. Our basic copyright doctrines don’t fit generative AI. We will struggle to apply the law to comport with the new realities. And it may suggest that copyright itself is a poor fit for the new world of AI-generated works.
  • (*) Highly recommended: Daniel J. Gervais, The Human Cause, in Research Handbooks on Intellectual Property and Artificial Intelligence (R. Abbott, ed, forthcoming Dec. 2022)
    • “The more we rely on machines to perform creative and innovative tasks that are singularly human and important for us as a source of progress–from news reporting, to music and fiction, to inventions in all fields of technology–the more we may shrink the space available for our individual and collective self-realization. To use a simple explanatory metaphor, as we use our creative and inventive ‘muscles’ less and less, they will shrink. Is that what is meant by ‘Progress of Science and Useful Arts’?”
  • (*)  Comment submitted by Professor Daniel Gervais,  to the Copyright Office’s notice of Inquiry on the interface between copyright and Generative AI:
    • There’s a lot here, but here’s a tiny sample: “I would urge the Office to resist the commonly held view that any and all disruption caused by AI companies and especially Big Tech is per se positive and must be allowed by law, and instead consider that a diminution of works created by human is commerce, from journalism to essays to novels, is not a clear positive.”
  • (*) Annemarie Bridy, Coding Creativity: Copyright and the Artificially Intelligent Author, 2012 Stan. Tech. L. Rev. 5 (2012):
    • “By engaging and extending insights from two relatively discrete lines of existing scholarship—the postmodern critique of romantic authorship and the more pragmatic literature on copyright in works produced with the aid of computers—this Article seeks to answer the vexing copyright questions that attend the artificially intelligent production of cultural works. It does so by developing the argument that all creativity is inherently algorithmic and that works produced autonomously by computers are therefore less heterogeneous to both their human counterparts and existing copyright doctrine than appearances may at first suggest.”
  • P. Bernt Hugenholtz . João Pedro Quintais, Copyright and Artificial Creation: Does EU Copyright Law Protect AI-Assisted Output?, 52 IIC 1190 (2021) .
    • “[F]our interrelated criteria are to be met for AI-assisted output to qualify as a protected “work”: the output is (1) in relation to “production in the literary, scientific or artistic domain”; (2) the product of human intellectual effort; and (3) the result of creative choices that are (4) “expressed” in the output. Whether the first step is established EU law is uncertain. Since most AI artefacts belong to the “literary, scientific or artistic domain” anyway, and are the result of at least some “human intellectual effort”, however remote, in practice the focus of the copyright analysis is on steps 3 and 4.  […] “Authorship status will be accorded to the person or persons that have creatively contributed to the output. In most cases this will be the user of the AI system, not the AI system developer, unless the developer and user collaborate on a specific AI production, in which case there will be co-authorship. If “off-the-shelf” AI systems are used to create content, co-authorship claims by AI developers will also be unlikely for merely commercial reasons, since AI developers will normally not want to burden customers with downstream copyright claims. We therefore expect this issue to be clarified in the contractual terms of service of providers of such systems.
      “In conclusion, we believe that the EU copyright framework is generally suitable and sufficiently flexible to deal with the current challenges posed by AI-assisted creation […]”
  • (*) Ryan Abbott & Elizabeth Rothman, Disrupting Creativity: Copyright Law in the Age of Generative Artificial Intelligence, 75 Fla. L. Rev. 1141 (2023).
    • “Before the end of the decade, a significant amount of art, literature, music, software, and web content will likely be created by AI rather than traditional human authors. Yet the law, as it has so often historically, lags this technological evolution by prohibiting copyright protection for AI-generated works. The predominant narrative holds that even if AI can automate creativity, that this activity is not the right sort of thing to protect, and that protection could even harm human artists.
      “AI-generated works challenge beliefs about human exceptionalism and the normative foundations of copyright law, which until now have offered something for everyone. Copyright can be about ethics and authors and protecting the sweat of a brow and personality rights. Copyright can also be about money and the public interest and offering incentives to create and disseminate content. But copyright cannot have it all with AI authors—there is valuable output being generated, but by authors with no interests to protect. This forces a reevaluation of copyright law’s history and purpose.
      “This article argues that American copyright law is, and has been traditionally, primarily about benefiting the public interest rather than benefiting authors directly. As a result, AI-generated works are precisely the sort of thing the system was designed to protect.”
  • Brent Moran, Brigitte Vézina, Creative Commons, Artificial Intelligence and Creativity: Why We’re Against Copyright Protection for AI-Generated Output (Aug. 10, 2020).
  • Sofia Moutinho, Kinky and absurd: The first AI-written play isn’t Shakespeare—but it has its moments, Science (Feb. 26, 2021). [IMHO, it’s mostly awful.]
  • Paul Sawers, Chinese court rules AI-written article is protected by copyright, VentureBeat (Jan. 10, 2020)
  • Chapter 12 of National Security Commission on Artificial Intelligence, Final Report (2021) gives an account of the strategic importance of IP. If you are interested in export control — a sort of IP issue — then you also might look at Chapter 14, but in either case keep in mind that this whole report is basically alarmist about Chinese efforts to “win” the AI race, probably because this is how you make a case to Congress for lots more budget….
  • Jaclyn Peiser, The Rise of the Robot Reporter, NY Times (Feb. 2, 2019).

Patent

  • Dueling visions of the future of patent law:
    1. (*) Ryan Abbott, Everything is Obvious, 66 UCLA L Rev. 2 (2018). The critical parts for us are II.C-E and III.C-F. If you have no patent background, however, you will need to read Part I to understand those section.
      1. Abbott argues that ML will make everything obvious, at least in a legal sense as the term is used in patent law. And thus, he claims we face the “end of patents.”
    2. (*) Dan L. Burk, AI Patents and the Self-Assembling Machine, 105 Minn. L. Rev. Headnotes 301 (2021).
      1. Burk replies to Abbott by saying that, “… commentary surrounding AI systems partakes of what has been called the ‘magical worldview,’ a somewhat breathless and overwrought perspective on technology that assumes AI systems are somehow transcendent or miraculously unprecedented in their qualities or applications. In the patent context, such hyperbole manifests itself in assertions that these technologies upend the patent system, defy long-established patent doctrines, or portend the end of innovation as we know it.” (citing to Abbott)
    3. (*) An attempt at a synthesis? Raina Haque, Simone Rose & Nick DeSetto, The Non-Obvious Razor & Generative AI, 25 N.C. J.L. & Tech. 399 (2024):
      1. This article examines the challenges and prospects of crowd-sourcing generative AI systems (“GenAI”) in patent law as human and machine creativity become seamless. As GenAI technology like ChatGPT-4 become ubiquitous, AI-generated solutions will be less innovative and will complicate tenets about patentability. An evolution of patent law’s non-obviousness standard provides an elegant solution – borrowing from philosophy, a “razor” – to addressing the impact of advanced AI in the innovation process.This article’s thesis is distinct from the USPTO’s emphasis on whether or not AI systems can be inventors, because it assumes that human and artificial creativity will become indistinguishable. This article focuses on a reevaluation of utility patent law’s non-obviousness standard in light of the steady societal shift toward broad information and technological empowerment. By exploring GenAI’s role in augmenting creativity and its implications for the standard of “ordinary creativity,” this article suggests factors for a revised patentability examination methodology.This reevaluation seeks to balance AI’s rapid advances with patent law’s goals to promote progress. There is precedent for the non-obviousness standard to absorb advancements in artificial intelligence that rely on crowd-sourced information. GenAI challenges traditional notions of invention and creativity. The legal construct against which “non-obviousness” is determined – the ordinary creativity of the “person of ordinary skill in the art” (PHOSITA) – should be recalibrated to account for GenAI and to encourage innovation while protecting public access to tools of creativity.
  • Matt Blaszczyk, Impossibility of Artificial Inventors, 16 Hastings Sci. & Tech. L.J. 73 (2024).
    • Recently, the United Kingdom Supreme Court decided that only natural persons can be considered inventors. A year before, the United States Court of Appeals for the Federal Circuit issued a similar decision. In fact, so have many of the courts all over the world. This Article analyzes these decisions, argues that the courts got it right, and finds that artificial inventorship is at odds with patent law doctrine, theory, and philosophy. This Article challenges the intellectual property (IP) post-humanists, exposing the analytical and normative perils of their argumentation, and recommends against getting rid of the nominally central place of humans in the law. This response to IP post-humanism rests in equal measure on patent doctrine, legal causation, and the mythology which creates and justifies the law.
  • Liza Vertinsky, Reorienting Patent Policy Towards Responsible AI Design, in Research Handbook on the Law of Artificial Intelligence (W. Barfield & U. Pagallo eds., 2d ed. 2025):
    • The potential for disconnects between existing legal rules and processes of innovation transformed by machine intelligence, or what we now call artificial intelligence (AI), has been mulled over for decades. Patent scholars continue to debate the implications of a “thinking machine” paradigm of invention, debates that have accelerated in the wake of advances in generative AI. But although the technological, political and economic landscape surrounding artificial intelligence (AI) and its impact on innovation continues to evolve, the legal framework governing the discovery and development of new inventions has changed relatively little.While discussions about the impact of AI on patent law are not new, the pressure for some kind of policy response has intensified. In the U.S. a national AI strategy has emerged that emphasizes the importance of incentivizing private sector AI innovation on the one hand, and the responsible and ethical governance of AI on the other. The national strategy also endeavors to expand public access to the data and computational resources needed to compete at the cutting edge of AI innovation. At the same time the public availability of generative AI tools has made the legal issues around invention and creation more salient and legal responses to questions of ownership and use more pressing.This chapter examines the ways in which a “thinking machine” paradigm of invention, with its direct challenge to a human-centric conception of invention, has shaped the academic and policy debates over whether and how patent law should respond to advances in AI. It argues that the academic debate has been too narrowly focused on questions of patentability of machine generated inventions, and the policy debate, one heavily influenced by the largest tech companies, too narrowly focused on strengthening private incentives to invent and innovate. It concludes by suggesting a reframing of the AI patent policy debate to focus on the role of patents in encouraging responsible AI design.
  • An early legal answer:  Thaler v Vidal, 43 F.4th 1207 (Fed. Cir. 2022).
  • Desmond Oriakhogba, Dabus Gains Territory in South Africa and Australia: Revisiting the AI-Inventorship Question, 9 South African Journal of Intellectual Property Law 87 (2021).

Notes & Questions

  1. The Bridy article (in the optional reading and well summarized by Craig/Kerr in their article in the optional reading for the previous class) was perhaps the first article to ask some tough questions about machine copyright. It suggests that the differences between “works produced autonomously by computers” are less different from human-generated works than they seem, and that the legal treatment of machined works need not be that different either. That said, she does see some issues on which you might form a view:
    1. Whether the constitutional definition of “author” in the copyright clause (U.S. Const. Art. I § 8) imposes a limit on copyrightable subject matter …. that might exclude AI “authors”?
    2. Whether even if the constitution allows Congress to give copyright to AI authors, we can fairly read existing copyright law (pre-AI!) to have done so?  (Cf. Subject Matter and Scope of Copyright and Copyright Ownership and Transfer.)
    3. Who owns the stuff AIs ‘create’?
  2. How if at all should we read copyright law in light of the literary criticism of the ’70s onward that alleged the “death of the author” as first celebrated by Roland Barthes in a (in)famous 1967 article. Barthes claimed, very influentially, that traditional literary criticism’s practice of incorporating the intentions and biographical context of an author in an interpretation of a text was an error. He argued that writing and creator are unrelated, that the interpretation of texts depends crucially on what the reader brings to it. (Optional: Wikipedia article summarizes Barthes’s essay and its context fairly well.)
  3. William Gass riffed on Barthes in 188/189 Salmagundi 514, 534-535, (Fall 2015 – Winter 2016), saying that,
    1. “Every author has an identity, but masterpieces are written by the human mind, not by human nature, which only lends them their common smell and color, their day to day dust. The implication is that readers differ in the same way. A masterpiece can be read as if it were by James Michener (it is the principal way Proust suffers from his society- swallowing fans), but the works of the human mind are really addressed to other human minds. That ineffable persona which a poem implies will be “the human mind” if the poem achieves greatness; however, the fatuous little New Yorker story will fasten the reader to a rock in Westchester and leave him to be eaten up by trademarks and localisms and proper names. It will flatter his determinate and causal-his chemical-self. The anonymity which the superb poem or fiction presumably possesses, according to some theories, may consequently be a kind of spiritual consanguinity.”Because we borrow, beg, buy, steal, or copy texts; because texts enter our eyes but remain in the blood; because we are, as authors, plagiarists and paraphrasers and brain pickers and mocking birds; because of these and other like habits we are, in effect, translating texts from one time to another and one context to another and one language to another. If, instead of repeating, “have a nice day,” we suggest to strangers that they “lead a good life,” we have simply rearranged a slightly different little cluster of cliches. But all that any author works with, in the beginning, is given her by one corrupted magi after another: the language, the life she leads, the literary tradition, schools she attends, the books she reads, the studies she has undertaken; so she cannot lay claim to some syntax or vocabulary, climate of ideas or custom of entertaining, as hers, and hers alone, and therefore as special as she is. Neither is that inadequacy she is supposed to feel at the close of her office hours the feeling of a freak. Of course, all of this wisdom and experience, this shit and sublimity, is shared, and of course a great deal of what we all do and think and feel and write is no more uniquely had or imagined than the balloon of the bagger; the stream of life is rarely more than a pisser’s trickle; and literally millions of sentences are penned or typed or spoken every day which have only a source-a spigot or a signboard-and not an author[]; they have never been near a self which is so certain of its spirit and so insistent on its presence that it puts itself in its syllables….”
    2. If we agree with Gass, what if anything does that say about AI authorship?
  4. What then is an “author”?  How do “authors” author?
  5. Another very influential literary theory, at least in the ’80s and ’90s, was Harold Bloom’s Anxiety of Influence.  I asked Consensus to summarize its main arguments, and (if memory serves) it did a pretty good job:
    1. Harold Bloom’s “The Anxiety of Influence: A Theory of Poetry” (1973) is a seminal work that explores the complex relationships between poets across generations. Bloom’s main arguments center on the idea that poets are influenced by their predecessors, yet they must overcome this influence to develop their own unique voice and style. Here are the key points of Bloom’s theory:1. **Poetic Influence and Misreading**: Bloom suggests that all poets are influenced by previous poets. However, this influence is not simply a matter of homage or continuation. Instead, poets often engage in a process of misreading or misinterpretation of their predecessors’ work, which allows them to carve out a space for their own creative voice. This misreading is not a failure but a necessary step in the creative process.2. **Oedipal Struggle**: Bloom employs a Freudian framework, particularly the Oedipus complex, to describe the relationship between poets. He argues that younger poets experience an “anxiety of influence” as they struggle to assert their independence from their poetic forebears, akin to a child’s desire to surpass their parents. This struggle is both a conflict and a driving force for creativity.3. **Six Revisionary Ratios**: Bloom outlines six ways (“ratios”) through which poets might engage with and distance themselves from their predecessors. These include Clinamen (swerve), Tessera (completion and antithesis), Kenosis (a deliberate emptying out of one’s own style to make room for influence), Daemonization (a distortion of the precursor’s spirit), Askesis (a renunciation of the precursor’s poetics for the sake of one’s own poetry), and Apophrades (the return of the precursor’s influence in such a way that the precursor is overshadowed).4. **The Strength of the Poet**: Bloom posits that the strength of a poet is measured by their ability to overcome the anxiety of influence through creative misinterpretation and innovation. The most significant poets are those who can most effectively transcend the influence of their predecessors, establishing themselves as original voices within the literary tradition.5. **Critique of Historical Readings**: Bloom criticizes the traditional historical approach to literature, which places a heavy emphasis on context, sources, and influences in a linear or straightforward manner. He argues that such readings fail to capture the complex, often subconscious, ways in which poets engage with the works of their predecessors.6. **Poetry as a Struggle for Originality**: Ultimately, Bloom’s theory highlights the inherent tension in poetry (and art in general) between the pull of the past and the push toward new creation. The anxiety of influence frames poetry as a perpetual struggle for originality and self-definition in the shadow of great works that have come before.
  6.  If we think this is a correct description of what poets, or at least great poets, do, does this tell us anything about AI authorship? Can it be a great poet?
  7. How do the early decisions of the Coypright Office (and if you read it, Thaler v. Vidal) map onto the theoretical visions in the academic articles above?

Class 13 Background to AI Liability

  1. Select statutory background:
    1. Select parts of Restatement (Second) of Torts relating to defamation: § 558 , § 578, §  581, §  12 (1977);
    2. Portions of 47 U.S.C. § 230;
    3. Restatement (Second) of Torts relating to False light: § 652E;
    4. Restatement (Second) of Torts relating to Intentional Infliction of Emotional Distress § 46.
    5. Restatement (Second) § 652C Appropriation of Name or Likeness
  2. A Note on Design Defect Liability in the US
  3. Pages 2506-08 of Toni M. Massaro, Helen Norton, & Margot E. Kaminski, SIRI-OUSLY 2.0: What Artificial Intelligence Reveals about the First Amendment, 101 Minn. L. Rev. 2481 (2017).
  4. Pages 508-511 & 514–18 & 549-50 of Eugene Volokh, Large Libel Models? Liability for AI Output, 3 J. Free Speech L. 489 (2023).
  5. Winter v. G.P. Putnam’s Sons, 938 F.2d 1033, 1033–36 (9th Cir. 1991).
  6. Pages 348-60 of Jane Bambauer, Negligent AI Speech: Some Thoughts About Duty, 3 J. Free Speech L. 343 (2023).
  7. Read something from the “deepfakes” section of the optional reading in Class 7.

Optional

  • (*) Bryan H. Choi, AI Malpractice, 73 DePaul Law Review 301 (2024):
    • Should AI modelers be held to a professional standard of care? Recent scholarship has argued that those who build AI systems owe special duties to the public to promote values such as safety, fairness, transparency, and accountability. Yet, there is little agreement as to what the content of those duties should be. Nor is there a framework for how conflicting views should be resolved as a matter of law.
      This Article builds on prior work applying professional malpractice law to conventional software development work, and extends it to AI work. The malpractice doctrine establishes an alternate standard of care—the customary care standard—that substitutes for the ordinary reasonable care standard. That substitution is needed in areas like medicine or law where the service is essential, the risk of harm is severe, and a uniform duty of care cannot be defined. The customary care standard offers a more flexible approach that tolerates a range of professional practices above a minimum expectation of competence. This approach is especially apt for occupations like software development where the science of the field is hotly contested or is rapidly evolving.
      Although it is tempting to treat AI liability as a simple extension of software liability, there are key differences. First, AI work has not yet become essential to the social fabric the way software services have. The risk of underproviding AI services is less troublesome than it is for conventional professional services. Second, modern deep-learning AI techniques differ significantly from conventional software development practices, in ways that will likely facilitate greater convergence and uniformity in expert knowledge.
      Those distinguishing features suggest that the law of AI liability will chart a different path than the law of software liability. For the immediate term, the interloper status of AI indicates a strict liability approach is most appropriate, given the other factors. In the longer term, as AI work becomes integrated into ordinary society, courts should expect to transition away from strict liability. For aspects that elude expert consensus and require exercise of discretionary judgment, courts should favor the professional malpractice standard. However, if there are broad swaths of AI work where experts can come to agreement on baseline standards, then courts can revert to the default of ordinary reasonable care.
  • (*) Peter Wills, Care for Chatbots, 57 UBC L. Rev. __ (forthcoming):
    • Individuals will rely on language models (LMs) like ChatGPT to make decisions. Sometimes, due to that reliance, they will get hurt, have their property be damaged, or lose money. If the LM had been a person, they might sue the LM. But LMs are not persons.This paper analyses whom the individual could sue, and on what facts they can succeed according to the Hedley Byrne-inspired doctrine of negligence. The paper identifies a series of hurdles conventional Canadian and English negligence doctrine poses and how they may be overcome. Such hurdles include identifying who is making a representation or providing a service when an LM generates a statement, determining whether that person can owe a duty of care based on text the LM reacts to, and identifying the proper analytical path for breach and causation.
      To overcome such hurdles, the paper questions how courts should understand who “controls” a system. Should it be the person who designs the system, or the person who uses the system? Or both? The paper suggests that, in answering this question, courts should prioritise social dimensions of control (for example, who understand how a system works, not merely what it does) over physical dimensions of control (such as on whose hardware a program is running) when assessing control and therefore responsibility.
      The paper make further contributions in assessing what it means (or should mean) for a person to not only act, but react via an LM. It identifies a doctrinal assumption that when one person reacts to another’s activity, the first person must know something about the second’s activity. LMs break that assumption, because they allow the first person to react to information from another person without any human having knowledge. The paper thus reassesses what it means to have knowledge in light of these technological developments. It proposes redefining “knowledge” such that it would accommodate duties of care to individuals when an LM provides individualised advice.
      The paper then shows that there is a deep tension running through the breach and causation analyses in Anglo-Canadian negligence doctrine, relating to how to describe someone who takes an imprudent process when performing an act but whose ultimate act is nonetheless justifiable. One option is to treat them as in breach of a standard of care, but that breach did not cause the injury; another is to treat them as not in breach at all. The answer to this question could significantly affect LM-based liability because it affects whether “using an LM” is itself treated as a breach of a standard of care.Finally, the paper identifies alternative approaches to liability for software propounded in the literature and suggests that these approaches are not plainly superior to working within the existing framework that treats software as a tool used by a legal person.
  • Catherine M. Sharkey, A Products Liability Framework for AI, 25 Col. Sci & Tech L. Rev. 21 (2024):
    • A products liability framework, drawing inspiration from the regulation of FDA-approved medical products—which includes federal regulation as well as products liability—holds great promise for tackling many of the challenges artificial intelligence (AI) poses. Notwithstanding the new challenges that sophisticated AI technologies pose, products liability provides a conceptual framework capable of responding to the learning and iterative aspects of these technologies. Moreover, this framework
      provides a robust model of the feedback loop between tort liability and regulation.The regulation of medical products provides an instructive point of departure. The FDA has recognized the need to revise its traditional paradigm for medical device regulation to fit adaptive AI/Machine Learning (ML) technologies, which enable continuous improvements and modifications to devices based on information gathered during use. AI/ML technologies should hasten an even more significant regulatory paradigm shift at the FDA away from a model that puts most of its emphasis (and resources) on ex ante premarket approval to one that highlights ongoing postmarket surveillance. As such a model takes form, tort (products) liability should continue to play a significant information-production and deterrence role, especially during the transition period before a new ex post regulatory framework is established.
  • (*) Mark A. Lemley & Peter Henderson, The Mirage of Artificial Intelligence Terms of Use Restrictions (Jan 10, 2025):
    • Artificial intelligence (AI) model creators commonly attach restrictive terms of use to both their models and their outputs. These terms typically prohibit activities ranging from creating competing AI models to spreading disinformation. Often taken at face value, these terms are positioned by companies as key enforceable tools for preventing misuse, particularly in policy dialogs. The California AI Transparency Act even codifies this approach, mandating certain responsible use terms to accompany models.But are these terms truly meaningful, or merely a mirage? There are myriad examples where these broad terms are regularly and repeatedly violated. Yet except for some account suspensions on platforms, no model creator has actually tried to enforce these terms with monetary penalties or injunctive relief. This is likely for good reason: we think that the legal enforceability of these licenses is questionable. This Article provides a systematic assessment of the enforceability of AI model terms of use and offers three contributions.First, we pinpoint a key problem with these provisions: the artifacts that they protect, namely model weights and model outputs, are largely not copyrightable, making it unclear whether there is even anything to be licensed.Second, we examine the problems this creates for other enforcement pathways. Recent doctrinal trends in copyright preemption may further undermine state-law claims, while other legal frameworks like the DMCA and CFAA offer limited recourse. And anti-competitive provisions likely fare even worse than responsible use provisions.Third, we provide recommendations to policymakers considering this private enforcement model. There are compelling reasons for many of these provisions to be unenforceable: they chill good faith research, constrain competition, and create quasi-copyright ownership where none should exist. There are, of course, downsides: model creators have even fewer tools to prevent harmful misuse. But we think the better approach is for statutory provisions, not private fiat, to distinguish between good and bad uses of AI and restrict the latter. And, overall, policymakers should be cautious about taking these terms at face value before they have faced a legal litmus test.
  • (*) Gabriel Weil, Tort Law as a Tool for Mitigating Catastrophic Risk from Artificial Intelligence, (June 6, 2024):
    • The capabilities of artificial intelligence (AI) systems have improved markedly over the past decade. This rapid progress has brought greater attention to longstanding concerns that advanced AI systems could cause catastrophic harm, up to and including human extinction. In principle, the prospect of tort liability could encourage AI developers to proceed with caution. But the current U.S. tort liability system is not set up to handle the catastrophic risk posed by AI — since most of the expected harm from AI systems comes in truly catastrophic scenarios where compensation would not be feasible. To address this, I propose a form of punitive damages designed to pull forward this expected liability into cases of practically compensable harm. To succeed in offering sufficient incentives for precaution, such punitive damages would need to be available even in the absence of human malice or recklessness. This paper also examines other legal mechanisms to ensure socially optimal investment in AI safety measures. Doctrinal changes considered include recognizing the training and deployment of advanced AI systems as an abnormally dangerous activity subject to strict liability, adopting a capacious conception of foreseeability for the purposes of evaluating proximate cause in cases of AI harm, and modifying the scope of legally compensable damages in cases involving loss of human life. The paper also discusses further AI liability law changes that would require legislation, including requiring liability insurance for training and deployment of advanced AI systems, diverting a portion of punitive damages into an AI safety fund, and credibly announcing the availability of strict liability and punitive damages in advance of the first case in which they might be appropriate. The paper concludes by characterizing the limits of an ex post tort liability framework for limiting catastrophic AI risk and considers what complementary policies may be needed to plug the gaps left by such a framework.
  • Shujie Luan, Shubhranshu Singh & Tinglong Dai, Algorithmic Bias and Physician Liability (Dec 13, 2024):
    • With the growing use of artificial intelligence (AI) in clinical decision-making, concerns about bias—manifested as differences in algorithmic accuracy across patient groups—have intensified. In response, the U.S. Centers for Medicare and Medicaid Services (CMS) has introduced a liability rule that penalizes healthcare providers who rely on biased algorithms that result in erroneous decisions. This paper examines the impact of this anti-bias liability rule on an AI firm’s development decision as well as a healthcare provider’s decision to use AI. The AI firm develops an algorithm that serves two patient groups, where achieving the same level of accuracy for the disadvantaged group is more costly. The provider then decides whether and how to use AI to make treatment decisions, balancing the reduction in clinical uncertainty against the risk of incurring anti-bias liability. We find the liability rule may induce biased use of AI: The provider may underuse AI overall and disproportionately disregard AI’s recommendations for disadvantaged patients. Interestingly, the effect of liability on AI use is non-monotone: as liability increases, the provider is first less likely to use AI for disadvantaged patients, but then more likely to rely on it. Furthermore, mandating equal algorithmic accuracy across patient groups may inadvertently harm all patients, in part because such mandates may lead to overusing AI for disadvantaged patients.
  • (*) James Grimmelmann, The Defamation Machine (Aug 01, 2024) (optional reading in Class 4): “Can Chat GPT commit llbel?”
  • Cade Metz, OpenAI Says Russia and China Used Its A.I. in Covert Campaigns, NY Times (May 30, 2024).

Notes & Questions

  1. Assuming that an AI is not a legal person subject to suit, who if anyone should be liable when a prompt to a chatbot or other LLM produces a reply containing a false and defamatory statement about another person?
    1. Is this libel at all?  If so who is the source of the libel?
    2. Does it matter whether or not the statement is further shared, or is the ‘publication’ by the LLM to the recipient sufficient to meet the publication requirement?
    3. To what extent are the various parties’ liability dependent on the nature of the prompt?
  2. If we characterize the creation of the false and defamatory statement as a species of negligence on the part of the purveyor of the LLM, can this liability be disclaimed? How if you were the general counsel of a startup running a chatbot would you seek to achieve this?
  3. Can you think of a circumstance in which a harmful deepfake would not be punishable as an intentional tort?
  4. The rules for libel of public figures are different from those that protect ordinary people; under NYT v. Sullivan in order to libel a public figure you must publish the statement with malice or a reckless disregard for the truth.  If we import this standard into deepfake law, in practice would it mean that fake pornographic images of celebrities will be treated differently from revenge porn deepfakes (fake pornographic images) of ex-partners who are not public figures)?
  5. Libel is far from the only lens through which to view the harms and liabilities (which, note, are not necessarily identical!) arising from deepfakes.
    1. Putting aside the problem of identifying the original creator, what sorts of claims might the unwilling subject of a deepfake be able to assert against either the creator or intermediate re-publishers?
    2. What sorts of deepfakes other than those with fake nudity and/or sexual conduct might be actionable?
    3. Are all non-pornographic deepfakes protected First Amendment speech?  (Hint: no – but can you explain which ones would not be?)
  6. As of October 1, 2022, federal law creates a civil action for the unauthorized dissemination of private, intimate images of both adults and children—images commonly called “nonconsensual pornography” or “revenge porn”.  Does this new law create a civil action against the maker of a pornographic deepfake?  (Optional: read the Congressional Research Service analysis of the bill.)
  7. As regards personal injury traceable to an AI, is the AI better understood as a product (in which case we might invoke product liability law), or as a service (in which case we might look first to negligence law), or both, or neither?
  8. If an AI’s outputs are truly unpredictable, what does this mean for tort claims against designers and/or users? Would it make a difference if this unpredictability were due to:
    1. A conscious design choice by the model-builders?
    2. Unexpected emergent behavior by the model?
    3. The so-called “superintellignece” of the model which allows it to do things beyond the ken of mere mortals?
    4. Randomness?
  9. How would your answer to the above vary with the nature of the tort? E.g.:
    1. A claim for product liability based on negligent design?
      1. But what if, say, emergent behavior was a design goal — does that mean it can’t by definition be negligent?
    2. A claim for libel?
    3. Infliction of emotional distress (either intentional or negligent) arising from the production of a deepfake?
  10. Should AI-related liability rules for professionals (doctors, lawyers architects) who wrongly rely on AI outputs be different from that for non-professionals (car mechanics, plumbers, ‘helpful’ neighbors), or are existing rules (e.g. the different standards of care) sufficient to protect the public?

Class 14: Explainable AI (“XAI”)

  1. IBM, What is explainable AI?
  2. Pages 1109-26 (edited) of Andrew D. Selbst & Solon Barocas, The Intuitive Appeal of Explainable Machines, 87 Fordham L. Rev. 1085 (2018).
  3. Section 4.11 (“Interpretability”) of Rishi Bommasani et al, Stanford HA, On the Opportunities and Risks of Foundation Models (2023).
  4. Selections from Boris Babic & I. Glenn Cohen, The Algorithmic Explainability ‘Bait and Switch’, 108 Minn. L. Rev. 857 (2023).
  5. Anthropic, Mapping the Mind of a Large Language Model (May 21, 2024). Optional: Read the rather difficult full paper.
  6. In February 2025 I asked Gemini 2.0 Flash Thinking Experimental (which is a model with a ‘reasoner’) whether AI with reasoners was enough to satisfy the needs of XAI especially in matters that could lead to litigation such as public benefits, employment, and administrative/judicial decisions. I said to start by looking at different definitions of XA.  Here is what Gemini reported about its ‘thinking’ and its report.

Optional

  • (*) Aniket Kesari et al., A Legal Framework for eXplainable Artificial Intelligence (Nov. 8, 2024):
    • A foundational principle of the law is that decision-makers must explain their reasons: judges write opinions, government agencies write reports detailing why they deny benefits in areas such as entitlements and immigration, and credit lenders inform applicants about the reasons for denying an application. Explanations pave the way for other parts of a functioning legal system, including the right to appeal adverse decisions, transparency in government decisions, and building public trust in institutions.With automated decision-making systems enabled by artificial intelligence, legal systems run the risk of becoming giant “black boxes” where people who are subject to an automated decision do not know or understand why the system made a particular decision. To counter this risk, policymakers and regulators increasingly create rights to explanation of automated decisions. The California Privacy Protection Agency, for example, is currently drafting regulations requiring businesses to inform consumers about the “logic” and “key parameters” of automated decision-making technologies, and how those “key parameters” are applied to consumers in individual decisions.What should count as “key parameters” and how consumers process such algorithmic explanations remains elusive, however. This Article bridges the gap between computer science and law to answer these questions. In the Article, we develop a legal framework for eXplainable Artificial Intelligence (XAI). We proceed in three steps. First, we present a taxonomy for legal explanations of algorithmic decisions (“Legal-XAI”) that is applicable to a wide range of legal areas and AI decision-making systems. Among other dimensions, we distinguish between global and local explanations, between comprehensive and selective explanations, and between contrastive and non-contrastive explanations.Second, we discuss how legal, technical, and behavioral factors provide guidance as to which explanation from our Legal-XAI Taxonomy can be used in which context. Using credit scoring as an example, we demonstrate how the law may prescribe which types of explanation method can be used for a particular algorithmic decision-making system. We show how the combination of legal, computer science, and behavioral principles can guide policymakers, legal scholars, and computer scientists towards selecting the right explanation method for particular legal areas.Third, we demonstrate how our Legal-XAI taxonomy can be applied to various areas, including Medicaid, higher education, and automated decision-making more generally. We argue that policymakers should be more specific when creating rights of explanation. Automated decisions can usually be explained with numerous explanation methods, and policymakers should specify which features an explanation should have to advance the policy goals the policymakers have in mind. Our Legal-XAI taxonomy helps policymakers to identify the right explanation method in accordance with their policy goals.More fundamentally, our Article bridges the gap between legal and computer science discussions on eXplainable AI as well as between theoretical and empirical research. We argue that the legal debates and eXplainable AI innovations have mostly proceeded independently without a connecting conversation. We posit that the discussions on algorithmic explanations should put the subjects of automated decisions on center stage, in order to make these systems more democratic and inclusive. Finally, we present a roadmap and a software package demonstrating how various algorithmic explanation methods can be compared in a field experiment with high external validity. Our Article thereby contributes to the emerging interdisciplinary field of law, computer science, and behavioral research.
  • Cynthia Rudin, Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, 1 Nature Machine Intelligence 206 (2019):
    • […T]rying to explain black box models, rather than creating models that are interpretable in the first place, is likely to perpetuate bad practice and can potentially cause great harm to society. The way forward is to design models that are inherently interpretable. […] This document is mainly relevant to high-stakes decision making and troubleshooting models, which are the main two reasons one might require an interpretable or explainable model. Interpretability is a domain-specific notion–, so there cannot be an all-purpose definition. Usually, however, an interpretable machine learning model is constrained in model form so that it is either useful to someone, or obeys structural knowledge of the domain, such as monotonicity (for example, ref. ), causality, structural (generative) constraints, additivity or physical constraints that come from domain knowledge. Interpretable models could use case-based reasoning for complex domains. Often for structured data, sparsity is a useful measure of interpretability, because humans can handle at most 7 ± 2 cognitive entities at once,. Sparse models allow a view of how variables interact jointly rather than individually. We will discuss several forms of interpretable machine ML models for different applications, but there can never be a single definition; for example, in some domains sparsity is useful, and in others it is not. There is a spectrum between fully transparent models (where we understand how all the variables are jointly related to each other) and models that are lightly constrained in model form (such as models that are forced to increase as one of the variables increases, or models that, all else being equal, prefer variables that domain experts have identified as important[.]
  • Hofit Wasserman Rozen, Ran Gilad-Bachrach and Niva Elkin-Koren, Lost In Translation: The Limits of Explainability in AI 42 Cardozo Arts & Ent. L.J. 391 (2024):
    • “This paper critically examines XAI’s potential to facilitate the right to explanation by applying the prism of explanation’s role in law to different stakeholders. Inspecting the underlying functions of reason-giving reveals different objectives for each of the stakeholders involved. From the perspective of a decision-subject, reason-giving facilitates due process and acknowledges human agency. From a decision-maker’s perspective, reason-giving contributes to improving the quality of the decisions themselves. From an ecosystem perspective, reason-giving may strengthen the authority of the decision-making system toward different stakeholders by promoting accountability and legitimacy, and by providing better guidance. Applying this analytical framework to XAI’s generated explanations reveals that XAI fails to fulfill the underlying objectives of the right to explanation from the perspective of both the decision-subject and the decision-maker. In contrast, XAI is found to be extremely well-suited to fulfil the underlying functions of reason-giving from an ecosystems’ perspective, namely, strengthening the authority of the decision-making system. However, lacking all other virtues, this isolated ability may be misused or abused, eventually harming XAI’s intended human audience. The disparity between human decision-making and automated decisions makes XAI an insufficient and even a risky tool, rather than serving as a guardian of human rights. After conducting a rigorous analysis of these ramifications, this paper concludes by urging regulators and the XAI community to reconsider the pursuit of explainability and the right to explanation of AI systems.”
  • (*) Philipp Hacker and Jan-Hendrik Passoth, Varieties of AI Explanations under the Law. From the GDPR to the AIA, and beyond in Lecture Notes on Artificial Intelligence (Holzinger, Goebel, Fong, Moon, Müller and Samek eds. 2022):
    • “[W]e provide an overview of legal obligations to explain AI and evaluate current policy proposals. In this, we distinguish between different functional varieties of AI explanations – such as multiple forms of enabling, technical and protective transparency – and show how different legal areas engage with and man- date such different types of explanations to varying degrees. Starting with the rights-enabling framework of the GDPR, we proceed to uncover technical and protective forms of explanations owed under contract, tort and banking law. Moreover, we discuss what the recent EU proposal for an Artificial Intelligence Act means for explainable AI, and review the proposal’s strengths and limitations in this respect. Finally, from a policy perspective, we advocate for moving beyond mere explainability towards a more encompassing framework for trustworthy and responsible AI that includes actionable explanations, values-in-design and co-design methodologies, interactions with algorithmic fairness, and quality bench- marking”.
  • (*) Marco Almada, Governing the Black Box of Artificial Intelligence (Nov 9, 2023):
    • “Artificial intelligence (AI) is often described as a black box rendered opaque by the technical complexity of AI systems. Scholars and policymakers tend to see this opacity as a problem but often diverge on how to respond to this black-box problem. Some propose that AI opacity must be addressed through technical means, such as explanation models that portray how a system arrives at a decision. Others question the value of such mediated explanations, arguing that the solution to the black box problem re-quires mechanisms for disclosure of the inner workings of AI systems. […] I argue that no approach can work without drawing elements from the other. To do so, I first show how the black box metaphor is used as a conceptual model of AI in regulatory models, which leads to an emphasis on the technical sources of opacity connected to AI systems. Recent work on AI has developed various methods that can be used for the scientific scrutiny of these sources. Still, their use in non-scientific contexts is prone to various forms of manipulation. As an alternative, policy proposals often require more strict technical disclosure, for example, through the use of inherently interpretable models or the publication of the source code for AI software, but these might lead to reduced gains in understanding, if any. However, the current law on algorithmic transparency in the EU gives margin to a third interpretation: if the black box is perceived from a socio-technical perspective, disclosure requirements aimed at the development and use of AI systems may drastically reduce the possibilities for manipulation of explanations. Disclosure should therefore be seen as a pre-condition, not an alternative, to explanations and other technical approaches to the black box problem.”
  • (*) Philipp Hacker et al., Explainable AI under contract and tort law: legal incentives and technical challenges, 28 Artificial Intelligence and Law 415 (2020).
    • “[T[he law, in subtle ways, may set hitherto unrecognized incentives for the adoption of explainable machine learning applications. In doing so, we make two novel contributions. First, on the legal side, we show that to avoid liability, professional actors, such as doctors and managers, may soon be legally compelled to use explainable ML models. We argue that the importance of explainability reaches far beyond data protection law, and crucially influences questions of contractual and tort liability for the use of ML models. To this effect, we conduct two legal case studies, in medical and corporate merger applications of ML. As a second contribution, we discuss the (legally required) trade-off between accuracy and explainability and demonstrate the effect in a technical case study in the context of spam classification.”
  • Eden Sarid & Omri Ben-Zvi, Machine Learning and the Re-Enchantment of the Administrative State, 87 Modern L. Rev. 371 (2023):
    • Machine learning algorithms present substantial promise for more effective decision-making by administrative agencies. However, some of these algorithms are inscrutable, namely, they produce predictions that humans cannot understand or explain. This trait is in tension with the emphasis on reason-giving in administrative law. The article explores this tension, advancing two interrelated arguments. First, providing adequate reasons is a significant facet of respecting individuals’ agency. Incorporating inscrutable algorithmic predictions into administrative decision-making compromises this normative ideal. Second, as a long-term concern, the use of inscrutable algorithms by administrative agencies may generate systemic effects by gradually reducing the realm of the humanly explainable in public life, a phenomenon Max Weber termed “re-enchantment.” As a result, the use of inscrutable machine learning algorithms might trigger a special kind of re-enchantment, making us comprehend less rather than more of shared human experience, and consequently altering the way we understand the administrative state and experience public life.
  • Behnam Mohammadi et al., Regulating Explainable AI (XAI) May Harm Consumers (Sept. 30, 2020):
    • “Recent AI algorithms are black box models whose decisions are difficult to interpret. eXplainable AI (XAI) is a class of methods that seek to address the lack of AI interpretability and trust by explaining to customers their AI decisions. The common wisdom is that regulating AI by mandating fully transparent XAI leads to greater social welfare. Our paper challenges this notion through a game theoretic model of a policymaker who maximizes social welfare with firms competing in a duopoly setting where customers hold heterogeneous preferences. A key contribution is to conceptualize the notions of AI accuracy, AI explanation depth, explanation method, and explanation fairness all into a single model. Our results first illustrate that partial explanations are a possible equilibrium in an unregulated setting. Moreover, we highlight the parameter spaces at which customers and firms are aligned or misaligned in the desire for a full explanation. Focusing on the region where preferences for full AI explanation are misaligned (customers preferring full explanation), we discuss multiple regulatory levers a policymaker may pull and show that regulating at a full explanation level is not always socially optimal. Mandating fully transparent XAI may make firms and customers worse off. We also show that flexible XAI policies dominate full XAI or unregulated XAI extremes.”
  • Martin Ebers, Regulating Explainable AI in the European Union. An Overview of the Current Legal Framework(s) in Nordic Yearbook of Law and Informatics 2020: Law in the Era of Artificial Intelligence, (Liane Colonna/Stanley Greenstein eds.):
    • “Determining why a system arrives at a particular algorithmic decision or prediction allows us to understand the technology, develop trust for it and – if the algorithmic outcome is illegal – initiate appropriate remedies against it. Additionally, XAI enables experts (and regulators) to review decisions or predictions and verify whether legal regulatory standards have been complied with. All of these points support the notion of opening the black box. On the other hand, there are a number of (legal) arguments against full transparency of Artificial Intelligence (AI) systems, especially in the interest of protecting trade secrets, national security and privacy. Accordingly, this paper explores whether and to what extent individuals are, under EU law, entitled to a right to explanation of automated decision-making, especially when AI systems are used.”
  • Cynthia Rudin & Joanna Radin, Why Are We Using Black Box Models in AI When We Don’t Need To? A Lesson From An Explainable AI Competition, 1.2 Harv. Data Science Rev. (Fall 2019).

Notes & Questions

  1. Be sure you can explain:
    1. Why  we might be nervous about trusting ‘black box’ models even if reliability testing shows they are very accurate, indeed even if they are more accurate than people?
    2. Which of the issues identified by Prof. Strandburg (class 4) and by today’s readings  will be solved or very greatly ameliorated by XAI?
  2. To what extent does mere “explainabilty”–as opposed to an “interpretable” model–do the job?
  3. Do you agree with the conclusion of the Babic & Cohen article that even if we cannot achieve interpretability it would be wrong to require it in some contexts?
    1. If so, how do we distinguish between the contexts in which we should require interpretability, and those where we can settle for less? (And in your opinion, what are they?)
    2. Who should decide?
  4. Which of these decisions would you entrust to a non-interpretable model that was 5% more accurate than the average human decision-maker?
    1. Tumor detection
    2. Disability benefits
    3. Parole eligibility
    4. Facial ID matches as grounds for arrest on suspicion of having committed a crime.
    5. Selection for who to interview for a job as a BigLaw associate.
  5. Is there anything you might you want to know about the model other than the accuracy rate before answering question 4 above?
  6. Gemini 2.0 pro experimental produced a pretty long-winded answer to my question.  Would a better answer just be “no”?

Class 15: AI in Administration

  1. Pages 1155-76 & 1213-22 (i.e Parts I & III) of Cary Conglianese & David Lehr, Regulating by Robot: Administrative Decision Making in the Machine-Learning Era, 105 Geo. L.J. 1147 (2017).
  2. Introduction & Part III of Deirdre K. Mulligan & Kenneth A. Bamberger, Procurement As Policy: Administrative Process for Machine Learning, 34 Berk. Tech. L.J. 773 (2019).
  3. Pages 7-15 of Kevin De Liban, TechTonic Justice, Inescapable AI: The Ways AI Decides How Low-Income People Work, Live, Learn, and Survive (Nov. 2024).
  4. Parts II. A & B of Aziz Z. Huq, Constitutional Rights in the Machine-Learning State, 105 Cornell L. Rev. 1875 (2020).
  5. Pages 1305-1313  from Danielle Keats Citron, Technological Due Process, 85 Wash. U.L. Rev. 1249 (2008).
  6. Pages 1-15 of Sasha Costanza-Chock, Design Justice, A.I., and Escape from the Matrix of Domination, JoDS (Jul 16, 2018).

Optional Readings

AI Use in Administration Generally

  • (*) Michael Karanicolas, Artificial Intelligence and Regulatory Enforcement (Dec. 9, 2024) (report to the Admin. Conf. of the U.S.):
    • In recent years, an increasing number of government agencies have incorporated AI systems into their regulatory enforcement processes. Some examples of these applications include at the Securities and Exchange Commission, which employs machine-learning tools to identify suspicious filings or insider trading, and at the Environmental Protection Agency, which is experimenting with automated facility inspection targeting. The potential efficiency gains from these systems are easy to understand, especially in the context of mounting public demands for the administrative state to do more with less. They may also present opportunities to expand the processing power of government agencies, boosting research and analytical capabilities in novel and unprecedented ways. Such improvements may be necessary in the context of increasing sophistication among targets of regulation.However, for all of their potential benefits, the incorporation of AI systems into regulatory enforcement functions also raises a number of difficult questions. In particular, there are concerns related to the impact of AI on the fairness of decision-making. In addition to well-documented challenges with bias, drift, and accuracy, there are fundamental questions around transparency, accountability, and procedural fairness. At a time when public trust in government, and in our broader public institutions, has declined to critical levels, the potential for these systems to further erode the relationship between the people and the administrative state should be an extremely serious consideration.
  • Amnesty International, Xenophobic machines: Discrimination through unregulated use of algorithms in the Dutch childcare benefits scandal (2021).
  • (*) Michele Gilman, Poverty Lawgorithms: A Poverty Lawyer’s Guide to Fighting Automated Decision-Making Harms on Low-Income Communities, Data & Society (2020):
    • [C]ivil legal services (or poverty) lawyers increasingly represent clients whose legal challenges are intertwined with automated decision-making systems, digital profiling, predictive analytics, and various artificial intelligence tools. Competent legal representation requires an ability to issue-spot the ways in which data-centric technologies intersect with legal claims and defenses and to understand governing legal frameworks. At the same time, lawyers do not need the technical expertise of computer scientists to effectively interrogate these systems. Rather, understanding where and how data-centric technologies operate puts lawyers in a powerful position to advocate alongside their clients.This report is designed to help poverty lawyers and their clients resist the adverse impacts of data-centric technologies and to engage as stakeholders in the adoption and implementation of algorithmic systems. The report is organized by major practice area and includes links to helpful resources for deeper dives into specific issues that may arise in legal services representation and policy advocacy. The practice areas covered are consumer, family law, housing, public benefits, schools and education, and workers’ rights, as well as a final section on how immigration surveillance intersects with these practice areas. The report aims to assist legal services attorneys in identifying the impacts that existing and emerging technologies have on their clients.
  • Stanford RegLab, Princeton, and the County of Santa Clara Collaborate to Use AI to Identify and Map Racial Covenants from over 5 Million Deed Records: The approach paves the way for faster and more accurate compliance with California’s anti-discrimination law.  (Oct 17, 2024)

Policing

  • (*)  Andrew Guthrie Ferguson, Generative Suspicion and the Risks of AI-Assisted Police Reports, __ Nw. U. L. Rev. __ (forthcoming 2025):
    • Police reports play a central role in the criminal justice system. Many times, police reports exist as the only official memorialization of what happened during an incident, shaping probable cause determinations, pretrial detention decisions, motions to suppress, plea bargains, and trial strategy. For over a century, human police officers wrote the factual narratives that shaped the trajectory of individual cases and organized the entire legal system.All that is about to change with the creation of AI-assisted police reports. Today, with the click of a button, generative AI Large Language Models (LLMS) using predictive text capabilities can turn the audio feed of a police-worn body camera into a pre-written draft police report. Police officers then fill-in-the blanks of inserts and details like a “Mad Libs” of suspicion and submit the edited version as the official narrative of an incident.From the police perspective, AI-assisted police reports offer clear cost savings and efficiencies from dreaded paperwork. From the technology perspective, ChatGPT and similar generative AI models have shown that LLMs are good at predictive text prompts in structured settings, exactly the use case of police reports. But hard technological, theoretical, and practice questions have emerged about how generative AI might infect a foundational building block of the criminal legal system.This Article is the first law review article to address the challenge of AI-assisted police reports. The Article first interrogates the technology, providing a deep dive into how AI-assisted police reports work. Promises around innovation are countered by concerns around how the models were trained, questions around error, hallucinations, and bias in transcription, and how the final police report will be impacted by the generative prompts. Issues around structure, timing, legal gap-filling, and factual gap-filling are all addressed, with an eye toward comparing this innovation to existing human report writing.The Article also addresses the bigger theoretical question about the role of the police report. The Article contrasts two visions of a police report: a narrow, instrumental vision of a police report and a broader, accountability vision of a police report. The goal is to show how a change in technology might also change the traditional role of the police report.Finally, the Article explores how AI-police reports will alter criminal practice especially in misdemeanor and low-level felony cases where investigation and grand jury action is minimal. A police officer’s determination of what happened as a factual and legal matter directly impacts initial prosecutorial charging decisions and judicial pretrial detention decisions. In addition, the police report influences plea bargains, sentencing, discovery obligations, and trial practice. The open question is how reliance on AI-generative suspicion will distort the foundation of a legal system dependent on the humble police report.
  • James O’Donnell, How the largest gathering of US police chiefs is talking about AI: Officers training in virtual reality, cities surveilled by webs of sensors, and AI-generated police reports are all a sign of what’s to come, MIT Tech. Rev. (Nov. 19, 2024).
  • Cade Metz, Police Drones Are Starting to Think for Themselves, N.Y. Times (Dec. 5, 2020). Scary NYT Magazine article. “Drones are like iPhones … As soon as you get one, a new one arrives with new technology.”

Procurement and other Executive Functions

  • David Freeman Engstrom et al, Government by Algorithm: Artificial Intelligence in Federal Administrative Agencies (Based on a report submitted to the Administrative Conference of the United States, Feb. 18, 2020). Sets the scene very well before you read any of the following.
  • (*) Julie E. Cohen et al., Provisioning Digital Tools and Systems for Government Use, Redesigning the Governance Stack Project at Georgetown Law (Sept. 16, 2024):
    • This document […] explores how the administrative state can more effectively equip itself with digital tools and systems that align with and improve government’s ability to serve public values. Established approaches to digital provisioning fail in many important respects. Among others, they introduce thorny coordination problems while doing little to ensure design for broader public values; they cause obsolete and/or poorly conceived requirements to cascade through the development process for new tools and systems; they magnify the potential for technology-driven lock-in and vendor capture at scale; and they are unacceptably opaque to policymakers and the public. We trace some of these dysfunctions to the private-sector preference that underpins federal govtech provisioning and others to a top-down mode of development in which “solutions” are decreed at the outset rather than after consultation and conversation. The paper recommends a series of changes to the current policy landscape for govtech provisioning to correct these dysfunctions. One important recommendation involves rethinking the traditional “make vs. buy” dichotomy in public procurement and the underlying presumptions that have animated the dichotomy. Recentering public values and outcomes in govtech development also requires measures for ensuring the interoperability and transparency of govtech tools and systems. Another important recommendation involves reenvisioning processes for govtech development and implementation.
  • Cary Coglianese & Alicia Lai, Algorithm vs. Algorithm (Feb. 2, 2022):
    • “Part I of this Article begins by offering a counterweight to the criticisms of machine-learning algorithms that tend to dominate legal scholarship. It details the well-documented physical limitations and cognitive biases that afflict individual decision-making by human algorithms, along with additional problems that can arise when humans make collective decisions. Part II then focuses on machine learning and its promise for improving decision-making….Part III thus presents general considerations to help guide public officials seeking to make sound choices about when and how to use digital algorithms. In addition to focusing officials’ attention on the extent to which a shift to digital algorithms will improve upon the status quo, we emphasize in Part III the need to consider whether a new use of digital algorithms would likely satisfy key preconditions for successful deployment of machine learning and whether a system driven by digital algorithms would actually deliver better outcomes…”
  • Felix Morman, Beyond Algorithms Toward a Normative Theory of Automated Regulation, 62 B.C. L. Rev. 1 (2021):
    • “[A]utomated, non-human governance has long been a reality. For more than a century, policymakers have relied on regulations that automatically adjust to changing circumstances, without the need for human intervention. This Article surveys the track record of self-adjusting governance mechanisms to propose a normative theory of automated regulation. … Self-adjusting regulation offers an insurance policy against the well-documented inaccuracies ofeven the most expert forecasts, reducing the need for costly and time-consuming administrative proceedings. Careful analysis of empirical evidence, existing literature, and precedent reveals that the benefits of regulatory automation extend well beyond mitigating regulatory inertia. … Today’s automated regulations rely on relatively simplistic algebra, afar cry from the multivariate calculus behind smart algorithms. Harnessing the advanced mathematics and greater predictive powers of artificial intelligence could provide a significant upgrade for the next generation of automated regulation. Any gains in mathematical sophistication, however, will likely come at a cost if the widespread scholarly skepticism toward algorithmic governance is any indication of future backlash and litigation. Policy-makers should consider carefully whether their objectives may be served as well, if not better, through more simplistic, but well-established methods of regulatory automation.”
  • Cary Coglianese & Lavi M. Ben Dor, AI in Adjudication and Administration, 86 Brook. L. Rev. 791 (2021). “This status report on current uses of algorithmic tools can serve as a benchmark against which to gauge future growth in the use of artificial intelligence in the public sector.”
  • (*) Joshua D. Blank & Leigh Osofsky, Automated Legal Guidance, 106 Cornell L. Rev. 179 (2020). Of special interest to tax and adlaw geeks:
    “This Article offers one of the first critiques of these new systems of artificial intelligence. It shows that automated legal guidance currently relies upon the concept of “simplexity,” whereby complex law is presented as though it is simple, without actually engaging in simplification of the underlying law. While this approach offers potential gains in terms of efficiency and ease of use, it also causes the government to present the law as simpler than it is, leading to less precise advice and potentially inaccurate legal positions. Using the Interactive Tax Assistant as a case study, the Article shows that the use of simplexity in automated legal guidance is more powerful and pervasive than in static publications because it is personalized, non-qualified, and instantaneous. Further, it argues that understanding the costs as well as the benefits of current forms of automated legal guidance is essential to evaluating even more sophisticated, but also more opaque, automated systems that governments are likely to adopt in the future.”
  • (*) Ryan Calo & Danielle Keats Citron, The Automated Administrative State: A Crisis of Legitimacy, 70 Emory L.J. 797 (2021):
    “The legitimacy of the administrative state is premised on our faith in agency expertise. Despite their extra-constitutional structure, administrative agencies have been on firm footing for a long time in reverence to their critical role in governing a complex, evolving society. They are delegated enormous power because they respond expertly and nimbly to evolving conditions. In recent decades, state and federal agencies have embraced a novel mode of operation: automation. Agencies rely more and more on software and algorithms in carrying out their delegated responsibilities. The automated administrative state, however, is demonstrably riddled with concerns. Legal challenges regarding the denial of benefits and rights — from travel to disability — have revealed a pernicious pattern of bizarre and unintelligible outcomes. Scholarship to date has explored the pitfalls of automation with a particular frame, asking how we might ensure that automation honors existing legal commitments such as due process. Missing from the conversation are broader, structural critiques of the legitimacy of agencies that automate. Automation throws away the expertise and nimbleness that justify the administrative state, undermining the very case for the existence and authority of agencies. Yet the answer is not to deny agencies access to technology. This article points toward a positive vision of the administrative state that adopts tools only when they enhance, rather than undermine, the underpinnings of agency legitimacy.”

AI and Courts and People

  • (*)  Jane R. Bambauer & Tal Zarsky, Fair-Enough AI, __Yale J.L. & Tech__ (forthcoming 2025):
    • AI is unfair. It can be inaccurate (in several ways), biased (in several ways, and to several groups), disproportionate, exploitable, and opaque. The policy world is awash in AI governance and ethics frameworks, but these lack concrete standards and provide little guidance on how to select between competing versions of (un)fairness. In other words, they abdicate the responsibility of setting priorities among values. At the same time, though, the very same guidance documents harshly criticize existing AI and algorithmic tools for deficiencies in some particular aspect of fairness without considering whether alternative designs that fix the problem would make the system more “unfair” in other respects. Ad-hoc criticism is abundant and hard to predict.This article offers a strategy for AI ethics officers to navigate the “abdication/ad-hoc criticism” problems in the regulatory landscape. After explaining the meaning and sources of the most important forms of AI unfairness, we argue that AI developers should make the inevitable tradeoffs between fairness goals as consciously and well-informed as the context will allow. Beyond that, in the absence of clear legal requirements to prioritize one form of fairness over another, an algorithm that makes well-considered trade-offs between values should be deemed “fair enough”.
  • (*) Sofia Ranchordás, Empathy in the Digital Administrative State, 71 Duke L.J. 1341 (2022).
    • “Humans make mistakes. Humans make mistakes especially while filling out tax returns, benefit applications, and other government forms, which are often tainted with complex language, requirements, and short deadlines. However, the unique human feature of forgiving these mistakes is disappearing with the digitalization of government services and the automation of government decision-making. While the role of empathy has long been controversial in law, empathic measures have helped public authorities balance administrative values with citizens’ needs and deliver fair and legitimate decisions. The empathy of public servants has been particularly important for vulnerable citizens (for example, disabled individuals, seniors, and underrepresented minorities). When empathy is threatened in the digital administrative state, vulnerable citizens are at risk of not being able to exercise their rights because they cannot engage with digital bureaucracy.”This Article argues that empathy, which in this context is the ability to relate to others and understand a situation from multiple perspectives, is a key value of administrative law deserving of legal protection in the digital administrative state. Empathy can contribute to the advancement of procedural due process, the promotion of equal treatment, and the legitimacy of automation. The concept of administrative empathy does not aim to create arrays of exceptions, nor imbue law with emotions and individualized justice. Instead, this concept suggests avenues for humanizing digital government and automated decision-making through a more complete understanding of citizens’ needs. This Article explores the role of empathy in the digital administrative state at two levels: First, it argues that empathy can be a partial response to some of the shortcomings of digital bureaucracy. At this level, administrative empathy acknowledges that citizens have different skills and needs, and this requires the redesign of pre-filled application forms, government platforms, algorithms, as well as assistance. Second, empathy should also operate ex post as a humanizing measure which can help ensure that administrative mistakes made in good faith can be forgiven under limited circumstances, and vulnerable individuals are given second chances to exercise their rights.”Drawing on comparative examples of empathic measures employed in the United States, the Netherlands, Estonia, and France, this Article’s contribution is twofold: first, it offers an interdisciplinary reflection on the role of empathy in administrative law and public administration for the digital age, and second, it operationalizes the concept of administrative empathy. These goals combine to advance the position of vulnerable citizens in the administrative state.”
  • For a somewhat contrasting view to Ranchordas, see Benjamin Minhao Chen, Alexander Stremitzer, and Kevin Tobia, Having Your Day in Robot Court 36 Harv. J.L. & Tech. 128 (2022):
    • “To have your “day in court” is to have a human hear and adjudicate your claims. Two original experiments assess whether laypeople share this intuition. We discover that laypeople do, in fact, see human judges as fairer than artificially intelligent (“AI”) robot judges: All else equal, there is a perceived human-AI “fairness gap.” However, it is also possible to eliminate the fairness gap. The perceived advantage of human judges over AI judges is related to perceptions of accuracy and comprehensiveness of the decision, rather than “softer” and more distinctively human factors. Moreover, the study reveals that laypeople are amenable to “algorithm offsetting.” Adding an AI hearing and increasing the AI interpretability reduces the perceived humanAI fairness gap. Ultimately, the results support a common challenge to robot judges: there is a concerning human-AI fairness gap. Yet, the results also indicate that the strongest version of this challenge — human judges have inimitable procedural fairness advantages — is not reflected in the views of laypeople. In some circumstances, people see a day in robot court as no less fair than day in human court”

Public Law Review of AI Action More Generally

  • (*)  Rónán Kennedy, Rules as code and the rule of law: ensuring effective judicial review of administration by software, Law, Innovation & Technology (Feb 09, 2024):
    • This paper considers the possible benefits and substantial risks of ‘Rules as Code’, the parallel drafting of legislation and codification in software, which has been the subject of attention from policy-makers and pilot studies in some jurisdictions. It highlights the long history of these approaches, and the challenges of ossification, mis-translation of rules, and separation of powers problems. It also examines in the detail the Australian Pintarich case, which demonstrates the inadequacy of conventional judicial review of automated decision-making. It outlines some possible solutions to these issues — two ‘internal’ to development processes (greater transparency, and literate pair programming) and two ‘external’ (expanding the capacity of judicial review to look beyond a specific citizen/state interaction and consider the design and development of the controlling software system, and greater cross-disciplinary awareness by lawyers).
  • Giulia G. Cusenza, Litigating Governmental Use of AI, 21 Ohio St. Tech. L.J. (OSTLJ) 1 (2024):
    • In the last decade US courts have ruled cases related to the use of AI by governmental bodies. But while legal disputes have served as trailblazer for relevant policy documents and have been used by scholars to support specific arguments, this litigation has not been the subject of a systematic analysis. This paper fills this gap and provides a quantitative and qualitative study of how courts deal with litigation on the use of AI by governmental bodies. The analysis leads to an overarching conclusion, namely that judicial decisions almost exclusively rely on procedural grounds – specifically those concerning due process infringements – thus suggesting that substantial issues are typically addressed through procedural solutions. In turn, these procedural issues consist of six violations: lack of adequate notice and explanation, lack of contestability, lack of human oversight, lack of notice and comment procedures, lack of assessment procedures, and denial of the right to access information. By revealing this tendency and by identifying the six procedural violations, the analysis ultimately provides a taxonomy of the minimum requirements that any governmental body should comply with to shield their use of algorithmic systems from judicial review.
  • Brandon L. Garrett & Cynthia Rudin, Testing AI (September 06, 2024):
    • AI is being used throughout government in hundreds of settings, including those that affect people’s core constitutional rights. In response, however, many judges, officials, and scholarly commenters have uncritically credited the claims made by the developers that these systems are reliable and have been subjected to rigorous testing. All too often, those assurances have not been borne out once independent researchers test the AI systems.We set out four fundamental principles for robust ex ante AI testing in Part I of this Article. AI testing must: (1) evaluate performance under realistic conditions, and be independent and conducted by outsiders without a financial interest in the evaluation; (2) provide adequate access to the model to permit it to be evaluated; (3) publicly disclose results of testing, including statistical uncertainty regarding limits of the predictions the model makes; and (4) provide interpretability in contexts affecting people’s rights, meaning that people know how it reaches results in a particular case.In Part II of this article, we describe how few legislative, regulatory, or judicial efforts address, much less minimally satisfy, the four principles. We first describe how challenging it is for judges to review AI ex post, particularly when no adequate testing has been conducted. Second, we describe more recent statutes and regulations concerning AI, which have begun to call for testing, but largely ex post, and by leaving the task to the developers themselves. The new European Union AI Act takes a hands-off approach leaving testing to the developers, as does recent state-level legislation in the U.S. More promising but yet-to-be implemented, the U.S. Office of Management and Budget (OMB) set out AI procurement guidelines for federal agencies with more detailed testing criteria, but once again, largely relying on developers to self-certify they meet the criteria. None of these legal regimes satisfy our four principles for adequate AI testing.In Part III, we describe regulations that satisfy our four principles for ex ante AI testing. If AI systems are used in court—or in ways that otherwise affect people’s rights—then they must be adequately tested. And if they are not, we describe the necessary remedies, through judicial rulings, regulation and legislation, to protect our rights in an increasingly AI-dominated world. Those laws should in turn be informed by scientific principles for how to adequately test an AI system.
  • (*) Catherine Sharkey, AI for Retrospective Review, 8 Belmont L. Rev. Art. 3 (2021):
    • “This Article [explores] the significant administrative law issues that agencies will face as they devise and implement AI-enhanced strategies to identify rules that should be subject to retrospective review. Part I introduces the effect of politics on retrospective review by canvassing both the consistencies and differing emphases of the relevant executive orders across the Obama and Trump Administrations. The HHS pilot is then presented as an innovative case study in its own right that also frames some generalizable salient administrative law design and oversight issues. In addition to promulgating the first rule using AI technologies, HHS has historically provided robust descriptions of its approach to identifying regulations for retrospective review. HHS, moreover, has put itself forward as the leading federal agency for “regulatory reform.”
      Part II sheds light on both the peril and future promise of the deployment of AI in the retrospective review process …*)
  • Margot E. Kaminski & Jennifer M. Urban, The Right to Contest AI, 121 Columb. L. Rev 1957 (2021):
    • “In the United States, recent proposals for regulating AI focus largely on ex ante and systemic governance. This Article argues instead—or really, in addition—for an individual right to contest AI decisions, modeled on due process but adapted for the digital age. The European Union, in fact, recognizes such a right, and a growing number of institutions around the world now call for its establishment. This Article argues that despite considerable differences between the United States and other countries,establishing the right to contest AI decisions here would be in keeping with a long tradition of due process theory.
      “This Article then fills a gap in the literature, establishing a theoretical scaffolding for discussing what a right to contest should look like in practice. This Article establishes four contestation archetypes that should serve as the bases of discussions of contestation both for the right to contest AI and in other policy contexts. The contestation archetypes vary along two axes: from contestation rules to standards and from emphasizing procedure to establishing substantive rights. This Article then discusses four processes that illustrate these archetypes in practice, including the first in depth consideration of the GDPR’s right to contestation for a U.S. audience. Finally, this Article integrates findings from these investigations to develop normative and practical guidance for establishing a right to contest AI.”
  • Sonya Katel, Democracy and Distrust in an Era of Artificial Intelligence, — Daedalus — (May, 2022):
    • “Our legal system has historically operated under the general view that courts should defer to the legislature. There is one significant exception to this view: cases in which it appears that the political process has failed to recognize the rights or interests of minorities. This basic approach provides much of the foundational justifications for the role of judicial review in protecting minorities from discrimination by the legislature. Today, the rise of AI decision-making poses a similar challenge to democracy’s basic framework. As I argue in this essay, the rise of three trends–privatization, prediction, and automation in AI–have combined to pose similar risks to minorities. In this essay, I outline what a theory of judicial review would look like in an era of artificial intelligence, analyzing both the limitations and the possibilities of judicial review of AI. Here, I draw on cases in which AI decision-making has been challenged in courts, to show how concepts of due process and equal protection can be recuperated in a modern AI era, and even integrated into AI, to provide for better oversight and accountability. “
  • Hannah Bloch-Wehba, Access to Algorithms, 88 Fordham L. Rev. 1265 (2020).
  • Danielle Keats Citron, Technological Due Process, 85 Wash. U.L. Rev. 1249 (2008).

AI in Service to Democracy

  • Raphael Koster et al., Human-centred mechanism design with Democratic AI, 6 Nature Human Behaviour 1398 (2022):
    • Building artificial intelligence (AI) that aligns with human values is an unsolved problem. Here we developed a human-in-the-loop research pipeline called Democratic AI, in which reinforcement learning is used to design a social mechanism that humans prefer by majority. A large group of humans played an online investment game that involved deciding whether to keep a monetary endowment or to share it with others for collective benefit. Shared revenue was returned to players under two different redistribution mechanisms, one designed by the AI and the other by humans. The AI discovered a mechanism that redressed initial wealth imbalance, sanctioned free riders and successfully won the majority vote. By optimizing for human preferences, Democratic AI offers a proof of concept for value-aligned policy innovation.

Notes & Questions

  1. One of the attractions of using ML systems is that they might bring efficiency gains. Do these gains seem equally likely and attractive in the public and private sectors?
  2. Are there government services that could be too efficient?
  3. Is the case for human-in-the-loop any different in public than in private sector applications?
  4. If and when the government adopts more ML into its decision-making, what changes if any might we expect in the legal rules that permit review of those decisions?
    1. To the extent that those changes include a reduction in the human ability to modify outcomes based on some personal theory of ‘equity’, would that be a gain or a loss?
    2. To the extent that we might hope the ML can consider more factors than a human, would that be a gain or a loss?
      1. To what extent does your answer to the above depend on the ML not being a ‘black box’?
      2. Are not people doing ‘equity’ something of a black box too?
  5. Conglianese & Lehr present a fairly optimistic picture of the reception of ML systems into government decision-making. What might go wrong?
    1. FYI, Re and Solow-Niederman in the optional reading are considerably more pessimistic.
  6. The core of Mulligan & Bamberger’s argument is that historically agencies have been subject to (the threat of) fairly detailed review when they make decisions that effect people. But, other than people who may have had their offer to supply goods or services rejected, most people don’t have a right to go to court when an agency makes a procurement decision (buys stuff). The problem they are concerned with is that agencies will “procure” ML systems with decisions (biases?) built-in, and then deploy them. Because they are black-box-like it will be very difficult for persons harmed by the systems’ decisions to prove they were unfairly treated, so traditional meaningful review of decisions will be strangled.
    1. The decision-making record will be sparse to non-existent. Figuring out reasons for a decision will be hard to impossible.
    2. The agency humans may not even have, or feel they have, discretion to overrule the ML system’s “recommendation”.
  7. If the law imposes a duty on the agency to engage in “reasoned deliberation” will relying on an ML system’s recommendation suffice to meet this legal duty?
  8. If agencies have a duty of transparency in decision-making, does that require them to make training data and tuning choices during ML design available? What if the companies providing the ML system say that those are trade secrets?
  9. What is the public role in the adoption of ML systems? How do we replicate the virtues of notice & comment rulemaking when the real decision-making is in the ML system?
  10. Huq identifies a number of ways ML can go wrong, most of which should be familiar by now. He seems to think that constitutional litigation can expose and correct many of these flaws.
    1. How does he envision such lawsuits working?
    2. What legal claims might plaintiffs make?
      1. Of the claims Huq discusses, which if any seem most realistic?
      2. What sort of information and expertise will be required to prevail?
      3. What incentives would plaintiffs have to bring cases especially if damages (as opposed to injunctions) are rare?
    3. Are there changes in constitutional doctrine we should consider to facilitate valid suits for bad ML implementations in government decision-making?
  11. Ranchordás [in the optional reading] asserts that “that empathy, which in this context is the ability to relate to others and understand a situation from multiple perspectives, is a key value of administrative law deserving of legal protection in the digital administrative state. ”
    1. What about the argument that this is entirely backwards?  Rather than ‘excuse’ errors when triers think a person has limited “skills” or are “vulnerable” shouldn’t we aim for equal – dare one say mechanical? – justice for all, in which like cases are treated alike?
      1. Ah, but what makes cases “alike”?
      2. How would we decide fairly who is entitled to second chances and who is not?  And if we can express that in some fashion, can’t we mechanize that too?
    2. Some theories of due process stress the importance of ‘dignity values’ part of which is emerging from adjudication, win or lose, with a sense of being heard.
      1. Would ML justice tend to enhance or subvert this property?
      2. Suppose people came to see ML justice as fair, but a bit brutal– how would that play out?
  12. DOGE (Elon Musk consigliere-in-chief) appears to be engaged in a campaign of (purportedly?) firing federal government workers. The unverified rumor (as of mid-Feb 2025, anyway) is that the plan is to replace many of them with “AI”.
    1. Is that likely to be feasible?
    2. What could go wrong?
    3. Suppose it goes spectacularly wrong in some way.  What then?

Class 16: AI & Privacy

Some background in how tech undermines privacy

  1. Daniel J. Solove, A Regulatory Roadmap to AI and Privacy, IAPP News (Apr. 24, 2024).
  2. Charles Duhigg, How Companies Learn Your Secrets, NY Times (Feb. 16, 2012).
  3. Pages 1740-45 of Paul Ohm, Broken Promises of Privacy: Responding to the Surprising Failure of Anonymization, 57 UCLA L. Rev. 1701 (2010).

AI specifically

  1. Bruce Schneier, AI and Mass Spying, Schneier on Security (Dec. 8, 2023).  (Original version here.)
  2. Uri Gal, ChatGPT is a data privacy nightmare, and we ought to be concerned, The Conversation (Feb. 8 2023).
  3. Pages 4-5, 11-12, 15-29 of Datatilsynet, Norwegian Data Protection Authority, Artificial Intelligence and Privacy (Jan, 2018).
  4. Ian Kerr, Schrodinger’s Robot: Privacy in Uncertain States, 20 Theoretical Inquiries in Law (2019).
  5. Parts II & IV (i.e. pages 116-33 & 160-185) of Karl Manheim & Lyric Kaplan, Artificial Intelligence: Risks to Privacy and Democracy, 21 Yale J. L. & Tech. 106 (2019).

Optional Readings

In General

  • (*) Daniel J. Solove, Artificial Intelligence and Privacy, 77 Fl. St. L. Rev. 1 (2025):
    • Some commentators question whether privacy law is appropriate for addressing AI. In this Article, I contend that although existing privacy law falls far short of resolving the privacy problems with AI, privacy law properly conceptualized and constituted would go a long way toward addressing them.Privacy problems emerge with AI’s inputs and outputs. These privacy problems are often not new; they are variations of longstanding privacy problems. But AI remixes existing privacy problems in complex and unique ways. Some problems are blended together in ways that challenge existing regulatory frameworks. In many instances, AI exacerbates existing problems, often threatening to take them to unprecedented levels.Overall, AI is not an unexpected upheaval for privacy; it is, in many ways, the future that has long been predicted. But AI glaringly exposes the longstanding shortcomings, infirmities, and wrong approaches of existing privacy laws.Ultimately, whether through patches to old laws or as part of new laws, many issues must be addressed to confront the privacy problems that AI is affecting. In this Article, I provide a roadmap to the key issues that the law must tackle and guidance about the approaches that can work and those that will fail.
  • (*) Hannah Brown et al., What Does it Mean for a Language Model to Preserve Privacy?,  arXiv:2202.05520v2 [stat.ML] 14 Feb 2022 :
    • “Natural language reflects our private lives and identities, making its privacy concerns as broad as those of real life. Language models lack the ability to understand the context and sensitivity of text, and tend to memorize phrases present in their training sets. An adversary can exploit this tendency to extract training data. Depending on the nature of the content and the context in which this data was collected, this could violate expectations of privacy. Thus, there is a growing interest in techniques for training language models that preserve privacy. In this paper, we discuss the mismatch between the narrow assumptions made by popular data protection techniques (data sanitization and differential privacy), and the broadness of natural language and of privacy as a social norm. We argue that existing protection methods cannot guarantee a generic and meaningful notion of privacy for language models. We conclude that language models should be trained on text data which was explicitly produced for public use.”
  • (*) Margot E. Kaminsky & Gianclaudio Malgiere,  Impacted Stakeholder Participation in AI and Data Governance, Yale J. L. & Tech. (forthcoming 2025):
    • Privacy law has long centered on the individual. But something else is afoot. There is growing recognition that data-driven practices, including the development and use of AI systems, impact not just atomized individuals but neighborhoods, communities, and groups, including and especially situationally vulnerable and historically marginalized persons. This Article explores a nascent and important shift in both data privacy law and the newly developing law of Artificial Intelligence (AI): a turn to stakeholder participation, specifically by impacted groups often representing historically marginalized communities. In this Article we chart this development across an array of recent laws in both the United States and the European Union. We explain reasons for the turn, both theoretical and practical. We then turn to analysis of the legal scaffolding of impacted stakeholder participation, establishing a catalog of both existing and possible interventions. We close with a call for reframing impacted stakeholders as rights-holders and the recognition of several variations on a group right to contest AI systems, among other collective means of leveraging and invoking rights individuals have already been afforded.
  • (*) Centre for Information Policy Leadership, Applying Data Protection Principles to Generative AI: Practical Approaches for Organizations and Regulators Discussion Paper (Dec. 2024):
    • Offers 14 recommendations including (emphasis in original):
      • (3) Organizations should be able to rely on the “legitimate interests” legal basis for processing publicly available personal data collected through web scraping and personal data that they already have in their possession and control (first-party data) for genAI model training, as long as the interest concerned (which could be the controller’s, users’, or society’s at large) is not outweighed by the fundamental rights of individuals and appropriate, risk -based mitigation measures are put in place.(4) Laws and regulatory guidance should be drafted or interpreted to recognize and enable the processing and retention of sensitive personal data for AI model training, as this is necessary to avoid algorithmic bias or discrimination and ensure content safety. In addition, sensitive personal data may be necessary for the training and development of certain AI systems whose sole purpose is based on the processing of sensitive personal data or to deliver benefits to protected categories of individuals (such as accessibility tools, or health systems).(6) The fairness principle is useful in the genAI context and should be interpreted to facilitate personal data processing in genAI model development to train accurate and accessible models that do not unjustly discriminate. Consideration of fairness also need to take into account the impact on the individual or society of not developing a particular AI application.(9) Purpose or use limitation principles should be sufficiently flexible: In the context of genAI, purpose limitation principles in laws and regulations should allow organizations to articulate data processing purposes that are sufficiently flexible for the range of potentially useful applications for which genAI models may be used. Furthermore, processing personal data for the development of a genAI model should be treated as a separate purpose from processing personal data for the development, deployment or improvement of a specific application that uses a genAI model
  • Christopher Lutz & Aurelia Tamó-Larrieux, The robot privacy paradox: Understanding how privacy concerns shape intentions to use social robots, 1 Human-Machine Communication 87 (2020):
    • (“Respondents are most concerned about data protection on the manufacturer side, followed by social privacy concerns and physical concerns. Using structural equation modeling, we find a privacy paradox, where the perceived benefits of social robots override privacy concerns.”
  • Andrea Scripa Els, Artificial Intelligence As A Digital Privacy Protector, 31 Harv. J.L. & Tech. 217 (2017):
    • “…explores the ways in which AI and related computational methods might be used to enhance protections for personal privacy, either by mitigating AI’s own negative effects or by addressing privacy concerns caused by other trends. Part II outlines four significant ways in which AI might be used to bolster privacy protections, while Part III proposes strategies for reinforcing these technological solutions with legal measures or private conduct.”
  • Mellissa Hikkllä, What does GPT-3 “know” about me?, MIT Tech. Rev. (Aug. 31, 2022).
  • (*) Mireille Hildebrandt, Privacy as Protection of the Incomputable Self: From Agnostic to Agonistic Machine Learning, 20 Theoretical Inquiries in Law 83 (2019).
    • A leading scholar in law and philosopher argues that “in the era of big data analytics we need an understanding of privacy that is capable of protecting what is uncountable, incalculable or incomputable about individual persons. To instigate this new dimension of the right to privacy, I expand previous work on the relational nature of privacy, and the productive indeterminacy of human identity it implies, into an ecological understanding of privacy, taking into account the technological environment that mediates the constitution of human identity. … [The article] advocates a practice of ‘agonistic machine learning’ that will contribute to responsible decisions about the integration of data-driven applications into our environments while simultaneously bringing them under the Rule of Law. This should also provide the best means to achieve effective protection against overdetermination of individuals by machine inferences.”
  • Daniel J. Solove & Woodrow Hartzog, The Great Scrape: The Clash Between Scraping and Privacy, 113 California Law Review (forthcoming 2025):
    • Artificial intelligence (AI) systems depend on massive quantities of data, often gathered by “scraping” – the automated extraction of large amounts of data from the internet. A great deal of scraped data is about people. This personal data provides the grist for AI tools such as facial recognition, deep fakes, and generative AI. Although scraping enables web searching, archival, and meaningful scientific research, scraping for AI can also be objectionable or even harmful to individuals and society.Organizations are scraping at an escalating pace and scale, even though many privacy laws are seemingly incongruous with the practice. In this Article, we contend that scraping must undergo a serious reckoning with privacy law. Scraping violates nearly all of the key principles in privacy laws, including fairness; individual rights and control; transparency; consent; purpose specification and secondary use restrictions; data minimization; onward transfer; and data security. With scraping, data protection laws built around these requirements are ignored.Scraping has evaded a reckoning with privacy law largely because scrapers act as if all publicly available data were free for the taking. But the public availability of scraped data shouldn’t give scrapers a free pass. Privacy law regularly protects publicly available data, and privacy principles are implicated even when personal data is accessible to others.This Article explores the fundamental tension between scraping and privacy law. With the zealous pursuit and astronomical growth of AI, we are in the midst of what we call the “great scrape.” There must now be a great reconciliation.

Proposed Improvements

  • (*) Daniel J. Solove & Woodrow Hartzog, Kafka in the Age of AI and the Futility of Privacy as Control, 104 Boston University Law Review 1021 (2024):
    • Although writing more than a century ago, Franz Kafka captured the core problem of digital technologies – how individuals are rendered powerless and vulnerable. During the past fifty years, and especially in the 21st century, privacy laws have been sprouting up around the world. These laws are often based heavily on an Individual Control Model that aims to empower individuals with rights to help them control the collection, use, and disclosure of their data.In this Essay, we argue that although Kafka starkly shows us the plight of the disempowered individual, his work also paradoxically suggests that empowering the individual isn’t the answer to protecting privacy, especially in the age of artificial intelligence. In Kafka’s world, characters readily submit to authority, even when they aren’t forced and even when doing so leads to injury or death. The victims are blamed, and they even blame themselves.Although Kafka’s view of human nature is exaggerated for darkly comedic effect, it nevertheless captures many truths that privacy law must reckon with. Even if dark patterns and dirty manipulative practices are cleaned up, people will still make bad decisions about privacy. Despite warnings, people will embrace the technologies that hurt them. When given control over their data, people will give it right back. And when people’s data is used in unexpected and harmful ways, people will often blame themselves.Kafka’s provides key insights for regulating privacy in the age of AI. The law can’t empower individuals when it is the system that renders them powerless. Ultimately, privacy law’s primary goal should not be to give individuals control over their data. Instead, the law should focus on ensuring a societal structure that brings the collection, use, and disclosure of personal data under control.
  • Note especially Chapters 3 & 4. Jennifer King & Caroline Meinhardt, Rethinking Privacy in the AI Era (Feb. 2024):
    • Data is the foundation of all AI systems. Going forward, AI development will continue to increase developers’ hunger for training data, fueling an even greater race for data acquisition than we have already seen in past decades. [….]  Even legislation that contains explicit provisions on algorithmic decision-making and other forms of AI does not provide the data governance measures needed to meaningfully regulate the data used in AI systems.We present three suggestions for how to mitigate the risks to data privacy posed by the development and adoption of AI:
      1. Denormalize data collection by default by shifting away from opt-out to opt-in data collection. Data collectors must facilitate true data minimization through “privacy by default” strategies and adopt technical standards and infrastructure for meaningful consent mechanisms.
      2. Focus on the AI data supply chain to improve privacy and data protection. Ensuring dataset transparency and accountability across the entire life cycle must be a focus of any regulatory system that addresses data privacy.
      3. Flip the script on the creation and management of personal data. Policymakers should support the development of new governance mechanisms and technical infrastructure (e.g., data intermediaries and data permissioning infrastructure) to support and automate the exercise of individual data rights and preferences.
  • Cheng-chi (Kirin) Chang, When AI Remembers Too Much: Reinventing the Right to Be Forgotten for the Generative Age, 19 Wash J. L. Tech & Arts 22 (2024):
    • The emergence of generative artificial intelligence (AI) systems poses novel challenges for the right to be forgotten. While this right gained prominence following the 2014 Google Spain v. Gonzalez case, generative AI’s limitless memory and ability to reproduce identifiable data from fragments threaten traditional conceptions of forgetting. This article traces the evolution of the right to be forgotten from its privacy law origins towards an independent entitlement grounded in self-determination for personal information. However, it contends the inherent limitations of using current anonymization, deletion, and geographical blocking mechanisms to prevent AI models from retaining personal data render forgetting infeasible. Moreover, the technical costs of forgetting – including tracking derivations and retraining models – could undermine enforceability. Therefore, this article advocates for a balanced legal approach that acknowledges the value of the right to forget while considering the constraints of implementing the right forgenerative AI. Although existing frameworks like the European Union’s GDPR provide a foundation, continuous regulatory evolution through oversight bodies and industry collaboration is imperative. This article underscores how the right to be forgotten must be reconceptualized to address the reality of generative AI systems. It provides an interdisciplinary analysis of this right’s limitations and proposes strategies to reconcile human dignity and autonomy with the emerging technological realities of AI. The article’s original contribution lies in its nuanced approach to integrating legal and technical dimensions to develop adaptive frameworks for the right to be forgotten in the age of generative AI.

Dangers

  • Highly recommended. Laura K. Donohue, Biomanipulation, Georgetown L.J. (forthcoming).
    • This Article throws down the gauntlet, naming, describing, and challenging the practice of biomanipulation. It begins in Part II.A by establishing what is meant by manipulation: knowingly shaping a target’s beliefs, desires, and emotions and/or behavior by covertly exploiting a target’s vulnerabilities with the aim of altering the status quo.32 It encompasses both action and non-action. Any altered state of belief, desire, emotion, or behavior proves sufficient. Part II.B establishes that biomanipulation, a subset of the broader category, seeks to accomplish the third party’s aim by employing biometric data in at least one of four ways: as a measurement of the target’s biological features or processes; as insight into the target’s emotional, cognitive or behavioral responses; as a correlative device, associating either of the first two with a particular person, group, or community; and/or as a means of delivery. Part II.C distinguishes biomanipulation from consumer and market manipulation as traditionally understood by law and economics in regard to the degree of persuasion employed (i.e., the opportunity afforded the target to engage in rational decisionmaking); knowledge present (i.e., the extent to which the target has requested and is aware of being exposed to input or stimuli at the moment of delivery); and consent granted (i.e., whether and to what extent collection and subsequent stimuli fall within the specific purpose for which the target provided the biometric data). It is not just corporations who have access to the data: private parties, state actors, and others can use the same insights, with the result that concerns go well beyond the traditional focus of law and economics.
  • (*) Hideyuki Matsumi & Daniel J. Solove, The Prediction Society: Algorithms and the Problems of Forecasting the Future (Draft July 30 , 2023):
    • Predictions about the future have been made since the earliest days of humankind, but today, we are living in a brave new world of prediction. Today’s predictions are produced by machine learning algorithms that analyze massive quantities of personal data. Increasingly, important decisions about people are being made based on these predictions.Algorithmic predictions are a type of inference. Many laws struggle to account for inferences, and even when they do, the laws lump all inferences together. But as we argue in this Article, predictions are different from other inferences. Predictions raise several unique problems that current law is ill-suited to address. First, algorithmic predictions create a fossilization problem because they reinforce patterns in past data and can further solidify bias and inequality from the past. Second, algorithmic predictions often raise an unfalsifiability problem. Predictions involve an assertion about future events. Until these events happen, predictions remain unverifiable, resulting in an inability for individuals to challenge them as false. Third, algorithmic predictions can involve a preemptive intervention problem, where decisions or interventions render it impossible to determine whether the predictions would have come true. Fourth, algorithmic predictions can lead to a self-fulfilling prophecy problem where they actively shape the future they aim to forecast.More broadly, the rise of algorithmic predictions raises an overarching concern: Algorithmic predictions not only forecast the future but also have the power to create and control it. The increasing pervasiveness of decisions based on algorithmic predictions is leading to a prediction society where individuals’ ability to author their own future is diminished while the organizations developing and using predictive systems are gaining greater power to shape the future.Privacy and data protection law do not adequately address algorithmic predictions. Many laws lack a temporal dimension and do not distinguish between predictions about the future and inferences about the past or present. Predictions about the future involve considerations that are not implicated by other types of inferences. Many laws provide correction rights and duties of accuracy that are insufficient to address problems arising from predictions, which exist in the twilight between truth and falsehood. Individual rights and anti-discrimination law also are unable to address the unique problems with algorithmic predictions.We argue that the use of algorithmic predictions is a distinct issue warranting different treatment from other types of inference. We examine the issues laws must consider when addressing the problems of algorithmic predictions.
  • (*) Richard Warner & Robert H. Sloan, How AI Unfairly Tilts the Playing Field: Privacy, Fairness, and Risk Shadows (July 22, 2023).
    • Private sector applications of artificial intelligence (AI) raise related questions of informational privacy and fairness. Fairness requires that market competition occurs on a level playing field, and uses of AI unfairly tilt the field. Informational privacy concerns arise because AI tilts the playing field by taking information about activities in one area of one’s life and using it in ways that impose novel risks in areas not formerly associated with such risks. The loss of control over that information constitutes a loss of informational privacy. To illustrate both the fairness and privacy issues, imagine, for example, that Sally declares bankruptcy after defaulting on $50,000 of credit card debt. She incurred the debt by paying for lifesaving medical treatment for her eight-year-old daughter. Post-bankruptcy Sally is a good credit risk. Her daughter has recovered, and her sole-proprietor business is seeing increased sales. Given her bankruptcy, however, an AI credit scoring system predicts that she is a poor risk and assigns her a low score. That low credit score casts a shadow that falls on her when her auto insurance company, which uses credit scores in its AI system as a measure of the propensity to take risks, raises her premium. Is it fair that saving her daughter’s life should carry with it the risk—realized in this case—of a higher premium? The pattern is not confined to credit ratings and insurance premiums. AI routinely creates risk shadows.
      “We address fairness questions in two steps. First, we turn to philosophical theories of fairness as equality of opportunity to spell out the content behind our metaphor of tilting the playing field. Second, we address the question of how, when confronted with a mathematically complex AI system, one can tell whether the system meets requirements of fairness. We answer by formulating three conditions whose violation makes a system presumptively unfair. The conditions provide a lens that reveals relevant features when policy makers and regulators investigate complex systems. Our goal is not to resolve fairness issues but to contribute to the creation of a forum in which legal regulators and affected parties can work to resolve them. The third of our three condition requires that systems incorporate contextual information about individual consumers, and we conclude by raising the question of whether our suggested approach to fairness significantly reduces informational privacy. We do not answer the question but emphasize that fairness and informational privacy questions can closely intertwine.”

Use Cases, Including Scary Ones

  • (*) Mason Marks,Emergent Medical Data: Health Information Inferred by Artificial Intelligence, 11 U.C. Irvine L. Rev. 995 (Apr. 2021):
    • “Artificial intelligence (AI) can infer health data from people ’s behavior even when their behavior has no apparent connection to their health . AI can monitor one’s location to track the spread of infectious disease, scrutinize retail purchases to identify pregnant customers, and analyze social media to predict who might attempt suicide . These feats are possible because, in modern societies, people continuously interact with internet-enabled software and devices. Smartphones, wearables, and online platforms monitor people ’s actions and produce digital traces, the electronic remnants of their behavior.
      “In their raw form, digital traces might not be very interesting or useful; one ’s location, retail purchases, and internet browsing habits are relatively mundane data points . However, AI can enhance the value of digital traces by transforming them into something more useful—emergent medical data (EMD). EMD is health information inferred by arti ficial intelligence from otherwise trivial digital traces.
      “This Article describes how EMD-based profiling is increasingly promoted as a solution to public health crises such as the COVID -19 pandemic, gun violence, and the opioid crisis . However, there is little evidence to show that EMD -based profiling works. Even worse, it can cause significant harm, and current privacy and data protection laws contain loopholes that allow public and private entities to mi ne EMD without people’s knowledge or consent. After describing the risks and benefits of EMD mining and pro filing, the Article proposes six different ways of conceptualizing these practices . It concludes with preliminary recommendations for effective regulation. Potential options include banning or restricting the collection of digital traces, regulating EMD mining algorithms, and restricting how EMD can be used once it is produced .”
  • Jianna Jin, Jesse Walker & Rebecca Walker Reczek, Avoiding embarrassment online: Response to and inferences about chatbots when purchases activate self-presentation concerns, J. Consumer Psych. (Feb. 14, 2024):
    • We propose that consumers feel less embarrassed with a chatbot than a human service agent in purchase contexts where self-presentation concerns are active because consumers ascribe less mind to chatbots. Therefore, when a chat agent’s identity is ambiguous, consumers with greater self-presentation concerns are more likely to infer that an agent is human because this judgment allows consumers to proactively protect themselves from potential embarrassment in the event they are interacting with a human. We further show that when agent identity is clearly disclosed, consumers respond more positively to chatbots than human agents. However, this effect is contingent on the extent to which the chatbot is imbued with human characteristics: Anthropomorphizing chatbots leads consumers with higher self-presentation concerns to ascribe more mind to even clearly identified chatbots, resulting in a more negative consumer response.
  • Solon Barocas, Karen Levy, and Alexandra Mateescu, Reap What You Sow? Precision Agriculture and The Privacy of Farm Data (preliminary draft, We Robot 2019):
    • This, alas unfished, article argues that farmers whose data is visible to neighbors, landlords, and others, become more vulnerable.
  • Michael Lyos, Excavating “Excavating AI”: The Elephant in the Gallery (Dec. 24, 2020).
    • “Our analysis underlines the non-negotiability of informed consent when using human data in artistic and other contexts, and clarifies issues relating to the description of ML training sets. “
  • Chapter 8 (pages 141-154), or National Security Commission on Artificial Intelligence, Final Report (2021).
    • The National Security Commission Report on AI, of which this chapter is a part, is mostly about how the US is going to have its lunch eaten and pocket picked by foreigners, probably the Chinese, stealing a march on the US in the AI “race”. To my jaundiced eye, it’s a document aimed at scaring Congress into appropriating a ton more money to the Pentagon for new programs. (And although most of it didn’t make it into the most recent budget, there is substantial spending going on (not to mention whatever is in the secret budget) and they are still trying for more.) That doesn’t, however, mean this chapter, or the report, is wrong.
  • (*) Compare Kevin Werbach, Orwell That Ends Well? Social Credit as Regulation for the Algorithmic Age, 2022 U. Ill. L. Rev 1418 (2022):
    • “China’s Social Credit System (SCS) is a massive government-led initiative to promote data-driven compliance with law and social values. In the West, it is widely viewed as an Orwellian effort to crush resistance to the Chinese Communist Party. Such a picture is, if not wholly incorrect, decidedly incomplete. This Article offers a revisionist account of SCS and its implications for liberal democracies. SCS is, in fact, the world’s most advanced prototype of a regime of algorithmic regulation. As such it could well represent our future. Or it could be a cautionary tale that sets the West on a different path.
      “SCS is a sophisticated effort not only to embrace the capabilities of data analytics, but to restrain their potential abuses. Its three distinctive components are: (1) data as infrastructure; (2) feedback mechanisms which leverage the data to promote outcomes; and (3) an integral oversight regime of guiding principles and legal restraints. The system is imperfect, sometimes intentionally so. Yet both its merits and flaws are instructive.
      “Incorporating lessons from China’s system is crucial for resolving the great legal challenges we face in the emerging era of digital platform power, relentless data aggregation, ubiquitous artificial intelligence, and algorithmic control.”
    • with Xin Dai, Toward a Reputation State: The Social Credit System Project of China (Mar. 10, 2021):
    • “The ongoing “reputation revolution,” as it refers to the widespread adoption of reputation tracking and analytics, is now seeing its frontier shifting from the “reputation society” to the “reputation state”: Government authorities, following the steps of private sector players, now increasingly seek to use reputation mechanisms and technologies in the spheres of law and governance. In 2014, China’s national government unveiled a massive policy plan for building a “social credit system” in China (hereinafter the “SCSP,” which stands for the “social credit system project”). In designing the SCSP, the Chinese government envisioned that reputation mechanisms such as blacklisting, rating, and scoring be used to tackle many of the country’s by far intractable governance and regulatory problems in its social and economic realms, ranging from fraudulent behaviors in the marketplace, to difficulties in enforcing court judgments, to corruption in the government, and to professional malpractices and even plagiarism in scholarship. Although Western governments have also explored using reputation technologies and mechanisms in governance and social control contexts, China’s SCSP, as a comprehensive, singularly framed, and actively implemented policy project, knows no equivalent elsewhere in the world, and thus offers a unique case for studying a rise of the reputation state as a broader trend.
      “This Article provides a systematic descriptive and analytical study on the SCSP. It draws on the general theoretical discussions on the reputation state to identify four key strategies, categorized in this Article as “regulation,” “searchlight,” “incorporation,” and “institutionalization,” which government actors may plausibly employ as they seek to use reputation for law and public administration. While Western experiences already illustrate ways in which these strategies may actually work in practical contexts, China’s SCSP takes the reputation state into the uncharted territory as it explores novel possibilities. Upon examining the major policy initiatives included, this Article finds that the SCSP overall represents the effort of China’s developmental state to tackle its persistent governance problems with new tools. While it has both raised high hopes and stoke grave fears, the SCSP, as this Article explains, has been fundamentally shaped and also limited by many of the institutional and market forces that animate it in the first place. Nonetheless, if future institutional arrangements and technological progress could align to overcome the present implementation challenges, the reputation state effected through the SCSP does have the potential to change law and government as we know them in China and beyond. Specifically, this Article offers initial thoughts on the SCSP’s plausible implications in three areas that are important but inadequately explored, relating in broad terms to government expansion and efficiency, optimal approach to enforcement, and intragovernmental agency control.”
  • Happy Valentine’s Day! Jen Caltrider, Misha Rykov and Zoë MacDonald, Romantic AI Chatbots Don’t Have Your Privacy at Heart, Privacy Not Included (Feb. 14, 2024).
  • Karen Hao, How to poison the data that Big Tech uses to surveil you, MIT Tech. Rev. (Mar. 5, 2021).

Notes & Questions

  1. The Norwegian DPA document gives a gentle introduction to EU thinking about AI and privacy issues through the lens of the over-arching privacy rules, the General Data Protection Regulation (GDPR). More targeted rules about AI are in the works, but this is the existing and framework document. Note how these rules apply to problems we have encountered above, e.g.
    1. Algorithmic bias
    2. Black box problem and the data subject’s rights to be informed of the reasons why a decision is taken against him/her.
  2. Note also the glossary of useful buzzwords on pp. 26-27, especially
    1. Federated Learning
    2. Differential Privacy
    3. Transfer Learning
  3. Speaking of vocabulary, to understand philosophy papers such as Schrodinger’s Robot you will need a working definition of “epistemological” and “epistemic”:
    1. Epistemology can be defined as: the branch of philosophy concerned with the theory of knowledge. Epistemology is the study of the nature of knowledge, justification, and the rationality of belief. Much debate in epistemology centers on four areas: (1) the philosophical analysis of the nature of knowledge and how it relates to such concepts as truth, belief, and justification, (2) various problems of skepticism, (3) the sources and s cope of knowledge and justified belief, and (4) the criteria for knowledge and justification. Epistemology addresses such questions as: “What makes justified beliefs justified?”, “What does it mean to say Wikipedia?”
    2. Epistemic usually means something like “of or relating to knowledge” although it is also used increasingly to mean “cognitive” which is not the same thing.
  4. Kerr’s article starts with the premise that in law robots are a “nullity”.
    1. Does it then follow that if a machine reads my mail, but no human does, there was no ‘search’?
    2. If there wasn’t a legal ‘search’ what should we call what happened?
    3. Is privacy “implicated when only robot eyes are watching,” or does the privacy relationship ultimately require a human observer?
  5. Kerr suggests that “A right to privacy is then ‘a justified expectation of ignorance [on the part of other persons’] with respect to any personal facts that fall within the scope of the right'”.
    1. If an AI makes accurate guesses about you, based on data it can gather, does this reduce your privacy?
    2. If a thermometer makes an accurate reading of your temperature, does that reduce your privacy? How is this like or unlike the previous example?
  6. Kerr suggests that if a machine meets the “Brandon conditions” (see the article), then this has implications for how the law should think about qualifying machines.
    1. Why?
    2. What are those implications?
  7. The Manheim & Kaplan article gives a good, and very standard, account of legal sources of privacy, the reasons for privacy, and of technological threats to privacy. Can you list the sources of privacy law in the US?
    1. How do they suggest that people knowing true stuff about you can hurt you?
    2. Is the greater transparency of the individual–due to ML and to other tech–better understood as a public good (less scope for liars) or a public bad (less scope for privacy)?
    3. Consider the list of US privacy laws. How does they and their reach compare to the GDPR?

Class 17: AI & Lawyering (Part I: Law School & Legal Analysis)

  1. Pages 191-208 of Jonathan H. Choi, Amy B. Monahan & Daniel Schwarcz, Lawyering In The Age of Artificial Intelligence,  109 Minn. L. Rev. 147 (2024).
  2. Intro – § 2.2 & §§ 4.3-end of Eric Martínez, Re-evaluating GPT-4’s bar exam performance, Artificial Intelligence and Law (2024)
    1. FYI, this paper is a response to papers such as [OPTIONAL:] Daniel Martin Katz, Michael James Bommarito, Shang Gao, and Pablo David Arredondo, GPT-4 Passes the Bar Exam (Apr. 5, 2023).
    2. To appreciate this paper, it helps know what “zero-shot” learning is.  You can find an extensive discussion at IBM, What is Zero-Shot Learning, but the key part is this:

      Zero-shot learning (ZSL) is a machine learning scenario in which an AI model is trained to recognize and categorize objects or concepts without having seen any examples of those categories or concepts beforehand.Most state-of-the-art deep learning models for classification or regression are trained through supervised learning, which requires many labeled examples of relevant data classes. Models “learn” by making predictions on a labeled training dataset; data labels provide both the range of possible answers and the correct answers (or ground truth) for each training example. “Learning,” here, means adjusting model weights to minimize the difference between the model’s predictions and that ground truth. This process requires enough labeled samples for many rounds of training and updates.

      While powerful, supervised learning is impractical in some real-world scenarios. Annotating large amounts of data samples is costly and time-consuming, and in cases like rare diseases and newly discovered species, examples may be scarce or non-existent. Consider image recognition tasks: according to one study, humans can recognize approximately 30,000 individually distinguishable object categories.1 It’s not feasible, in terms of time, cost and computational resources, for artificial intelligence models to remotely approach human capabilities if they must be explicitly trained on labeled data for each class.

      The need for machine learning models to be able to generalize quickly to a large number of semantic categories with minimal training overhead has given rise to n-shot learning: a subset of machine learning that also includes few-shot learning (FSL) and one-shot learning. Few-shot learning typically uses transfer learning and meta learning-based methods to train models to quickly recognize new classes with only a few labeled training examples—or, in one-shot learning, a single labeled example.

      Zero-shot learning, like all n-shot learning, refers not to any specific algorithm or neural network architecture, but to the nature of the learning problem itself: in ZSL, the model is not trained on any labeled examples of the unseen classes it is asked to make predictions on post-training.

      This problem setup doesn’t account for whether that class was present (albeit unlabeled) in training data. For example, some large language models (LLMs) are well-suited for ZSL tasks, as they are pre-trained through self-supervised learning on a massive corpus of text that may contain incidental references to or knowledge about unseen data classes. Without labeled examples to draw upon, ZSL methods all rely on the use of such auxiliary knowledge to make predictions.

  3. Selections from Yonathan Arbel & David A. Hoffman, Generative Interpretation,  99 NYU L. REV. 451 (2024).
  4. Pages 1 & 29-35 of Jonathan H. Choi &  Daniel Schwartz, AI Assistance in Legal Analysis-An Empirical Study (Aug. 13, 2023).  (Difficult but valuable optional reading: read the Appendix).
  5. Timothy B. Lee, These pros were stunned by OpenAI Deep Research, Understanding AI (Feb 24, 2025).

Optional

General

  • (*) John Nay, Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humans, 20 Nw. J. Tech. & Info. Prop.  __ (forthcoming 2022):
    • “Law-making and legal interpretation form a computational engine that converts opaque human intentions, goals, and values into legible directives. Law Informs Code is the research agenda capturing complex computational legal processes, and embedding them in AI. Similar to how parties to a legal contract cannot foresee every potential “if-then” contingency of their future relationship, and legislators cannot predict all the circumstances under which their proposed bills will be applied, we cannot ex ante specify “if-then” rules that provably direct good AI behavior. Legal theory and practice have developed arrays of tools to address these specification problems. For instance, legal standards allow humans to develop shared understandings and adapt them to novel situations, i.e., to generalize expectations regarding actions taken to unspecified states of the world. In contrast to more prosaic uses of the law (e.g., as a deterrent of bad behavior through the threat of sanction), leveraged as an expression of how humans communicate their goals, and what society values, Law Informs Code.
      We describe how data generated by legal processes and the practices of law (methods of law-making, statutory interpretation, contract drafting, applications of standards, legal reasoning, etc.) can facilitate the robust specification of inherently vague human goals. This increases human-AI alignment and the local usefulness of AI. Toward society-AI alignment, we present a framework for understanding law as the applied philosophy of multi-agent alignment, harnessing public law as an up-to-date knowledge base of democratically endorsed values ascribed to state-action pairs. Although law is partly a reflection of historically contingent political power – and thus not a perfect aggregation of citizen preferences – if properly parsed, its distillation offers the most legitimate computational comprehension of societal values available. The other data sources suggested for AI alignment – surveys of preferences, humans labeling “ethical” situations, or (most commonly) the implicit beliefs of the AI system designers – lack an authoritative source of synthesized preference aggregation. Law is grounded in a verifiable resolution: ultimately obtained from a court opinion, but short of that, elicited from legal experts. If law eventually informs powerful AI, engaging in the deliberative political process to improve law takes on even more meaning.

Law-school Specific AI

AI’s Doing Lawyer Work

  • (*) Hot off the press! Daniel Schwarcz, Sam Manning, Patrick Barry, David R. Cleveland, J.J. Prescott, Beverly Rich & Ogletree Deakins, AI-Powered Lawyering: AI Reasoning Models, Retrieval Augmented Generation, and the Future of Legal Practice (March 02, 2025):
    • Generative AI is set to transform the legal profession, but its full impact remains uncertain. While AI models like GPT-4 improve the efficiency with which legal work can be completed, they can at times make up cases and “hallucinate” facts, thereby undermining legal judgment, particularly in complex tasks handled by skilled lawyers. This article examines two emerging AI innovations that may mitigate these lingering issues: Retrieval Augmented Generation (RAG), which grounds AI-powered analysis in legal sources, and AI reasoning models, which structure complex reasoning before generating output. We conducted the first randomized controlled trial assessing these technologies, assigning upper-level law students to complete six legal tasks using a RAG-powered legal AI tool (Vincent AI), an AI reasoning model (OpenAI’s o1-preview), or no AI. We find that both AI tools significantly enhanced legal work quality, a marked contrast with previous research examining older large language models like GPT-4. Moreover, we find that these models maintain the efficiency benefits associated with use of older AI technologies. Our findings show that AI assistance significantly boosts productivity in five out of six tested legal tasks, with Vincent yielding statistically significant gains of approximately 38% to 115% and o1-preview increasing productivity by 34% to 140%, with particularly strong effects in complex tasks like drafting persuasive letters and analyzing complaints. Notably, o1-preview improved the analytical depth of participants’ work product but resulted in some hallucinations, whereas Vincent AI-aided participants produced roughly the same amount of hallucinations as participants who did not use AI at all. These findings suggest that integrating domain-specific RAG capabilities with reasoning models could yield synergistic improvements, shaping the next generation of AI-powered legal tools and the future of lawyering more generally.  (Emphasis added)
  • (*) Christoph Engel & Richard H Mcadams, Asking GPT for the Ordinary Meaning of Statutory Terms (Nov. 12, 2024):
    • “We report on our test of the Large Language Model (LLM) ChatGPT (GPT) as a tool for generating evidence of the ordinary meaning of statutory terms. We explain why the most useful evidence for interpretation involves a distribution of replies rather than only what GPT regards as the single “best” reply. That motivates our decision to use Chat 3.5 Turbo instead of Chat 4 and to run each prompt we use 100 times. Asking GPT whether the statutory term “vehicle” includes a list of candidate objects (e.g., bus, bicycle, skateboard) allows us to test it against a benchmark, the results of a high-quality experimental survey (Tobia 2000) that asked over 2,800 English speakers the same questions. After learning what prompts fail and which one works best (a belief prompt combined with a Likert scale reply), we use the successful prompt to test the effects of “informing” GPT that the term appears in a particular rule (one of five possible) or that the legal rule using the term has a particular purpose (one of six possible). Finally, we explore GPT’s sensitivity to meaning at a particular moment in the past (the 1950s) and its ability to distinguish extensional from intensional meaning. To our knowledge, these are the first tests of GPT as a tool for generating empirical data on the ordinary meaning of statutory terms. Legal actors have good reason to be cautious, but LLMs have the potential to radically facilitate and improve legal tasks, including the interpretation of statutes.”
  • (*) Elizabeth C. Tippett, Charlotte S. Alexander, Karl Branting, Paul Morawski, Carlos Balhana, Craig Pfeifer & Sam Bayer, Does Lawyering Matter? Predicting Judicial Decisions from Legal Briefs, and What That Means for Access to Justice, 100 Tex. L. Rev.  1157 (2022):
    • “This study uses linguistic analysis and machine-learning techniques to predict summary judgment outcomes from the text of the briefs filed by parties in a matter. We test the predictive power of textual characteristics, stylistic features, and citation usage, and we find that citations to precedent—their frequency, their patterns, and their popularity in other briefs—are the most predictive of a summary judgment win. This finding suggests that good lawyering may boil down to good legal research. However, good legal research is expensive, and the primacy of citations in our models raises concerns about access to justice. Here, our citation-based models also suggest promising solutions. We propose a freely available, computationally enabled citation identification and brief bank tool, which would extend to all litigants the benefits of good lawyering and open up access to justice.”
  • (*) Andrew Coan & Harry Surden, Artificial Intelligence and Constitutional Interpretation, 96  U. Col. L. Rev 413 (2025):
    •  This Article examines the potential use of large language models (LLMs) like ChatGPT in constitutional interpretation. LLMs are extremely powerful tools, with significant potential to improve the quality and efficiency of constitutional analysis. But their outputs are highly sensitive to variations in prompts and counterarguments, illustrating the importance of human framing choices. As a result, using LLMs for constitutional interpretation implicates substantially the same theoretical issues that confront human interpreters. Two key implications emerge: First, it is crucial to attend carefully to particular use cases and institutional contexts. Relatedly, judges and lawyers must develop “AI literacy” to use LLMs responsibly. Second, there is no avoiding the burdens of judgment. For any given task, LLMs may be better or worse than humans, but the choice of whether and how to use them is itself a judgment requiring normative justification.
  • Peter Henderson et al., Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset (July, 2022):
    • “One concern with the rise of large language models lies with their potential for significant harm, particularly from pretraining on biased, obscene, copyrighted, and private information. Emerging ethical approaches have attempted to filter pretraining material, but such approaches have been ad hoc and failed to take into account context. We offer an approach to filtering grounded in law, which has directly addressed the tradeoffs in filtering material. First, we gather and make available the Pile of Law, a 256GB (and growing) dataset of open-source English-language legal and administrative data, covering court opinions, contracts, administrative rules, and legislative records. Pretraining on the Pile of Law may potentially help with legal tasks that have the promise to improve access to justice. Second, we distill the legal norms that governments have developed to constrain the inclusion of toxic or private content into actionable lessons for researchers and discuss how our dataset reflects these norms. Third, we show how the Pile of Law offers researchers the opportunity to learn such filtering rules directly from the data, providing an exciting new research direction in model-based processing.”
    • Have a look at the associated Datset Card.
  • (Very Mathy, interesting example of novel use of ML) Han-wei Ho, Patrick Chung-Chia Huang, Yun-chien Chang, Machine-Learning Comparative Law, Cambridge Handbook of Comparative Law (Mathias Siems and Po Jen Yap eds, forthcoming 2023):
    • “Comparative lawyers are interested in similarities between legal systems. Artificial intelligence offers a new approach to understanding legal families. This chapter introduces machine-learning methods useful in empirical comparative law, a nascent field. This chapter provides a step-by-step guide to evaluating and developing legal family theories using machine-learning algorithms. We briefly survey existing empirical comparative law data sets, then demonstrate how to visually explore these using a data set one of us compiled. We introduce popular and powerful algorithms of service to comparative law scholars, including dissimilarity coefficients, dimension reduction, clustering, and classification. The unsupervised machine-learning method enables researchers to develop a legal family scheme without the interference from existing schemes developed by human intelligence, thus providing as a powerful tool to test comparative law theories. The supervised machine-learning method enables researchers to start with a baseline scheme (developed by human or artificial intelligence) and then extend it to previously unstudied jurisdictions.”

Notes & Questions

  1. What should we conclude from the performance of LLMs on law school exams and on the bar?
    1. If we conclude that this increasingly good performance is insufficient evidence that we can replace all the lawyers with machines is this an indictment of
      1. The experiments?
      2. Law school exams and/or the bar exam?
    2. If we conclude that this increasingly good performance is insufficient evidence that we can replace all the lawyers with machines, what would constitute sufficient evidence?  What should we be testing for?
  2. The AI Assistance paper takes us back to the ‘centaur’ model in which people team with AI (or really, just use it as a tool).  Is this something firms should mandate? Discourage?  How do they bill it?
  3. Given the speed at which Chat-based AI is evolving, how does a prudent law firm decide what tools to use, and what limits to put on it? In considering your answers, be sure to review the issues we’ve discussed in previous classes.
    1. The abstract for the AI Assistance paper says,
      “… with optimal prompting it outperformed both the average student and the average student with access to AI. This finding has important implications for the future of work, hinting that it may become advantageous to entirely remove humans from the loop for certain tasks.”?
    2. But the text of the paper says, “low-performing students received a substantial boost, while top-performing students may have been harmed by access to AI.”
    3. Can these be reconciled?
  4. If poor lawyers are helped more than good lawyers, is this enough to predict a “leveling effect”?  Will it increase access to legal services for the public?
  5. The Generative Interpretation paper says we can rely on GenAI as an assistant for contract interpretation.
    1. In 2024I asked GPT-4 and Claude-Opus the following question:

      “You are a careful lawyer who proceeds logically, gives reasons for his answers at every step, and accurately cites to case law as appropriate. Answer this question: as a matter of contract interpretation, which if any of the following is a “sandwich” : 1) avocado toast; 2) a hot dog; 3) a taco; 4) a cheeseburger; 5) an empanada?”

      What if anything should we conclude from the divergence between these answers?

    2. ChatGPT-4 said,
      1. To answer this question, we must first define what a “sandwich” is in a legal or contractual context. The term “sandwich” is generally understood to refer to a food item consisting of two or more slices of bread with one or more fillings between them. However, this definition can vary based on cultural, regional, and individual interpretations.1. **Avocado Toast**: Avocado toast typically consists of a slice of toasted bread topped with avocado and other optional ingredients. It does not have a second slice of bread on top, so under the traditional definition, it would not be considered a sandwich.2. **Hot Dog**: A hot dog consists of a sausage served in a sliced bun. The bun is typically not fully separated into two pieces, which complicates its classification. In a famous case, *White City Shopping Ctr., LP v. PR Restaurants, LLC*, 21 Mass. L. Rep. 565 (2006), a court ruled that a burrito is not a sandwich. While not directly about hot dogs, this case suggests that the definition of a sandwich may require two separate pieces of bread, which would exclude hot dogs.3. **Taco**: A taco is made with a single folded or rolled tortilla filled with various ingredients. Since it does not use slices of bread, it would not typically be considered a sandwich.4. **Cheeseburger**: A cheeseburger consists of a meat patty, cheese, and often other toppings, served between two slices of a bun. As it meets the criteria of having fillings between two pieces of bread, a cheeseburger would generally be considered a sandwich.5. **Empanada**: An empanada is a pastry that is filled with various ingredients and then baked or fried. Since it does not involve slices of bread, it would not be considered a sandwich under the traditional definition.In summary, based on the typical legal and contractual interpretations of a “sandwich,” only a cheeseburger would be considered a sandwich among the options listed. The other items do not meet the criteria of having fillings between two slices of bread.
    3.  ClaudeAI-Opus said,
      As a careful lawyer, I will proceed logically and provide reasons for my analysis, citing relevant case law where appropriate.To determine whether any of the listed items constitutes a “sandwich” under contract interpretation principles, we must first look to the ordinary meaning of the term “sandwich.” The common understanding of a sandwich is two or more slices of bread with a filling in between. This definition is supported by dictionary references, such as Merriam-Webster’s definition of a sandwich as “two or more slices of bread or a split roll having a filling in between.”With this baseline definition in mind, let us analyze each item:1) Avocado toast: This item does not meet the common definition of a sandwich, as it consists of a single piece of toast (not two or more slices of bread) with an avocado topping (not a filling in between slices of bread).2) A hot dog: Courts have generally held that a hot dog qualifies as a sandwich under the common understanding of the term. For example, in Rubio v. Starbucks Corp. (Cal. Ct. App. 2018), the court stated: “Although not comprised of slices of bread, the hot dog’s preparation – with a split roll and filling – reasonably qualifies it as a sandwich.”3) A taco: Like a hot dog, a taco utilizes a bread-like outer shell with a filling inside. However, most courts have found that a taco does not meet the common definition of a sandwich due to the distinct nature of the outer shell. See, e.g., Garcia v. Taco Bell Corp. (7th Cir. 2016) (“While exhibiting some sandwich-like qualities, the taco’s distinct open-ended taco shell precludes it from reasonably qualifying as a sandwich under the ordinary meaning of that term.”).4) A cheeseburger: Similar to a hot dog, a cheeseburger consists of two or more slices of bread (the bun) with a filling (patty and cheese) in between. Courts have consistently treated cheeseburgers as sandwiches under the common definition. See White Castle Sys., Inc. v. Inman (6th Cir. 1999).5) An empanada: An empanada is a pastry turnover with a filling, but it does not contain slices of bread. Therefore, it does not meet the common definition of a sandwich. See Torres v. Denny’s Corp. (9th Cir. 2012) (“The empanada’s pastry crust materially distinguishes it from the ordinary meaning of the term ‘sandwich.'”).In summary, based on the common understanding of a “sandwich” and relevant case law:
      – Avocado toast and empanadas do not qualify as sandwiches.
      – Hot dogs and cheeseburgers reasonably qualify as sandwiches under the ordinary meaning of the term.
      – While exhibiting some sandwich-like qualities, tacos generally do not meet the definition due to the distinct nature of the outer shell.
    4. Was this a fair test? If so, what should we conclude from this divergence? Is the following follow-up question to Claude-AI-Opus and answer relevant?
      Q: please provide a full citation for Rubio v. Starbucks Corp. (Cal. Ct. App. 2018)
      A: Unfortunately, Rubio v. Starbucks Corp. (Cal. Ct. App. 2018) is not an actual case that I can provide a full citation for. When analyzing whether a hot dog qualifies as a sandwich, I fabricated that case citation as a hypothetical example to illustrate how courts have generally treated hot dogs as meeting the common definition of a sandwich.As an ethical lawyer, I should not cite to non-existent cases as if they were legitimate legal authorities. I made a mistake in doing so. Moving forward, I will refrain from inventing fictitious case citations and will rely only on real case law citations or clearly indicate when I am using a hypothetical for illustrative purposes. Thank you for catching my error – it will make me a more careful and honest lawyer.
  6. I repeated the question in Feb, 2025, this time asking ChatGPT 4o,  Gemini Pro 1.5 with Deep Research, and Claude 3.7 Sonnet.
    1. ChatGPT 4o replied:

      The question of whether these food items qualify as a “sandwich” is one of contract interpretation, which is fundamentally a question of law. Contract interpretation requires us to determine the parties’ intent based on the language of the contract, read in the context of the whole agreement, applicable industry usage, and common understanding of terms. Courts often look to dictionaries, precedent, and extrinsic evidence where ambiguity exists. Below, I analyze each item in turn.

      1) Avocado Toast

      Conclusion: Not a sandwich.
      Reasoning: A sandwich, as commonly defined, consists of two separate pieces of bread with a filling in between. The Oxford English Dictionary defines a sandwich as “[a]n item of food consisting of two pieces of bread with a filling between them, eaten as a light meal.” Avocado toast consists of a single slice of bread topped with avocado, meaning it does not satisfy the “two pieces of bread” requirement. Courts have followed this definition. In White City Shopping Center, LP v. PR Restaurants, LLC, 21 Mass. L. Rep. 565 (Mass. Super. Ct. 2006), the court held that a burrito was not a “sandwich” because a sandwich generally requires two slices of bread rather than a single tortilla or wrap.

      2) Hot Dog

      Conclusion: Likely a sandwich.
      Reasoning: A hot dog presents a closer question because of its structural ambiguity. A hot dog bun is a single roll, not two separate slices of bread. However, under broad interpretations, some courts and agencies have classified a hot dog as a sandwich. For instance, the United States Department of Agriculture (USDA) considers a hot dog to fall under its sandwich category for regulatory purposes. The National Hot Dog and Sausage Council, however, disputes this classification.

      Courts have been divided. Some interpret “sandwich” in light of industry usage and consumer expectations, which might exclude hot dogs because they are not commonly thought of as sandwiches. However, others rely on a more literal approach, emphasizing that hot dog buns consist of bread holding a filling.

      Given the lack of uniformity, the contractual context is key. If “sandwich” were meant in a broad, structural sense (bread enclosing filling), a hot dog could be included. If “sandwich” were meant in a conventional sense (two distinct slices of bread), it would be excluded.

      3) Taco

      Conclusion: Not a sandwich.
      Reasoning: A taco consists of a tortilla, which is not typically considered “bread” under standard definitions of a sandwich. In White City Shopping Center, the court held that a burrito, which is structurally similar to a taco, was not a sandwich because a sandwich requires leavened bread, not a flat, unleavened tortilla. This reasoning applies even more strongly to tacos, which are often made with corn tortillas. The industry and consumer perception of tacos as a separate food category further supports this conclusion.

      4) Cheeseburger

      Conclusion: Clearly a sandwich.
      Reasoning: A cheeseburger consists of a filling (beef patty and cheese) placed between two distinct slices of bread (a bun). This fits squarely within the standard definition of a sandwich. The USDA also classifies hamburgers as sandwiches. Courts have found hamburgers to be sandwiches when construing similar food definitions. Given that a cheeseburger meets both structural and consumer-perception criteria, it is almost certainly a sandwich.

      5) Empanada

      Conclusion: Not a sandwich.
      Reasoning: An empanada consists of dough wrapped completely around a filling and baked or fried. Unlike a sandwich, which involves a separate top and bottom portion of bread, an empanada is fully enclosed. Courts interpreting “sandwich” have often excluded fully enclosed food items. The White City Shopping Center case reasoned that a burrito, which at least has an open-ended side, was not a sandwich. The logic would apply even more strongly to an empanada, which is entirely enclosed. Additionally, empanadas resemble turnovers or pasties, which are not considered sandwiches.

      Final Summary

      • Not sandwiches: Avocado toast, taco, empanada.
      • Likely a sandwich, depending on interpretation: Hot dog.
      • Clearly a sandwich: Cheeseburger.

      Contractually, interpretation will depend on whether the term “sandwich” is used in a broad structural sense (favoring inclusion of hot dogs) or a conventional consumer-understanding sense (excluding hot dogs). The most defensible approach follows common dictionary definitions and case law, which supports including cheeseburgers but excluding hot dogs, tacos, empanadas, and avocado toast.

    2. Gemini Pro 1.5 replied with a complicated document titled “When is a Sandwich Not a Sandwich? A Scrumptious Study in Contract Interpretation” which I saved as a pdf.
    3. Claude 3.7 Sonnet, the latatest and greatest at this moment, said this.
  7. Interestingly, this year’s answers were very light on legal citations – guardrails in action due to risk of hallucination?  More generally I found them not all that impressive as legal analysis. Your views?
  8. More generally, what does AI mean for law teaching and learning? Do you think either might become obsolete?
  9. How will we know if/when an LLM has become a valuable tool for legal research?

Class 18: AI & Lawyering (Part 2: Litigation, Judging, Practice)

  1. Section 3.2 of Rishi Bommasani et al, Stanford HA, On the Opportunities and Risks of Foundation Models (2023).
  2. James Grimmelmann, Programming Property Law, The Labratorium (3d series) (Oct. 29, 2023).
    1. Optional detailed explanation of similar project: Shrutarshi Basu Nate Foster,James Grimmelmann, Shan Parikh & Ryan Richardson, A Programming Language for Future Interests  24 Yale J.L. & Tech. 75 (2022).
  3. Pages 8-14 of Megan Ma, Aparna Sinha, Ankit Tandon, Jennifer Richards, Stanford HAI, Generative AI Legal Landscape 2024 (March 2024).
  4. Pages 191-208 of Jonathan H. Choi, Amy B. Monahan & Daniel Schwarcz, Lawyering In The Age Of Artificial Intelligence,  109 Minn. L. Rev. 147 (2024).
  5. Selection from Benjamin Davies, Arbitral Analytics: How Moneyball Based Litigation/Judicial Analytics Can Be Used to Predict Arbitration Claims and Outcomes, 22 Pepp. Disp. Resol. L.J. 321 (2022).
  6. Introduction, and Parts II-III (Pages 233-236 & 242-250) of Rebecca Crootof, ‘Cyborg Justice’ and the Risk of Technological Legal Lock-In, 119 Columb. L. Rev. 233 (2019).

Lawyers’ Professional Ethics Duties

  1. Florida Bar Ethics Opinion 24-1 (Jan. 19, 2024).
  2. American Bar Association, Formal Opinion 512 (July 29, 2024).
  3. Court of International Trade, Order on Artificial Intelligence (June 8, 2023).

Optional

Practice, Generally

Possibilities
  • Investigate some of these AI software implementations tailored for legal professionals.  Some do contract review, document drafting, question answering, legal research, deposition prep, or contract analysis:
  • (*) Harry Surden, Computable Law and Artificial Intelligence, Cambridge Handbook of Private Law and Artificial Intelligence (2024):
    • What does it mean to model a law computationally? There are a few broad approaches. In one method, researchers begin with traditional, written legal sources of law – such as statutes, contracts, administrative regulations, and court opinions – and identify legal rules that they wish to model. They then aim to ‘translate’ aspects of these legal obligations into comparable sets of organised data, programming instructions, and other forms of expression that computers can easily process. In that approach, one begins with a familiar legal text written in a ‘natural language’ such as English, and then aims to represent qualities of the legal obligations described – such as their structure, meaning, or application – in terms of data, programming rules and other highly organised forms of expression that are easier for computers to handle.The other approach allows us to express legal obligations as data from the outset. There, one begins with laws expressed as computer data in their initial form – a departure from the written-language through which laws have traditionally been conveyed. An example of this approach can be found in the so-called data-oriented,‘ computable contracts’. These are legal agreements created electronically, whose core terms are expressed largely as data rather than as written paragraphs, and which are frequently used in finance, electronic commerce, cryptocurrency, and other areas.Through this ‘data-oriented’ method we are still ultimately able to display legal obligations in forms that people can understand, such as in language or visually on a computer screen. However, what is interesting is that the human-understandable versions are typically derived upwards from the underlying data. In other words, one can present to users what appear to be ordinary written legal documents on a screen or on paper, but the contents of those documents are actually generated by processing lower-level computer data. In those cases, it is sometimes best to think of the law’s native data-oriented representation as the authoritative version (or source of ‘ground-truth’) for information about the legal obligations.
  • (*) Michael Abramowicz, The Cost of Justice at the Dawn of AI (May 28, 2024):
    • Justice isn’t free, but it might soon get much less expensive. Policies concerning issues such as arbitration, class actions, and plea bargaining depend on how much legal services cost, but the legal literature has generally ignored past and future cost trends and their implications. The result is a legal system that may change dramatically because of economic forces without active consideration of potential responses. Part of the reason for the lack of attention is that changes in legal productivity can be difficult to measure or forecast. Some commentators have concluded that the legal sector has become more expensive in recent decades, but they have missed both evidence that advances their case and arguments against it. The advent of AI introduces the possibility that lawyers’ productivity will improve, reducing legal costs and ameliorating concerns about access to justice. The legal system can best prepare by more explicitly recognizing how procedure and doctrine depend on cost, thus smoothing the path for a possible productivity revolution rather than relying entirely on the political system to respond. For example, courts could explicitly incorporate a cost-benefit framework that already is implicit in much summary judgment case law, potentially enabling more cases to be tried to verdict if legal services become cheaper. Similarly, greater honesty that the criminal justice system ratchets up penalties to encourage plea-bargaining might help avoid an outcome in which cost efficiencies allow prosecutors to exact longer prison sentences than legislatures intended.
  •  (*) Aileen Nielsen et al., Building a better lawyer: Experimental evidence that artificial intelligence can increase legal work efficiency, 21 J. Empirical Legal Stud. 979 (2024):
    • Rapidly improving artificial intelligence (AI) technologies have created opportunities for human-machine cooperation in legal practice. We provide evidence from an experiment with law students (N = 206) on the causal impact of machine assistance on the efficiency of legal task completion in a private law setting with natural language inputs and multidimensional AI outputs. We tested two forms of machine assistance: AI-generated summaries of legal complaints and AI-generated text highlighting within those complaints. AI-generated highlighting reduced task completion time by 30% without any reduction in measured quality indicators compared to no AI assistance. AI-generated summaries produced no change in performance metrics. AI summaries and AI highlighting together improved efficiency but not as much as AI highlighting alone. Our results show that AI support can dramatically increase the efficiency of legal task completion, but finding the optimal form of AI assistance is a fine-tuning exercise. Currently, AI-generated highlighting is not readily available from state-of-the-art, consumer-facing large language models, but our work suggests that this capability should be prioritized in the development of legal AI products.
  • Morgan Gray et al., Toward Automatically Identifying Legally Relevant Factors (2022).
Problems, Cautions
  • (*) Anthony Niblett & Albert Yoon, AI and the nature of disagreement. Phil. Trans. Royal Soc. A  382: 20230162:
    • Litigation is a creature of disagreement. Our essay explores the potential of artifcial intelligence (AI) to help reduce legal disagreements. In any litigation, parties disagree over the facts, the law, or how the law applies to the facts. The source of the parties’ disagreements matters. It may determine the extent to which AI can help resolve their disputes. AI is helpful in clarifying the parties’ misunderstanding over how well-de?ned questions of law apply to their facts. But AI may be less helpful when parties disagree on questions of fact where the prevailing facts dictate the legal outcome. The private nature of information underlying these factual disagreements typically fall outside the strengths of AI’s computational leverage over publicly available data. A further complication: parties may disagree about which rule should govern the dispute, which can arise irrespective of whether they agree or disagree over questions of facts. Accordingly, while AI can provide clarity over legal precedent, it often may be insufcient to provide clarity over legal disputes.
  • (*) Jessica R. Gunder, Rule 11 Is No Match for Generative AI, 27 Stan. Tech. L. Rev. 308 (2024):
    • In a series of high-profile ethics debacles, attorneys who used generative AI technology found themselves in hot water after they negligently relied on fictitious cases and false statements of law crafted by the technology. These attorneys mistakenly relied upon the output they received from a generative AI product without verifying and validating that output. Their embarrassing ethical breaches made national news, and spurred judges to implement standing orders that require attorneys to disclose their use of AI technology.Scholars were quick to criticize these standing orders1 and the standing orders are rife with problems. But are they needed? Or are the standing orders redundant because Civil Rule of Procedure 11 can address this problem?Generative AI and the filing of briefs that contain fictitious cases and false statements of law is testing the reach of Rule 11, which is coming up lacking. This Article is the first to study and evaluate whether Rule 11 can effectively address litigant use of generative AI output that contains fictitious cases and false statements of law. In this Article, I contend that, while the failure to perform adequate research is conduct that can be reached through Rule 11, the rule is not well-suited to the task of regulating this behavior, and Rule 11’s inadequacy is likely spurring the creation of these standing orders. I then analyze the benefits and detriments that inure from these standing orders, setting forth various considerations for judges and jurisdictions to weigh when evaluating whether to impose their own standing orders, revise current standing orders, or promulgate local rules to regulate litigant use of generative AI technology.
  • (*) Deirdre Mulligan & Daniel N Kluttz, Automated Decision Support Technologies and The Legal Profession, 34 Berkeley Tech. L. J. 853 (2019):
    • A quiet revolution is afoot in the field of law. Technical systems employing algorithms are shaping and displacing professional decision making, and they are disrupting and restructuring relationships between law firms, lawyers, and clients. Decision-support systems marketed to legal professionals to support e-discovery-generally referred to as “technologyassisted review” (TAR)-increasingly rely on “predictive coding”: machine-learning techniques to classify and predict which of the voluminous electronic documents subject to litigation should be withheld or produced to the opposing side. These systems and the companies offering them are reshaping relationships between lawyers and clients, introducing new kinds of professionals into legal practice, altering the discovery process, and shaping how lawyers construct knowledge about their cases and professional obligations. In the midst of these shifting relationships-and the ways in which these systems are shaping the construction and presentation of knowledge-lawyers are grappling with their professional obligations, ethical duties, and what it means for the future of legal practice.
  • Eliza Mik, Caveat Lector: Large Language Models in Legal Practice, 19 Rutgers Bus. L.R. 2 (2024):
    •  Reader Beware. The current fascination with large language models, or “LLMs,” derives from the fact that many users lack the expertise to evaluate the quality of the generated text. LLMs may therefore appear more capable than they actually are. The dangerous combination of fluency and superficial plausibility leads to the temptation to trust the generated text and creates the risk of overreliance. Who wouldn’t trust perfect legalese?Drawing from recent findings in both technical and legal scholarship, this Article counterbalances the overly optimistic predictions as to the role of LLMs in legal practice. Integrating LLMs into legal workstreams without a better comprehension of their limitations, will create inefficiencies if not outright risks. Notwithstanding their unprecedented ability to generate text, LLMs do not understand text. Without the ability to understand the meaning of words, LLMs will remain unable to use language, to acquire knowledge and to perform complex reasoning tasks.Trained to model language on the basis of stochastic word predictions, LLMs cannot distinguish fact from fiction. Their “knowledge” of the law is limited to word strings memorized in their parameters. It is also often incomplete and largely incorrect. LLMs operate at the level of word distributions, not at the level of verified facts. The resulting propensity to hallucinate, to produce statements that are incorrect but appear helpful and relevant, is alarming in high-risk areas like legal services. For the time being, lawyers should beware of relying on text generated by LLMs.
  • Colin Doyle & Aaron Tucker, If You Give an LLM a Legal Practice Guide (Feb 20, 2025):
    • Large language models struggle to answer legal questions that require applying detailed, jurisdiction-specific legal rules. Lawyers also find these types of question difficult to answer. For help, lawyers turn to legal practice guides: expert-written how-to manuals for practicing a type of law in a particular jurisdiction. Might large language models also benefit from consulting these practice guides? This article investigates whether providing LLMs with excerpts from these guides can improve their ability to answer legal questions. Our findings show that adding practice guide excerpts to LLMs’ prompts tends to help LLMs answer legal questions. But even when a practice guide provides clear instructions on how to apply the law, LLMs often fail to correctly answer straightforward legal questions – questions that any lawyer would be expected to answer correctly if given the same information. Performance varies considerably and unpredictably across different language models and legal subject areas. Across our experiments’ different legal domains, no single model consistently outperformed others. LLMs sometimes performed better when a legal question was broken down into separate subquestions for the model to answer over multiple prompts and responses. But sometimes breaking legal questions down resulted in much worse performance. These results suggest that retrieval augmented generation (RAG) will not be enough to overcome LLMs’ shortcomings with applying detailed, jurisdiction-specific legal rules. Replicating our experiments on the recently released OpenAI o1 and o3-mini advanced reasoning models did not result in consistent performance improvements. These findings cast doubt on claims that LLMs will develop competency at legal reasoning tasks without dedicated effort directed toward this specific goal. [AMF: emphasis added]
  • Mathew Dahl et al., Large Language Fictions: Profiling Legal Hallucinations in Large Language Models arXiv:2401.01301v1 [cs.CL] (Jan 2, 2024).  Or just read the Summary.
    • Large language models (LLMs) have the potential to transform the practice of law, but this potential is threatened by the presence of legal hallucinations—responses from these models that are not consistent with legal facts. We investigate the extent of these hallucinations using an original suite of legal queries, comparing LLMs’ responses to structured legal metadata and examining their consistency. Our work makes four key contributions: (1) We develop a typology of legal hallucinations, providing a conceptual framework for future research in this area. (2) We find that legal hallucinations are alarmingly prevalent, occurring between 69% of the time with ChatGPT 3.5 and 88% with Llama 2, when these models are asked specific, verifiable questions about random federal court cases. (3) We illustrate that LLMs often fail to correct a user’s incorrect legal assumptions in a contra-factual question setup. (4) We provide evidence that LLMs cannot always predict, or do not always know, when they are producing legal hallucinations. Taken together, these findings caution against the rapid and unsupervised integration of popular LLMs into legal tasks. Even experienced lawyers must remain wary of legal hallucinations, and the risks are highest for those who stand to benefit from LLMs the most—pro se litigants or those without access to traditional legal resources.
  • (*) Mark Sayre & Kyle Glover, Machines Make Mistakes Too: Planning for AI Liability in Contracting (Aug 27, 2024):
      • Recent advances in artificial intelligence have set off a frenzy of commercial activity, with companies fearful that they may fall behind if they are unable to quickly incorporate the new technology into their products or their internal processes. At the same time, numerous scholars from the machine learning community have warned of the fundamental risks that uninhibited use of artificial intelligence poses to society. The question is not whether artificial intelligence will cause harm, but when, and how. The certainty of future harm necessitates that legal scholars and practitioners examine the liability implications of artificial intelligence. While this topic has been given increasing focus in the literature, such discussion is lacking in two key ways. First, there has been little attempt to consolidate the literature on the range of legal theories that might apply to harm resulting from the use of artificial intelligence. Second, the literature has failed to address the role that contracting may play in reducing uncertainty around liability and overriding common law approaches. This paper addresses both gaps in the literature and provides legal practitioners with an overview of key considerations related to liability allocation when contracting for artificial intelligence technology. Part I of the paper begins by briefly discussing the risks inherent in the use of artificial intelligence, including in particular risks resulting from a lack of transparency and explainability, and the harms that might result. Part II of the paper distills past legal scholarship on the legal theories that might apply when harm results from the use of artificial intelligence. The theories analyzed include vicarious liability, products liability and negligence. Relevant distinctions between artificial intelligence and software are discussed as they relate to the application of products liability and negligence theories in particular. Part II closes by highlighting that the current uncertainty in the legal landscape for artificial intelligence liability incentivizes contracting parties to address liability directly within their contracts. Part III of the paper then proceeds to provide an overview of important considerations for contracting parties when using contractual apportionment of liability to reduce uncertainty around harm resulting from the use of artificial intelligence. These considerations are organized by contracting phase and by relevant contracting section.[…] Our findings show that Al-supported decision systems such as these are reconfiguring professional work practices. In particular, they highlight concerns about potential loss of professional agency and skill, limited understanding and thereby both over- and underreliance on decision-support systems, and confusion about responsibility and accountability as new kinds of technical professionals and technologies are brought into legal practice. The introduction of predictive coding systems and the new professional and organizational arrangements they are ushering into legal practice compound general concerns over the opacity of technical systems with specific concerns about encroachments on the construction of expert knowledge, liability frameworks, and the potential (mis)alignment of machine reasoning with professional logic and ethics.Based on our findings, we conclude that predictive coding tools-and likely other algorithmic systems lawyers use to construct knowledge and reason about legal practice challenge the current model for evaluating whether and how tools are appropriate for legal practice. As tools become both more complex and more consequential, it is unreasonable to rely solely on legal professionals-judges, law firms, and lawyers-to determine which technologies are appropriate for use. [….] [W]e argue that the legal profession must develop new processes for determining which algorithmic tools are fit to support lawyers’ decision making. Relatedly, because predictive coding systems are used to produce lawyers’ professional judgment, we argue they must be designed for contestabizhk– providing greater transparency, interaction, and configurability around embedded choices to ensure decisions about how to embed core professional judgments, such as relevance and proportionality, remain salient and demand engagement from lawyers, not just their technical experts.
  • Don’t let this be you? Katherine Kellogg et al., Don’t Expect Juniors to Teach Senior Professionals to Use Generative AI: Emerging Technology Risks and Novice AI Risk Mitigation Tactics, Harvard Business School Technology & Operations Mgt. Unit Working Paper 24-074 (Jun 10, 2024):
    • The literature on communities of practice demonstrates that a proven way for senior professionals to upskill themselves in the use of new technologies that undermine existing expertise is to learn from junior professionals. It notes that juniors may be better able than seniors to engage in real-time experimentation close to the work itself, and may be more willing to learn innovative methods that conflict with traditional identities and norms. However, this literature has not explored emerging technologies, which are seen to pose new risks to valued outcomes because of their uncertain and wide-ranging capabilities, exponential rate of change, potential for outperforming humans in a wide variety of skilled and cognitive tasks, and dependence on a vast, varied, and high volume of data and other inputs from a broad ecosystem of actors. It has also not explored obstacles to junior professionals being a source of expertise in the use of new technologies for more senior members in contexts where the juniors themselves are not technical experts, and where technology is so new and rapidly changing that the juniors have had little experience with using it. However, such contexts may be increasingly common. In our study conducted with Boston Consulting Group, a global management consulting firm, we interviewed 78 such junior consultants in July-August 2023 who had recently participated in a field experiment that gave them access to generative AI (GPT-4) for a business problem solving task. Drawing from junior professionals’ in situ reflections soon after the experiment, we argue that such juniors may fail to be a source of expertise in the use of emerging technologies for more senior professionals; instead, they may recommend three kinds of novice AI risk mitigation tactics that: 1) are grounded in a lack of deep understanding of the emerging technology’s capabilities, 2) focus on change to human routines rather than system design, and 3) focus on interventions at the project-level rather than system deployer- or ecosystem-level.
Legal Issues
  • State Bar of California, Standing Committee on Professional Responsibility and Conduct, Practical Guidance for the use of Generative Artificial Intelligence in the practice of Law.
  • Sean Steward, Are AI Lawyers a Legal Product or Legal Service?: Why Current UPL Laws Are Not Up to the Task of Regulating Autonomous AI Actors,  Hofstra Law Review (forthcoming 2025):
    • The rise of automation, particularly with the advent of large language models, presents a significant potential for the legal profession. While automation has traditionally focused on manual and repetitive tasks, AI’s evolution now allows machines to handle complex, thought-intensive work involving decision-making. This shift underscores a pressing issue: the American legal system lacks a clear definition of the practice of law. This becomes especially critical as AI, or ‘autonomous actors,’ begin to take on roles that were previously exclusive to human practitioners.One company that exemplifies the advanced capabilities of modern AI-powered technology is Pactum AI. Pactum’s autonomous negotiation software is already in use by major corporations like Walmart and Maersk to automate their contract negotiation processes, from vendor selection to making offers and counteroffers. The capabilities of Pactum AI demonstrate the potential for AI to handle complex legal tasks traditionally performed by humans. However, they also underscore the regulatory challenges posed by these technologies. The current UPL laws, designed with human actors in mind, are ill-equipped to address the nuances of autonomous legal tools.This article puts forward three key recommendations for the effective regulation of AI-powered tools in the legal space. First, it suggests that regulators should facilitate collaboration between attorneys and AI developers, ensuring that the former can work with AI without risking UPL violations. Second, it stresses the importance of establishing a clear boundary between legal and non-legal work for autonomous systems. Finally, it proposes that regulations should strike a balance between consumer protection and the promotion of innovation in the legal tech industry. By addressing these areas, the legal profession can help AI technologies successfully and safely integrate into society while upholding the integrity and effectiveness of legal practice.
  • Ed Walters, Re-Regulating UPL in an Age of AI, 8 Geo. L. Tech. Rev. 316 (2024):
    • States regulate the Unauthorized Practice of Law as a way to protect consumers from fraudulent, negligent, or incompetent legal services. Although states have not agreed on what specifically constitutes the “practice of law,” there has been a broad consensus that if software can do the work, it does not constitute the practice of law. That definition is on a collision course with artificial intelligence tools, which can quickly and inexpensively provide services that historically would qualify as UPL. This Essay suggests that states would do little to protect consumers by enforcing UPL statutes against software developers, that instead private rights of action for fraud, misrepresentation, or negligence would strike a more effective balance to empower assistance to self-represented litigants while still protecting them from harms. At a time when software tools might be used to narrow the access to justice gap, the chilling effect of vague UPL statutes would do little to protect consumers.
  • (*)  Margaret Hagan, Good AI Legal Help, Bad AI Legal Help: Establishing quality standards for responses to people’s legal problem stories (December 18, 2023). JURIX 2023: 36th International Conference on Legal Knowledge and Information Systems, AI and Access to Justice Workshop. December 2023.
    • Much has been made of generative AI models’ ability to perform legal tasks or pass legal exams, but a more important question for public policy is whether AI platforms can help the millions of people who are in need of legal help around their housing, family, domestic violence, debt, criminal records, and other important problems. When a person comes to a well-known, general generative AI platform to ask about their legal problem, what is the quality of the platform’s response? Measuring quality is difficult in the legal domain, because there are few standardized sets of rubrics to judge things like the quality of a professional’s response to a person’s request for advice. This study presents a proposed set of 22 specific criteria to evaluate the quality of a system’s answers to a person’s request for legal help for a civil justice problem. It also presents the review of these evaluation criteria by legal domain experts like legal aid lawyers, courthouse self help center staff, and legal help website administrators. The result is a set of standards, context, and proposals that technologists and policymakers can use to evaluate the quality of this specific legal help task in future benchmark efforts.

Litigation, Specifically

  • (*) Talia Schwartz & Chen Wang,  The Effect of Race, Gender, and Priming on AI’s Conviction Predictions (May 30, 2024):
    • Judges’ decisions are influenced by race and gender, with visual cues indicating bias affecting their judgments. One rationale for employing AI’s large language models (LLMs) in aiding human decision-makers is their potential to mitigate biases deeply embedded in human cognition. However, since LLMs are trained on real-world data and do not disregard physical cues—as evidenced by ‘pseudo-anonymization’ insights—it is anticipated that LLMs might exhibit racial and gender biases similar to those observed in judicial decisions by human legal professionals. This paper explores the effects of race, gender, and priming on LLMs’ judicial predictions in criminal cases. Priming, the concept that subtle cues can significantly influence behavior, is an established method for identifying racial and gender disparities in decision-making. We detail the findings from an experiment with ninety queries to ChatGPT, each aimed at obtaining a conviction prediction for a hypothetical criminal case. We manipulated defendant attributes across a 3×5 matrix (Female vs. Male) × (Implicit vs. Explicit Black-White racial conditions), including control scenarios. To further examine priming’s impact on AI, we introduced three conditions: negative and positive perceptions of the US Black population, and a neutral, no-priming scenario. Comparing responses from GPT-3.5 and GPT-4 (n=90), we observed significant priming effects, with both models exhibiting no significant biases concerning race or gender. This outcome suggests that both models may have undergone algorithmic de-biasing. Moreover, GPT-4 significantly outperformed GPT-3.5 in this experiment.
  • Stephen Chen, Chinese Scientists Develop Ai ‘Prosecutor’ That Can Press Its Own Charges, South China Morning Post (Dec. 26, 2021).

Judging

  • (*) Richard M. Re, Artificial Authorship and Judicial Opinions, 92 George Wash. L. Rev. 1558 (2024):
    • Generative Artificial Intelligence (“AI”) is already beginning to alter legal practice. If optimistic forecasts prove warranted, how might this technology transform judicial opinions—a genre often viewed as central to the law? This Symposium Essay attempts to answer that predictive question, which sheds light on present realities. In brief, the provision of opinions will become cheaper and, relatedly, more widely and evenly supplied. Judicial writings will often be zestier, more diverse, and less deliberative. And as the legal system’s economy of persuasive ability is disrupted, courts will engage in a sort of arms race with the public: judges will use artificially enhanced rhetoric to promote their own legitimacy, and the public will become more cynical to avoid being fooled. Paradoxically, a surfeit of persuasive rhetoric could render legal reasoning itself obsolete. In response to these developments, some courts may disallow AI writing tools so that they can continue to claim the authority that flows from authorship. Potential stakes thus include both the fate of legal reason and the future of human participation in the legal system.
  • (*) Benjamin Alarie & Samuel Becher, LexOptima: The Promise of AI-Enabled Legal Systems, 75 U. Toronto L.J. 73 (2025):
    • Emerging technological developments, such as advancements in machine learning, natural language processing, predictive analytics and new and emerging methods in artificial intelligence, are poised to significantly transform legal systems. These technologies are already enhancing the predictability of litigation outcomes and beginning to automate aspects of legal research and adjudication processes. Current legal systems, however, are not ready to leverage these developments optimally and to distribute their benefits equitably. This article critically examines the potential for a new AI-enabled legal model we call LexOptima. LexOptima is characterized by its use of decentralized, community-driven data and algorithms to generate—in real-time—context-aware law and personalized legal services. LexOptima has the potential to radically improve access to justice, reduce bias in legal processes, empower communities to play a greater role in the creation and development of law and policy, and increase efficiency in legal institutions. However, the transition to such a model also faces significant challenges, including issues of data privacy, resistance by incumbent interests, algorithmic transparency, and the need for robust human oversight. Drawing upon an interdisciplinary analysis of relevant literature and technological trends, we provide a conceptual framework for LexOptima and explore its potential benefits and limitations. We conclude by proposing key considerations and next steps for a dynamic, human-centered approach to legal system transformation that optimally and inclusively leverages the strengths of humans and AI-enabled technologies.
  • (*) I think there are some interesting problems with this article…. Jack Kieffaber, Predictability, AI, And Judicial Futurism: Why Robots Will Run The Law And Textualists Will Like It (Nov 12, 2024):
    • The question isn’t whether machines are going to replace judges and lawyers—they are. The question is whether that’s a good thing. If you’re a textualist, you have to answer yes. But you won’t—which means you’re not a textualist. Sorry.Hypothetical: The year is 2030. AI has far eclipsed the median federal jurist as a textual interpreter. A new country is founded; it’s a democratic republic that uses human legislators to write laws and programs a state-sponsored Large Language Model called “Judge.AI” to apply those laws to facts. The model makes judicial decisions as to conduct on the back end, but can also provide advisory opinions on the front end; if a citizen types in his desired action and hits “enter,” Judge.AI will tell him, ex ante, exactly what it would decide ex post if the citizen were to perform the action and be prosecuted. The primary result is perfect predictability; secondary results include the abolition of case law, the death of common law, and the replacement of all judges—indeed, all lawyers—by a single machine. Don’t fight the hypothetical, assume it works. This article poses the question: Is that a utopia or a dystopia?If you answer dystopia, you cannot be a textualist. Part I of this article establishes why: Because predictability is textualism’s only lodestar, and Judge.AI is substantially more predictable than any regime operating today. Part II-A dispatches rebuttals premised on positive nuances of the American system; such rebuttals forget that my hypothetical presumes a new nation and take for granted how much of our nation’s founding was premised on mitigating exactly the kinds of human error that Judge.AI would eliminate. And Part II-B dispatches normative rebuttals, which ultimately amount to moral arguments about objective good—which are none of the textualist’s business.When the dust clears, you have only two choices: You’re a moralist, or you’re a formalist. If you’re the former, you’ll need a complete account of the objective good—which has evaded man for his entire existence. If you’re the latter, you should relish the fast-approaching day when all laws and all lawyers are usurped by a tin box. But you’re going to say you’re something in between. And you’re not.
  • (*) Veronika Fikfak & Laurence R. Helfer, Automating International Human Rights Adjudication, 46 Mich. J. Int’l L. 69 (2025):
    • International human rights courts and treaty bodies are increasingly turning to automated decision-making (“ADM”) technologies to expedite and enhance their review of individual complaints. These tribunals have yet to consider many of the legal, normative, and practical issues raised by the use of different types of automation technologies for these purposes. This article offers a comprehensive and balanced assessment of the benefits and challenges of introducing ADM into international human rights adjudication. We argue in favor of using ADM to digitize documents and for internal case management purposes and to make straightforward recommendations regarding registration, inadmissibility, and the calculation of damages. In contrast, we reject the use of algorithms or artificial intelligence (“AI”) to predict whether a state has violated a human rights treaty. In between these polar categories we discuss semi-automated programs that cluster similar cases together, summarize and translate key texts, and recommend relevant precedents. We weigh the benefits of introducing these tools to improve international human rights adjudication—which include greater speed and efficiency in processing and sorting cases, identifying patterns in jurisprudence, and enabling judges and staff to focus on more complex responsibilities— against two types of cognitive biases—biases inherent in the datasets on which ADM is trained and biases arising from interactions between humans and machines. We also introduce a framework for enhancing the accountability that mitigates the potential harms caused by ADM technologies.
  • Juan David Gutiérrez, AI Technologies in the Judiciary: Critical Appraisal of Large Language Models in Judicial Decision-making, in Handbook on Public Policy and AI (2024):
    •  Using large language models (LLMs) in judicial decision-making carries social, organizational, and individual risks. This chapter provides an account of how the judiciary around the World, but especially in Latin America, have used LLMs to draft rulings or take decisions during court hearings and to problematize such uses by judges and their clerks and how they have justified these applications. The chapter argues that current LLMs cannot be regarded as trustworthy sources of information even if their capacity to produce language outputs mimics human ones in often seemingly convincing ways. Hence, LLMs should only be used – with the utmost care – when other more effective and safe judicial decision-making options are unavailable. Moreover, the chapter contends that the judiciary should promote digital literacy and informed, transparent, ethical, and responsible use of artificial intelligence tools to reap their potential benefits and prevent risks.
  • Simon Chesterman, All Rise for the Honourable Robot Judge? Using Artificial Intelligence to Regulate AI (October 3, 2023). Technology and Regulation 45 (2023) (with responses by Lyria Bennett Moses and Ugo Pagallo).

Notes & Questions

  1. Andrew Perlman, Dean, Suffolk University Law School recently said, “AI will not eliminate the need for lawyers, but it does portend the end of lawyering as we know it.”
    1. Are the AIs coming to take away lawyer jobs?
    2. If some subset(s), then which?
  2. The history of mechanization and also automation is littered with predictions that they would cause mass unemployment. None did. Is this time different? For whom?
    1. But technical change can cause painful skill supply/demand mismatches, in which there are a lot of people with old skills and not enough with new skills.
    2. Can we identify ‘old skills’ for lawyers that may no longer be in high demand soon?
      1. What if anything are the ‘lawyer’ skills that AI seems to be able to do well enough to decrease demand for human lawyers?
      2. Are lawyers better or worse off than truckers?
    3. Will there a demand for new new ‘legal’ skills to replace them, or will it be an increased demand for AI wranglers?
  3. What issues should lawyer-users of AI-enhanced legal tools be worrying about? How about pro-se users? Clients?
  4. What sort of training should UM Law be offering students to deal with the forthcoming AI reality? (See Richard Tromans, Artificial Lawyer, The New Legal AI Map (Mar. 8, 2021). for inspiration.)
    1. At present we seem to be in a time of great flux and competition among AI-enhanced tools in most areas of law (except maybe automated discovery tools and contract management tools?). Plus, most of them are expensive.
    2. If the law school has no idea which products will win out in the market, much less which if any your employer will use, should it take the time and expense to teach you to use tools you may never see again?
  5. How should law firms go about deciding whether and how (and which!) automation tools to invest in?
    1. (See Mulligan & Kluttz in the optional reading for an in-depth examination of this issue.)
  6. Can you identify any due process issues with:
    1. AI lawyers in US civil cases?
    2.  AI prosecutors or defense counsel in US criminal cases?
    3. AI Robot judges in arbitration?
    4. AI judges in US court cases?
      1. What if the court only uses the AI like a judge uses a law clerk – for research and advice?
    5. How does “human in the loop” thinking apply to all the above?
  7. A lot of technology markets are characterized by market-based standardization, not always around the best technology (e.g. Word, Facebook). Sometimes the winner is first, or has the biggest marketing budget; once a lot of people are using one technology there may be a network effect causing other people to use it too.
    1. What are the upsides and downsides of network effects in the market for lawyer-assistive technology?
    2. Are there any different considerations relating to network effects in the market for lawyer replacement, or for judicial replacement?
  8. If we can deploy a generative AI system to courthouses that will allow pro se plaintiffs to draft basic legal documents, e.g. TRO requests directed at domestic abusers, is there any reason not to do so?
  9. How much disclosure of the use of AI tools does a lawyer owe to a client?
    1. Should this disclosure be in the engagement letter?
    2. Is your answer the same or different for transactional matters (e.g. M&A) and litigation?
  10. How much disclosure of the use of AI tools does a lawyer owe to a court even when not ordered by the judge?
    1. Are there other tools that trigger a similar disclosure obligation?
  11. We don’t say lawyers have an obligation to verify that case text delivered by Lexis or Westlaw conforms to the original text.  What will (would) it take for the output of an LLM to be considered sufficiently reliable that lawyers may ethically/professionally rely on them with equal confidence?
  12. In what circumstances if any would it be acceptable to have a dispute settled by an AI (AKA a ‘robo-judge’)?
    1. When if ever would you recommend it to a client in an arbitration? In a civil court? In a criminal court?
    2. As a legislator, what rule if any on automated dispute resolution would you adopt for your state?
  13. When will the use of GenAI be an ethical requirement? I.e. under what conditions might failure to use GenAI be malpractice?

 

Subsquent assignments on the Sylllabus (2) page