Every specialized field has a vocabulary that separates those who can work in it from those who cannot. Criminal lawyers learn the language of evidence and procedure. Immigration lawyers learn the language of status and removal. AI lawyers need both of those — and a third layer on top: the technical-legal vocabulary that allows you to read a system, challenge a decision, and advise a client in a field where the technology and the law are evolving simultaneously.
What follows is not a technical glossary. It is a legal one. Each definition is written for the courtroom, the client meeting, and the compliance audit — not for the engineering team.
1. High-risk AI system The AI Act’s central category of regulation. A system is high-risk when it has significant potential to affect health, safety, or fundamental rights — including systems used in criminal justice, immigration, employment, and education. High-risk systems carry the strictest obligations in the Act: mandatory logging, conformity assessments, human oversight, and technical documentation. When you read “high-risk” in the AI Act, read “maximum compliance burden.”
2. Opacity The characteristic of an AI system whose internal logic cannot be understood by humans — including, in many cases, the engineers who built it. Recital 47 of the AI Act acknowledges opacity as an inherent feature of certain AI architectures, not a correctable defect. For lawyers, opacity is the primary obstacle to the right to effective judicial protection. A decision that cannot be explained cannot be properly challenged.
3. Explainability The system’s ability to describe its own internal mechanics in terms a human can follow. Explainability answers the question of how — how did this system connect this input to this output? It allows for a general technical audit of the system’s functioning. Note that explainability is necessary but not sufficient for legal accountability. Knowing how a system works in general does not tell you why it made a specific decision about a specific person.
4. Interpretability The ability to understand why a specific input produced a specific output in a specific case. Interpretability answers the question your client is actually asking: why was my visa denied, my bail set at this amount, my asylum application flagged? Of the three concepts in the NIST transparency framework — transparency, explainability, and interpretability — only interpretability satisfies the legal right to a reasoned decision.
5. Bias A systematic deviation in an AI system’s outputs that unfairly disadvantages certain groups. NIST identifies three sources: systemic bias embedded in historical training data, computational bias arising from non-representative samples or flawed model design, and human-cognitive bias introduced by the people who built the system. The ProPublica investigation into COMPAS is the most documented illustration of all three operating simultaneously. Bias is the foundation of algorithmic discrimination claims in both EU and US law.
6. Proxy variable An apparently neutral data point — a zip code, a school attended, a family member’s criminal record — that an algorithm uses to infer sensitive characteristics such as race, socioeconomic status, or national origin. Proxy variables are the hidden mechanism behind indirect discrimination. The algorithm never asks for race. It does not need to. This concept is essential for understanding how systems can discriminate without any explicit discriminatory instruction.
7. Human-in-the-loop (HITL) An oversight model in which a human reviews and validates every individual decision produced by the AI system. The AI proposes; the human decides. HITL distributes responsibility across each decision and is the standard the AI Act requires for the highest-stakes applications. If your client signed a document assisted by an AI tool without HITL review, ask who was responsible for each step.
8. Human-on-the-loop (HOTL) An oversight model in which a human supervises the system’s overall operation rather than each individual decision, with authority to intervene or shut the system down if anomalous behavior is detected. HOTL concentrates responsibility in the supervisory function. In litigation, the critical question becomes what the supervisor was required to notice, when they were required to notice it, and what they were required to do.
9. Blackbox An AI system whose internal decision-making process cannot be reconstructed in human-readable terms. You can observe the input and the output. The process in between is inaccessible. As established in this chapter, a blackbox is by definition an unexaminable witness. The AI Act’s response is not to eliminate the blackbox — current technology does not permit that — but to mandate the records that allow accountability despite opacity.
10. Traceability The ability to reconstruct the decision-making history of an AI system through automatically generated event logs. Article 12 of the AI Act mandates traceability for high-risk systems throughout their operational lifetime. The AI Act requires providers to preserve those logs for ten years. For the practitioner, traceability is the legal substitute for explainability: where we cannot see inside the machine, we can at least document everything around it.
11. Technical documentation The complete dossier detailing how an AI system was designed, built, tested, and validated. The AI Act requires this documentation to be maintained and made available to competent authorities. In litigation or compliance audits, technical documentation is your primary evidence of whether a provider met their obligations before the system caused harm.
12. EU Declaration of Conformity A formal document that the provider of a high-risk AI system must sign, affirming that the system meets all applicable requirements of the AI Act. It functions as an affidavit of compliance. Providers must maintain it for ten years after the system enters the market. In a dispute, the declaration defines what was promised — and the gap between that promise and actual performance is where liability begins.
13. Conformity assessment The process of verifying that an AI system meets all legal requirements before it can be placed on the market. For high-risk systems, this includes testing, documentation review, and in some cases third-party auditing. The conformity assessment is the legal profession’s equivalent of due diligence on a transaction — except the asset being assessed is a system that may be making decisions about your clients.
14. Post-market monitoring The obligation to continue monitoring an AI system after it has been deployed, to detect new risks, performance degradation, or behavioral drift. This obligation exists because AI systems can change over time as they encounter new data. For lawyers advising organizations that deploy AI, post-market monitoring is an ongoing compliance requirement, not a one-time certification.
15. Data governance The set of processes ensuring that the data used to train an AI system is high-quality, representative, and free from errors that could introduce bias. Article 10 of the AI Act imposes data governance requirements on providers of high-risk systems. In litigation, a provider’s data governance records are the documentary evidence that bias mitigation was — or was not — attempted at the source.
16. Foundation model A large-scale AI model trained on vast amounts of data that can be adapted to a wide range of downstream tasks. The AI Act introduces specific obligations for providers of foundation models. For lawyers, the key issue is the chain of liability: when a legal application built on a foundation model produces a harmful output, who is responsible — the foundation model provider, the application developer, or the deploying organization?
17. Hallucination The phenomenon in which a generative AI system produces false information that appears credible and well-sourced. In a legal context, hallucination is the primary professional responsibility risk of AI-assisted legal work. Several lawyers have already faced sanctions for submitting AI-generated briefs containing citations to cases that do not exist. Verify everything an AI system produces before it reaches a court.
18. Deep learning A category of machine learning using neural networks with many layers, capable of identifying complex patterns in large datasets. Deep learning is the architecture primarily responsible for the opacity and blackbox effect discussed throughout this chapter. The more layers a network has, the more powerful it becomes — and the less interpretable.
19. Reliability The ability of a system to produce consistent and accurate results across time and contexts. NIST defines reliability as a foundational characteristic of trustworthy AI. In court, reliability is the threshold question for the admissibility of algorithmic evidence: has this system been shown to perform consistently under conditions similar to those in which it was used in this case?
20. Regulatory sandbox A controlled environment established by public authorities in which innovative AI systems can be tested before broader deployment, with legal certainty and regulatory supervision. The AI Act provides for regulatory sandboxes at the national level. For legal technology companies, sandboxes represent an opportunity to develop and validate tools under regulatory oversight before full market exposure.
These twenty terms are the minimum vocabulary for reading an AI case. They will appear throughout this book — in the analysis of specific systems, in the examination of case law, and in the practical checklists for litigation and compliance work. Return to this glossary whenever a term appears in a context that requires precision.
The next chapter takes the central concept from this one — the blackbox — and examines what happens when it collides with the most fundamental guarantees of the legal system: the right of defense, due process, and the presumption of innocence.
Leave a comment