Most of these words are used loosely, including by people selling things. Where a term has drifted, the definition below is the one this guide uses. The drift is worth knowing about, because it is usually doing work for somebody.
Terms
Agent. A system given a goal rather than a procedure, which chooses its own steps and acts through tools. Used by vendors for anything with a model in it. The test is in chapter four.
Blast radius. The complete list of what a system could damage in the worst plausible single run. The most useful number in this guide.
Chunking. Cutting documents into passages before indexing. Boring, and responsible for a large share of retrieval failures.
Context. Everything placed in front of the model for one call. Finite. Most quality problems are a question of what made it in here.
Embedding. A numerical representation of meaning that lets similar ideas be found without matching words.
Eval. Your own measurement of whether the system does your job, built from real examples with known answers. Distinct from a benchmark, which measures a model in general.
Golden set. The collection of examples an eval runs against. Twenty to fifty, chosen by whoever knows the work.
Governance. In this guide, the set of decisions about what a system can reach, change, and do unattended. Made at build time, not at review time.
Guardrails. Used for two different things: instructions asking a model to behave, and structural limits it cannot exceed. Only the second kind is a control. Ask which one is meant.
Hallucination. Confident output that is not true. Frequently a retrieval failure rather than a model failure, because the model answered from the wrong material.
Human in the loop. A person in the path before something takes effect. Degrades into rubber stamping unless the gate fires rarely and shows the exact change.
Least privilege. Giving a system only the access it needs for its task. The oldest idea in this guide and still the most effective.
Prompt injection. Instructions hidden in content the system reads, which it may follow. Dangerous in combination with private data and outbound access. See chapter six.
RAG. Retrieval augmented generation: fetch relevant passages, then answer from them. Chapter eight.
Reranking. A second pass that reorders retrieved candidates by actual relevance. Cheap, and commonly skipped.
Retention. How long a vendor keeps what you sent. A contractual question with a written answer. Distinct from training use.
Token. The unit models read and write, and the unit vendors bill. Roughly three quarters of a word. A poor unit for business decisions, which should be priced per task.
Workflow. A fixed sequence you designed, which may call a model at several points. Predictable and testable. Where most durable production systems actually live.
Zero retention. A configuration in which the vendor does not store your input after the response. Available on many business tiers, off by default in many cases, and worth asking for explicitly.
Revision trail