Reading an AI Impact Assessment

A credible AI IA is not defined by length. Specificity is what matters. Five areas should appear in every document worth reading. Intended purpose and scope: the assessment must state precisely what the system is designed to do and, critically, what it is not designed to do. A candidate-screening tool trained on data from one industry sector is not validated for another, and if the scope section is vague, every subsequent section becomes unverifiable. Known limitations and failure modes: no AI system performs perfectly across all populations and conditions, so a well-formed IA names the specific conditions under which the system degrades, such as low-data scenarios, edge cases, and input types outside the training distribution. Vague language such as "performance may vary" is a warning sign; look for quantified error rates and named test conditions. Data sources and bias considerations: AI systems reflect the data they were trained on, so the IA should identify the origin, age, and composition of training data, and describe how the team assessed it for bias. Fairness sections typically use proxy metrics, indirect measures used as stand-ins for characteristics that cannot be measured directly, and a transparent document acknowledges the limits of those metrics. Equal error rates do not guarantee equitable outcomes in every real-world context, and a sound IA says so. Human oversight mechanisms: who reviews the system's outputs before consequential decisions are made, and how are appeals handled? The IA should name roles, not just principles. "A human will always be in the loop" is not an oversight mechanism; "the line manager receives a confidence score and must document their rationale before proceeding" is. Residual risk is the section that demands your closest attention. Gross risk is what the system could do before any controls are applied. Residual risk is what remains after all stated mitigations. When you sign off on an AI IA, you are accepting the residual risk as it will exist in your environment, with your staff and your processes, rather than the tidier version the vendor imagines. If mitigations rely on training that has not yet happened, or oversight processes that are not yet resourced, the residual risk is higher than the document suggests.