In plain English
Explainability is the degree to which a person can understand why an AI system produced a particular decision or prediction. Some models are easy to interpret, while many high-performing ones are effectively black boxes. Techniques exist to shed light on which inputs influenced an output, but explanations are often partial rather than complete.
Why it matters
When a decision affects someone, a loan refusal, a flagged transaction, a medical recommendation, being able to explain the reasoning is often a legal and ethical requirement, not a nicety.
A worked example
A credit model that can show the main factors behind a refusal, such as income and existing debt, is more explainable than one that simply outputs a score with no reasons.
Common confusion
Explainability is not the same as auditability. Explaining why one output happened differs from keeping a reliable, verifiable record of what the system did over time.