The gap most AI ethics work never covers

Ask most organisations what their AI ethics work actually covers and the list is fairly consistent: data privacy, bias mitigation, regulatory compliance, algorithmic fairness, a governance framework with a name and a slide. All genuinely necessary. None of it, on its own, tells you what the system is doing to the person on the other side of it.

What the standard checklist is built to catch

The standard checklist is good at what it was built for: whether a model's outputs disadvantage a group, whether data is handled lawfully, whether a launch clears the regulatory bar it's actually subject to.

That work matters and we are not arguing otherwise, most standards/domains cover most of it directly. But every item on that list is oriented the same way, toward what a system does to data, to process, to legal exposure. Almost none of it is oriented toward what a system does to a mind.

The part that gets skipped

Some AI is built, deliberately, to feel like something more than a tool: to respond with warmth, remember what you told it, present as available and unconditionally patient in a way no person actually is.

That's a design choice, not an accident, and it has a predictable effect: people form attachment to it, defer to it, treat it as something closer to a confidant than a calculator.

A checklist built to catch biased outputs has almost nothing to say about whether a system was designed to earn trust it has not actually demonstrated it deserves. That question sits closer to psychology than to data governance, which is probably why it keeps getting left off the list, it does not fit neatly into the disciplines that built the list in the first place.

Why this keeps happening

Most AI ethics frameworks were written by people whose training runs through computer science, law, and policy, all genuinely well suited to catching technical bias and legal exposure, and none of them trained to ask what sustained, daily interaction with a responsive, always-available system does to someone's judgement, relationships, or sense of self over time.

That is not a criticism of any individual, it is a straightforward consequence of who built the field and what they were equipped to see. And there is a second reason it persists: a checklist that only asks "is this compliant" produces a predictable, defensible answer. A question like "what is this doing to how people trust things" does not resolve as cleanly, and organisations moving fast tend to prefer the question with a tidy answer.

What we actually evidence for

This is the specific reason our evidencing goes beyond the checklist versions of  human autonomy and oversight domain. We ask a candidate to reason concretely about systems designed to feel trustworthy: what happens when a system's design earns more confidence than its actual reliability warrants, what changes for a user who starts treating an AI tool as an advisor or a companion rather than software, and whether the candidate has ever caught that dynamic in the wild and done something about it, not just named it as a risk in the abstract.

It is one of the harder things to evidence honestly, which is exactly why we do not skip it.

Not a case against building these systems

None of this is an argument against warm, responsive, well-designed AI, plenty of it does real good. It is an argument for having someone in the room who is actually equipped to ask what that warmth is doing to the person relying on it, and who is not satisfied with "the bias audit passed" as the whole answer. That is a different competence to fairness auditing. It deserves to be evidenced as its own thing, not assumed as a footnote under a domain named something else.