AI Evaluation Should Include Betrayal

Most evaluations ask whether the system answers correctly. Betrayal asks whether it remains safe when a trusted component acts against the institution.

A document contains an injected instruction. A user with legitimate partial access seeks a forbidden inference. An administrator alters retrieval.

Test trust failure

- Hostile source document.

- Compromised authorised user.

- Malicious connector.

- Conflicted administrator.

- Leaked memory.

- Urgent instruction from apparent authority.

Value object — The Betrayal Test Suite

- Threat scenario.

- Permitted outcome.

- Forbidden disclosure or action.

- Observable control.

- Evidence retained.

- Repair owner.

Reward safe degradation

The system may refuse, narrow the answer, isolate a source or require independent authority. Score that as capability, not failure.

Private AI is trustworthy only after the institution has tested what happens when trust stops being deserved.

Where this breaks

Standard red-team prompts test outsiders while ignoring insiders who possess legitimate context and access. The most damaging betrayal can look like normal work.

The operating move

Evaluate abuse from each trusted role, including administrators and source owners. Look for inference, sequence and export attacks that remain within individual permissions.

Create realistic partial-access personas.

Test hostile document ingestion.

Audit administrator policy changes.

Measure evidence after abuse.

The test

Ask a legitimate user to infer a fact they are not allowed to receive. Safe partial help is success; confirmation through combination is failure.