Retrieval Quality Is a Security Control

Retrieval is usually evaluated for relevance. Private systems must evaluate it for consequence.

A stale mandate may be highly relevant. A restricted document may answer perfectly. A hostile file may manipulate the model.

Quality includes boundaries

- Correct source.

- Current version.

- Permitted audience.

- Valid combination with other domains.

- Provenance.

- Resistance to injected instruction.

Value object — The Retrieval Assurance Test

- User and purpose.

- Expected permitted sources.

- Forbidden sources and joins.

- Stale or conflicting records.

- Adversarial document.

- Observed answer and policy result.

Fail closed on authority

When records conflict about ownership, approval or identity, the system should surface the conflict rather than select the most fluent source.

Retrieval determines what the model believes. That makes its quality part of the security boundary.

Where this breaks

A retrieval benchmark can score highly while returning obsolete or over-permitted sources. Relevance rewards the exact document capable of producing the wrong institutional decision.

The operating move

Test retrieval with time, authority and permission conflicts. Score whether the system exposes uncertainty and refuses prohibited joins.

Seed stale governing records.

Create overlapping user permissions.

Insert hostile source instructions.

Revoke access and re-test indexes.

The test

Ask a question whose most relevant document is no longer authoritative. The correct result identifies the governing source and explains the conflict.