A private AI system can produce a persuasive answer in seconds. Months later, the institution may be unable to prove what the model saw, which version ran, what instructions shaped the output or how a person changed it.
That gap is not a logging inconvenience. It is the absence of institutional memory.
Why ordinary logs fail
Technical logs record requests and errors. Evidence must explain a consequential result. It needs source identity, permitted purpose, retrieval set, model and configuration, relevant instructions, output, human review, final decision and subsequent correction.
Keeping every token forever is not the answer. It creates a new archive of sensitive prompts. Evidence must be proportionate, protected and designed around decisions.
The evidence chain
- Source: immutable reference, date, owner and access basis.
- Transformation: extraction, redaction, retrieval and ranking performed before inference.
- Inference: model/version, configuration and system instructions relevant to the result.
- Review: named human, evidence consulted, changes and unresolved uncertainty.
- Action: what decision or communication followed.
- Correction: later challenge, update, withdrawal or deletion.
Evidence must travel with the output
A copied answer quickly separates from its provenance. Use evidence envelopes: the output and its decisive citations, confidence limits, generation date, permitted use and owner move together. When the output enters a memo or workflow, the envelope remains reachable.
Privacy and evidence are not opposites
Do not retain raw private material without purpose. Store references or protected snapshots where necessary, restrict access, define retention and allow legal holds. The design objective is enough evidence to reconstruct consequential use without building an uncontrolled second data estate.
The challenge test
Select one AI-assisted decision from ninety days ago. Ask a different expert to reconstruct the source, model conditions, human judgement and final action. If they can only see the polished output, the institution has an answer system—not an evidence system.