A provider says customer prompts are not used to train the model. That statement matters. It does not mean nobody learns from the deployment. Usage creates metadata: who asks, when, from where, about which systems, with which volume, latency and tool sequence. Connectors reveal repositories. Administrators see errors. Support tickets contain examples. Employees observe which questions senior people ask. The model may not be training on the content while the surrounding system learns the institution.
Map the observers
- Model provider and its subprocessors.
- Cloud, identity, logging and security platforms.
- Application vendor and support personnel.
- Internal administrators, developers and analysts.
- Connected source-system owners.
- Recipients of generated output.
Each observer sees a different projection. Combined, those projections can reveal strategy, pressure, relationships and capability.
Metadata can be more revealing than prompts
A spike in translation, due-diligence and employment queries may reveal an acquisition. Repeated health-related retrieval may expose a personal event. Use of a new jurisdiction’s records can signal relocation. Protect query content, but also assess identity, timing, destination, model choice and tool calls.
Value object — The AI Observation Map
For each AI workflow, record:
- Data content visible at each component.
- Metadata and telemetry generated.
- People with administrative or support access.
- Retention and secondary-use terms.
- Cross-customer or product-improvement uses, including evaluation.
- Exports to monitoring, analytics and ticketing systems.
- Controls: minimisation, pseudonymisation, isolation, deletion and contractual restriction.
Update the map when a new connector, agent tool or monitoring platform is added.
Training is not the only secondary use
Data may be retained for abuse monitoring, quality review, debugging, legal compliance or product analytics. Some uses are legitimate and necessary. The institution should know the scope, duration and access. Avoid the binary question “used for training?” Ask what is retained, who can see it and what decisions or products it can influence.
Internal observation needs governance too
A private deployment can expose executives to internal administrators. Query logs may reveal legal advice, investment interest or family matters. Limit raw-content logging. Separate security telemetry from semantic content where possible. Audit administrator access and create a protected process for investigations.
Reduce semantic exhaust
Users paste more context than necessary because the interface makes it easy. Retrieve minimum sources, redact identifiers where they do not affect the task and avoid sending entire threads when a controlled extract is sufficient. Generated outputs also become new data. Route them into governed systems or expire them; do not let every answer become another unmanaged record.
Privacy is about observers
A private AI architecture is not defined only by whether the model trains on prompts. It is defined by the total set of observers and the inferences available to them. Map who learns from the workflow. The answer is almost always larger than the model.