“It runs locally” is presented as the end of the privacy discussion. Local execution can be valuable. It can reduce data sent to external providers, improve offline capability and give the institution more control over retention. It can also concentrate sensitive context on an unmanaged workstation, expose prompts through logs, retrieve documents without permission and depend on updates from an opaque supply chain. Location is one architectural property, not a privacy guarantee.
Follow the entire request
A local model still depends on:
- The device, operating system and user session.
- Source documents and indexes.
- Embedding, retrieval and memory stores.
- Model files, update channels and management tools.
- Logs, crash reports and performance telemetry.
- Generated outputs and applications that receive them.
- Administrators and support personnel.
Any component can create exposure or retain the information longer than expected.
Value object — The Local AI Boundary Test
Answer:
- What data enters the workflow and from which governed sources?
- Which components process or store raw content, embeddings and output?
- Can every user retrieve only what they are authorised to see?
- Who can administer the model, device and indexes?
- Does any telemetry, update or support path leave the local boundary?
- How are memory, logs and generated files expired or deleted?
- What happens when the device is lost, replaced or compromised?
If the answer relies on the phrase “air-gapped” or “on premises” without evidence, the design is incomplete.
Local can weaken central control
A cloud platform may provide mature identity, logging, retention and incident controls. Moving the model to personal devices can bypass those controls. Local deployment is strongest when the institution can manage the devices, verify software, enforce permissions and observe consequential use without collecting unnecessary semantic content.
Protect the indexes
Retrieval indexes and embeddings may contain or reveal private information even when the original document is encrypted elsewhere. They can survive source deletion and may be copied for performance. Classify, encrypt, back up and delete them as governed records. Test that source permission changes propagate.
Secure the update chain
Models, runtimes and extensions change. Verify sources, signatures and approval for updates. Restrict plugins and tools that can access the network, file system or other applications. A private model with an unrestricted tool layer can disclose more than a hosted chat interface.
Plan the device event
Local capability makes the endpoint more valuable. Define loss response, remote revocation, encrypted storage, backup exclusions and recovery. Avoid creating a unique private intelligence system that exists only on one principal’s laptop.
Choose architecture by consequence
Some work belongs in a local model. Some can use a well-governed hosted service. Some should not enter an AI workflow. Make that decision from data, authority, administration and recovery. Locality can support privacy. It does not manufacture it.