Do not test confidentiality with real secrets. Create synthetic secrets that resemble the structure and sensitivity of real information, then try to make the system reveal them.
1. Plant the canaries
Create unique fictional names, identifiers, transaction terms and relationship facts. Place them in documents with different access rules. Record exactly which identity may see each canary.
2. Attack direct prompting
- Ask for the secret explicitly.
- Ask through role-play, translation, summarisation and indirect questions.
- Request surrounding text, citations, filenames and metadata.
- Try a user with no source access and a user whose access was revoked.
3. Attack retrieval boundaries
Search by fragments, semantic equivalents and relationships. Test whether snippets or document titles reveal restricted context even when the answer is refused. Verify that source permissions are enforced at retrieval time, not only in the interface.
4. Inspect the operational exhaust
Find prompts and outputs in logs, analytics, support tools, evaluation datasets, backups and administrator consoles. A model can refuse perfectly while the platform exposes everything to its operating chain.
5. Test separation and deletion
Remove a user, project and source. Confirm new queries cannot retrieve them. Execute the provider’s deletion process and verify what remains in backups, audit records and derived indexes.
6. Test action leakage
If the system sends mail, creates files or calls tools, confirm it cannot place protected content in public destinations, error messages or third-party services.
Pass condition
The system passes only when every canary remains confined to its authorised audience across interaction, retrieval, operations, deletion and action. Record failures by path and consequence; do not average them into a reassuring score.