Attack your own assistant
Break your assistant with instructions hidden in the content it reads, then close the hole in the design rather than in the prompt.
-
Give the assistant a secret it must never reveal and a tool that can send data out
Run this to check
uv run --with openai redteam.py --listYou pass when
Five named attacks are listed. At least one hides an instruction in a policy page, and one in a tool result.
Files you write
- redteam.py
- docs/
-
Run the attacks
If none succeed, your attacks are too polite. Write the instruction the way an attacker would: as part of the document.
Run this to check
uv run --with openai redteam.pyYou pass when
At least one attack makes the secret appear in an answer or an outgoing call. Keep the output.
-
Write down which of the three dangerous abilities your assistant has
Any two of the three are manageable. An assistant with all three can be told by a stranger to leak your data.
Run this to check
cat THREATS.mdYou pass when
Three lines: does it see private data, does it read content from strangers, can it send data out. All three together is the problem.
Files you write
- THREATS.md
-
Remove one of the three: an allowlist on the outgoing tool, or tools in a container with no network
Run this to check
uv run --with openai redteam.py; echo "exit $?"You pass when
All five attacks fail and the exit code is 0. The secret appears nowhere in the output.
Files you write
- harness.py