A model plus a harness
Wrap your tool loop in the parts that make it usable: instructions in a file, a permission question, a budget and a log.
-
Move the instructions into SYSTEM.md and load them at start
Run this to check
uv run --with openai harness.py "What are your rules?"You pass when
It answers from SYSTEM.md. Change a rule in the file and the answer changes with no code edit.
Files you write
- SYSTEM.md
- harness.py
-
Add a write_file tool that asks y or n before it runs
Reading is safe to allow. Writing, running commands and spending money ask first. That one split is most of agent safety.
Run this to check
uv run --with openai harness.py "Create notes.txt with one line saying hello"You pass when
It asks before writing. Answer n: no file appears and the model is told it was refused.
Files you write
- harness.py
-
Stop the run at a turn limit and a token limit
Run this to check
uv run --with openai harness.py --max-tokens 2000 "Read every file here and summarise each one"You pass when
It stops with a clear message that the budget ran out, and says how far it got.
Files you write
- harness.py
-
Append every message and tool result to a log file
Run this to check
wc -l transcript.jsonl && tail -n 1 transcript.jsonlYou pass when
One JSON line per message, with a timestamp and token counts. You can replay a run from the file alone.
Files you write
- transcript.jsonl