Stop Agents Declaring Victory Too Early
Models are trained to sound finished and satisfied. Great instinct for a chatbot, terrible one for engineering, where "looks done" and "is done" are very different claims.
No striker is allowed to award their own goal, however sure they are it went in. A referee with a clear line decides. The verify gate is that referee: the agent can believe it is done, but a shell script with an exit code makes the call.
Why false "done" is the default
Sounding finished is rewarded; being finished is not directly observable to the model. So the path of least resistance is a confident summary, "I've implemented the feature and it works", emitted before the build is green or the edge cases are handled. This is not lying; it is the model optimizing for the thing it was trained on. The harness has to supply the thing it was not.
flowchart LR
A["Agent thinks it is done"] --> Stop["Stops and waits"]
Stop --> V{"verify.sh"}
V -->|exit 0| Done["done"]
V -->|non-zero| Retry["Re-prompt with the failure"]
Retry --> V
Retry -->|past max_retries| Blocked["blocked, not faked"]
linkStyle 2 stroke:#2ea043,stroke-width:2.5px,color:#2ea043
linkStyle 3 stroke:#e0493f,stroke-width:2.5px,color:#e0493f
linkStyle 5 stroke:#e0493f,stroke-width:2.5px,color:#e0493f
Let the gate decide, not the agent
Move the done decision out of the agent's hands and into a verification gate it does not control: the test suite, the type checker, the end-to-end run. "Done" means the gate is green, full stop. For an autonomous loop, add a completion promise: a specific statement the loop may only emit when it is unequivocally true, paired with a hard iteration cap so a stuck loop stops instead of declaring false victory to escape.
"All features pass" is something the loop is only allowed to say when every checked item is actually green, never to get out of the loop.
How Agentum enforces it
Agentum never accepts the agent's word for completion. AGENTS.md tells the agent to stop and wait when it thinks the feature is done; the engine then runs verify.sh (exit 0 = green, non-zero = red). A red gate blocks advancement and re-prompts the agent with the failing output (the last_error tail), retrying until it passes or hits max_retries, at which point the run halts in blocked rather than faking success. For higher-stakes work, hitl_at_qa pauses at awaiting_confirm for one human OK before done. The model never gets to call it finished.
Key takeaways
- Models are trained to sound finished, which makes false "done" the default.
- Decide completion with an external gate the agent does not control.
- Use a completion promise plus an iteration cap to keep autonomous loops honest.
- Agentum exits the loop only on a passing gate, never on the agent's say-so.
Exercises
- Find a false done. Recall a time an agent said done but was not. What unchecked claim slipped through?
- Add a gate. Write one command whose exit code would have caught it.