Put Observability Inside the Harness

An agent that can't watch the runtime is debugging in the dark, guessing at why the program does what it does. Observability is what turns the loop's feedback from a hunch into evidence.

Debugging by speculation

When the agent cannot see logs, stack traces, or the live program, it falls back on plausible-sounding speculation: "this is probably a race condition, " "maybe the config is wrong." Sometimes the guess is right; often it sends the loop down a wrong branch that wastes iterations. The feedback stage is only as good as the evidence flowing into it.

flowchart LR
  Pane["Live tmux pane over WS"] --> Watch["Watchdog reads it"]
  Watch -->|context low| Compact["/compact"]
  Watch -->|crash signature| Crash["session.crashed"]
  FailTail["verify.sh failing tail"] --> Err["surfaced as last_error"]

Make the runtime visible from inside

Observability is not a separate ops concern bolted on afterward, it is part of the harness. The agent needs first-class access to logs, command output, and the live state of what it is running, available within the loop so each iteration's feedback is grounded. When the runtime is visible, "why did this fail?" has an answer the agent can read instead of invent.

The terminal is the evidence

In Agentum the agent works inside a live tmux pane, streamed over WebSocket, and that pane is both where the work happens and where the evidence lives. A per-session watchdog reads the same pane for signals, context-low triggers /compact, crash signatures emit session.crashed, and the failing tail of verify.sh is surfaced as the feature's last_error. For web work, a host-resident Chromium runs on the host (so it survives your client sleeping) and you watch it live via a CDP screencast tunnelled over ssh -L. The loop's feedback, your supervision, and the real runtime are one source of truth.

Exercises

Further reading