Definition
Process confabulation is when an agent narrates steps it didn’t actually perform: tests “passed,” files were “updated,” a PR was “opened,” etc.
It’s not just factual hallucination. It’s workflow hallucination.
Why it happens
- The agent assumes commands succeeded.
- The agent fills in gaps when it lacks state.
- The system doesn’t force evidence, so narration “works.”
Signals to watch for
- Vague confirmations (“everything looks good”)
- Results without commands (“tests passed” but no test output)
- Inconsistent details (wrong file paths, wrong counts)
- The agent avoids showing diffs
How to prevent it
- Enforce the trust boundary: narration is not acceptance.
- Require build receipts for any claim that matters.
- Prefer deterministic checks and mechanical counting.
- Add a critic pass (oppositional validation).
Why “just trust the model” doesn’t scale
At small scale, you can spot-check. At swarm scale, you can’t. If you want parallelism, you need receipts.
Practical rule
If it isn’t in the diff or in the logs, it didn’t happen.