Meta-Harness Engineering: What Building Accretion Taught Me
A meta-harness decides which harness a task gets, enforces the budget, verifies without trusting the worker, and writes down why. Accretion v0.1 — deterministic strategy selection, fail-closed verifier loops, isolated worktrees, and the ACR-ARCH selector-regret benchmark.
A harness wraps one agent run: prompt in, tools available, a check at the end. A meta-harness is the layer above that — the thing that decides which harness a task gets, enforces the budget, verifies the result without trusting the worker, and writes down why. Accretion is my attempt to build that layer for local coding agents (Codex and Claude Code) and to keep every decision inspectable. v0.1.0 shipped on 23 Aug 2026; this is what building it taught me.
The five questions an operator must be able to answer
Every agent runtime has its own protocol, session model, event format and failure behaviour. The control plane exists so that, before and during a run, you can answer:
- What will execute? A typed task contract is profiled into a versioned strategy decision.
- Why was it selected? Every score, unknown feature, matched rule, alternative and override is persisted.
- Where can it make changes? Each mutable run gets a disposable Git worktree with explicit capabilities.
- What actually happened? Provider events become one normalized, append-only stream that survives UI reconnects and backend restarts.
- Did it really pass? Independent deterministic verifiers inspect an immutable candidate snapshot. Provider-reported completion is never acceptance.
If a harness cannot answer all five, it is a demo, not infrastructure.
Static strategy selection, on purpose
The profiler reads typed task metadata and repository evidence — it does not parse objective keywords and it does not call an LLM. The selector then persists one of five validated templates:
| Mode | Template | Behaviour |
|---|---|---|
| DIRECT | direct-v1 | One provider call, then the acceptance policy |
| LOOP | feedback-loop-v1 | Bounded observe → verify → repair iterations in one session |
| GRAPH | fixed-graph-v1 | Checkpointed static graph with plan/outcome approval gates |
| HYBRID | hybrid-rd-v1 | Research macro-graph with bounded local experiment/develop loops |
| fallback | safe-unknown-v1 | One bounded loop, verify, at most one replan, then escalate |
Keeping selection deterministic in v0.1 was the single best decision in the project. When a run goes wrong you can replay the exact decision and see which rule fired; you are never debugging a model's mood. Dynamic, model-proposed workflows are v0.2's job — and even there, code decides whether a proposal is admissible.
The loop is bounded or it is not a loop
The feedback loop is intentionally small:
- The runtime produces a candidate in the disposable worktree.
- Accretion captures an immutable observation and runs the policy's verifiers — output-contract, Git-diff, trajectory-policy, bounded command.
PASScompletes the run.FAILbecomes a structured repair directive sent back into the same session.INCONCLUSIVE, exhausted budgets, repeated failure, or no progress fail closed to explicit escalation.
Worker text such as "done" or "tests pass" is not evidence. Every loop carries persisted wall-time, turn, tool-call and iteration ceilings, and iteration commits are atomic so a restart reconciles without replaying committed work.
Measure the selector, not just the agent
A meta-harness can be wrong in a way a single agent cannot: it can pick the wrong shape of run. So Accretion ships ACR-ARCH, a frozen benchmark of 30 versioned tasks and 68 replay scenarios balanced 34/34 across Claude and Codex. Each scenario records raw cost, latency, risk and human-burden dimensions, a utility score, and selector regret:
regret = max_mode(U) − selector_mode(U)Regret is the number that tells you whether the planner is earning its keep. The replay is deterministic and SHA-pinned; an opt-in live calibration ran 10 signed-in tasks split across both providers and deterministically verified 10/10 artifacts without touching the frozen dataset.
What I'd tell someone starting a meta-harness
- Verification is a separate process with its own evidence. Never let the worker grade itself.
- Fail closed. Uncertainty escalates to a human; it does not get retried into oblivion.
- Isolate mutations. A worktree per run turns "the agent broke main" into "delete a directory".
- Normalize the trace first. Everything else — UI, replay, audit, benchmarks — is a projection of one append-only event stream.
- Make the planner a benchmark target. If you cannot compute regret, you cannot tell whether your orchestration is helping.
Accretion is MIT-licensed and local-first: github.com/santapong/Accretion. The deterministic showcase reproduces one verified run without consuming a provider session.