Loop Engineering: The Check Is the Craft

Strip any working agent down and you find the same unit: do, check, decide. Do some work. Check whether it is actually better. Decide to continue or stop. The whole craft of loop engineering — the term Addy Osmani put on it in mid-2026 — lives in one uncomfortable fact: the do step is nearly free to build, and the check step is nearly everything.

Self-grading does not work

The tempting shortcut is to ask the model that produced the work whether the work is good. The peer-reviewed record on this is unusually clear, and it is negative: intrinsic self-correction — model critiques itself, revises, repeats — makes net accuracy drop on reasoning tasks. Models talk themselves out of right answers as often as into them, and they carry a measurable bias toward preferring their own output.

The subtler finding: the fix is not merely using a separate model as judge. A second LLM opinion is still an opinion. What moves the needle is an external, grounded signal — test execution, tool output, a trained process reward model scoring each step. Ground the check and iteration starts paying for itself; leave it ungrounded and more loops just mean more confident drift.

Stopping is an unsolved problem, so budget for it

There is no published formalism for when an agent loop should stop. None. And the largest multi-agent failure study shows what fills that vacuum: step repetition and unaware of termination are among the top failure modes across every model tier — agents circling a finished task, burning budget, sometimes undoing their own work.

So the decide step gets engineered, not hoped for:

MAX_ITERS = 6

for i in range(MAX_ITERS):
    artifact = agent.do(task, context)

    # The check is external and grounded. Not "does the model like it" -
    # something the agent cannot argue with.
    result = run_tests(artifact)          # or: tool output, reward model

    if result.passed:
        break                             # stop on success
    if not result.improved_over(previous):
        break                             # stop on stall
    context = context.with_feedback(result.failures)
    previous = result

Three stop conditions, none clever: success against the external check, stall against a progress measure, and a hard iteration budget. Sampling more attempts has the same shape — quality gains from best-of-N flatten by roughly N of four to sixteen, so budgets are not just safety, they are where the economics already stop paying.

The toolkit that makes it a loop

When Osmani expanded the idea into a field guide, the definition got concrete. A real loop, in his telling, has five components: automations (scheduled runs — without them you have a session, not a loop), worktrees (isolated parallel copies so attempts do not trample each other), skills (codified project knowledge the loop reads instead of rediscovering), connectors (wired access to real tools), and subagents for independent verification — plus external state that outlives any single run. One line from that guide compresses this whole post: an agent asked to grade its own output tends to praise it, and tuning an independent skeptical evaluator is far more tractable than making a generator critical of its own work.

The operator role changes accordingly. The bottleneck has moved from writing code to proving it works — so the practitioner owns the outer loop (what counts as done, what the evaluator checks, when to stop paying) while agents run the inner one. Loop engineering does not teach you to do the work better; it removes you from the position of doing the work, and relocates your judgment into the checks.

The loop is a contract

Written this way, a loop is a small contract: what counts as better, who measures it, and when we stop asking. Every layer above — graphs wiring loops together, harnesses deciding what loops may touch — inherits its reliability from this unit. Get the check wrong here and no topology above it can fix the drift; get it right and even a single dumb loop is a useful machine.