Direct Joint Copy Beat QP IK — and the Habit That Caught It

I wanted better movement math. The humanoid mirror in RoboLLM copied my joint angles straight from MediaPipe Pose onto a PAL Talos in RViz, and "direct joint copy" sounded like the naive baseline you replace with a proper solver. So I built the proper solver — pink QP differential IK on top of Pinocchio — ran it head-to-head against the baseline, and it lost. This post is about why, and about the habit that caught it.

The setup

Full-body mirror of a 32-DOF Talos (neck, waist, two 7-DOF arms, two 6-DOF legs), pelvis pinned, kinematic only. Input is one webcam and MediaPipe Pose landmarks. The control loop ticks at 50 Hz, so each tick has a 20 ms budget. Two retargeting modes:

• direct — per-joint angle retargeting from landmark vectors, with slew limits.
• qp — pink's differential IK: track wrist position and palm orientation targets under joint limits, solved with quadprog each tick.

Both were judged by an independent A/B harness, not by me eyeballing RViz.

What the judge found

The only metric with a real ground truth — wrist position, checked against MoveIt /compute_fk — regressed. The tick budget was blown by 1.5–3×. And the one place QP was supposed to shine, palm orientation, could not be scored at all: MediaPipe Pose has no palm landmarks, so the orientation target the solver was tracking was itself invented from wrist-elbow geometry. A solver that beautifully tracks a made-up target is not a win; it is a circular argument.

The default mode was flipped back to direct. mode:=qp stays in the tree as an opt-in experiment.

Why direct copy is the ceiling here

The lesson generalises: retargeting fidelity is capped by the information in the input, not by the sophistication of the solver. With Pose-only landmarks you have joint positions and a visibility score per joint. Direct copy spends all of that information and nothing else. QP adds degrees of freedom the input cannot constrain, and the solver fills them with whatever minimises its cost — plausible, smooth, and unmeasurable.

QP IK earns its keep in exactly two situations:

1. Hand landmarks exist (MediaPipe Hands/Holistic), so palm orientation is observed rather than guessed.
2. Real hardware needs velocity- and limit-aware IK — then the constraints are the point, and 30 ms ticks are worth paying for at a lower rate.

Neither held for a webcam mirror in RViz.

The habit that caught it

None of this would have surfaced without three things being true before the comparison ran:

• A ground-truthed metric. The retarget harness verifies commanded joints against /compute_fk to 0.00° — and it had already proved it catches wrong formulas and corrupted constants, so a regression from it is believable.
• A hard budget. 50 Hz means 20 ms; "feels smooth" is not a number.
• An independent judge. The A/B verdict came from a separate model-run critic with a refute-first brief, not from the author of the QP branch.

That is the same shape as the verifier gate in Accretion: the worker's claim that it is better is never the evidence.

What it cost

The Talos phases ran as orchestrated workflows with explicit gates — RViz demo, live mirror session, push approval. Real spend was about 4× the estimate (1.16M tokens for C1, 1.58M for C2), dominated by container verification rounds. That is the price of being able to say "refuted" with a table instead of "felt worse" with a shrug — and it was the cheaper outcome, because the alternative was shipping a slower, less accurate default.

Code: github.com/santapong/RoboLLM → examples/talos_mirror.