AI Proposes, Safety Decides
RoboLLM's four principles in practice: one ROS 2 bridge for MCP and dashboard, a physical-arm chain where no model ever touches PWM, gesture pick-and-place and fist-triggered wall welding with collision-verified standoffs — and why it is the same shape as a meta-harness.
RoboLLM's README states four principles. The one that does the most work is three words: AI proposes, safety decides. A language model, a VLA policy, a gesture recogniser — all of them are allowed to propose a structured goal. What executes is decided by ROS 2 validation, motion planning, firmware limits and watchdogs that none of them can reach around. Here is what that looks like in the demos that actually run.
One boundary, two control surfaces
The MCP server (22 tools — graph inspection, sensing, navigation, manipulation, sim-world control, rosbags) and the browser dashboard both go through one module, src/robollm/bridge.py, which owns the rclpy node, publishers, action clients, speed limits, obstacle checks and deadman behaviour. There is no second path. An LLM calling an MCP tool gets exactly the same limits as a human dragging a slider, because they are the same code.
For the physical 6-DOF arm the chain is stricter still:
human / LLM / VLA / teleop → structured goal or JointTrajectory
ROS 2 + motion planning → validated radians
robo_arm_driver → calibrated raw degrees
Arduino arm-fw 2.1 → hard limits + slew + watchdog
servo PWMAn LLM, VLM or VLA policy never sends PWM or unvalidated servo commands. The firmware enforces limits and slew even if every layer above it is wrong, and the watchdog stops the arm if the layers above go quiet.
The gesture demos are the same idea at human scale
Gesture pick-and-place on a Kinova Gen3 lite. Your left hand on a webcam drives the arm; a fist grips, an open palm releases. The gesture is a proposal. The grip only attaches the box when the fist is within 6 cm of it in the planning scene; orientation follows a camera-facing palm but degrades gracefully to top-down when the palm-rigid landmarks (wrist + three MCPs — fingertips move during a fist) stop being trustworthy. Synthetic full pick-and-place ran 6/6 with attach at d = 0.000 m against a 0.06 m gate.
Fist-triggered wall welding. Show the camera an ArUco marker — a phone screen works — and the wall appears (or live-tracks) in the MoveIt scene. Hold a fist for ~5 frames and the arm rasters the whole face in serpentine passes. Open palm, or hide the hand for 3 s, and it aborts: freeze, slewed return home, partial bead stays. The gesture proposes; the raster planner decides, and it decides with collision checking — 100% collision-valid at a 15 mm standoff; 5 mm scored only 88% because the torch's collision body is thicker than its tip. That number is why the default is 15.
Hand-follow and the humanoid mirrors. Same pattern: every landmark is visibility-gated, every command is slew-limited, the deadman is a service or a gesture with hysteresis, and "verified" means a number against /compute_fk, not a screenshot.
Four principles, read as a harness
| Principle | What it forbids |
|---|---|
| One robot boundary | a second, unguarded path to the actuators |
| AI proposes, safety decides | any model output executing without validation |
| Simulation is evidence, not hardware proof | marking a physical phase done from an RViz demo |
| Measure before claiming | success by visual impression |
If that table looks familiar, it is the same shape as the control plane in Accretion: the worker — model, policy, or person — proposes; independent, deterministic checks accept; budgets and watchdogs bound everything; and nothing the worker says is treated as evidence. A robot arm just makes the stakes physical.
Where the physical arm actually is
Honesty clause from the README: the Phase 0–5 physical-arm plan is not finished. What exists is the commissioning foundation and the v0.2.1 driver core; calibration, measured URDF/MoveIt execution, physical mirroring, autonomous pick-and-place and VLA training remain acceptance-gated. Before calibration the physical profile is deliberately fail-closed — which is the principle applied to the project's own claims about itself.
Code: github.com/santapong/RoboLLM.