Skip to main content
The residual stack is Cadenza’s answer to a hard deployment problem: you have a capable but frozen base policy (or VLA), the real robot’s dynamics don’t quite match what it was trained on, and you can’t afford to re-train the whole thing on-device. Instead you learn a small residual that nudges the base’s actions, govern it, and then distill the base+residual pair into a compact student that runs onboard without the base at all.
Residual RL and distillation need the rl extra: pip install -e ".[rl]" (installs torch). Without cadenza-lab they run on a deterministic proxy task (labelled as such); --real targets the gated cadenza-lab sim seam. The governed commands (train, eval, bench) require sign-in.

​
The control law

The residual never replaces the base — it corrects it:
The frozen base/VLA picks a_base; the small residual head emits Δa, scaled by α (action scale) and an optional gate. The base never receives a gradient.

​
env residual init: scaffold + profile

Establishes the residual architecture and dry-run profiles the head — params, per-step latency, peak memory — before you commit to a training run. No training happens here.
Writes rescue-dog/residual/residual.json.

​
env residual train: governed PPO on the frozen base

PPO-trains the residual head against the frozen base under a perturbation curriculum (sparse reward). The Cadenza API governs the run: it picks the hyperparameters, decides when to stop, and returns the verdict — the client only collects rollouts and runs the raw PPO step each round.
On DEPLOY the policy is saved and promoted as the residual baseline; on BLOCK it rolls back to the previous residual. The trained head lands at rescue-dog/residual/residual_policy.pt.

​
env residual eval: govern the residual

Re-scores the trained residual on its own — success / collision / residual-sanity / regression — and returns DEPLOY | BLOCK | NEEDS_DATA. --promote sets it as the baseline if it passes. See Governance.

​
env residual bench: residual vs full RL

A head-to-head benchmark of the current full-RL stack against cadenza-cli’s residual RL — reporting compute, dollar cost, and accuracy, then a verdict on whether the residual arm wins.

​
Distillation: a base-free onboard student

Once a residual is deployed, env distill collects teacher (base + residual) rollouts and trains a compact student that reproduces the teacher’s behaviour without loading the base — so it runs on CPU/MPS at the control-loop rate, optionally quantized to int8.
The report prints the teacher→student success gap, student param count, and the onboard latency against the 50 Hz control budget (it tells you whether the student meets or MISSES the floor). Artifacts land in rescue-dog/student/.

​
Govern the student

Scores the student on gap / success / regression and returns the same DEPLOY | BLOCK | NEEDS_DATA verdict, with rollback on BLOCK.

​
Full loop

Every torch path here runs through Cadenza’s transparent acceleration layer — bf16, thread tuning, matmul precision, and torch.compile are applied automatically per hardware target.