Robots act, then find out. This one finds out first: it imagines the next few seconds of every action it could take, refuses the ones it does not survive, and reports the moment its own model of the world stops being valid. The controller is receding-horizon MPC in which the transition function is a generative video model and the state is pixels — no state estimator, no metric map, no explicit geometry. The contribution is not the loop, which dates to 2017, but the monitor: model-based control earned epistemic uncertainty from ensembles of small dynamics models, and the field replaced them with one enormous video model without replacing that mechanism. We measure disagreement in decision space instead of pixel space — K stochastic rollouts under a fixed action, scored by the same critic, and the entropy of the resulting outcome distribution — which is invariant to appearance by construction, where the obvious pixel-residual monitor fires on scene noise and is therefore ignored.
Six steps, each forced by the one above it.
Receding-horizon MPC, except the transition function is generative and the state is pixels. No state estimator, no metric map, no explicit geometry anywhere in the stack.
action set 5 discrete · horizon 2–3 s · cadence target 3 Hz · samples K = 5 · committed 1 step
forward · left · right · reverse · hold — in parallel.collision | drop-off | clear, plus a progress scalar and one line of reasoning.hold above threshold.The loop replans every period; the model integrates one step, never the episode. Because it is re-conditioned on a fresh real observation each cycle rather than integrated forward from an initial condition, error does not compound across the episode and the usual transfer problem does not arise. The domain gap is bounded by a single step of prediction — not by the divergence between a synthetic asset and the world.
The constraint that dominates every other decision is closed-loop latency: imagination, scoring, arg-max and actuation must finish inside one control period. That is what forces the small action set, the short horizon, the aggressive resolution reduction, and concurrent rather than serial rollouts. Branch count is a free parameter traded directly against cadence.
We will not claim novelty where there is none. Planning by imagining futures is a decade-old line, and saying so is what makes the rest credible.
The obvious substitute for ensemble disagreement is to compare the imagined frame at t+k against the observation that actually arrived. Everyone reaches for it and it is broken by construction: perceptual residual superimposes genuine epistemic error on ordinary aleatoric variation. A person walking through frame, or a lighting shift, produces a large residual while the model's physics was perfectly correct — so a monitor built on it fires on scene noise and is therefore ignored.
DÉJÀ-VU runs a second pass instead. Hold the committed action fixed, draw K stochastic rollouts under identical conditioning, label each with the same critic, and take the entropy of the resulting outcome distribution. Agreement in decision space is invariant to appearance by construction — the question asked is not whether two futures look alike, but whether they end in the same physical consequence.
| K critic verdicts, action held fixed | Entropy | Reading | Controller |
|---|---|---|---|
| clear · clear · clear · clear · clear | 0.00 | confident, and confident it is safe | commit |
| collide · collide · collide · collide · clear | 0.72 | confident, and confident it is not | veto branch |
| clear · collide · clear · collide · collide | 0.97 | the model does not know | latch hold |
Five rollouts that all end in contact are a confident model. Five that split three-two are a model with no opinion — precisely the state in which a robot must stop. The residual is kept and displayed on the same timeline, purely as the baseline it is meant to replace, so the failure mode of the naive method is visible next to the proposed one.
That turns the demo into a claim we can falsify in a weekend: across ~20–30 logged approach trials, does decision-space disagreement predict real failure — at what precision, and how many milliseconds of lead time before contact? If it does, it is the missing calibration primitive for world-model control. If it does not, that is a real negative result about a signal the field is about to reach for, and we will say so on stage.
Each of these is a weekend well spent on nothing.
| The easy version | Why it is out |
|---|---|
| Prompt-to-world scene generator | The world model is a renderer. No claim is being made. |
| "Robot dreams its next move" | Done in 2018. Being faster at it does not make it new. |
| Pixel-residual surprise meter | Measures the weather, not the model. Ships only as the baseline we beat. |
| Longer horizon, prettier frames | Fidelity theatre. Says nothing about whether the model can be trusted. |
| A rover doing obstacle avoidance | A demo, not a finding. If we get wheels, the wheels serve the measurement. |
Built top-down, cut bottom-up. Nothing above a cut line is ever sacrificed for anything below it.
If Sunday runs short, cut from the bottom — never reorder.
One probe, two questions, answered before the first line of the controller is written. The entire project's viability is decided here rather than discovered at 3 AM.
Q1 — does the model obey the action signal? Fire one frame under four different actions and check that the scene yaws the way it was told to, not merely plausibly.
Q2 — do K samples under a fixed action produce usable variance? If the rollouts come back near-identical, outcome entropy is always zero and the contribution dies.
| Outcome | Response |
|---|---|
| Q1 ✓ Q2 ✓ | Build the full loop. Playing for the win. |
| Q1 ✓ Q2 ✗ | Drop the doubt meter, keep everything else. Still a strong project. |
| Q1 ✗ | Pivot immediately, do not negotiate: counterfactual black box. Replay a recorded clip's decision points under alternate actions and score which futures survive. Same framing, same split screen, zero live-control dependency. |
These are actual rollouts from the live system: one real photograph — a real robot arm and a real device on a desk — expanded into three candidate futures by LingBot World 2 under a fixed noise seed, so the action is the only variable. Claude judged each transition; the shove was vetoed, the grab was chosen.
Measured on the way: our inverse-dynamics model reads the real robot at r = 0.999 (MAE 1.58 units, held-out episode) and collapses to r = 0.02 on the world model's imagined footage — the policy goes still, as if no robot were on screen. Achieved control cadence: ~2 Hz (measured 1.97–2.33 Hz), not the 3 Hz we originally targeted.
"We didn't want to build another robot that hallucinates the future — that's 2018 with better GPUs. We wanted to fix the thing that broke when the field swapped dynamics ensembles for one giant video model: nobody can tell you anymore when the model is wrong. So we built the loop as a test rig, and spent the weekend measuring whether disagreement between a world model's own futures predicts real failure. Here are the trials, here's the precision, here's the lead time — and here's exactly where the obvious pixel-residual method fails."
Pixel-space MPC carries no guarantees. The critic is a soft constraint, not a certificate — it can be wrong, and its false-negative rate is unmeasured. A 2–3 s horizon at reduced resolution bounds what the system can possibly catch. K is small, so the entropy estimate is coarse. And decision-space agreement can be low precisely because the model is confidently and uniformly wrong — a failure mode no ensemble method escapes either. None of this is fatal to the claim; all of it belongs in the claim.