🤖 Robotics arXiv Digest

Thursday, October 8, 2026

📄 30 papers 📂 6 research areas Generated by Claude

🔭 Research Landscape

Today's 30 papers are dominated by a single question: what should a robot predict about the future? Seven papers build world-action models (WAMs) and, read together, they reject the default of predicting future video. UNITAS (#3) argues for metric 3D scene flow because pixel distances are not physical distances (49% lower displacement error than PointWorld, 99.8% on LIBERO); MiniWAM (#10) shows that compact inverse-dynamics-shaped targets with 65× fewer tokens outperform native DINOv3 or video-VAE futures and train 8× faster; PlanWAM (#13) applies the same idea to driving, letting the planning loss shape the future latent (93.8 PDMS on NAVSIM); LiteNWM (#16) gets a 128× speedup by scoring candidate trajectories in latent space; and Spatial Language Modeling (#2) goes furthest, predicting explicit contour tokens. DreamTrue (#21) is the exception that proves the rule: it stays in video, but needs counterfactual post-training and a learned defect reward to cut physically implausible outcomes from 48% to 6%. The emerging consensus is that a world model's future should be shaped by what control needs, not by what a video generator happens to produce.

The second theme is human data at scale, bridged by a shared latent space instead of kinematic retargeting. VioLA (#8) trains a humanoid generalist on 140M frames, 93% of them human, by predicting body and hand motion latents that a pretrained controller executes. It reaches 100% zero-shot locomotion success where GR00T N1.7 manages 16.7%. Being-M0.7 (#14) puts the bridge earlier, in a visual-motion prior learned from 10k hours of mixed human video and motion. Generative Neural Retargeting (#22) replaces per-demo MPC with flow-matching sampling, using 8.5% of the samples and doubling success. TACROSS (#24) does the same for touch, aligning a $11 glove with robot sensors at the contact-event level. Dex-One2Many (#7) and YOCO (#28) round this out: one abstracts a single human video into scene-graph constraints, and the other removes per-user glove bias. Across these papers, cheap human data plus a learned translation layer is replacing expensive teleoperation.

A third, more contrarian thread asks how much a frozen foundation model can do with better scaffolding. ARC (#26) improves π_0.5's real-robot success by 82 points using only auto-labeled, action-grounded reasoning traces. SpatialHarness (#9) lifts a frozen GPT-6 Astra from 0% to 100% on Tower of Hanoi by rendering virtual camera views from a digital twin, and Rendering-Free Lookahead (#20) learns that same 'where to look' judgement offline. SuperNav (#4) builds an agent harness around an un-fine-tuned MLLM for navigation. The sharpest challenge is Embodied Turing Machines (#15): it removes the model from the loop entirely, uses coding agents to grow a code-only policy library (70% on 42 bimanual tasks), and proposes that library as a data engine for VLAs. RoboAware (#29) sits in between, learning only when to hand control to code and when to an end-to-end policy. On the classical side, the controlled benchmark in #5 offers a useful rule of thumb that applies to this debate too: hand-built structure wins when dynamics are predictable, and learning wins when they are stochastic.

World-Action Models & Predictive Representations

Models that jointly predict actions and future states — and today's sharp debate over what that 'future' should look like: pixels, 3D flow, compact latents, or planning-shaped codes.

#3 UNITAS
#10 MiniWAM
#13 PlanWAM
#14 Being-M0.7
#21 DreamTrue
#16 LiteNWM
#2 Unifying Policy Learning and State Prediction through Spa…

Foundation-Model Harnesses, Reasoning & Code Policies

Getting more out of frozen or lightly adapted foundation models at test time: virtual views, reasoning traces, code-as-policy libraries and learned skill coordinators.

#26 ARC
#9 SpatialHarness
#15 Embodied Turing Machines
#29 RoboAware
#30 Rewiring Semantics, Dynamics, and Control: A Simple yet E…

Human-to-Robot Transfer, Dexterity & Touch

Turning abundant human motion, video and touch into robot-executable data — via shared latents, generative retargeting, scene graphs and cheap gloves.

#8 VioLA
#7 Dex-One2Many
#22 Generative Neural Retargeting for Human-to-Robot Dexterou…
#24 TACROSS
#28 YOCO
#1 How Firm Should a Grasp Be?
#27 From Solo to Ensemble

Navigation, Active Perception & Localization

Agentic MLLM navigation, learning where to look, and infrastructure-free relative localization for robot teams.

#4 SuperNav
#20 Rendering-Free Lookahead for Question-Guided Active Vision
#11 Distributed Relative Localization for Homogeneous Multi-R…
#12 Distributed Relative Localization Based on Ultra-WideBand…

RL & Policy Learning Foundations

General-purpose training recipes: privileged-to-RGB-D curricula, higher-order action losses, and a controlled classical-vs-RL planning comparison.

#18 FOCUS
#19 Higher-Order Action Supervision Makes A Strong Policy Class
#5 Real-Time Motion Planning with Dynamic Hazards: Classical…

Control & Field Systems

Hybrid model/learning controllers on racing cars, quadrotors in wind and surface vessels — plus catalog-scale orbital conjunction screening.

#6 MAP2
#25 WAND
#23 Sim-to-Real RL for ASVs using SysID
#17 Scalable LEO Conjunction Screening using Adaptive Synthet…

World-Action Models & Predictive Representations

h=62

UNITAS: A 3D-Native World Action Model for Embodied Manipulation

📅 2026-10-08cs.RO👤 K. Jia (h=62)

Ruixiang Wang, Yongyi Su, Wenlve Zhou, Bo Yue, Hengyan Liu

Core Contributions

Argues that pixel-space WAMs are mismatched to robotics: images are view-dependent projections, so UNITAS puts observations, actions and scene dynamics in one shared metric 3D frame.
Represents both robot grippers and human hands as 3D point trajectories ('action flow'), giving an embodiment-agnostic action interface that can absorb human data.
A physical-time trajectory tokenizer compresses each point track into a single token anchored at its 3D position, and world-aligned 3D positional embeddings work with or without depth input.
At 1.7B params: up to 49% lower scene-displacement error than PointWorld on RoboTwin, 99.8% on LIBERO and 85% average real-world success — evidence that 3D-native dynamics helps both prediction and control.

Show abstract

World action models (WAMs) aim to answer a coupled physical question: given a task instruction, what motion should the robot execute, and how will that motion change the surrounding world? Most existing WAMs build on pretrained video generators and represent world evolution through images or visual latents. Robotic interaction, however, takes place in metric three-dimensional space, while images are view-dependent projections whose pixel distances do not directly encode physical distances. We introduce UNITAS, to our knowledge the first 3D-native world action model that unifies observations, actions, and scene dynamics in a shared metric 3D frame within each interaction, using a common representation across robot embodiments and human hands. Action flow represents human hands and robot grippers as 3D point trajectories, while scene flow describes scene-point displacements conditioned on these trajectories. World-aligned 3D positional embeddings ground visual tokens with or without depth input, and a physical-time trajectory tokenizer encodes each point trajectory as one token anchored at its current 3D position. This interface supports both direct action execution and action-conditioned scene prediction. With 1.7B parameters, UNITAS achieves the best action-conditioned scene prediction on RoboTwin among the compared methods, with up to 49% lower displacement errors than PointWorld, and state-of-the-art manipulation success, including 99.8% on LIBERO and an average of 85% across real-world tasks. The code is available at https://github.com/DexForce/UNITAS.

#10

h=36

MiniWAM: Learning Compact Future Targets for Efficient World-Action Modeling

📅 2026-10-08cs.RO👤 Wei-Yun Yau (h=36)

Jie Chen, Ruofei Bai, Yuxin Cai, Yifeng Zhang, Chengyang He

Core Contributions

Questions the assumption that WAMs should predict futures in native backbone space (DINOv3 / video VAE features), which is high-dimensional and expensive.
PRISM learns compact future targets from privileged current→future pairs by combining inverse-dynamics supervision with feature reconstruction, so targets emphasize control-relevant change.
Uses 65× fewer future tokens yet outperforms native-feature prediction for both DINOv3 and WAN2.1 targets, with up to 8× faster world-action training.
At only 0.25B params it is competitive with much larger WAMs on LIBERO, LIBERO-Plus and RoboTwin 2.0; analyses show the inverse-dynamics term adds behavioral structure beyond reconstruction.

Show abstract

World modeling has emerged as an effective co-training objective for robot policies, giving rise to World Action Models (WAMs) that jointly predict actions and future states. However, most WAMs predict future states in the native representation space of pretrained visual backbones, resulting in high-dimensional targets with substantial training cost. We introduce MiniWAM, which instead predicts compact future representations learned from privileged current-future transitions. To construct these targets, we propose Predictive Representations via Inverse Spatiotemporal Modeling (PRISM), which combines inverse-dynamics supervision with feature reconstruction to emphasize control-relevant transition information while preserving useful future-state information. With the learned PRISM encoder frozen, MiniWAM is trained to jointly predict the resulting targets and robot actions from current observations. With 65$\times$ fewer native future feature tokens, MiniWAM consistently outperforms native future-feature prediction with both DINOv3 and WAN2.1 VAE features, while achieving up to an 8$\times$ speedup in world-action training. At 0.25B parameters, MiniWAM is already competitive with substantially larger WAMs on LIBERO, LIBERO-Plus, and RoboTwin 2.0 simulation benchmarks. Representation analyses further show that PRISM contributes behavioral structure beyond feature reconstruction alone. These results demonstrate that effective world-action modeling does not require predicting native visual futures, and that compact predictive representations provide a strong and substantially more efficient target for policy learning. The project page is available at: https://j1dan.github.io/MiniWAM.

#13

h=32

PlanWAM: Planning-Shaped Future Representations for End-to-End Autonomous Driving

📅 2026-10-08cs.ROcs.LG👤 Wei-Chao Zhuang (h=32)

Jinchang Xu, Hongda Yu, Fengwei Dong, Wenhui Huang, Xi Wei

Core Contributions

Asks a question most driving world models skip — which future representation helps planning? — and answers by letting the planning loss shape the latent future.
A privileged posterior branch sees true future frames and is trained with trajectory objectives; Hindsight-to-Foresight Distillation teaches a history-only prior to predict that planning-shaped latent.
A Temporal Register Pyramid compresses history in a recency-aware way, keeping inference compact.
93.8 PDMS / 90.9 EPDMS on NAVSIM v1/v2 and 38.7 HD-Score zero-shot on closed-loop HUGSIM — strong across open- and closed-loop settings. Conceptually a twin of MiniWAM (#10) in driving.

Show abstract

World models in end-to-end autonomous driving predict future scene evolution to provide foresight for trajectory planning. Existing methods mainly study how to predict the future and how to use it, but less often ask which future representation is actually most useful for planning. To this end, we propose PlanWAM, a Planning-Shaped World Action Model. The key idea is to let the planning task shape the future-state representation, so that it retains the information most useful for planning. A latent world model then predicts this planning-shaped future latent representation from historical observations and uses it for planning, enabling foresighted planning. Specifically, we first use a Temporal Register Pyramid to compress multi-frame historical information in a recency-aware manner, learning a compact history representation oriented toward future reasoning and planning. We then introduce a privileged future posterior branch that observes ground-truth future frames, and shape its future latent representation with trajectory-planning objectives to obtain a planning-shaped future latent representation. Hindsight-to-Foresight Distillation trains a prior branch that depends only on history to predict this future latent representation. The predicted future latent representation serves as planning context and guides trajectory generation and selection. PlanWAM achieves 93.8 PDMS / 90.9 EPDMS on NAVSIM-v1/v2 navtest and reaches 38.7 HD-Score on closed-loop HUGSIM in a zero-shot setting, demonstrating leading planning performance across both open-loop and closed-loop evaluations. Extensive experiments further demonstrate that planning-shaped future representations provide an effective and deployable form of foresight for world-action models.

#14

h=31

Being-M0.7: A Latent World-Action Model for Humanoid Robots

📅 2026-10-08cs.ROcs.CVcs.LG👤 Zong-Qing Lu (h=31)

Junpeng Yue, Boyuan Li, Yuxuan Wang, Zepeng Wang, Yuhui Fu

Core Contributions

Learns from a 10,000+ hour human corpus that mixes video-only, motion-only and paired video-motion data — exploiting heterogeneous sources that most methods discard for lack of pairing.
Jointly predicting future visual latents and whole-body motion forces visual features to encode future kinematics, giving a humanoid-relevant dynamics prior.
Three-stage pipeline (human pre-training → robot mid-training for viewpoint/body → action post-training) with a gated cross-attention action expert reading from the frozen prior.
Highest aggregate success among compared methods on SIMPLE and matches the strongest baseline on real Unitree G1 loco-manipulation. Contrasts with VioLA (#8), which bridges human data at the action rather than visual-prior level.

Show abstract

Humanoid loco-manipulation requires coordinated locomotion and manipulation informed by future scene evolution and whole-body motion, yet learning these capabilities is constrained by scarce robot demonstrations. Human video and motion datasets offer scalable supervision, but many contain only video or motion rather than paired video-motion data. Moreover, human motion does not directly specify executable robot actions. We present Being-M0.7, a latent world-action model that transfers visual-motion priors learned from mixed-modality human data to humanoid control through pre-training, robot mid-training, and action post-training. We curate a corpus from more than 10,000 hours of raw human-centric data, integrating video-only, motion-only, and paired video-motion streams to learn complementary visual dynamics and whole-body kinematic structure. Joint prediction of future latent visual states and motion encourages visual representations to encode future kinematics. Robot mid-training adapts this coarse-grained prior to robot viewpoints and body dynamics. During action post-training, an action expert combines visual predictive representations from the frozen, adapted prior with current images and proprioception through gated cross-attention, grounding predictive context in executable whole-body commands. Being-M0.7 achieves the highest aggregate success rate among the compared baselines on SIMPLE and matches the strongest baseline on real-world Unitree G1 loco-manipulation tasks.

#21

h=23

DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Training

📅 2026-10-08cs.ROcs.CV👤 Lue Fan (h=23)

Junyan Li, Ruizhi Li, Yu Liu, Xiangshuo Liu, Mingchao Sun

Core Contributions

Targets two silent failure modes of robot video world models: poor action following from miscalibrated data, and optimism bias from datasets dominated by successful episodes.
Renders action trajectories into image-space conditions with offline geometric calibration, improving action fidelity across embodiments and views.
Counterfactual post-training perturbs recorded actions to generate futures for unseen contacts and failures, scored by an embodied video reward model trained on human defect labels and optimized with RL.
Human-assessed interaction defect rate falls from 48.12% to 6.25% on AgiBot, and it won the world-model track of the AgiBot World Challenge 2026.

Show abstract

We present DreamTrue, a multi-view, cross-embodiment robot world model for action-faithful and physically plausible video prediction. Training such a model on existing robot datasets faces two obstacles: imprecise calibration can impair action following, while limited coverage of unsuccessful interactions can bias predictions toward successful outcomes. To improve action following across embodiments, we render action trajectories into image-space conditions and introduce offline geometric calibration to align these conditions with the target videos. To broaden interaction coverage, we introduce counterfactual post-training, modifying recorded action trajectories and generating future videos under a wider range of actions and contact configurations. To provide feedback on these predictions without paired ground-truth futures, we construct a human-annotated video dataset covering robot, object, and interaction defects and use it to train an embodied video reward model. Its scores guide reinforcement-learning post-training toward more physically plausible interaction outcomes. On AgiBot, DreamTrue attains state-of-the-art action following, while reducing the human-assessed interaction defect rate from from 48.12% to 6.25%. Notably, our model ranks first in the world model track of the AgiBot World Challenge 2026. The project page can be found at https://brave-eai.github.io/DreamTrue.

#16

h=27

LiteNWM: Efficient Latent World Models for Onboard Visual Navigation in the Wild

📅 2026-10-08cs.RO👤 Zhen-Shan Bing (h=27)

Linkai Liu, Yuntian Zhang, Zhenshan Bing, Chen Chen, Lingjuan Lyu

Core Contributions

Makes navigation world models deployable: instead of generating pixel rollouts per candidate, it shares one visual encoding and predicts latent futures for all candidates at multiple horizons in one pass.
A learned scorer ranks candidate trajectories from those predictions, turning the world model into a plug-in evaluator for any proposer.
17.56% lower trajectory error than NoMaD+NWM-XL with a 128× end-to-end speedup; transfers to the MBRA proposer without retraining (−16.2% error).
Real-robot success in unseen indoor/outdoor scenes rises from 43.3% to 83.3% over NoMaD — foresight finally fits onboard.

Show abstract

Direct visual navigation policies generate trajectories efficiently but do not explicitly evaluate their future consequences. Generative navigation world models provide this foresight through visual rollouts, which are costly when evaluating multiple candidates. We present LiteNWM, a latent navigation world model that shares visual encoding across candidates and jointly predicts their action-conditioned future representations at multiple horizons, while a learned scorer uses these predictions to select trajectories. In offline evaluations on RECON, SCAND, and SACSoN, LiteNWM reduces macro-averaged trajectory error by 17.56% relative to NoMaD+NWM-XL and achieves a 128.00-fold end-to-end speedup on an RTX 5090. The same evaluator transfers from NoMaD to MBRA without proposer-specific retraining, reducing MBRA's macro-averaged trajectory error by 16.2%. In real-robot experiments in unseen indoor and outdoor environments, LiteNWM improves navigation success from 43.3% to 83.3% relative to NoMaD. These results demonstrate that LiteNWM can be deployed for future-aware planning and closed-loop navigation on a physical robot.

h=77

Unifying Policy Learning and State Prediction through Spatial Language Modeling

📅 2026-10-08cs.ROcs.AI👤 T. Tuytelaars (h=77)

Minye Wu, Zehao Wang, Tinne Tuytelaars

Core Contributions

Proposes a 'spatial language': contours, goals, action targets and future states all become tokens in one discrete coordinate vocabulary, so a single autoregressive Transformer learns both control and action-conditioned prediction via next-token loss.
Unlike WAMs that bolt a video decoder onto a policy, the world-prediction target here is explicit geometry (scene contours) — cheap, interpretable and directly tied to pushing outcomes.
Random-play pretraining (actions as conditioning, excluded from the loss) teaches physics before any expert data — ablations show joint action+state sequences improve control, and random-play adds further gains.
On Push-T it is competitive in simulation and beats real-robot baselines in success and target coverage, while the same model can roll out scene states given an action plan.

Show abstract

Learning how actions change scene geometry can provide complementary supervision for goal-directed manipulation. We introduce Spatial Language Modeling, which represents scene contours, goals, action targets, and future states with a shared vocabulary of discrete coordinates and semantic tokens. A task-specific grammar organizes these elements into spatial sequences, allowing one autoregressive Transformer to learn action generation and action-conditioned state prediction through a common next-token objective. We train the model from scratch using random-play transition pretraining followed by joint action and state training on expert demonstrations. During pretraining, recorded action coordinates condition subsequent state predictions and are excluded from the prediction loss. During control, the model decodes only executable action targets and updates its history with newly observed states. We evaluate the approach on Push-T in simulation and on a real robot. The model achieves competitive simulation performance and higher task success and target coverage than the evaluated real-robot policy baselines. Training ablations show improved control with joint action and state sequences, with further gains from random-play pretraining. Given supplied action trajectories, the same model also predicts successive scene states, capturing the geometric effects of pushing.

Foundation-Model Harnesses, Reasoning & Code Policies

#26

h=20

ARC: A Reasoning Recipe for Robot Foundation Models

📅 2026-10-08cs.ROcs.AI👤 Ankit Goyal (h=20)

Gokul Puthumanaillam, Tao Sun, Elie Aljalbout, Moritz Reuss, Zhaoshuo Li

Core Contributions

Shows that a reasoning recipe, not more data or scale, can drive large zero-shot gains in existing robot foundation models.
Key finding: useful traces are grounded in the robot's next action and explain its causal structure — why it's appropriate and what effect it should have — rather than generic scene descriptions.
Traces are auto-labeled from existing demos (ARC-Trace-DROID), so no new robot data is needed; adaptation is tailored to both VLAs (π_0.5) and WAMs (Cosmos3-Nano-Policy).
New SOTA on RoboLab-120 and MolmoSpaces, up to +50 points on RoboLab-Reasoning-50, and +82.2 points real-robot success for π_0.5.

Show abstract

The prevailing approach to improving robot foundation models (RFMs) relies on larger models, more robot demonstrations, and costly training at scale. We show that there exists an effective and efficient complementary approach: the right reasoning recipe can substantially improve the zero-shot task performance of existing state-of-the-art RFMs. We refer to this recipe as ARC. It consists of three key ingredients: a reasoning trace, a scalable automatic labeling pipeline, and a strategy for adapting pretrained RFMs to use these traces for control. First, we find that effective reasoning traces should be grounded in the robot's next action and explain its causal structure: why the action is appropriate and what effect it should produce. Second, we show that these traces can be generated automatically from existing demonstrations, enabling us to construct ARC-Trace-DROID from DROID without collecting new robot data. Third, we show how state-of-the-art VLAs such as $π_{0.5}$ and WAMs such as Cosmos3-Nano-Policy can learn to use these traces for control, with fine-tuning and inference tailored to each model's architecture and capabilities. Using ARC, we obtain gains in zero-shot RFM performance that, to our knowledge, are unprecedented without additional robot demonstrations or foundation-scale training. The adapted models establish a new state of the art on RoboLab-120 and MolmoSpaces, with gains of up to 50 percentage points on RoboLab-Reasoning-50. On real robots, ARC improves $π_{0.5}$'s task success by 82.2 percentage points. Project website: https://arc-robot-reasoning.github.io/

h=38

SpatialHarness: Test-Time Spatial Scaffolding for Fine Robotic Manipulation

📅 2026-10-08cs.RO👤 Jing-Jing Chen (h=38)

Jiayu Wang, Yue Yu, Bin Zhu, Zhiyao Yang, Jingjing Chen

Core Contributions

Diagnoses fine-manipulation failures of frontier multimodal models as a spatial observability problem, not a capability problem: the physical cameras simply don't show the critical relation.
Keeps a simulated twin synchronized with execution and renders task-specific virtual views (e.g., looking down a plug socket) for a frozen policy — no fine-tuning, no extra hardware.
Interaction-aware synchronization distinguishes static, held and transition modes so the twin stays aligned while objects are grasped and moved.
With the same frozen GPT-6 Astra policy: plug insertion 26.7% → 66.7%, Tower of Hanoi 0% → 100% — suggesting latent ability is being bottlenecked by viewpoint.

Show abstract

Frontier multimodal foundation models (e.g., GPT-6 Astra) have recently shown strong potential for direct robotic control, yet their performance on fine manipulation remains limited. We argue that an important source of failure is not necessarily insufficient policy capability, but insufficient spatial observability, where task-critical spatial relationships may be poorly revealed by the existing physical camera setup. We introduce SpatialHarness, a test-time embodied harness that provides test-time spatial scaffolding for fine robotic manipulation without policy fine-tuning or changes to the physical sensing setup. SpatialHarness maintains an online simulated scene synchronized with real-world execution, identifies task-critical spatial relationships, and renders complementary virtual views that expose them to a frozen multimodal policy. To keep the simulated scene aligned during interaction, we develop interaction-aware scene synchronization that distinguishes static, held, and transition modes. We evaluate SpatialHarness on four real-robot manipulation tasks spanning precise geometric alignment, object-relative placement, and articulated-object interaction. Using the same frozen GPT-6 Astra policy, SpatialHarness substantially improves task success, including from 26.7% to 66.7% on plug insertion and from 0% to 100% on Tower of Hanoi. These results indicate that improving spatial observability at test time can unlock fine-manipulation capabilities already present in strong multimodal foundation models. Project website: https://emilia113.github.io/SpatialHarness/.

#15

h=27

Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement

📅 2026-10-08cs.ROcs.CV👤 Fangzhou Hong (h=27)

Kairui Hu, Siyuan Hu, Fangzhou Hong, Zhaoxi Chen, Ziwei Liu

Core Contributions

A provocative counterpoint to VLAs and agent harnesses: treat the world as a Turing machine whose tape is state, and write the entire policy as code that measures and tracks that state — no model in the test-time loop.
Code makes state explicit, decisions inspectable, failure recovery programmable, and runtime fast and cheap; new tasks inherit from a shared library so capabilities accumulate.
Positions coding agents as the engine of recursive self-improvement: they iteratively extend the library in a closed loop with every change auditable.
A single library reaches 70.24% across RoboDojo's 42 bimanual tasks, and the authors pitch it as a cheap data engine for training VLAs.

Show abstract

Most robot policies keep a model in the control loop: a VLA maps observations to actions, and an Agent Harness, such as Agent-as-Policy or Harness VLA queries a VLM for decision making at run time. We propose a different view: the embodied world is an Embodied Turing Machine, whose tape is the robot and environment state and rules are the policy. If this state can be represented accurately, the decision making can be written entirely in code. We therefore propose Code-Only-as-Policy (COAP): code measures and tracks the robot, environment, and task state from camera images and proprioception, and makes every decision from it. The same code applies across episodes, and different tasks share one library without a VLM or VLA in the loop. Compared with VLAs and Agent Harnesses, we analyze three advantages of COAP: (i) Explicit State: the state can be stored in code; (ii) Execution: code makes decision making controllable, recovers from failures flexibly, and runs fast and cheaply online; (iii) Extensibility: new tasks reuse, inherit, or extend the shared library, so capabilities can accumulate over tasks. These advantages make COAP a suitable medium for recursive self-improvement (RSI): coding agents develop the library in a closed loop, and each change is explicit and controllable. On RoboDojo's 42 bimanual tasks, the resulting library reaches a success rate of 70.24% without a model at test time. The upper bound of COAP lies in how accurately the state is represented for decision making and how robust the code logic is. We thus propose COAP as a new paradigm for embodied tasks; since it applies across episodes, it can also serve as an efficient data engine for VLAs and Agent Harnesses.

#29

h=20

RoboAware: Learning to Coordinate Embodied Skills from Counterfactual Outcomes

📅 2026-10-08cs.ROcs.AI👤 Yin-Chuan Li (h=20)

Bohan Zhou, Xingbei Chen, Emily Huang, Weilin Ruan, Haojian Huang

Core Contributions

Keeps the coding agent frozen and learns only a small state-conditioned coordinator that decides which policy family (scripted skill vs. end-to-end policy) should act next.
State-Locked Counterfactual Branching restores the exact training state and executes every admissible branch — exposing the outcomes of choices not taken.
Execution-Aware Learning combines MCTS with Q-learning to distill those outcomes into family-conditioned values, organized by the five-stage P⁵ schema.
77.0% over 100 tasks, with 90.0% on RoboSuite, 73.8% on LIBERO-Pro clusters and 90.0% on RoboTwin bimanual, beating code-as-policy and VLA-harness baselines.

Show abstract

Embodied coding agents can combine modular robot skills with frozen end-to-end policies, yet effective composition requires anticipating which policy family will succeed in the current physical state. We present RoboAware, which builds on coding agents' skill orchestration by learning only a state-conditioned responsibility coordinator from counterfactual outcomes. Inspired by the success of REPL, we propose the $P^5$ schema and formulate a hierarchical MDP based on it. $P^5$ organizes skills uniformly into five semantic stages, defining where responsibility can be compared. To address the lack of counterfactual branch outcomes in existing work, we introduce State-Locked Counterfactual Branching (SCB), which restores the same training state to generate and execute a code block from each admissible family, exposing outcomes that selected-branch experience leaves unobserved. Building on this, we propose Execution-Aware Learning (EAL), which combines Monte Carlo tree search with Q-learning to distill these outcomes into family-conditioned values. At deployment, the coordinator selects the policy family according to observable context, and the frozen coding agent generates the next local code block. Comprehensive single-episode evaluations on 100 tasks show that RoboAware reaches a 77.0% overall success rate, with SOTA averages of 90.0% on RoboSuite, 73.8% on diverse LIBERO-Pro task clusters, and 90.0% on challenging RoboTwin bimanual tasks, outperforming existing code-as-policy and VLA-harness baselines.

#30

h=20

Rewiring Semantics, Dynamics, and Control: A Simple yet Effective Action-Centric Tri-Stream Transformer

📅 2026-10-08cs.ROcs.AI👤 Shunyu Liu (h=20)

Shuang Luo, Yilun Kong, Yunpeng Qing, Yihang Jiao, Zhi Hou

Core Contributions

Tackles how to combine a VLM (semantics) and a video world model (dynamics) for control without one drowning out the other.
ACT³ uses three streams: a dedicated action expert attends layer-wise to both backbones, while each backbone attends only within itself — preserving their distinct roles.
Both backbones still receive gradients from control supervision, so they adapt toward action-relevant features without being fused.
Reports gains over fusion counterparts on simulated and real manipulation benchmarks — a simple architectural answer to the VLA-vs-WAM divide.

Show abstract

Vision-Language-Action (VLA) models have emerged as a prominent framework for complex robotic manipulation, building on the strong semantic understanding of pretrained Vision-Language Models (VLMs). However, such VLM backbones offer insufficient physical dynamics priors, which limits the generalization capabilities of robot policies. Recent efforts therefore integrate video-generation World Models (WMs) into robot policies through various strategies, using predictive dynamics to facilitate action generation. Despite these advances, harnessing semantic understanding and dynamics prediction as complementary guidance for action generation remains challenging. In this paper, we introduce $\mathrm{ACT}^3$, a simple yet effective Action-Centric Tri-Stream Transformer that fuses semantic and dynamics information into control actions while preserving the distinct roles of context streams. Specifically, $\mathrm{ACT}^3$ enables the dedicated action expert to access VLM and WM representations through layerwise attention, with each backbone attending only within its own stream. This straightforward interaction design maintains independent forward propagation in the context streams while allowing both backbones to be updated through control supervision. Experiments on both simulated and real-world robotic manipulation benchmarks show that the proposed $\mathrm{ACT}^3$ yields results superior to its counterparts.

Human-to-Robot Transfer, Dexterity & Touch

h=39

VioLA: Learning Generalist Humanoid Control Policies from Human Data

📅 2026-10-08cs.ROcs.LG👤 Wieland Brendel (h=39)

Mert Albaba, Jens Beißwenger, Anna Manasyan, Daniel Marta, Michael J. Black

Core Contributions

Changes what a humanoid generalist predicts: body and hand motion latents executed by pretrained controllers, instead of joint commands — sidestepping the coupled, balance-critical joint space.
Because motion encoders map both human and robot motion into the same latents, any human recording is automatically labeled in the policy's action space; 93.2% of its 140.6M training frames are human.
Zero-shot real-robot locomotion following hits 100% success where GR00T N1.7 gets 16.7% and Ψ₀ gets 0%, and manipulation reaches 88.6% with no task-specific fine-tuning.
Backbone-agnostic: works with two VLA and one world-action model backbones, and a policy trained on human data alone can still walk on the real robot.

Show abstract

Teaching a humanoid to follow instructions with its whole body runs into two obstacles. Its action space is large and tightly coupled: legs, arms, and fingers must move together while the robot keeps its balance, which makes joint-level actions hard to learn. And humanoid demonstrations are scarce, so current humanoid generalist policies do not follow new instructions out of the box and are fine-tuned on teleoperated demonstrations of each task before deployment. Human demonstrations exist in far larger numbers, but a person's motion is not a robot command. We remove both obstacles by changing what the generalist policy predicts. We introduce VioLA, a generalist humanoid policy that predicts body and hand motion latents instead of joint commands. A pretrained body- and hand-controller execute these latents on the robot. Their corresponding motion encoders map human and robot motion into the same latent spaces. A human recording is therefore labeled in the policy's action space, and the training demonstration pool contains 140.6 million frames, 93.2% of them human. As a result, VioLA follows locomotion instructions on the real robot zero-shot, without task-specific fine-tuning, reaching 100% success where GR00T N1.7 and $Ψ_0$ reach 16.7% and 0%, respectively. It also reaches 88.6% manipulation success without task-specific fine-tuning. The same approach works across two VLA and one world-action model backbones. A generalist policy trained on human demonstrations alone performs locomotion tasks on the real robot zero-shot. Code and checkpoints will be released.

h=43

Dex-One2Many: Learning Dexterous Manipulation from a Single Human Demonstration

📅 2026-10-08cs.ROcs.CV👤 Jia-Bin Huang (h=43)

Jusuk Lee, Sungha Kim, Yeonsoo Park, Jonguk Cheon, Yoonkyo Jung

Core Contributions

Targets the gap between pure imitation of a single human video (brittle to new poses) and RL from scratch (can't explore multi-stage tasks) by abstracting the video into sequential scene graphs.
The graphs constrain object relations, not exact poses — so they double as generators of diverse reset states (beyond the video's poses and grasps) and as dense per-stage rewards.
Stage-wise resets keep RL exploration short and guided, making long-horizon tool use with a multi-fingered hand tractable in simulation.
Zero-shot sim-to-real on a dexterous hand across five tasks: +6.5% over baselines on seen configurations, widening to +71% on unseen ones — generalization is where the abstraction pays off.

Show abstract

While learning dexterous manipulation from a single human video offers a promising alternative to costly robot demonstrations, many recent methods predominantly imitate demonstrated motions. Such strict motion matching often limits generalization to initial object poses, goal poses, and grasps not shown in the video. Alternatively, discovering a policy via reinforcement learning (RL) allows for broad generalization, but without prior guidance, it struggles with high-dimensional exploration in complex, multi-stage tasks. To address these coupled generalization and exploration challenges, we present Dex-One2Many, a real-to-sim-to-real framework that learns a generalizable dexterous manipulation policy from a single human video. Our key insight is to abstract the video into sequential scene graphs that guide RL, enabling efficient exploration while preserving broad generalizability. The graphs serve as generative constraints for sampling diverse reset states and provide dense rewards for each stage. Because the graphs constrain relations rather than exact poses, these reset states cover object poses and grasps beyond the video, while initializing each stage from them with dense rewards keeps exploration short and guided. Trained entirely in simulation, Dex-One2Many transfers zero-shot to a real multi-fingered hand. Across five tool-use and manipulation tasks, Dex-One2Many exceeds baselines by 6.5% in seen configurations, while its robust generalization widens this gap to 71% in unseen scenarios.

#22

h=21

Generative Neural Retargeting for Human-to-Robot Dexterous Manipulation

📅 2026-10-08cs.RO👤 Zhiyang Dou (h=21)

Dechen Gao, Yue Yang, Ben Abbatematteo, Nathan Godwin, Pengcheng Wang

Core Contributions

Hypothesizes that dynamically feasible retargeted trajectories lie near a shared low-dimensional manifold — so retargeting becomes conditional sampling instead of a fresh optimization per demo.
Uses flow matching conditioned on human motion; unlike per-trajectory MPC, solving earlier demos makes later ones cheaper.
56.2% success vs 27.2% for sampling MPC while using only 8.5% of MPC's samples.
Powers a real-to-sim data engine producing 223k dexterous demonstrations across 3.3k objects with dense contact-force labels — a sizable new resource.

Show abstract

Human demonstrations are a scalable data source for learning dexterous manipulation, but the embodiment gap prevents human motion from being executed directly on robots. Inverse kinematics (IK) retargets human motion to robots efficiently but ignores dynamics, often producing infeasible motions. Reinforcement learning (RL) and sampling-based model predictive control (MPC) are commonly employed to yield dynamically feasible motions, but both are sample-inefficient and sensitive to hyperparameters. RL suffers from costly and unstable training and tedious reward engineering; MPC avoids policy optimization, yet retargets each trajectory in isolation, and solving one does not make the next easier. Sampling cost grows rapidly with dataset size and task difficulty. We hypothesize that dynamically feasible trajectories concentrate near a low-dimensional manifold shared across demonstrations, so that retargeting can be reduced to sampling from that manifold, conditioned on human motion, rather than solving a fresh optimization problem for every demonstration. We propose \textbf{Generative Neural Retargeting} (GNR), which uses a flow matching model to sample feasible trajectories. GNR outperforms MPC with only $8.5\%$ of the samples required by MPC, achieving a success rate of $56.20\%$ compared to $27.20\%$ for MPC. GNR can be used for scalable and efficient retargeting of large-scale, long-horizon, and millimeter precision human demonstrations: by applying GNR within a real-to-sim data engine, we produce a dexterous manipulation dataset with dense contact-force labels, spanning $223$k demonstrations and $3.3$k object geometries.

#24

h=21

TACROSS: An Efficient and Low-Cost Scalable Human Touch System Across Heterogeneous Tactile Sensors for Dexterous Robot Learning

📅 2026-10-08cs.ROcs.LG👤 Zhong-Liang Jiang (h=21)

Bo Chen, Huanzhang Hu, Junyang Ma, Bo Yue, Fangdi Yu

Core Contributions

Recognizes that glove and robot tactile sensors differ in transduction, layout and dynamics, so it aligns at the level of contact events rather than raw channels.
Hardware: a 5-layer, 285-taxel piezoresistive glove costing USD 10.86; canonicalizers + residual adapters + a cross-finger temporal Transformer map heterogeneous sensors into a shared 256-D tactile latent.
Policy learning stays robot-grounded — robot demos provide action labels while human data shapes tactile representations and adds confidence-weighted auxiliary targets.
3.5× demonstration efficiency vs teleoperation at 95.7% lower equipment cost, with 150+ hours of tactile data to be released.

Show abstract

Collecting tactile demonstrations on robots is costly and slow, motivating the use of lower-cost human tactile gloves for scalable data collection. However, human capacitive/piezoresistive gloves and robotic tactile sensors differ fundamentally in transduction principle, sensor layout, spatial resolution, and dynamic response, making alignment of raw sensor channels ill-posed. To address this problem, we present TACROSS, a scalable system for learning from human touch and transferring it to robots that bridges this heterogeneity by aligning tactile streams at the level of contact events rather than raw sensor values. The hardware component of TACROSS integrates a piezoresistive glove with five layers and a cost of USD 10.86 with 285 sensing points. To align contact semantics, we design canonicalizers and residual adapters that map heterogeneous signals into a shared tactile latent with 256 dimensions via a temporal Transformer with attention across fingers. We further introduce a robot-grounded policy learning scheme in which robot demonstrations provide the sole source of ground-truth action supervision, while human demonstrations support tactile representation learning and provide confidence-weighted auxiliary supervision through valid retargeted hand targets. We evaluate our system on four contact-rich manipulation tasks. Compared to conventional teleoperation, our proposed system achieves a 3.5-fold efficiency improvement while reducing demonstration acquisition equipment cost by 95.7%. We will open-source the TACROSS hardware and software system and publicly release a tactile dataset comprising over 150 hours of recordings. Project page: https://tacross-touch-project.github.io/.

#28

h=20

YOCO: You Only Calibrate Once! Fast Mocap Calibration for Dexterous Teleoperation

📅 2026-10-08cs.RO👤 Yan-Chao Yang (h=20)

Yu Zhang, Yunqi Li, Yushi Du, Yi Ma, Yanchao Yang

Core Contributions

Addresses a mundane but costly issue in dexterous data collection: glove and markerless hand-tracking biases that shift per user, fit and session.
A HyperNet conditioned on a few paired raw/target poses predicts LoRA-style updates to a frozen MANO estimator — calibration becomes one feed-forward pass, no per-user optimization.
Preserving the MANO prior keeps poses anatomically plausible; synthetic drift augmentation on InterHand2.6M provides training signal.
Improves calibration efficiency, hand-state accuracy and downstream teleoperation compared with uncalibrated input and standard calibration baselines.

Show abstract

Dexterous teleoperation requires reliable human-hand state estimations. However, common low-cost motion-capture gloves and markerless trackers often exhibit biases that vary across users, glove fit, and recording sessions, degrading retargeting and demonstration quality. We present YOCO, a fast few-shot, fine-tuning-free calibration framework that corrects biased hand-pose streams from a small set of paired raw and target poses. Instead of optimizing a separate model for every operator or session, YOCO conditions a calibration HyperNet on the paired examples and predicts LoRA-style updates for a frozen MANO hand-estimation module, turning per-user calibration into a lightweight feed-forward adaptation step while preserving the geometric prior of MANO and the efficiency of a compact estimator. We train YOCO with synthetic drift augmentations on InterHand2.6M and evaluate on augmented InterHand sequences, offline real glove data, and dexterous teleoperation tasks. Across these settings, YOCO improves calibration efficiency, hand-state estimation quality and teleoperation performance compared with uncalibrated input and standard calibration baselines.

h=108

How Firm Should a Grasp Be?

📅 2026-10-08cs.ROcs.CV👤 S. Nayar (h=108)

Matthew Beveridge, Shree K. Nayar

Core Contributions

Reframes grasping as an inverse problem: instead of a fixed force or slip-reactive controller, it estimates mass, elasticity and friction during closure and computes the minimum force that still holds — directly targeting the bruise-vs-drop trade-off for delicate produce.
Fuses vision (shape/size) with tactile deformation response, so material properties are inferred from the grasp itself rather than from a prior object database.
Contributes a measured-property dataset of real fruits and vegetables (shape, mass, elasticity, friction) — a rare ground-truth resource for physically grounded grasp research.
The force model is built in simulation from the measured properties, then validated on a real parallel-jaw gripper across many objects — a physics-first route that needs no large real grasp dataset.

Show abstract

An ideal robot grasp is firm enough to securely handle an object, yet gentle enough to avoid damaging it. Achieving this balance requires knowledge of the object's material properties, such as its mass, elasticity, and surface friction. These properties, however, are seldom precisely known a priori. In this work, we propose a visuotactile approach to estimating material properties in real time, during the process of grasping. Our method uses these estimated properties to determine the minimum grasp force required to handle the object. We contribute a new dataset of real-world objects (fruits and vegetables) with measured physical properties (shape, mass, elasticity, and friction), which we use to construct our force estimation model via simulations. We experimentally validate our approach to grasp force control using a robot with a parallel-jaw gripper. We demonstrate our system's ability to gently grasp a wide variety of objects, in each case adapting to their unique physical properties.

#27

h=20

From Solo to Ensemble: A Hierarchical Framework for Composable Multi-Agent Human-Object Interaction

📅 2026-10-08cs.RO👤 Ye Shi (h=20)

Zekai Deng, Kangyi Chen, Ye Shi, Jingya Wang

Core Contributions

Converts a single-agent physics-based HOI policy into a reusable Object-oriented Motion Skill that executes short-horizon object-proxy motions.
Separates concerns: the distilled low-level skill is frozen, and a high-level policy coordinates multiple agents by issuing region-wise object-level actions.
Shifts multi-agent learning from contact-rich full-body control to compact object-level coordination, so the same skill composes across object shapes, interaction types and team sizes.
Avoids per-task fine-tuning that entangles contact execution with coordination in prior multi-agent HOI work.

Show abstract

Physics-based human-object interaction has achieved robust single-agent manipulation skills, yet extending them to multi-agent cooperative tasks remains challenging. Existing approaches typically adapt interaction policies through task-specific fine-tuning, which entangles low-level contact-rich execution with high-level coordination and limits reuse across object geometries, interaction types, and team sizes. We propose a hierarchical framework that converts a single-agent HOI policy into a reusable Object-oriented Motion Skill. Specifically, we reinterpret teacher rollouts as object-oriented action supervision by extracting short-horizon object-proxy motions from executed trajectories, and distill task-specific teachers into a low-level skill operating in an Object-oriented Action Space. For downstream tasks, the distilled skill is frozen as a reusable executor, while a high-level policy coordinates multiple agents by generating region-wise object-oriented actions conditioned on the shared object, task goal, agent states, and local manipulation regions. This formulation shifts multi-agent HOI learning from direct contact-rich full-body control to compact object-level proxy-motion coordination. Experiments on diverse HOI tasks show that the distilled Object-oriented Motion Skill supports robust proxy-motion execution and enables composable policy learning across different interaction types, object geometries, and team sizes.

Navigation, Active Perception & Localization

h=60

SuperNav: An Agentic Navigation System for Any Task in Any Scene

📅 2026-10-08cs.ROcs.CV👤 Xiao-Wei Zhou (h=60)

Jinkai Zhang, Jingyi Xu, Yuanhong Yu, Jiarui Guo, Ruizhen Hu

Core Contributions

Inverts the dominant recipe of fine-tuning MLLMs to emit navigation actions: SuperNav keeps the MLLM frozen and general, and builds an agent harness around it — skills, tools, progress tracking and context management.
A visual-point interface lets the model specify goals by pointing in images, while classical navigation tools handle motion — preserving the MLLM's open-world reasoning instead of overwriting it with narrow nav data.
Execution feedback lets the agent revise decisions mid-task, supporting long, multi-object and demand-driven ('I'm thirsty') requests that single-shot policies struggle with.
Beats four baselines on instance-level, multi-object and demand-driven tasks, plus HM3D category-level evaluation and a real quadruped deployment.

Show abstract

General-purpose service robots need navigation systems that can handle diverse human requests in unfamiliar environments, combining task generality with scene generality. Some existing methods fine-tune multimodal large language models (MLLMs) to predict navigation actions, making their behavior dependent on the coverage of navigation training data and potentially limiting generalization to new requests and environments. Our key insight is to let the MLLM focus on interpreting requests, understanding scenes, and making decisions while preserving its general-purpose capabilities and delegating motion execution to navigation tools. To realize this idea, we introduce SuperNav, which equips a pretrained MLLM with a specialized agent harness without navigation-specific fine-tuning of the MLLM. Our harness supports these decisions with Navigation Skills, agent-oriented Tools for physical interaction, and task-progress and context management. A unified visual-point interface connects decision-making to motion by allowing the model to specify destinations directly in images and revise its decisions from execution feedback. Together, these components support sustained navigation across different task requirements and environments. SuperNav outperforms four evaluated baselines on instance-level, multi-object, and demand-driven tasks. Category-level evaluation on HM3D and deployment on a real quadruped robot further demonstrate its applicability across environments. Project Page: https://zju3dv.github.io/SuperNav/

#20

h=24

Rendering-Free Lookahead for Question-Guided Active Vision

📅 2026-10-08cs.CVcs.RO👤 Yusuke Iwasawa (h=24)

Koya Sakamoto, Daichi Azuma, Shuhei Kurita, Naoya Chiba, Yusuke Iwasawa

Core Contributions

Frames active vision for VQA as predicting answerability — a VLM's estimate that a future view will suffice to answer the question.
Moves expensive lookahead to training: a privileged teacher renders candidate views in 3D Gaussian Splatting scenes and scores 1- and 2-step answerability with a frozen VLM.
Two-stage distillation yields a student that predicts these action values from the question, recent frames and a candidate motion — no rendering at deployment.
On 377 unseen-environment E3VS-Bench episodes, mean judge score improves 43% over a direct-action baseline with the same VLM. Shares SpatialHarness's (#9) premise that the right view unlocks the model.

Show abstract

Active robot vision requires controlling the camera to reveal task-relevant information that is hidden from the current viewpoint. For example, determining what is inside a box may require raising the camera and looking down into it. For viewpoint-dependent question answering, the challenge is to select camera motions that expose the visual evidence needed to answer the question. Although vision-language models (VLMs) can interpret observed images, selecting such motions requires anticipating the usefulness of unseen views. We quantify this usefulness as answerability, a VLM's estimate that a view suffices to answer the question, and present Rendering-Free Lookahead (RFL), a viewpoint-selection policy that ranks candidate camera motions by predicted future answerability. RFL transfers visual lookahead from deployment to offline training. At training, a privileged teacher renders candidate future views in 3D Gaussian Splatting (3DGS) scenes and uses a frozen VLM to compute one- and two-step answerability targets. Through two-stage distillation, a student learns to predict these action values from the question, recent visual observations, and a candidate camera motion. At deployment, RFL uses these predicted values to select camera motions without rendering future views. On 377 E3VS-Bench test episodes in unseen environments, RFL improves the mean judge score by 43\% over a direct-action baseline using the same VLM. These results support learning camera-control policies from privileged visual lookahead for viewpoint-dependent question answering.

#11

h=35

Distributed Relative Localization for Homogeneous Multi-Robot Systems through UWB Ranging and Limited Communications

📅 2026-10-08cs.RO👤 Chau Yuen (h=35)

Zhiqiang Cao, Ran Liu, Billy Pik Lik Lau, Chau Yuen, U-Xuan Tan

Core Contributions

Solves the 'which blob is which teammate?' problem for identical-looking robots by combining anonymous LiDAR cluster tracking with UWB ranges and odometry.
A joint matching strategy associates tracked clusters to robot IDs, then fuses LiDAR, UWB and odometry for full relative pose estimation (not just position).
Only odometry is exchanged, over the UWB radios themselves — no Wi-Fi, mesh network or external anchors — fitting GPS-denied search-and-rescue settings.
Validated in simulation and real experiments; a direct evolution of the same group's graph-matching approach (#12).

Show abstract

Accurate and reliable relative localization is crucial for multi-robot applications like exploration, search, and rescue missions. LiDAR-based solutions offer high accuracy in localizing surrounding objects; however, distinguishing homogeneous robots with similar appearances remains challenging due to the lack of distinctive identification features. In this paper, we propose a fully distributed relative pose estimation approach by integrating LiDAR, UWB, and odometry measurements, allowing each robot to accurately and continuously localize its teammates without external infrastructure. Specifically, potential anonymous teammate robot clusters from LiDAR scans are tracked by a dynamic tracker. We then identify teammate robots from these tracked anonymous clusters using a joint matching strategy, ensuring reliable data association between robots and clusters. Finally, by combining the corresponding LiDAR observations, UWB ranging, and odometry measurements, each robot precisely localizes others while minimizing data exchange. The system requires only odometry data exchange through onboard UWB, eliminating the need for additional communication infrastructure like WiFi routers or mesh networks. Extensive simulation and real-world experiments demonstrate the effectiveness and reliability of the proposed relative localization approach.

#12

h=35

Distributed Relative Localization Based on Ultra-WideBand and LiDAR for Multi-robot with Limited Communication

📅 2026-10-08cs.RO👤 Chau Yuen (h=35)

Zhiqiang Cao, Ran Liu, Billy Pik Lik Lau, Chau Yuen, U-Xuan Tan

Core Contributions

Formulates teammate identification as common-subgraph matching between a UWB connection graph (who is how far from whom) and a LiDAR graph of anonymous objects in line of sight.
Explicitly detects false matches caused by obstacles blocking LiDAR, then re-localizes those robots using correctly matched neighbors plus UWB ranges.
Fully distributed with limited information exchange, aimed at homogeneous teams in GPS-denied, communication-poor environments.
Read together with #11: this graph-based position estimator is the precursor to the tracking-based full-pose system — useful for seeing how the design evolved.

Show abstract

Relative localization is crucial for a multi-robot system to collaboratively perform tasks, such as exploration and formation. However, this is highly challenging for homogeneous robots with similar appearance in GPS-denied and communication-limited environments. In this paper, we propose a fully distributed relative position estimation approach for a team of robots based on onboard UWB and LiDAR sensors, in which LiDAR is utilized to obtain the position of anonymous objects in Line-of-Sight (LOS), and UWB is used for ranging between robots. We construct two graphs, namely UWB connection graph and LiDAR connection graph, to represent the spatial relationship among objects (robots and obstacles) based on UWB and LiDAR measurements. Identification and relative position estimation are formulated as a common subgraph matching problem. A falsely-matched robot identification approach is designed to recognize the falsely-matched results caused by obstacle blockage in LiDAR field of view. These robots are then localized by leveraging the well-matched robots and the UWB ranging measurements in the UWB connection graph. We conducted experiments to evaluate the performance of our approach. The results show that the proposed approach is capable of achieving satisfactory positioning accuracy for a team of robots in a distributed manner with only exchanging limited information.

RL & Policy Learning Foundations

#18

h=25

FOCUS: From Privileged States to RGB-D with Controlled Modality Switching and Representation Alignment

📅 2026-10-08cs.RO👤 N. Vijaykumar (h=25)

Filip Grigorov, Kourosh Darvish, Nandita Vijaykumar

Core Contributions

Bridges the privileged-state / RGB-D gap within a single-stage PPO run rather than the usual teacher-student two-stage distillation.
The actor's rollout modality is regulated automatically by the KL divergence between its RGB-D and privileged-state action distributions: lean on state while they disagree, shift to RGB-D as they align.
A representation-alignment term pushes the two latents to induce the same actions, so the curriculum self-paces without hand-tuned schedules.
Average test success 0.71 → 0.93 across five tasks; on Pick-and-Place 0.47 → 0.86, with a 5× gain in budget-normalized learning efficiency.

Show abstract

Vision-based reinforcement learning for robotic manipulation is sample-inefficient because RGB-D observations are high-dimensional and noisy. Privileged state information available in simulation can accelerate training, but its absence at test time creates a train-test modality gap. We propose FOCUS, a single-stage PPO framework that trains the critic on privileged state while automatically regulating whether the actor collects rollouts from RGB-D or privileged-state latents. Regulation is driven by the KL divergence between the action distributions induced by the two modalities, while representation alignment encourages consistent action selection across them. Together, these mechanisms limit RGB-D rollouts when the actor's action distributions from RGB-D and privileged-state latents disagree. As they align, RGB-D exposure increases, shifting on-policy training toward the RGB-D inputs used at test time. Across five manipulation tasks, FOCUS raises average test success from 0.71 to 0.93 relative to the strongest RGB-D-at-test baseline on each task. When accounting for each method's complete training pipeline, budget-normalized training-success AUC increases from 0.47 to 0.65. On Pick-and-Place, test success rises from 0.47 to 0.86, while AUC increases from 0.12 to 0.61, a 5.0x improvement in learning efficiency over the fixed interaction budget.

#19

h=24

Higher-Order Action Supervision Makes A Strong Policy Class

📅 2026-10-08cs.ROcs.AI👤 Xianyuan Zhan (h=24)

Peng Cheng, Yunxian Hou, Zhi Zhou, Qian Zhang, Chang Huang

Core Contributions

Attributes the jitter and fragility of IL/RL policies to supervising only zeroth-order actions, ignoring how actions change over time.
Introduces a loss that jointly supervises zeroth- and first-order actions with formal guarantees, applicable to deterministic, stochastic and flow policies without architectural changes.
Acts as a plug-and-play module for existing offline RL frameworks — a cheap smoothness/robustness prior rather than a new algorithm.
Reports consistent gains on OGBench and D4RL, including better OOD generalization in low-data regimes where temporal consistency matters most.

Show abstract

Modern data-driven decision-making methods, such as imitation learning (IL) and reinforcement learning (RL), have achieved great success in solving many complex tasks. However, these methods often suffer from serious control instability and robustness issues when applied in real-world applications such as robotics and autonomous driving, posing notable challenges for their practical deployment. We argue that this instability issue stems largely from their limitations in solely supervising and optimizing zeroth-order actions (i.e., the action labels), failing to account for higher-order action dynamics and temporal consistency. In this paper, we show that simultaneously supervising both zeroth- and first-order actions can dramatically enhance policies' performance and control robustness. To achieve this, we introduce a novel and elegant loss scheme supported by formal theoretical guarantees that can equip any off-the-shelf policy model (e.g., deterministic, stochastic, or flow policies) with the capability for higher-order action supervision, without requiring any structural modifications. Moreover, our proposed method can serve as a lightweight plug-and-play module that seamlessly integrates with a broad spectrum of existing offline RL frameworks. Extensive evaluations on OGBench and D4RL demonstrate that our approach yields substantial performance and robustness improvements across a wide range of continuous control environments. Notably, our method can also enhance policies' out-of-distribution (OOD) generalization capability in the challenging low-data regime, making it an ideal tool in tackling many real-world control problems.

h=58

Real-Time Motion Planning with Dynamic Hazards: Classical vs. Learning-Based Methods

📅 2026-10-08cs.ROcs.AI👤 A. Bruckstein (h=58)

Eran Iceland, Alexander Tuisov, Oren Gal, Ariel Barel, Alfred M. Bruckstein

Core Contributions

A deliberately non-method paper: same environments, constraints, observability and metrics for classical planners and PPO, with sweeping 'sprinkler' hazards creating time-varying forbidden zones.
Finds a clean regime shift — classical planners are near-perfect with better paths in deterministic worlds, but under stochastic hazard dynamics online search becomes budget-bound (fail at low budget, slow and long at high budget).
PPO wins on latency, success and path quality precisely when obstacle evolution is random — and the authors isolate dynamics uncertainty, not partial observability, as the deciding factor.
Provides a practical decision rule for practitioners choosing between search and learned policies, rather than another leaderboard claim.

Show abstract

We study real-time motion planning in dynamic hazard fields through a controlled comparison between classical planning and learning-based methods. Rather than introducing a new planner, we construct a unified benchmark in which representative classical and learning-based methods face the same environments, motion constraints, information assumptions, and evaluation metrics. The test environment consists of planar domains populated with rotating sprinkler-like hazards that generate time-varying forbidden regions via sweeping angular sectors. Our results show a clear regime shift. In deterministic environments, classical planners achieve near-perfect success and higher-quality paths, though sometimes at the cost of substantial planning or replanning time. Under stochastic obstacle dynamics, however, online search becomes strongly budget-sensitive: low budgets lead to frequent failure, while high budgets improve success at the cost of latency and longer trajectories. PPO-based policies, trained under the same scenario distribution, consistently outperform in latency, success rate, and path quality in these stochastic regimes. Overall, the results indicate that uncertainty in obstacle evolution, more than partial observability, is the dominant factor determining which planning paradigm is practically effective for the problem at hand.

Control & Field Systems

h=49

MAP2: Model- and Acceleration-Based Pursuit with MPC and Gaussian Process Residual Learning for Autonomous Racing

📅 2026-10-08cs.RO👤 Michele Magno (h=49)

Weizhan Huang, Cheng Hu, Edoardo Ghignone, Nicolas Baumann, Yiqin Wang

Core Contributions

Extends geometric pursuit beyond its breaking point: at the handling limit, Ackermann geometry fails, so MAP2 replaces it with a curvature-based kinematic MPC over a horizon.
MPC outputs are mapped to steering through a tire model corrected by a Sparse Gaussian Process residual, capturing dynamics the analytic tire model misses while keeping computation light.
Keeps the simplicity and low latency of pursuit controllers while adding look-ahead — a middle ground between Pure Pursuit and full nonlinear dynamic MPC.
On a real vehicle: 37.99% lower lateral error than MAP and 44.65% lower than Pure Pursuit, with lap times improved by at least 1.5%.

Show abstract

Autonomous racing requires accurate trajectory tracking near the handling limits of a vehicle while maintaining low computational latency. Geometric controllers are computationally efficient, but the Ackermann steering geometry becomes invalid under limit-handling conditions. Model- and Acceleration-based Pursuit (MAP) preserves the simplicity of geometric approaches while leveraging tire dynamics. Yet MAP remains fundamentally a geometric controller that relies on Ackermann steering geometry, and its tire model may not fully capture the vehicle's actual dynamic response. This paper presents MAP2, a model-based pursuit controller that combines a curvature-based kinematic MPC and a Sparse Gaussian Process (SGP) residual correction. The proposed algorithm uses MPC to optimize kinematic control inputs over a prediction horizon and maps them to steering commands through a tire dynamics model augmented with SGP residual correction. Real-world vehicle experiments demonstrate substantial reductions in lateral tracking error and lap time. Compared with MAP and Pure Pursuit (PP), MAP2 reduces the average lateral tracking error by 37.99% and 44.65%, respectively, while reducing average lap time by at least 1.5%.

#25

h=21

WAND: Learning Robust Navigation under Complex Wind Disturbances and Dense Obstacles for Quadrotors

📅 2026-10-08cs.RO👤 Zhi-Gang Wang (h=21)

Zhonghan Tang, Chenhui Li, Shuai Liang, Zhongrui You, Jianan Li

Core Contributions

Instead of making the policy implicitly infer wind, WAND explicitly estimates disturbance acceleration from proprioceptive history with a TCN.
The estimate is used twice: fed into the policy via a zero-initialized residual WindAdapter, and as feedforward compensation in the low-level controller — coupling disturbance-aware planning with rejection.
Zero-initialization lets the adapter grow from a pretrained navigation policy without destabilizing it.
+8.3 points average success across 12 wind settings over feedforward-only, direction-dependent avoidance in crosswind tests, and 18/20 successes in fan-driven indoor flights.

Show abstract

Robust navigation in cluttered environments remains a fundamental challenge for quadrotors, particularly when strong wind disturbances arise, which perturb vehicle dynamics, limit control authority, and substantially increase collision risk. Existing learning-based navigation policies typically rely on obstacle perception and proprioceptive observations, requiring the policy to infer time-varying disturbance effects implicitly and thereby limiting robustness under partial observability. This paper proposes WAND (Wind-Aware Navigation with Disturbance Estimation), a reinforcement learning framework for navigation under time-varying wind disturbances in dense obstacle fields. Specifically, WAND estimates wind-induced disturbance acceleration from historical proprioceptive states using a Temporal Convolutional Network (TCN). This estimation is integrated into the policy via a zero-initialized residual module, \emph{WindAdapter}, while simultaneously providing feedforward compensation for low-level control. The dual use of the estimate couples disturbance-conditioned navigation with feedforward disturbance rejection. Across 12 wind-disturbed simulation settings, WAND improved the observed success rate by 8.3 percentage points on average relative to feedforward compensation alone. Controlled opposite-crosswind experiments further showed wind-direction-dependent trajectory adaptation. In indoor fan-induced flight tests, WAND succeeded in 18 of 20 trials, demonstrating the feasibility of real-time onboard navigation.

#23

h=21

Sim-to-Real RL for ASVs using SysID

📅 2026-10-08cs.RO👤 Maani Ghaffari (h=21)

Cody Sheltraw, Tsimafei Lazouski, Maani Ghaffari, Alan Papalia

Core Contributions

Builds a GPU-parallel ASV simulator for RL, filling a gap left by marine simulators that rarely support batched training.
Identifies hydrodynamic and thruster parameters from only a CAD model and a brief set of open-water trajectories — no CFD or towing-tank testing.
The SysID pipeline lets RL start from unknown vehicle dynamics, lowering the barrier for new hulls and small teams.
Zero-shot sim-to-real on a BlueBoat for path following and station keeping.

Show abstract

Autonomous Surface Vehicles (ASVs) operating in dynamic marine environments require robust control policies for tasks such as path following and station keeping, making reinforcement learning (RL) a promising alternative to classical controllers. However, existing ASV simulators rarely support parallel environments for RL training. Such existing simulators require accurate hydrodynamic modeling from computational fluid dynamics solvers or towing tank tests for setting hydrodynamic parameters to address the sim-to-real gap. To address these challenges, we present an ASV simulator and accompanying pipeline that enables training policies starting from unknown vehicle dynamics. Our framework uses only a CAD model and brief set of open-water field trajectories for approximating and refining both hydrodynamic and thruster parameters. Real-world deployments on a BlueBoat ASV demonstrate successful zero-shot sim-to-real transfer in path following and station-keeping tasks without prior hydrodynamic and propeller information.

#17

h=25

Scalable LEO Conjunction Screening using Adaptive Synthetic-Covariance Thresholds

📅 2026-10-08cs.RO👤 M. Kochenderfer (h=25)

Vedant Srinivas, Grace Ra Kim, Duncan Eddy, Mykel J. Kochenderfer

Core Contributions

Replaces the one-size-fits-all 50 km apsis filter in collision screening with object- and time-specific thresholds derived from propagated radial uncertainty.
Well-tracked objects get tight filters, poorly tracked ones get wider ones — less conservatism where it's wasteful, more where it's needed.
A binary-search altitude sweep keeps heterogeneous intervals tractable at catalog scale (log-linear), 27× faster than exhaustive evaluation with identical output.
On 33,791 Space-Track objects over 7 days: candidate pairs cut 28.78% vs fixed threshold and 71.48% vs brute force, while retaining every close approach in two independent reference sets.

Show abstract

As orbital populations grow, conjunction screening must search more object pairs for potential collision risks. Existing pipelines use early-stage filters, such as an apsis filter, to eliminate pairs whose orbital altitude ranges cannot produce a close approach. However, these filters apply the same fixed threshold to every object, making the screening unnecessarily conservative for well-constrained objects while potentially insufficient for objects with greater orbit-prediction uncertainty. In this work, we present an uncertainty-aware conjunction screening method that replaces a single fixed apsis threshold with object-specific thresholds derived from propagated radial uncertainty. The resulting thresholds vary across both objects and prediction time, becoming narrower when propagated uncertainty is small and wider when uncertainty grows. We pair this adaptive screening rule with a binary-search altitude sweep that makes resulting heterogeneous intervals efficient to evaluate at catalog scale with log-linear search complexity. We evaluate the method on 33,791 objects from the Space-Track General Perturbations (GP) catalog using the preceding eight public ephemeris snapshots and a seven-day prediction horizon. The adaptive sieve retains 162.8 million candidate pairs, reducing the candidate set by 28.78% relative to a fixed 50 km apsis threshold and by 71.48% relative to brute-force all-pairs screening. It retains all close approaches identified in two independently propagated reference sets. At the largest tested catalog size, the binary-search sweep is 27 times faster than exhaustive interval evaluation while returning the same candidate set. These results demonstrate that uncertainty-aware screening can substantially reduce the candidate burden passed to downstream conjunction analysis, providing a scalable path toward managing increasingly large and heterogeneous orbital populations.