🤖 Robotics arXiv Digest

Tuesday, October 6, 2026

📄 30 papers 📂 7 research areas Generated by Claude

🔭 Research Landscape

The headline of today's 30 papers is world-action models growing up — and being caught out. OpenWAM (#21) builds the clean experimental testbed the area has lacked, a single causal robot-video backbone with switchable video/action generation orders, and shows causal robot-video pretraining lifts LIBERO-Long video-then-action success from 68.4% to 97.8%. AutodidactWAM (#11) exposes the flip side: after adapting Cosmos 3 to a G1 humanoid, the generated video is plausible while the co-generated action succeeds only 7% of the time — so it mines actions back out of the video via hand-pose estimation and self-distills. Both papers reach the same conclusion from opposite directions: in WAMs the video channel currently carries more reliable knowledge than the action head, and inverse dynamics is the bridge (OpenWAM's counterfactual-trained inverse model reaches 84% vs 21.5% on held-out tasks). Strikingly, the same idea surfaces in theory: the LeCun co-authored JEPA paper (#23) proves that an inverse-dynamics loss is what stops world-model encoders from collapsing the unstable modes a controller needs.

A second theme is self-improvement loops that update the evaluator, not only the policy. VeriFine (#7) co-evolves the policy and its rubric judge, spending human effort only on judge disagreements; RIWANav (#30) co-trains a world model and a navigation policy, with the world model scoring GRPO rollouts and the policy curating the world model's next curriculum; EmbodiedSmith (#14) lets scene and task generators refine each other; and Silicon Language (#24) imagines robot fleets trading skill packets with rollback. The common premise is that a frozen judge or simulator becomes the ceiling as policies improve. Meanwhile VLA papers keep pushing to new embodiments and lower cost: WareFly-VLA (#1) finds that π0, GR00T and OpenVLA transfer poorly to drones and that continuous action heads beat tokenization, EgoLAP (#9) shows language-level motion intent bridges egocentric human data (2.3× gain), and VLA-ACL (#27) prunes 87.5% of visual tokens on a frozen VLA.

The third cluster is a strong safety-under-model-uncertainty thread with real hardware. LBA-CBF (#20) runs a bank of 250,000 candidate dynamics models to keep CBFs valid through wind reversals and payload drops; context-conditioned HJ reachability (#13) carries one certificate across unseen tracks and frictions on a full-scale car at 0.99 g; latent-disturbance robust optimization (#16) brings worst-case reasoning into world-model latent spaces (−70% failures); and belief-informed dual control (#3) proves convergence even when the controller must first move away from the goal to learn. Read alongside the WAM papers, the day suggests the field's next step: pairing learned world models with the calibrated uncertainty and certificate machinery that classical control already provides — #16 and #23 are early examples of exactly that merger. On the learning side, QF3 (#2) makes off-policy RL for flow policies fast enough (10× over FPO++) to train humanoid locomotion from scratch, tightening the link between generative imitation and RL fine-tuning.

VLA & World-Action Models

Vision-language-action and world-action models: new embodiments (UAVs, humanoid hands), human-video pretraining, composable video-action designs, and cheaper inference.

#1 WareFly-VLA: A Vision-Language-Action Framework for...
#9 EgoLAP: Learning from Egocentric Human Data through...
#11 AutodidactWAM: Cross-Modal Self-Distillation from Ge...
#21 OpenWAM: An Open Framework for Composable World-Acti...
#27 VLA-ACL: Action-Consistent Visual Token Pruning for...

Self-Improvement & Data Flywheels

Closed loops where policies, judges, world models, simulators or robot peers improve one another instead of relying on fixed data or evaluators.

#7 VeriFine: Scaling Verification for Self-Improvement...
#14 EmbodiedSmith: Scaling Embodied Data through Recursi...
#24 Silicon Language: A Robot-Native Knowledge Exchange...
#30 RIWANav: Recursive World-Action Models with Self-Imp...

Safety Filters & Certified Control

Provable or calibrated safety under uncertainty: dual control with convergence guarantees, context-conditioned reachability, adaptive CBFs, and robust latent world-model steering.

#3 Belief-Informed Hybrid Control with Almost-Sure Targ...
#13 Context-Conditioned Hamilton-Jacobi Reachability for...
#16 Modeling Latent Disturbances for Robust Decision-Mak...
#20 LBA-CBF: Rapidly Adaptive Safety Filters via Paralle...

Policy Learning, Optimization & Representations

Flow-policy RL, population-based trajectory optimization, coordinated diffusion, geometric imitation, control-aware JEPA representations, and role-specialized multi-robot policies.

#2 QF3: Fast Flow RL with Filtered Q-Gradients
#4 Pareto-Optimal Entropy-Regularized Trajectory Optimi...
#8 One for All, All for One: Coordinated Multi-Agent Di...
#15 Fast Non-Parametric Heteroscedastic Imitation Learni...
#23 Preserving Unstable Modes Through Inverse Dynamics i...
#28 CoRE: Learning Collaboration-Role Experts for Decent...

Legged & Humanoid Control

Whole-body loco-manipulation over long horizons, magnetic-field compensation, multi-quadruped pushing, and force-aware payload carrying.

#5 Humanoid Horizon: Extending Task Horizon in Whole-Bo...
#18 Magnet-Aware Control of Legged Robots
#19 Safe Multi-Robot Collaborative Transport Using Densi...
#25 Feeling Through the Load: Compliant Quadruped Locomo...

Aerial Robotics & Simulation

Agile quadrotor planning with hard obstacle constraints and a differentiable JAX simulator with realistic degraded feedback.

#6 NMPP: Nonlinear Model Predictive Planning for Agile...
#26 UWB Meets Crazyflow: Simulating Degraded Feedback at...

Driving, Teleoperation & HRI

Infrastructure around autonomy: end-to-end driving costs, map-quality metrics, network QoS for teleoperation, attestation deadlines, and a spoken-dialogue HRI benchmark.

#10 Vector Map Quality Metrics for Contextual Autonomous...
#12 Towards an Extensible Benchmark for Spoken Dialogue...
#17 Mitigating Concept Drift in QoS Prediction for Teleo...
#22 Beyond Waypoint Regression: Query-Based Cost Learnin...
#29 Mission-Aware Attestation Envelopes for Time-Critica...

VLA & World-Action Models

#1 h=85

WareFly-VLA: A Vision-Language-Action Framework for UAV Navigation and Human Tracking in Smart Warehouses

2026-10-06 cs.ROcs.CV 👤 H. Nguyen-Xuan (h=85)

Thinh D. Le, Son T. Nguyen, Duong Q. Nguyen, Dung D. Le, Ngo Anh Vien

Core Contributions

Fills a real gap: VLA work has focused on arms and ground robots, and there was no UAV benchmark pairing continuous 4-DoF flight commands with fine-grained, human-written target descriptions in an industrial setting.
507 teleoperated episodes / 8,504 transitions in Isaac Sim covering target approach and person following under occlusion, long-range search and altitude change — scenarios that matter for warehouse safety and inventory drones.
Benchmarks SmolVLA, GR00T N1.7, π0 and OpenVLA under a leakage-free episode-level split; strict generalization makes all four collapse, a useful counterweight to optimistic manipulation-centric VLA numbers.
Two transferable lessons: continuous action heads beat discrete action tokenization for flight, and only the forward channel is reliably learnable from a single frame — evidence that aerial VLAs need temporal context and embodiment-specific interfaces.

Show abstract

Vision-Language-Action (VLA) models have achieved impressive results in robotic manipulation and ground-mobile navigation, yet language-conditioned control of unmanned aerial vehicles (UAVs) in smart warehouses remains largely unexplored, hindered by the lack of benchmarks that jointly provide continuous low-level flight actions, fine-grained natural-language target descriptions, and realistic industrial environments. This paper introduces WareFly-VLA, a photorealistic UAV VLA framework and dataset for language-guided human search, localization, and tracking in warehouse environments. It contains 507 human-teleoperated flight episodes and 8,504 high-resolution RGB transitions collected in NVIDIA Isaac Sim, each paired with a human-written appearance description of the target worker and a synchronized four-degree-of-freedom control command. Two aerial tasks are covered: target approach and person following, under occlusion, long-range search, altitude variation, and clutter. A unified benchmark of four open-source VLA architectures (SmolVLA, GR00T N1.7, pi_0 and OpenVLA) is established under a leakage-free episode-level protocol at two control rates. The results show that language-conditioned aerial control in warehouses is far from solved: performance drops substantially under strict generalization settings, continuous action modeling consistently outperforms discrete action tokenization, only the forward channel is reliably learnable from a single frame, and current foundation-model interfaces transfer poorly from ground and humanoid embodiments to aerial platforms. The synchronized video, language, action, pose, and difficulty annotations further support world-model research. The dataset, baselines, and evaluation protocol are released to support language-grounded aerial autonomy in smart warehouses.

#9 h=32

EgoLAP: Learning from Egocentric Human Data through Language-Action Reasoning

2026-10-06 cs.ROcs.AI 👤 Dhruv Shah (h=32)

Lihan Zha, Shresth Grover, Tenny Yin, Samuel M. Bateman, Hengkai Pan

Core Contributions

Key insight: low-level human and robot actions differ, but their motion intent is shared — so supervise on structured, temporally abstracted language actions rather than raw hand trajectories.
Adds motion-level chain-of-thought grounded in geometry, physics and affordances, used as a common interface for co-training on egocentric human and robot data.
80.1% mean real-world task progress — a 2.3× gain over alternative action representations for human-data transfer.
Motion-level reasoning beats a richer composite format (subtasks + boxes + visual traces), suggesting that the right abstraction matters more than stacking reasoning types.

Show abstract

Egocentric human data offer a path to scaling robot learning beyond costly robot demonstrations, yet the embodiment gap makes raw human trajectories a poor supervisory target for control. Our key insight is that, although low-level actions are embodiment-specific, their underlying motion intent can capture task-relevant structure that transfers across humans and robots. We introduce EgoLAP, a VLA pre-training framework that jointly learns from human and robot trajectories through a shared language-based action chain-of-thought. EgoLAP expresses motion intent as structured, temporally abstracted language actions and pairs them with motion-level reasoning grounded in scene geometry, physics, and object affordances. Across extensive real-world and simulated experiments, EgoLAP transfers human experience to robot control more effectively than alternative action representations and reaches 80.1% mean real-world task progress, a 2.3x performance gain over alternative action representations. Motion-level reasoning also outperforms a composite reasoning format that combines subtask, object-box, and visual-trace reasoning.

#11 h=28

AutodidactWAM: Cross-Modal Self-Distillation from Generated Video to Robot Actions

2026-10-06 cs.RO 👤 D. Tsetserukou (h=28)

Sergei Kurchev, Iaroslav Kolomiets, Miguel Altamirano Cabrera, Artem Lykov, Dzmitry Tsetserukou

Core Contributions

Identifies a 'video-action asymmetry' in world-action models: after LoRA-adapting Cosmos 3 to a G1 humanoid with dexterous hands, generated videos look right while co-generated actions are mis-targeted (≈17/10/7% pre-grasp/grasp/pick-place).
Exploits the stronger modality: a teleop-free hand-pose estimator + IK recovers actions from the model's own generated video, which then serve as preferred targets for fine-tuning only action layers (self-distillation).
Recovered actions alone reach ≈75/47/42%; the best hybrid DPO+SFT+DTW objective hits 90% pre-grasp on the training object and 30% full-task success on a held-out object.
Cautionary finding: plain Flow-DPO scores 0% success despite perfect validation preference accuracy — preference metrics can be misleading for robot action fine-tuning.

Show abstract

World-action models (WAMs) such as Cosmos 3 jointly generate future video and robot actions from an observation and instruction. Adapting one such model with a lightweight LoRA fine-tune to a previously unseen robot, a Unitree G1 humanoid with five-fingered BrainCo hands, exposes a video-action asymmetry: the video renders plausible task executions, while the co-generated action is systematically mis-targeted. We evaluate closed-loop real-robot trials at three cumulative stages: pre-grasp, grasp, and pick-and-place. The native action succeeds only approximately 17%, 10%, and 7% of the time, respectively, and performs worse on held-out objects. We propose AutodidactWAM, a hand-pose estimator trained without teleoperation, followed by inverse kinematics, that runs on the model's generated video to recover action estimates. Paired with the native prediction, these recovered actions provide preferred targets for fine-tuning only the action-related layers, while the generated video is teacher-forced. We compare supervised relabeling with a rectified-flow adaptation of Diffusion-DPO. After one-time embodiment adaptation, self-distillation requires no additional task-specific teleoperation. The recovered-action gate reaches approximately 75%, 47%, and 42% pre-grasp, grasp, and pick-and-place success, compared with 17%, 10%, and 7% for the native action. A hybrid objective combining preference supervision, supervised target fitting, and Cartesian trajectory anchoring (DPO+SFT+DTW) performs best: on Oreo, the training object, it reaches 90% pre-grasp and 20% full-task success; on a held-out object, it reaches 80% and 30%. Plain Flow-DPO reaches 0% success despite 1.000 validation preference accuracy, indicating that the combination of training objectives, rather than the contrastive objective alone, drives the observed gains.

#21 h=17

OpenWAM: An Open Framework for Composable World-Action Models

2026-10-06 cs.ROcs.CV 👤 Ehsan Adeli (h=17)

Heng Yu, David D. Yuan, Juze Zhang, Changan Chen, Yao Feng

Core Contributions

Controls the confounds in WAM research: one causal robot-video backbone (Wan2.2-5B, >10k hours) with a Mixture-of-Transformers action expert that supports joint, video→action, action→video and decoupled generation.
Shows causal robot-video pretraining matters: video-then-action success on LIBERO-Long rises from 68.4% to 97.8%.
Counterfactual transitions help dynamics models: a frozen local-context inverse model reaches 84.0% on held-out LIBERO-90 tasks vs. 47.0% (full-context) and 21.5% (demos only).
For forward dynamics, counterfactual supervision cuts RGB error 34.5% and raises outcome identification from 21.1% to 71.3% — learning from non-successful data has measurable value.

Show abstract

World-action models (WAMs) couple future prediction with robot control, yet existing systems often vary the video backbone, interaction structure, supervision, and inference procedure simultaneously, making their design choices difficult to compare. We introduce OPENWAM, an open world-action modeling framework built around a common causal robot-video foundation and configurable video-action interaction. Starting from Wan2.2-5B, we perform causal robot-video pretraining on over 10,000 hours of video, then integrate an action expert through a shared Mixture-of-Transformers architecture that supports joint, video-then-action, action-then-video, and decoupled generation. OPENWAM achieves high success rates on four LIBERO suites and real-world bimanual tasks; robot-video training with causal adaptation improves VTA success on LIBERO-Long from 68.4% to 97.8%. The same configurable architecture naturally extends to inverse and forward dynamics, allowing us to study how counterfactual transitions improve independently trained dynamics models beyond demonstrations alone. When only the video predictor is adapted to a new task, a frozen local-context inverse dynamics model trained on counterfactual data and demonstrations achieves 84.0% mean success across four held-out LIBERO-90 tasks, compared with 47.0% for a full-context inverse model and 21.5% for a local-context model trained only on demonstrations. For forward dynamics, counterfactual supervision reduces RGB prediction error by 34.5% and raises outcome identification from 21.1% to 71.3% among 16 same-state outcomes. OPENWAM provides a common testbed for comparing WAM interaction designs and for studying dynamics learning from video data beyond successful demonstrations.

#27 h=13

VLA-ACL: Action-Consistent Visual Token Pruning for Efficient Vision-Language-Action Models

2026-10-06 cs.ROcs.CV 👤 Yang Yue (h=13)

Owen Du, Yang Yue, Jie Zhang, Jiaqi Pi, Chi Bene Chen

Core Contributions

Unlike attention-score or motion heuristics, learns a lightweight visual-token pruner supervised by its effect on actions while keeping the base VLA frozen.
Training enforces consistency between actions from pruned vs. full context (teacher), with ground-truth actions as auxiliary supervision.
Prunes up to 87.5% of visual tokens, cuts compute up to 75% and gives 1.5× inference speedup on LIBERO and real tasks with competitive success.
A frozen-model approach means it can be retrofitted to existing VLAs without costly re-training.

Show abstract

Vision-Language-Action (VLA) models achieve strong robotic manipulation performance but incur high computational costs from processing long token sequences at every control step, limiting real-time deployment. Visual token pruning offers a direct solution, as visual patches dominate the input sequence and contain considerable redundancy. Existing approaches, however, either rely on indirect training-free heuristics, such as attention scores and motion thresholds, or require costly fine-tuning of the base VLA model. We introduce VLA-ACL (Action Consistency Learning), which learns a lightweight visual token pruning policy through action-level supervision while keeping the base VLA model entirely frozen. The training objective encourages actions produced from pruned visual contexts to remain consistent with the full-context teacher, with ground-truth actions as auxiliary supervision. This directly ties token selection to its effect on the downstream control output. Experiments on LIBERO and real-world manipulation tasks show that VLA-ACL prunes up to 87.5% of visual tokens while retaining competitive performance, reduces computation by up to 75%, and achieves a 1.5x inference speedup. These results establish a stronger performance-efficiency trade-off than existing frozen-VLA pruning methods and demonstrate the value of action-level supervision for visual token selection. Code is available at https://github.com/du-owen/VLA-ACL.

Self-Improvement & Data Flywheels

#7 h=37

VeriFine: Scaling Verification for Self-Improvement in Embodied Reasoning

2026-10-06 cs.AIcs.RO 👤 B. Ivanovic (h=37)

Zewei Zhou, Rachel Luo, Yulong Cao, Chaowei Xiao, Chensheng Peng

Core Contributions

Argues fixed LLM/VLM judges become the bottleneck of self-improvement: as policies get better, their failures change, and a static judge can neither detect them nor select useful training data.
Two coupled loops: a Policy Improvement Loop that clusters failures into an adaptive curriculum, and a Judge Improvement Loop triggered on plateaus that queries humans only on informative disagreements.
'Coactive calibration' has humans and agents jointly resolve disagreements toward an objective physical-reasoning rubric (spatial grounding, causality, safety), so human effort refines the verifier rather than labeling data.
Shows continued improvement of both policy and judge across RL and SFT on driving and robot navigation — a template for scaling verification alongside policy capacity.

Show abstract

Self-improving policies continually expose new failure patterns, changing what their judges must be able to verify. However, current fixed judges constrain both optimization feedback and the discovery of useful training examples, limiting further self-improvement. This challenge is even more acute in embodied reasoning, where reliable evaluation must account for spatial grounding, causal reasoning, and safety-aware decision-making. We introduce VeriFine, an agent harness framework that scales verification through the co-evolution of the policy, training curriculum, and judge. The Policy Improvement Loop uses a rubric judge to diagnose recurring failures, construct an adaptive curriculum, and optimize the policy. When progress plateaus and verification becomes a bottleneck, the Judge Improvement Loop selectively queries human guidance on informative failure cases and refines the judge through coactive calibration, in which humans and agents resolve disagreements and converge toward the objective rubric of physical reasoning. The revised judge then guides the next stage of data selection and policy optimization. Experiments on driving and robot navigation tasks demonstrate continuous self-improvement in both policy and judge capability across reinforcement and supervised fine-tuning. These results show how scaling verification supports continuous self-improvement as policy failure patterns evolve.

#14 h=25

EmbodiedSmith: Scaling Embodied Data through Recursive Self-Improvement Flywheel in Simulation

2026-10-06 cs.CVcs.RO 👤 Pengxiang Ding (h=25)

Yikai Qin, Yifei Deng, Mingjian Liang, Wenxuan Song, Zepeng Lin

Core Contributions

Unifies asset, scene and task generation in one simulation pipeline, removing the usual dependency on fixed asset libraries and hand-scripted skills.
Core novelty is mutual refinement: scene generation anticipates task needs, and task generation requests targeted scene edits — improving long-horizon task generation success.
Supports mobile manipulators, humanoids, dexterous hands, deformables and fluids, broadening the physics represented in synthetic training data.
Downstream policies generalize better as generated data diversity grows, positioning it as both a pretraining data source and an evaluation engine.

Show abstract

Scaling robotic foundation models requires diverse training data and reliable evaluation environments. Simulation offers a scalable solution, yet existing generation pipelines remain constrained by predefined assets and skills, a disconnect between scene generation and task generation, and limited support for complex embodiments and physics. We introduce EmbodiedSmith, a framework for scalable embodied data generation through recursive self-improvement (RSI). EmbodiedSmith unifies asset, scene, and task generation in a pipeline that supports autonomous creation and language-driven customization. Its core is an agentic refinement loop: scene generation anticipates downstream task requirements, while task generation guides targeted scene edits, allowing scenes and tasks to iteratively improve one another. This joint refinement improves task generation success, including for long-horizon tasks. The framework further supports mobile manipulators, humanoids, and dexterous hands, as well as interactions involving deformable objects and fluids, broadening the range of behaviors and physical phenomena represented in generated data. Together, these capabilities provide a flexible simulation engine for both robot pretraining and evaluation. Extensive experiments validate the quality, diversity, and generation efficiency of the resulting data, while downstream policy experiments demonstrate that increased data diversity improves generalization.

#24 h=14

Silicon Language: A Robot-Native Knowledge Exchange Framework for Heterogeneous Robots

2026-10-06 cs.RO 👤 China (h=14)

Yi Liu, Xianglin Meng, Chang Chen, Jingjing Fan

Core Contributions

Reframes skill transfer across heterogeneous robots as robot-driven 'knowledge exchange': robots package experience, publish it, retrieve peers' packets and translate them to their own sensors/actuators.
Receiver-side usability evaluation plus progressive blending with automatic rollback aims to guard against negative transfer.
Infrastructure stack: edge agent, Silicon Transfer Protocol and a knowledge hub, modeled on the human scholarly system.
30-day mine-lab deployment reports adaptation time dropping from days to hours; the authors explicitly flag this as descriptive, not a controlled comparison.

Show abstract

Reusing a capability across heterogeneous robots still requires substantial human adaptation and verification: transferring a skill often means re-engineering interfaces, retuning parameters, and re-validating safety. We introduce Silicon Language, a robot-native knowledge exchange framework that treats the robot as the active subject of its own capability evolution. In this framework, a robot that wants a capability encodes its own experience into knowledge packets, publishes them, retrieves peer packets, translates them for its own sensors and actuators, and reviews them through independent local trial. A receiver-side usability evaluation procedure lets each robot decide for itself whether an external packet is useful, and progressive blending with automatic rollback is designed to reduce the risk of negative transfer when adopting it. The system combines three infrastructure layers (edge agent, Silicon Transfer Protocol (STP), and knowledge hub) with a capability stack inspired by the human scholarly system. We report a 30-day proof-of-concept deployment at an above-ground simulated-mine laboratory in Yulin, with following trials on an outdoor sand road and an indoor factory floor. Two heterogeneous robots encoded and translated three capabilities across embodiments through operator-assisted file copies mediated by the Silicon Language translation layer; source-side trials of the dust-locked following behavior were recorded on Taurus. Project records indicate that per-capability adaptation time dropped from days to hours; we present these figures as descriptive deployment records rather than controlled measurements. The deployment provides initial evidence for cross-embodiment knowledge exchange; fleet-level autonomous evolution and controlled with/without-packet comparisons remain future work.

#30 h=12

RIWANav: Recursive World-Action Models with Self-Improvement for Urban Navigation

2026-10-06 cs.RO 👤 Shou-Wei Ruan (h=12)

Jing Xie, Shouwei Ruan, Yubin Wang, Yuxiang Zhang, Haitao Yang

Core Contributions

Counters a frozen world model becoming stale as the policy improves: co-adapts world model and navigation policy in a recursive self-improvement loop.
World model imagines outcomes of candidate actions to provide relative rewards for GRPO, avoiding costly real-world trial and error.
The improved policy then curates a self-curriculum of expert-consistent action-video pairs chosen by behavioral novelty and prediction error to refine the world model.
Outperforms training baselines and prior urban navigation methods, with real-world trials demonstrating practicality.

Show abstract

Long-horizon urban navigation requires sequential local decisions whose errors can compound over time. Imitation learning (IL) rarely learns from failures, while physical trial-and-error reinforcement learning (RL) is costly. Action-conditioned world models can provide imagined feedback by predicting visual consequences for candidate actions. However, a frozen world model may become less reliable as the policy evolves. In this paper, we introduce RIWANAV, a post-training framework that casts the coupled adaptation of a world model and an action model (policy) as task-specific recursive self-improvement (RSI). Each cycle alternates two updates. The world model evaluates policy actions through imagined outcomes, providing comparative feedback for group-relative policy optimization (GRPO). The improved policy then constructs a grounded self-curriculum, selecting expert-consistent action-video pairs by behavioral novelty and prediction error. The refined world model supplies feedback for the next policy update, closing the recursive self-improvement loop. Experiments show that RIWANAV outperforms training baselines and prior methods, validating the proposed recursive self-improvement loop between the policy and world model. Real-world trials further demonstrate its practical applicability.

Safety Filters & Certified Control

#3 h=63

Belief-Informed Hybrid Control with Almost-Sure Target-Set Convergence

2026-10-06 eess.SYcs.ROmath.OC 👤 C. Belta (h=63)

Clinton Enwerem, Saleh Kemal, John S. Baras, Calin Belta

Core Contributions

Addresses the dual-control dilemma formally: reaching a target set under parameter uncertainty may require deliberately moving away from it to learn, which naive Lyapunov-style decrease constraints forbid.
Introduces a scalar controller state that budgets cumulative slack on an expected-decrease constraint, plus a two-step lookahead that updates the belief with predicted observations before checking admissibility.
Proves almost-sure target-set convergence and bounds expected neighborhood-entry time; UCB extensions carry guarantees to sampled models with summable error probabilities.
Concrete wins: with unknown control sign the two-step method reaches ‖x‖<0.01 in 11 decisions while one-step loses feasibility; in bimanual assembly it cuts ∞-norm relative-position error by 96.4% and correctly detects infeasibility at low friction.

Show abstract

Controlling a hybrid system to a target set under parameter uncertainty can require informative actions that temporarily drive the system state away from the specified set. We propose a belief-informed dual-control algorithm that combines belief-space receding-horizon selection with an expected-decrease constraint on a nonnegative target-set progress function. To accommodate exploratory deviations, a scalar controller state bounds the constraint's cumulative slack. Our algorithm's two-step lookahead selection uses predicted observations to update the parameter belief before evaluating the subsequent action's admissibility. Under distance-comparison bounds, correct conditional prediction, and recursive feasibility, we prove almost-sure target-set convergence at decision times and bound both the sum of expected progress-function values and the expected neighborhood-entry time. Upper confidence bounds extend these convergence guarantees to bounded model samples with summable error probabilities. In planar regulation with an unknown control direction, we verify recursive feasibility: the proposed two-step selection reduces the Euclidean state norm below 0.01 within 11 decisions for either sign, whereas a myopic one-step selection loses admissibility immediately after zero input. In a simulated bimanual assembly task, our algorithm's one-step implementation uses clearance feedback to complete the assembly under nominal friction, yielding a 96.4% lower infinity-norm relative-position error than the reference-tracking baseline at the method's completion time. At lower friction, its posterior conditioning successfully detects an empty admissible set, whereas a fixed-prior alternative admits an action that violates the conditional decrease constraint. Project page: https://clintonenwerem.com/belief-hybrid-control/.

#13 h=25

Context-Conditioned Hamilton-Jacobi Reachability for Adaptive Safety Filtering

2026-10-06 eess.SYcs.RO 👤 Somil Bansal (h=25)

Ali Fuat Sahin, Yunus Yazoglu, John Talbot, Zeyuan Feng, James Dallas

Core Contributions

Rather than synthesizing a Hamilton-Jacobi certificate per deployment, learns one backward reachable tube conditioned on local boundary geometry, friction (0.4–2.0) and disturbance scale.
On 11 unseen track geometries it maintains containment across all frictions while staying within 1.2 pp intervention rate and 0.09 m/s speed of certificates re-synthesized with test-geometry knowledge.
Deployed as a sampled-data CBF filter on a full-scale vehicle at the handling limit: containment held in every session, with 99th-percentile accelerations of 0.99 g.
Lap-time variation of only 3.1% across certificates — reachability-based safety can generalize with little performance cost.

Show abstract

Hamilton-Jacobi reachability constructs safety certificates for specified dynamics and safety constraints, tying each certificate to the deployment context for which it is synthesized. We ask whether a single certificate can instead represent a family of context-dependent safety problems and be queried across deployment conditions without re-synthesis. We learn a backward reachable tube for an eight-state vehicle model conditioned on local boundary geometry, friction coefficient, and adversarial disturbance scale. Geometry enters through an ego-frame boundary observation that defines the local containment constraint, while friction and disturbance scale enter as explicit operating-condition variables. This allows the same value function to be queried across friction coefficients from 0.4 to 2.0 and on geometries absent from synthesis. On 11 held-out evaluation geometries, the certificate maintains containment across the full tested friction range, including simultaneous geometry and grip shifts, while remaining within 1.2 percentage points in intervention rate and 0.09 m/s in speed of certificates re-synthesized with knowledge of the test geometry. We then deploy the certificate as a sampled discrete-time control barrier function filter on a full-scale vehicle near the handling limit. Lateral containment holds in every hardware session under both adversarial driving and autonomous racing, with 99th-percentile acceleration magnitude reaching 0.99 g. Across three certificates evaluated under a fixed autonomous racing controller, lap time varies by only 3.1%, demonstrating that a context-conditioned reachability certificate can transfer to deployment geometries absent from synthesis with modest performance cost.

#16 h=23

Modeling Latent Disturbances for Robust Decision-Making in World Models

2026-10-06 cs.ROcs.AIcs.LG 👤 A. Bajcsy (h=23)

Junwon Seo, Andrea Bajcsy

Core Contributions

Defines what a 'disturbance' means inside a learned world-model latent space — a perturbation to latent dynamics producing pessimistic yet plausible transitions.
Plausibility comes from a dynamics-aware similarity metric plus OOD detection, calibrated with conformal prediction so the uncertainty set isn't overly conservative.
Robust actions and worst-case latent disturbances are solved jointly as a game, applied to latent safety filtering and sample-and-verify steering of generative policies.
On a Franka, failures drop 70% in safety filtering and 54% in sampling-based steering — robust control without hand-specified physical models.

Show abstract

In this paper, we study robust decision-making in the latent space of world models (WMs). Robust optimization is a mathematical framework where, given explicitly specified dynamics and physically meaningful disturbances, a robot can select actions that remain effective even under worst-case disturbances. However, applying this principle to the learned latent space of WMs introduces a fundamental challenge: because WMs have fully learned state spaces and dynamics inferred from high-dimensional observations, it is unclear how to define latent-space disturbances that faithfully represent uncertainty in the underlying system. Our key idea is to model a latent-space disturbance as a perturbation to the learned latent dynamics that induces pessimistic but plausible transitions. Specifically, we construct a set of plausible latent dynamics by combining a dynamics-aware similarity metric that captures plausible transitions with out-of-distribution detection that excludes implausible latent states. We calibrate this uncertainty set over latent dynamics using conformal prediction, ensuring that WM imaginations induced by the latent disturbance remain plausible without becoming overly pessimistic. We then jointly optimize robust robot actions and the worst-case latent disturbances through game-theoretic optimization. We leverage this latent-space robust optimization to robustify policy steering, considering two paradigms: latent safety filtering and sample-and-verify steering of a generative control policy. Our controlled simulation experiments show that our latent disturbance enables robust decision-making directly in WM latent spaces, and hardware experiments with a Franka manipulator show that modeling latent disturbances enables robust policy steering, reducing failures by 70% in safety filtering and 54% in sampling-based policy steering. Project website: https://junwon.me/LatentDisturbance/.

#20 h=17

LBA-CBF: Rapidly Adaptive Safety Filters via Parallel Dynamics Inference

2026-10-06 cs.ROeess.SY 👤 F. Maurelli (h=17)

Maitham F. AL-Sunni, Timeea-Andreea Radu, Hassan Almubarak, Henry Z. Liao, Michael Görner

Core Contributions

Fixes a practical CBF weakness: certificates assume a model, so abrupt unmeasured changes (wind reversal, payload drop) break safety exactly when it matters.
Ranks a finite bank of candidate dynamics by recent prediction error and enforces the HOCBF against all models within tolerance of the best — interpolating between adaptive and fully robust filtering, with no switching model needed.
Proves safety whenever a safety-representative model remains in the retained set; banks up to 250,000 models run in-loop thanks to parallel inference.
In quadrotor sim it matches an oracle with 100% safe success while baselines achieve 0–88%; hardware on Crazyflie 2.1 and F1TENTH adapts to wind, payload release and friction changes.

Show abstract

Control barrier functions (CBFs) certify commands through an assumed dynamics model, so an abrupt, unmeasured regime change can undermine the certificate exactly when safety matters most. We present Look-Back Adaptive Control Barrier Functions (LBA-CBF), which rank a finite bank of candidate dynamics by recent prediction error over a short look-back window and enforce the high-order CBF condition against every model within a tolerance of the best, spanning best-fit adaptation to full-bank robust filtering. The dynamics may depend nonlinearly on the unknown parameters, and no switching model or continuously parameterized estimator is required. We prove that any feasible filtered input satisfies the true CBF condition whenever a safety-representative candidate is retained. In quadrotor simulation with abrupt wind reversals and an unknown payload, LBA-CBF is safe and reaches the goal from all random initial conditions, matching an oracle, while adaptive and robust baselines achieve 0-88% success. Banks of up to 250,000 models run inside the control loop, and Crazyflie 2.1 and F1TENTH experiments demonstrate adaptation to wind, payload release, and varying tire-road friction. Code, videos, and project details are available at: https://lla-control.github.io

Policy Learning, Optimization & Representations

#2 h=63

QF3: Fast Flow RL with Filtered Q-Gradients

2026-10-06 cs.ROcs.LG 👤 Angjoo Kanazawa (h=63)

Chung Min Kim, Brent Yi, David McAllister, Hongsuk Choi, Himanshu Gaurav Singh

Core Contributions

Makes off-policy RL practical for flow policies: rather than differentiating through the whole multi-step sampler, QF3 backpropagates the critic's action gradient through a one-step prediction of the flow output, alongside a standard flow-matching loss.
The 'filtered Q-gradient' trick applies critic gradients only on action dimensions that stay near the replay action — limiting updates to where both the critic and the one-step approximation are trustworthy.
Claimed first off-policy flow-RL method to train humanoid locomotion from scratch and transfer zero-shot to hardware, with ~10× wall-clock speedup over on-policy FPO++.
Same algorithm also fine-tunes demonstration-pretrained flow manipulation policies (ABC-Sim, Robomimic), so one method covers both RL-from-scratch and post-training of imitation policies.

Show abstract

Flow policies have become a standard policy class for learning robot behaviors from demonstrations, but reinforcement learning is still critical for improving pre-trained flow policies or learning them from scratch through interaction. We introduce QF3 (Fast Flow RL with Filtered Q-Gradients), an online off-policy RL algorithm that trains a flow policy with flow matching plus the critic's action gradient, backpropagated through a one-step prediction of the flow's output. To keep updates where the critic and this prediction are reliable, QF3 applies the critic gradient only to action dimensions that stay near the replay action. To our knowledge, QF3 is the first off-policy flow RL method to train humanoid locomotion policies from scratch and transfer them zero-shot to hardware. Paired with a high-throughput off-policy training recipe, it trains humanoid locomotion and motion-tracking policies with a 10x wall-clock speedup over FPO++, a recent on-policy flow RL method. We further apply QF3 to fine-tune pretrained flow-based manipulation policies on both ABC-Sim and Robomimic tasks. These results suggest that QF3 can both learn robot policies from scratch and refine those acquired from demonstrations. Website: https://qf3-rl.github.io/

#4 h=49

Pareto-Optimal Entropy-Regularized Trajectory Optimization

2026-10-06 cs.ROeess.SY 👤 Evangelos A. Theodorou (h=49)

Dimitrios S. Georgiou, Augustinos D. Saravanos, Evangelos A. Theodorou

Core Contributions

Targets DDP's main weakness — getting stuck in poor local basins on non-convex, collision-constrained problems — without giving up its second-order efficiency.
Derived from the free-energy/relative-entropy inequality, PER-DDP keeps a population of trajectories and uses prior-guided sampling around each one, rather than sampling only near a few cost-best survivors like earlier sampling-augmented DDP.
Decouples sampling effort from population size via expanded rollout evaluations, and uses Pareto filtering on (task, constraint) so feasible-but-costly alternatives aren't discarded too early.
Evaluated across hundreds of environments, it reports higher success than state-of-the-art sampling-augmented TO and solves instances no baseline can.

Show abstract

Trajectory optimization (TO) under nonlinear dynamics, actuation limits and collision avoidance constraints is a fundamental problem in robotics, albeit especially challenging due to its highly non-convex nature. For this setting, Differential Dynamic Programming (DDP) is an efficient second-order shooting method, yet its local structure renders it vulnerable to suboptimal basins. Sampling-augmented variants mitigate this susceptibility through stochastic exploration, but often sample only around the few trajectories they retain for reoptimization, based solely on their cost which restricts exploration breadth. We introduce Pareto-Optimal Entropy-Regularized DDP (PER-DDP), an entropy-regularized population framework derived from the free-energy/relative-entropy inequality. Our method combines prior-guided sampling that shapes exploration around each retained trajectory, with expanded rollout evaluations that probe these sampling policies beyond the few retained candidates, and Pareto filtering for preserving task-constraint alternatives across iterations. This decouples sampling effort from the optimization population size and broadens exploration without sacrificing the second-order structure that makes DDP effective. Across multiple systems and hundreds of environments, PER-DDP achieves higher success rates than state-of-the-art sampling-augmented TO methods and finds reliable solutions in environments beyond the reach of all baselines.

#8 h=36

One for All, All for One: Coordinated Multi-Agent Diffusion Steering via Stochastic Optimal Control

2026-10-06 cs.ROcs.AI 👤 Stefan Bauer (h=36)

Riccardo Barbano, Vincent Pauline, Runchang Li, George Webber, Alexander Denker

Core Contributions

Takes a modular stance on generative modeling: instead of training one model of a structured output, reuse frozen pretrained diffusion models as components and learn only how to coordinate them.
Coordination is posed as stochastic optimal control over the joint reverse processes — an assembly-level reward traded off against KL-like deviation from each model's pretrained dynamics.
The learned control is amortized, so one trained controller handles new task instances and different spatial constraints without re-optimization.
Demonstrated on multi-agent maze navigation, articulated robot planning and text-conditioned human motion — relevant for composing per-robot or per-limb generative planners.

Show abstract

Deep generative models often produce structured outputs composed of interacting components. Modelling these outputs with a single model requires learning both the component distributions and their interactions. We pursue a modular alternative: reuse independently trained component generators and learn only how to coordinate them to produce coherent structured outputs. Our framework, Coordinated Multi-Agent Diffusion Steering (CMDS), treats frozen pretrained diffusion models as reusable generative primitives and coordinates their reverse processes through a learned control. We formulate coordination as a stochastic optimal control problem, balancing an assembly-level reward that specifies the desired properties of the combined output against deviations from the pretrained dynamics. The learned control amortises this optimisation, allowing reuse across new task instances. Experiments show that CMDS can recover a known target distribution, satisfy different spatial constraints with the same trained control, and recover individual sources from degraded mixtures. Across multi-agent maze navigation, articulated robot planning, and text-conditioned human motion, CMDS turns frozen models into coordinated multi-agent generators.

#15 h=24

Fast Non-Parametric Heteroscedastic Imitation Learning With Geometric Priors

2026-10-06 cs.RO 👤 Alin Albu-Schäffer (h=24)

Maximilian Mühlbauer, Arne Sachtler, Markus Knauer, Cem Küçükgenç, Yanlong Huang

Core Contributions

Brings Riemannian geometry and heteroscedastic (input-dependent) uncertainty together in a kernel imitation method, where prior work had one or the other or required retraining to adapt.
Non-separable diagonal kernels capture uncertainty coupling between DoFs, supporting manifold-valued inputs and outputs with large orientation changes.
Updates take <3 ms for position+orientation trajectories, enabling online adaptation — e.g., task-parameterized adaptation to new object poses.
Validated on real-robot manipulation in both autonomous and shared-control modes, where calibrated uncertainty tells the system how much to defer to the human.

Show abstract

When learning probabilistic policies from human demonstrations, data-efficient learning and fast adaptations to new scenarios are key requirements. One popular way to achieve intuitive and reliable adaptations is through non-parametric, typically kernel-based, methods. However, existing solutions either fail to account for the geometry of manifolds common in robotics, limiting data efficiency, or, when geometry-aware, provide unreliable uncertainty estimates or require retraining to adapt. We propose a non-parametric approach leveraging geometric priors in scenarios of data scarcity and heteroscedastic uncertainties for probabilistic modeling. We utilize the method to formulate policies based on time or robot state, where non-separable diagonal kernels allow capturing uncertainty relations between degrees of freedom for same-sized in- and outputs. Fast updates, requiring less than 3 ms for a trajectory involving both position and orientation are possible through an optimized formulation. Our approach supports both manifold-valued input and manifold-valued output with large orientation changes. Using task parameterization, adaptation to different object poses is easily possible. We evaluate the approach on a set of toy examples and on real robot manipulation tasks both in autonomous execution and in shared control scenarios.

#23 h=16

Preserving Unstable Modes Through Inverse Dynamics in JEPA World Models

2026-10-06 cs.LGcs.ROeess.SYmath.OC 👤 Yann LeCun (h=16)

Leonardo F. Toso, Yann LeCun, James Anderson, Oumayma Bounou

Core Contributions

Shows a hidden failure in JEPA world models: next-step prediction + anti-collapse regularization can reach low loss while collapsing controllable unstable modes, making stabilization impossible.
Fix is simple — add an inverse-dynamics (action reconstruction) loss to make representations control-aware.
Proves exact action reconstruction makes the encoder injective on the H-step reachable subspace, and that the Gramian's dominant eigenspace converges to the controllable unstable subspace as H grows.
Theory is for linear systems, but CartPole, Walker2D and PointMaze experiments show the benefit carries to nonlinear visual control.

Show abstract

Robotic systems often exhibit unstable modes, along which small perturbations and disturbances can cause unbounded growth unless corrected through feedback. Controlling such systems from high-dimensional visual observations requires representations that preserve these modes. Joint-embedding predictive architectures (JEPAs) provide a natural framework for learning such representations and their dynamics from visual data. However, we demonstrate that next step prediction combined with anti-collapse regularization does not guarantee that controllable unstable modes are preserved: the training loss can be minimized while these modes are collapsed, making stabilization from the learned representation impossible. To address this, we augment world-model training with an action reconstruction objective (i.e., an inverse dynamics loss) that encourages control-aware representations, namely, visual representations that preserve crucial features for control. We prove that exact action reconstruction makes the encoder injective on the finite-horizon reachable subspace. Thus, the encoder cannot discard any state direction reachable by an action sequence within $H$ steps. Moreover, we show that, as $H$ grows, the dominant eigenspace of the finite-horizon controllability Gramian converges to the controllable unstable subspace. We establish our theoretical results for linear systems and demonstrate empirically that our findings extend to nonlinear visual control tasks (CartPole, Walker2D, and PointMaze), highlighting the benefits of control-aware representation learning.

#28 h=13

CoRE: Learning Collaboration-Role Experts for Decentralized Collaborative Manipulation with One Policy

2026-10-06 cs.RO 👤 Wei-Ming Zhi (h=13)

Yanan Zhou, Zhaoyan Qian, Zihao Li, Mingyuan Ba, Ranpeng Qiu

Core Contributions

Tackles decentralized collaboration where every robot runs the same policy with no IDs, prompts or communication — roles must be inferred from local observations.
Mixture of query-conditioned cross-attention 'collaboration-role experts' with a local router per action-chunk position.
Expert selection is supervised without role labels via an action-expert alignment loss based on relative forced-route prediction errors.
Best average among decentralized methods in sim and robust in real tests to partner delays and slowdowns.

Show abstract

Collaborative manipulation requires robots to perform complementary actions as interactions unfold. We study single-policy decentralized collaboration: every robot runs the same policy from its visual observations and proprioception, without task prompts, identity labels, or inter-robot messages. The challenge is to learn complementary team behaviors within shared parameters and select appropriate actions from each robot's local observations. We introduce CoRE, which learns Collaboration-Role Experts from pooled multi-task, multi-robot demonstrations. Fused appearance and geometry provide local interaction evidence. Query-conditioned cross-attention experts provide adaptable prediction paths, which a local router combines at each action-chunk position. During training, an action-expert alignment loss supervises expert selection using relative forced-route prediction errors against demonstrations under fixed inputs, without role labels. Across simulation benchmarks, CoRE achieves the highest average performance among evaluated decentralized methods. Physical experiments demonstrate effective collaboration across diverse manipulation tasks and robustness to partner delays and slowdowns. Project page: https://aus.bot/research/core/.

Legged & Humanoid Control

#5 h=48

Humanoid Horizon: Extending Task Horizon in Whole-Body Loco-Manipulation via Parallel Training, Dynamic Starting, and Reward Gating

2026-10-06 cs.ROcs.GR 👤 A. Cangelosi (h=48)

Haozhuo Zhang, Qiang Zhang, Jian Tang, Mingzhe Ni, Michele Caprio

Core Contributions

Names two failure modes in long-horizon humanoid loco-manipulation RL: 'easy-reward bias' (early stages dominate training) and catastrophic forgetting when curricula shift to later stages.
Parallel Training runs S stage streams concurrently under one shared policy, so every stage gets continuous gradients instead of sequential curriculum handoff.
Dynamic Starting seeds each stream from terminal states of upstream rollouts, covering the messy, realistic stage boundaries; Reward Gating zeroes reward if the previously placed object gets knocked, teaching the policy to preserve earlier work.
>80% per-stage success on two-object LHM-Humanoid (66 held-out scenes); with more objects, success degrades gracefully whereas all baselines drop sharply.

Show abstract

Cluttered indoor environments, where large and heavy objects are scattered across diverse surfaces, require humanoid robots to sequentially navigate, grasp, transport, and accurately place each item at its target location within a single uninterrupted episode. This long-horizon, whole-body loco-manipulation task remains a significant challenge for current methods. Previous approaches often suffer from two main issues: easy-reward bias, where training overemphasizes early transport stages at the expense of later ones, and catastrophic forgetting, where focusing on later stages leads to a decline in earlier-stage performance. In this work, we introduce Humanoid Horizon, a unified policy framework designed to overcome these limitations through three interrelated mechanisms. The Parallel Training Strategy organizes $N$ scenes into $S$ concurrent stage streams governed by a shared policy, ensuring all transport stages receive continuous gradient updates and removing the bottleneck of sequential optimization. The Dynamic Starting Mechanism updates each environment's initial state with terminal states from upstream rollouts, gradually broadening transition coverage and enhancing robustness at stage boundaries. Reward Gating sets the reward to zero for the rest of the episode in later-stage streams when the immediately preceding object is displaced beyond a set threshold, so the shared policy learns not to disturb a just-placed object and earlier placements are preserved throughout the episode. Collectively, these strategies achieve per-stage success rates exceeding 80\% on the two-object LHM-Humanoid benchmark (350 training scenes, 66 held-out scenes). As the number of sequentially transported objects grows beyond two, success declines with the horizon, but the degradation is graceful relative to the sharp drop seen in all baselines.

#18 h=19

Magnet-Aware Control of Legged Robots

2026-10-06 cs.RO 👤 Majid Khadiv (h=19)

J. Playan Garai, S. B. Djuve, C. McGreavy, M. Khadiv

Core Contributions

Motivated by robots in Big Science facilities (accelerators, fusion), where strong fields both corrupt sensors and exert pose-dependent wrenches that topple legged robots.
Adds a custom MuJoCo plugin for magnetic forces on rigid bodies and an inverse estimator that infers the latent field from the quadruped's own dynamic response plus any sensors.
Magnet-aware MPC + WBC anticipates disturbances, increasing rejectable magnetic force 2.5× and torque 1.6–2×.
Expands operable area by 29.34%, of which 10.84% would immediately collapse an uncompensated controller.

Show abstract

Autonomous robots can increase uptime and reduce human exposure in Big Science facilities, but strong magnetic fields needed for their operation corrupt sensors and induce pose-dependent mechanical wrenches that destabilize robots and challenge conventional reactive controllers. This paper presents a control framework for modeling, estimating, and dynamically compensating for spatially varying magnetic wrenches acting on legged robots to improve robustness in these fields. We introduce a custom physics plugin for the MuJoCo simulator to model magnetic forces on rigid-body elements, alongside an inverse field-estimation framework to infer the latent magnetic field directly from quadruped dynamic responses and any number of sensor readings. Furthermore, we develop a Magnet-Aware Model Predictive Control (MPC) and Whole-Body Control (WBC) architecture that predicts and counteracts magnetic perturbations during locomotion to increase the range of magnetic fields in which the robot can operate. The effectiveness of the framework is validated through both simulation and physical hardware experiments. We show our method increases the the maximum rejectable disturbances from magnetic field in the force space by a factor of 2.5 and and between 1.6-2 times in torque space compared to a non-compensated system. Within the proposed magnetic field, this constitutes an increase in the area in which the robot can operate by 29,34% of which 10,84% would have previously caused an immediate collapse to non-compensated controllers.

#19 h=18

Safe Multi-Robot Collaborative Transport Using Density Functions

2026-10-06 cs.RO 👤 J. Sahoo (h=18)

Jagannath Prasad Sahoo, Sriram S. K. S. Narayanan, Umesh Vaidya

Core Contributions

Lets multiple quadrupeds push a shared object from only start and goal poses — no precomputed reference trajectory.
Hierarchical design: centralized box-level MPC with a control-density constraint (density functions as an alternative to CBFs) for convergence and obstacle avoidance, then distributed whole-body MPC per robot.
Each robot tracks its moving contact point while avoiding static obstacles and neighbors under a shared-information assumption.
In MuJoCo with 2–3 Unitree Go2s through narrow passages, outperforms matched CBF and RRT* baselines on push-only, force/torque-coupled tasks.

Show abstract

This paper presents a hierarchical density-based model predictive control framework for safe collaborative manipulation by multiple quadrupedal robots. The framework enables a team of robots to push a shared object to a desired pose using only the initial and goal poses, without requiring a precomputed reference trajectory. A centralized box-level MPC optimizes contact forces while enforcing a control-density constraint for goal convergence and obstacle avoidance. Each robot then solves its own distributed robot-level whole-body MPC, under a stated shared-information assumption, to track its moving contact location while accounting for static obstacles and the time-varying positions of neighboring robots. The approach is evaluated in MuJoCo using whole-body contact dynamics for two and three Unitree Go2 quadrupeds collaboratively pushing rigid objects through narrow passages. Comparisons with matched Control Barrier Function and RRT* based tracking baselines demonstrate the effectiveness of the proposed density-based formulation for push-only, force- and torque-coupled manipulation tasks. Implementation videos are available at https://jaggu2606.github.io/go2-density-mpc-pushing/

#25 h=13

Feeling Through the Load: Compliant Quadruped Locomotion under Payload Interactions

2026-10-06 cs.RO 👤 Koichiro Niinuma (h=13)

Shaunak A. Mehta, Mayank Mishra, Prajit KrisshnaKumar, Sebastian Scherer, Koichiro Niinuma

Core Contributions

Studies a new interaction mode: a human pushes an unrestrained payload on a quadruped, so the robot must separate intentional force from ordinary load sloshing.
Decouples compliant force-aware locomotion from force estimation, then couples them via estimator-in-the-loop data aggregation and fine-tuning to reduce mismatch.
Uses only onboard proprioception, avoiding force sensors on the payload.
Real-world tests show stable carrying, compliant yielding and human-guided trajectory changes — payload interaction as an intuitive command channel.

Show abstract

Quadruped robots are increasingly expected to carry objects while moving through human environments. But what happens when a person interacts directly with the payload rather than with the robot? If the payload is unrestrained, the robot must distinguish intentional external interactions from ordinary payload motion, while still keeping the load balanced and maintaining stable locomotion. How can a quadruped infer and compliantly respond to such interactions using only onboard measurements? In this work, we develop a force-aware locomotion framework that treats payload interactions as commands that shape the motion of the combined robot-payload system. Our approach separates the learning of force-aware locomotion and force estimation on an unrestrained payload. We combine a compliant load-carrying policy with a causal force estimator, trained through estimator-in-the-loop data aggregation and finetuning, to predict interactions from onboard robot measurements. Our simulations and real-world experiments show that the resulting controller can maintain stable payload-carrying locomotion, yield compliantly to external interactions, and use the inferred force to support human-guided changes in the robot's trajectory.

Aerial Robotics & Simulation

#6 h=44

NMPP: Nonlinear Model Predictive Planning for Agile UAV Flight in Cluttered Environments

2026-10-06 cs.RO 👤 M. Saska (h=44)

Ondřej Procházka, Juraj Pauko, Robert Pěnička, Martin Saska

Core Contributions

Moves obstacles from soft costs or convex corridors into hard geometric constraints of a full nonlinear MPC planner, then hands a full-state reference to an obstacle-blind SE(3) controller — separating safety from tracking.
Planned references are dynamically feasible by construction, so tracking error drops 58–67% vs. a linear-MPC planner and 41–70% vs. polynomial planners.
Collision-free forest flights up to 9.5 m/s; under an aggressive profile it succeeds 86% of the time versus 26% for a state-of-the-art planner.
Real-world flights reach 5.5 m/s in unknown clutter, showing the approach runs onboard rather than only in simulation.

Show abstract

Flying a quadrotor through a cluttered environment requires not only planning a collision-free reference trajectory based on perceived obstacles, but the reference also needs to be dynamically feasible and within the actuation limits of the vehicle, so that the controller can track it precisely. Existing methods either optimize a smooth polynomial inside a convex corridor, which limits agility, or treat obstacles as soft costs traded against tracking performance. We propose a Nonlinear Model Predictive Planning (NMPP) that imposes perceived obstacles as hard geometric constraints and hands a full-state reference to an obstacle-blind SE(3) controller. Our planner achieves a 58-67 % lower position RMSE than a linear Model Predictive Control trajectory planner and a 41-70 % lower RMSE than a polynomial trajectory planner. It also completes all forest flights with up to 9.5 m/s speed without collisions, and achieves 86 % flight success rate under a more aggressive speed profile where a state-of-the-art planner has only 26 % success rate. The real-world deployment showed reliable execution flying up to 5.5 m/s in an unknown cluttered environment.

#26 h=13

UWB Meets Crazyflow: Simulating Degraded Feedback at Scale for Aerial Robotics

2026-10-06 cs.RO 👤 Angela P. Schoellig (h=13)

Martin Schuck, Marcel P. Rath, Yufei Hua, Abhishek Goudar, SiQi Zhou

Core Contributions

JAX simulator for Crazyflie quadrotors that fuses physics and control into one jit-compiled, differentiable graph — order-of-magnitude speedups and deployable RL policies in seconds.
Adds a full UWB + IMU sensing pipeline with an EKF, so thousands of controllers can be evaluated under realistic degraded state estimates rather than perfect state.
Closes a common sim-to-real gap: policies are usually trained with ground-truth state but deployed on noisy estimators.

Show abstract

In this work, we introduce Crazyflow, an accurate, differentiable simulator built on JAX. By leveraging jit compilation via XLA, Crazyflow unifies physics and control into a single differentiable computation graph, enabling massive parallelization on accelerated hardware without sacrificing modeling accuracy. This architecture achieves order-of-magnitude speedups over existing baselines, capable of training deployable reinforcement learning agents in seconds. To highlight its highly modular design, we demonstrate how easily Crazyflow can be extended by integrating a complete, high-fidelity Ultra-Wideband (UWB) and Inertial Measurement Unit (IMU) simulation pipeline coupled with a full-state Extended Kalman Filter (EKF). This capability allows for massive parallel controller evaluation under realistic, degraded state feedback with minimal impact on GPU throughput. By combining speed, accuracy, and extensibility, Crazyflow serves as a foundational tool for the next generation of aerial robotics research.

Driving, Teleoperation & HRI

#10 h=31

Vector Map Quality Metrics for Contextual Autonomous Driving Systems

2026-10-06 cs.RO 👤 P. Bonnifait (h=31)

Marie-Ngoïe Badibanga Kalenda, Philippe Bonnifait, Marie-Anne Mittet

Core Contributions

Frames map quality as an operational decision — when to trigger crowdsourced updates and when to deploy a recompiled HD map — rather than a one-off accuracy number.
Proposes GOSPAM, an OSPA-family metric that unifies localization error, false positives (existence) and false negatives (completeness) for point and polyline features.
Systematic sensitivity study under bias, spurious/missing features and coordinate noise shows the metric responds interpretably to each degradation type.
Gives automotive map pipelines a single interpretable signal for map-maintenance triggers.

Show abstract

Ensuring safety in autonomous driving requires continuous map maintenance supported by reliable quality indicators. In this context, it is crucial to identify when and where map updates should be triggered, for instance through crowdsourced data, and under which conditions a new map compilation should be deployed. This paper focuses on effective metrics for assessing the quality of vector maps and guiding such decisions. We present a new metric called GOSPAM designed to measure map discrepancies in terms of location errors, existence, and completeness. Through detailed simulations on both point and polyline feature maps, we analyze its sensitivity to common map degradation such as bias, false positives, false negatives, and coordinate errors. The results demonstrate that GOSPAM offers a unified and interpretable measure that effectively captures various forms of map deviation, making it a strong candidate for map quality assessment in automotive applications.

#12 h=27

Towards an Extensible Benchmark for Spoken Dialogue with Social Robots

2026-10-06 cs.RO 👤 Jesse Thomason (h=27)

Casey Kennington, Ross Mead, Saad Elbeleidy, Jesse Thomason

Core Contributions

Points out that LLM-powered robot dialogue is evaluated mostly on turn-based text, ignoring real-time artifacts: clarification requests, interruptions, nods/facial cues and timing constraints.
Proposes an extensible community benchmark centered on these spoken-interaction phenomena, with a roadmap toward broader HRI aspects.
Recommends Retico, an incremental real-time dialogue framework, as the shared infrastructure, lowering the barrier to latency-aware evaluation.

Show abstract

Language models provide a plug-and-play interface between humans and robots, but important challenges remain when speech, dialogue, fast interaction, and collaboration are required. We propose a benchmark for the community to use as a way to explore common spoken dialogue artifacts between robots and humans, including requests for clarification, interruptions, embodied signals (e.g., head nods or facial cues), and time constraints. We also explain our vision to extend the benchmark for other aspects of human-robot interaction that are important to the larger research community. To facilitate the benchmark, we further propose using \textit{Retico}, a real-time communication framework that fulfills important technical requirements to enable robots to have spoken dialogue capabilities.

#17 h=20

Mitigating Concept Drift in QoS Prediction for Teleoperation of Autonomous Vehicles Using Historic Data

2026-10-06 cs.ROcs.AI 👤 Frank Diermeyer (h=20)

Xiyan Su, Jianning Gao, Mahmoud Ashri, Frank Diermeyer

Core Contributions

Treats network QoS as a first-class safety input for remote driving: predicts uplink data rate and round-trip latency from a real measurement campaign.
Tackles concept drift — models degrading on new routes/times — by incorporating historic measurements at the same locations into the prediction pipeline.
Introduces 'critical scenario detection' as an evaluation metric, scoring whether the model flags moments when teleoperation would actually fail, rather than average error.

Show abstract

Teleoperation serves as the fallback solution to autonomous driving but reliable functions of the teleoperation require a certain amount of mobile network resources, which cannot be guaranteed at all times. Therefore, predictive quality of service (pQoS) is introduced as a concept to increase the resilience of the teleoperation. In this paper, based on a data measurement campaign, we propose a prediction framework to prediction two important network KPIs of teleoperation: uplink data-rate and round-trip latency. Furthermore, we introduce a method to alleviate the performance degradation of machine-learning-based prediction models on previously unseen data due to concept drift by incorporating historic data into the prediction pipeline. Additionally, we introduce the metric of critical scenario detection to evaluate the prediction performance specifically for teleoperation.

#22 h=16

Beyond Waypoint Regression: Query-Based Cost Learning over Reachable Ego Futures for End-to-End Driving

2026-10-06 cs.ROcs.AIcs.LGeess.SY 👤 J. Zöllner (h=16)

Ahmed Abouelazm, Rupert Polley, Qingyuan Zhang, Yin Wu, Philip Schörner

Core Contributions

Moves away from waypoint regression (imitating expert geometry) toward learning bounded costs over dynamically reachable ego-trajectory queries, which can accept deployment-time safety constraints.
Compact joint scene tokens capture coherent multi-agent futures; contingency-aware cost aggregation and cost-guided MPPI mixing turn the learned cost landscape into feasible plans.
On nuScenes beats prior cost-learning planners (ST-P3, NMP) and most regression baselines in collision rate while staying competitive in L2.
On real logs, lowers collision rates vs. SparseDrive and Alpamayo without fine-tuning while keeping diverse candidate trajectories and an interpretable cost interface.

Show abstract

End-to-end planners based on waypoint regression achieve strong open-loop accuracy, but they primarily learn to mimic expert geometry and remain difficult to adapt to deployment-time safety constraints. We propose a query-based cost-learning framework that estimates bounded costs for dynamically reachable ego trajectory queries, rather than dense BEV cells or a small regressed trajectory set. Compact joint scene tokens capture coherent multimodal agent futures, while contingency-aware cost aggregation and cost-guided intra-cluster MPPI mixing convert the learned cost topology into feasible ego plans. On nuScenes, our method improves over prior cost-estimation planners such as ST-P3 and NMP, outperforms most regression baselines in collision rate, while remaining competitive in L2, and retaining an interpretable cost interface. On real-world driving logs, the proposed planner reduces collision rates compared with SparseDrive and Alpamayo without fine-tuning, while maintaining a diverse set of candidate trajectories.

#29 h=12

Mission-Aware Attestation Envelopes for Time-Critical Autonomous Action: A Hardware-in-the-Loop V2I Study

2026-10-06 cs.CRcs.ROeess.SY 👤 Stylianos Basagiannis (h=12)

Dimitrios Nikou, Nikolaos Kekatos, Sophia Petridou, Stylianos Basagiannis

Core Contributions

Argues attestation gates for privileged physical actions (e.g., V2I authorization) shouldn't be binary: evidence ages, decisions take time, and the physical deadline depends on current state.
Formulates a runtime assurance contract yielding four outcomes, distinguishing refusal due to tampering, stale evidence or late decision.
HIL study (driving sim + MCU on-board unit + TPM-backed roadside unit): every refusal by the hardware-informed model failed freshness, not response time.
Practical takeaway: the attestation interval is the key tunable, as impactful as a 5× latency change; and freshness bounds can't be set from security requirements alone.

Show abstract

An autonomous system that asks for a privileged physical action is usually gated on integrity evidence: a platform proves what it is running, and the request is granted or refused on that basis. Such a gate is normally treated as a predicate, yet the evidence behind it has an age, the decision that consumes it has a latency, and the physical system that waits for it has a deadline. We formulate mission-aware attestation as a runtime assurance contract that holds only when integrity is valid, the evidence is fresh enough, and the decision completes inside a budget derived from the current physical state. The contract yields four operational outcomes where a binary gate yields two, separating a refusal caused by tampering from one caused by stale evidence and from one caused by a late decision. We evaluate it on a hardware-in-the-loop vehicle-to-infrastructure platform: a driving simulator supplies the physical state and the authorisation deadline, while a microcontroller on-board unit and a TPM-backed roadside unit running Linux integrity measurement supply the assurance evidence. A security-blind model admits the whole operating space and a hardware-informed one three quarters of it, and every point it refuses fails the freshness margin rather than the response margin. Moving the attestation interval across the range the verifier permits costs about as much as a fivefold scaling of the latency distribution, and the interval is directly configurable, which makes it the immediately actionable deployment parameter. If the freshness bound does not exceed the authorisation budget, every late decision is also stale and lateness becomes unobservable, so the attestation interval and the freshness bound cannot be chosen from security requirements alone.