🤖 Robotics arXiv Digest

Tuesday, June 23, 2026

📄 30 papers 📂 7 research areas Generated by Claude

🔭 Research Landscape

Today's 30 papers are dominated by the push to give manipulation policies structure and geometry rather than raw scale, and to free them from the demonstration-data ceiling. InSight (#1) makes a VLA steerable at the primitive level so it can autonomously acquire and compose new skills with zero target-skill demonstrations; G³VLA (#13) injects calibrated multi-camera geometry into the visual-token stream of a pretrained VLA; GRA (#17) formalizes an "Asymmetric Preservation Principle" — generated robot video keeps geometry but loses control, so geometry should supervise perception while control comes only from real demos; and RoBoSR (#23) recasts manipulation as state transitions over object-centric scene graphs for long-horizon reasoning. The shared thesis is that the binding constraint on generalist policies is now the right inductive structure and where supervision is routed, not more teleoperation.

A second current is learning robustly from imperfect, mixed-quality data and grounding generative policies in physical feasibility. World Value Models (#4) argues value estimation needs the temporal modeling of world models rather than static-image VLM backbones, and ships Suboptimal-Value-Bench to evaluate beyond expert data; ReTVL (#6) turns the retries and mistakes in human demos into supervision for a mistake-sensitive value function; RE4 (#19) rebuilds imitation from interpretable, mode-aware building blocks instead of opaque end-to-end policies; and Optimization-Guided Diffusion (#25) closes the gap where diffusion outputs are distributionally valid yet infeasible, enforcing reachability and trackability at sampling time for up to 20-23pp task-success gains. World models recur as the connective tissue across the batch — powering value estimation (#4), closed-loop navigation planning in NavWM (#29), and dynamics-aware terrain encoding for stair locomotion in DynaWM (#30).

The third theme is robustness in the physical world paired with a candid streak of evaluation honesty. On the sensing-and-control side, TACTFUL (#5) and NoContactNoWorries (#16) treat touch — or a vision-plus-proprioception substitute for it — as a primary modality, the PDS joint (#21) embeds tunable compliance into dexterous hands, FT-WBC (#14) keeps legged manipulators upright through actuator failures, SlipSense (#22) catches slips 3.3x earlier than kinematic baselines, and perception work hardens localization in adverse domains (semantic intralogistics mapping #2, communication-robust multi-robot SLAM #12, maritime place recognition #24). Running alongside is a sober reminder not to trust the easy benchmark: a 340,000-fit BCI study (#20) shows no single decoder dominates and per-subject selection beats any fixed pipeline by ~7 points, an HRI validation (#18) documents legibility collapsing from online to real-world studies, and CPS failure explanation via actual causality (#11) builds tools to explain why black-box autonomous systems fail. Together with classical-systems contributions — parallel conic LQ control (#7), multi-floor planning (#8), reactive task allocation (#15), V2X intent-sharing (#26), evolutionary specialization budgets (#27), and USV seakeeping simulation (#28) — the batch reads as a field simultaneously broadening its reach and consolidating around structure, feasibility, and trustworthy evaluation.

VLA & Embodied Foundation Models

Primitive-steerable skill acquisition, calibrated camera geometry, geometry-vs-control supervision from generated video, and scene-graph structured embodied reasoning.

#1 InSight: Self-Guided Skill Acquisition via Steerable...
#13 G$^3$VLA: Geometric inductive bias for Vision-Langua...
#17 Supervise What Survives: Geometry-Guided VLA Adaptat...
#23 RoBoSR: Structured Scene Representations for Embodie...

Robot Learning: Value, Imitation & Generative Policies

World-model value estimation over mixed-quality data, retry-aware value learning, interpretable mode-based imitation, physics-constrained diffusion, and cloth manipulation via simulator-in-the-loop.

#4 World Value Models for Robotic Manipulation
#6 Beyond Monotonic Progress: Retry-Supervised Value Le...
#19 RE4: Transformation-aware Imitation of Object Intera...
#25 Grounding Generative Policies in Physics: Optimizati...
#10 Enabling Robust Cloth Manipulation via Inference-Tim...

Tactile, Contact & Dexterous Hardware

Vision-free tactile exploration, vision-plus-proprioception contact inference as a tactile substitute, and a parametric compliant joint for dexterous hands.

#5 TACTFUL: Tactile-Driven Exploration For Object Local...
#16 NoContactNoWorries: Estimating Contact through Visio...
#21 PDS Joint: A Parametric Double-Spiral Joint Tailored...

Legged Locomotion & Whole-Body Control

Fault-tolerant whole-body loco-manipulation under actuator failures and force-based early slip detection for quadrupeds.

#14 FT-WBC: Learning Fault-Tolerant Whole-Body Control f...
#22 SlipSense: Multimodal Sensing for Online Slip Detect...

Perception, SLAM & Visual Localization

Open-vocabulary semantic mapping, object-level relocalization, articulated Gaussian-splat digital twins, decentralized multi-robot SLAM, and cross-scene maritime place recognition.

#2 Vision-Language Model Reasoning for Contextual Seman...
#3 Compact Object-Level Representations with Open-Vocab...
#9 ArtiTwinSplat: Interactable Digital Twin Reconstruct...
#12 Decentralized Pose Graph Riemannian Optimization for...
#24 From Open Waters to Enclosed Cabins: ProteusVPR for ...

Navigation, Planning & World Models

Multi-floor optimization-based trajectory planning, a unified navigation world model, and a dynamics-aware world model for stair locomotion.

#8 Optimization-based Safe Trajectory Planning for Auto...
#29 NavWM: A Unified Navigation World Model for Foresigh...
#30 DynaWM: Dynamics-Aware Distillation with World Model...

Control, Coordination, HRI & Systems

Parallel conic LQ control, causal failure explanation for cyber-physical systems, reactive multi-task assignment, robot intent communication, BCI decoder benchmarking, V2X intent-sharing, evolutionary specialization, and USV motion simulation.

#7 Parallel Dynamic Programming for Conic Linear Quadra...
#11 Explaining Failures of Cyber-Physical Systems with A...
#15 Varying Bundle Size Reactive Multi-Task Assignment u...
#18 Legible and Intuitive Multi-modal Robot State and In...
#20 Average Rankings Mask Per-Subject Optimality: A Frie...
#26 Importance of Intent-Sharing for V2X-based Maneuver ...
#27 The Evaluation Cost of Task Specialization in Evolut...
#28 Efficient Time-Domain Simulation of USV Motions in S...

VLA & Embodied Foundation Models

#1 h=n/a

InSight: Self-Guided Skill Acquisition via Steerable VLAs

2026-06-23 cs.RO cs.AI cs.LG 👤 Maggie Wang

Maggie Wang, Lars Osterberg, Stephen Tian, Ola Shorinwa, Jiajun Wu

Core Contributions

Attacks the core VLA ceiling — a policy can only do skills present in its demo data — by making the policy steerable at the primitive-action level ("move gripper to the bowl", "lift upward", "pour the bottle"), so new behavior can be requested rather than re-collected.
Two coupled stages: a VLM auto-segments demonstrations into labeled primitives via plan decomposition plus end-effector poses, then a VLM-guided data flywheel identifies primitives missing for a novel task, autonomously attempts them with VLM-proposed low-level control, and folds successful trials back into training.
Learns block flipping, drawer closing, sweeping, twisting, and pouring with zero human demonstrations of those target skills, then composes the acquired primitives into novel long-horizon tasks without further human data.
Why it matters: reframes continual skill growth as primitive steerability rather than ever-larger teleoperation campaigns, giving VLAs a practical self-improvement loop.

Show abstract

Vision-language-action (VLA) models can learn manipulation skills from demonstrations, but their capabilities are bounded by the skills in the training data. We present InSight, a framework that unlocks autonomous skill acquisition by rendering VLAs steerable at the primitive-action level (e.g., "move gripper to the bowl", "lift upward", "pour the bottle"). InSight consists of two primary stages: (1) an automated segmentation pipeline that partitions demonstrations into labeled primitives via VLM plan decomposition and end-effector poses to enable VLA primitive steerability, and (2) a VLM-guided data flywheel that identifies missing primitives required to accomplish a novel task, autonomously attempts demonstrations of the missing primitives with VLM-proposed low-level control, and automatically labels, stores, and integrates successful demonstrations into the VLA training set. We evaluate InSight across simulation and real-world manipulation tasks, including block flipping, drawer closing, sweeping, twisting, and pouring, without any human demonstrations of these target skills. Once learned, these primitives can be composed to execute novel, long-horizon tasks without additional human demonstrations. Our findings demonstrate that primitive steerability provides a practical foundation for continual skill acquisition in VLA policies. Project website: https://insight-vla.github.io.

#13 h=n/a

G$^3$VLA: Geometric inductive bias for Vision-Language-Action Models

2026-06-23 cs.RO cs.AI 👤 Yue Peng

Yue Peng, Yongzhe Zhao, Artur Habuda, Khuyen Pham, Yanheng Zhu

Core Contributions

Diagnoses that VLA visual tokens stay anchored in 2D image coordinates instead of the cameras' calibrated geometry — a mismatch worst in multi-camera rigs, where views share known intrinsics and extrinsics yet are processed as independent images.
Injects calibrated structure into the visual-token stream without altering the action space or imitation objective, via intrinsic-conditioned ray embeddings, projective positional encoding (PRoPE), and bidirectional cross-view fusion.
Geometric supervision comes from ground-truth point maps when available or confidence-gated teacher predictions, needing no depth sensors or manual annotation.
Built on a strong VLA, delivers consistent gains across LIBERO, RoboCasa24, RoboTwin2.0, and real robots (largest on spatial and object-sensitive tasks), and validates that geometry transfers best when geometry-aware tokens have direct access to the action-generation pathway.

Show abstract

Vision-language-action (VLA) models have made rapid progress in generalist robot manipulation by harnessing semantic knowledge from pretrained vision-language backbones, but their visual tokens remain grounded in 2D image coordinates rather than the calibrated geometry of the robot's cameras -- a mismatch especially pronounced in multi-camera setups, where views are coupled by known intrinsics and extrinsics yet processed as independent images. We propose G$^3$VLA, a camera-aware geometric module that injects calibrated structure into the visual-token stream of a pretrained VLA without altering its action space or imitation objective, combining intrinsic-conditioned ray embeddings, projective positional encoding (PRoPE), and bidirectional cross-view fusion. Geometric supervision is provided either from ground-truth point maps when available, or from confidence-gated $π^3$X teacher predictions, requiring no depth sensors or manual annotations. Instantiated on $π_0$, G$^3$VLA yields consistent gains across the LIBERO suites, RoboCasa24, RoboTwin2.0, and real-robot settings, with the largest improvements on spatially and object-sensitive tasks. We further validate on $π_{0.5}$ and GR00T 1.5, with results suggesting that geometric transfer is most effective when geometry-aware tokens have direct access to the action generation pathway. Our project page is at https://sites.google.com/view/g3vla

#17 h=n/a

Supervise What Survives: Geometry-Guided VLA Adaptation from Synthetic Robot Videos

2026-06-23 cs.RO 👤 Danze Chen

Danze Chen, Yanzhe Chen, Qiming Huang, Zhijun Cao, Chen Gao

Core Contributions

Challenges the practice of treating generated robot videos like real data by recovering pseudo-actions from synthesized pixels, calling it a mismatched abstraction.
Articulates an Asymmetric Preservation Principle: video generation preserves geometry (the spatial "where") but loses control signals (the motor "how"), so geometry should supervise only visual perception while control comes from real demonstrations.
GRA extracts future 2D end-effector waypoints from source human video (via pose estimation, retargeting, simulation, and calibrated projection) and routes them to the VLA vision backbone through an auxiliary 2D head, while the action head trains on real demos only and the waypoint loss acts as a spatial-representation anchor.
Why it matters: outperforms pseudo-action baselines under matched data budgets and narrows the gap to policies trained with substantially more real demonstrations.

Show abstract

Vision-Language-Action (VLA) models require large-scale video-action pairs, yet real teleoperation remains scarce. While generated robot videos offer a scalable alternative, existing methods treat them as real robot data by recovering pseudo-actions from synthesized pixels. We argue that deriving low-level control from generated visuals is a mismatched abstraction. A video captures only \emph{geometry}: the spatial trajectory representing the \emph{where} of a task. A real demonstration captures \emph{control}: the exact motor commands representing the \emph{how}. Human-to-robot video generation preserves these unequally: the visible geometry survives the generation process, while the underlying control signals are lost. This \textbf{Asymmetric Preservation Principle} dictates a clean rule: this surviving geometry should solely supervise visual perception, leaving control to real demonstrations. Following this principle, we propose \textbf{GRA} (\textbf{G}eometry-guided \textbf{R}epresentation \textbf{A}lignment), which extracts the geometric content as future 2D end-effector waypoints, computed from the source human video through pose estimation, retargeting, simulation, and calibrated projection, and routes them to the VLA vision backbone via an auxiliary 2D head. The action head is trained on real demonstrations only. During fine-tuning, the waypoint loss persists as a \textbf{spatial representation anchor} that prevents the backbone from losing its geometric grounding. On real-robot tasks, GRA outperforms pseudo-action baselines under matched data budgets and narrows the gap to policies trained with substantially more real demonstrations, suggesting that correctly routed geometry bridges generated videos to robot policies more reliably than recovered actions.

#23 h=n/a

RoBoSR: Structured Scene Representations for Embodied Robotic Reasoning

2026-06-23 cs.RO 👤 Kewei Hu

Kewei Hu, Wanchan Yu, Fangwen Chen, Jing Jiajian, Zimeng Li

Core Contributions

Introduces an intermediate structural representation that formulates manipulation as step-wise state transitions over semantically grounded, object-centric scene graphs, disentangling high-level reasoning from raw inputs.
By modeling object states and spatial relations at the perception-action interface, it enables explicit reasoning over preconditions, effects, and goals and enforces subtask dependencies for coherent long-horizon planning.
Builds Manip-Cognition-1.6M, an open-world dataset that jointly supervises scene understanding, instruction interpretation, and subtask planning across diverse tasks.
Why it matters: outperforms prompting-based methods and classical TAMP baselines in zero-shot generalization and long-horizon tasks, positioning structured intermediate representations as a key inductive bias for embodied reasoning.

Show abstract

Despite rapid progress, embodied reasoning under real-world variability remains challenging. Existing approaches rely on demonstration-driven sequential biases, limiting flexibility in open-ended and long-horizon tasks that require structured reasoning over evolving states. We introduce RoBoSR, an intermediate structural representation that formulates manipulation as step-wise state transitions over semantically grounded, object-centric scene graphs. By modeling object states and their spatial relations at the perception-action interface, RoBoSR disentangles high-level task reasoning from raw inputs and enables structured reasoning over preconditions, effects, and goal states. This representation endows the agent with causal reasoning capability, enforcing subtask dependencies and supporting coherent long-horizon task planning. To learn such structure-aware reasoning, we construct Manip-Cognition-1.6M, an open-world dataset that jointly supervises scene understanding, instruction interpretation, and subtask planning across diverse tasks. Across several benchmarks and real-world demonstrations, our method consistently outperforms prompting-based methods and classical TAMP baselines in zero-shot generalization and long-horizon tasks. The results underscore structured intermediate representations as a critical inductive bias for scalable embodied reasoning.

Robot Learning: Value, Imitation & Generative Policies

#4 h=n/a

World Value Models for Robotic Manipulation

2026-06-23 cs.RO 👤 Zhihao Wang

Zhihao Wang, Jianxiong Li, Yu Cui, Yuan Gao, Xianyuan Zhan

Core Contributions

Argues most robotic value models inherit VLM backbones trained on static or temporally sparse images and therefore lack the temporal modeling accurate value estimation needs; world models, built for temporal prediction, are the better foundation.
Marries world-model dynamics with value estimation to produce accurate task-progression signals that assess data quality across mixed-quality trajectories.
Introduces Suboptimal-Value-Bench — 800 suboptimal multi-embodiment trajectories with high-fidelity human-labeled frame annotations — filling the gap left by expert-only evaluation suites.
Achieves SOTA Value-Order Correlation on both standard and suboptimal benchmarks and improves downstream manipulation across multiple policy-extraction approaches in simulation and the real world.

Show abstract

Generalist value models play a pivotal role in scaling robotic policy learning from large-scale, mixed-quality data. Mathematically, accurate value estimation demands deep temporal understanding, requiring models to both ground the current belief using historical context and plan over future outcomes. However, most existing robotic value models are built on Vision-Language Model (VLM) backbones that are pretrained primarily on static or temporally sparse visual observations, lacking the requisite temporal modeling capabilities for value estimation. Unlike VLMs, world models naturally excel at temporal modeling and future planning, making them ideal foundations for learning generalizable value functions. Driven by this insight, we marry world models with value estimation to construct a new generalist robotic value model, World Value Model (WVM), that offers accurate task progressions to assess data quality. On standard benchmarks, WVM delivers state-of-the-art (SOTA) Value-Order Correlation (VOC) results. Complementing standard evaluation suites that contains only expert data, we further introduce Suboptimal-Value-Bench, a multi-embodiment benchmark consisting of 800 suboptimal trajectories with high-fidelity, human-labeled frame annotations. Our evaluations show that WVM maintains its SOTA performance on Suboptimal-Value-Bench, establishing its robustness in handling both expert and suboptimal data. When deployed for policy learning, WVM improves manipulation performance across various policy extraction approaches in both simulated and real-world deployment, providing robust guidance for learning from mixed-quality data.

#6 h=n/a

Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation

2026-06-23 cs.RO 👤 Xinyao Qin

Xinyao Qin, Junjie Lu, Kaixin Wang, Chuheng Zhang, Sinjae Kang

Core Contributions

Treats the mistakes and retries in human demos — imprecise grasps, misalignment, unstable contact, repeated attempts — as informative evidence about feasibility recovery rather than noise to discard.
Learns a mistake-sensitive value function by combining global progress calibration with local pairwise preference learning induced by sparsely annotated retry keypoints, capturing the degradation-and-recovery structure around errors.
Uses the learned value to reweight demonstration chunks for behavior cloning, down-weighting harmful execution errors while preserving useful corrective behavior.
Why it matters: yields finer-grained value estimates than monotonic-progress baselines and measurably improves imitation from imperfect, real-robot demonstrations.

Show abstract

Human demonstrations for robot imitation learning often contain mistakes and corrective behaviors, such as imprecise grasps, object misalignment, unstable contact, and repeated attempts. While these segments are commonly treated as noisy or suboptimal data, they provide valuable evidence about when execution deviates from a desirable path and how task feasibility can be restored. However, existing reward and value models often rely on monotonic progress assumptions, which capture coarse task advancement but may overlook local execution errors and corrective behaviors in imperfect demonstrations. In this work, we propose ReTVL (ReTry-Supervised Value Learning), a framework for learning mistake-sensitive value functions from mixed-quality robot demonstrations by leveraging retry events as sparse supervision. ReTVL captures the local degradation-and-recovery structure around mistakes by combining global progress calibration with local pairwise preference learning induced by sparsely annotated retry keypoints. The learned value model is then used to reweight demonstration chunks for downstream behavior cloning, reducing the influence of harmful execution errors while preserving useful corrective behaviors. Experiments on real-robot manipulation tasks show that ReTVL produces more fine-grained value estimates than progress-based baselines and improves imitation learning from imperfect demonstrations.

#19 h=n/a

RE4: Transformation-aware Imitation of Object Interactions Using Manipulation Modes

2026-06-23 cs.RO cs.LG 👤 Arsh Chawla

Arsh Chawla, Rahul Shome

Core Contributions

Pushes back on opaque end-to-end diffusion/flow imitation by reconstructing object-interaction skills from principled manipulation theory while preserving competitive performance.
Composes four interpretable steps: self-supervised model-free pose estimation from demo images, manipulation-mode-aware demonstration retrieval, a mode-preserving transformation plus replan to the retrieval point, and rollout of the transformed demo.
Evaluated on state- and image-based Push-T and Robomimic, with an adversarial sparse-data Push-T benchmark and low-data-regime experiments highlighting robustness.
Why it matters: shows simple, inspectable building blocks can rival black-box policies, restoring interpretability to learned manipulation.

Show abstract

Object interaction tasks have been a focus of advances in imitation learning. End-to-end methods, dominated by diffusion and flow-based variants have shown leaps in performance while sacrificing interpretability. Object-centric and pose-informed variants have had a role in learning from demonstration in manipulation tasks. In this paper, we revisit a few modern imitation learning benchmarks for object interactions, with the aim of composing a framework that repurposes principled theories of manipulation, preserving both performance and interpretability. For image observations, lightweight training is proposed for model-free pose estimation of the target object, using self-supervision over the demonstration data available for imitation learning. This information is then used to inform a manipulation mode-aware retrieval of a demonstration, a mode-aware transformation, a replan step that connects to the retrieval point while preserving mode constraints, and finally rolling out the transformed demonstration. These compose four key steps of the proposed RE4 framework, evaluated over state-based and image-based benchmarks in Push-T and Robomimic. An adversarial benchmark that evaluates sparse data regions of image-based Push-T showcases the robustness, further bolstered by indications from low-data regime experiments. The current work shows promise in using simple interpretable building blocks to learn manipulation skills.

#25 h=n/a

Grounding Generative Policies in Physics: Optimization-Guided Diffusion for Robot Control

2026-06-23 cs.RO 👤 Sabrina Bodmer

Sabrina Bodmer, René Zurbrügg, Tifanny Portela, Hao Ma, Alexandre Didier

Core Contributions

Targets the gap where diffusion-generated grasps, waypoints, or trajectories are distributionally valid yet physically infeasible — violating reachability, collision-avoidance, or closed-loop trackability — which blocks zero-shot cross-embodiment deployment.
Reformulates diffusion guidance as constrained optimization: replaces the random sampling perturbation in the reverse process with an optimized correction that imposes hard constraints or soft penalties without retraining the model.
Keeps generated samples close to the learned prior, so feasibility is enforced without sacrificing behavior quality.
Quantitative: matches projection- and gradient-guidance feasibility while better preserving grasp quality, improving task success by up to 20pp on dexterous grasping and 23pp on visuomotor manipulation over the best baseline.

Show abstract

Diffusion models sample effectively from high-dimensional, multimodal distributions, but their outputs may violate deployment constraints. For task-space robot policies, generated grasps, waypoints, or trajectories can be distributionally valid yet infeasible, violating reachability, collision-avoidance, or closed-loop executability requirements. This embodiment gap limits zero-shot deployment across robots, even when the task-space behavior itself is transferable. We propose an inference-time optimization framework that couples the behavior generation to physical feasibility by formulating diffusion guidance as a constrained optimization problem. Our key insight is to replace the sampling perturbation in the backward process with an optimized correction, allowing hard constraints or soft penalties to be imposed during sampling without the need to retrain the diffusion model, while keeping samples close to the learned prior. We evaluate the method on dexterous grasp synthesis with reachability and collision-avoidance constraints, and dynamic manipulation with controller-level trackability constraints. Across settings and robot embodiments, optimization-guided denoising matches the feasibility of projection- and gradient-guidance baselines while better preserving grasp quality, and improving controller-level executability and task success, with task success improving by up to 20pp. on dexterous grasping and 23pp. on visuomotor manipulation over the best baseline.

#10 h=n/a

Enabling Robust Cloth Manipulation via Inference-Time Simulator-in-the-Loop Refinement

2026-06-23 cs.RO 👤 Xin Liu

Xin Liu, Yulin Li, Ziming Li, Pengyu Jing, Zhenhao Huang

Core Contributions

Extends inference-time simulator-in-the-loop refinement — proven for rigid bodies — to real-world cloth manipulation from a single RGB image, a far harder state-and-contact regime.
Three pillars: a synthetic-data and rollout pipeline on the FLASH deformable simulator; a real-to-sim module trained purely on synthetic data that maps one RGB frame to simulation-ready cloth state by fusing pretrained features with learnable canonical tokens; and online planning coupling sparse-mesh rollouts with prior-guided MPPI anchored to an offline-distilled policy.
Design preserves manipulation-relevant deformation and contact while keeping enough parallel rollout batches to be practical.
Why it matters: real-robot experiments show higher success rates and stronger robustness than baselines on deformable tasks where state estimation is notoriously hard.

Show abstract

Simulator-in-the-loop optimization offers a promising inference-time mechanism for robot manipulation. It uses a physical simulator as a backend rollout engine to evaluate candidate trajectories in parallel and refine nominal actions online, a paradigm proven effective in rigid-body manipulation where state and contact are relatively tractable. We bring this paradigm to real-world cloth manipulation from a single RGB input through three pillars. (i) We design a scalable synthetic-data generation and inference-time rollout pipeline built on FLASH, a deformable-object simulator that provides a practical balance among physical fidelity, numerical stability, and rollout efficiency. (ii) We develop a real-to-sim module, trained purely on synthetic data, that maps a single RGB observation to simulation-compatible cloth state by fusing pretrained visual features with learnable canonical tokens. (iii) We perform online planning by coupling a sparse-mesh rollout backend with prior-guided MPPI, anchored at an offline-distilled policy trajectory, preserving manipulation-relevant deformation and contact while enabling sufficient parallel rollout batches. Real-robot experiments show higher success rates and stronger robustness than baseline methods.

Tactile, Contact & Dexterous Hardware

#5 h=n/a

TACTFUL: Tactile-Driven Exploration For Object Localization and Identification in Confined Environments

2026-06-23 cs.RO cs.AI 👤 Shivani Kamtikar

Shivani Kamtikar, Chung Hee Kim, Camilla Tabasso, Tye Brady, Joshua Migdal

Core Contributions

Builds a fully vision-free tactile exploration framework: a multi-fingered robot autonomously explores confined workspaces, discovers objects through contact, and identifies them via tactile reconstruction.
Trained entirely on real hardware without simulation, learning a single policy that balances global workspace exploration with local surface refinement through a dynamic reward schedule.
Demonstrates that touch, paired with structured learning, can serve as a primary modality for object-level reasoning where vision fails — clutter, confined or occluded spaces.
Achieves 77% success with 0.015 m average reconstruction error, outperforming baseline approaches on real-world objects.

Show abstract

Humans effortlessly locate and identify objects by touch alone, even without vision. In contrast, robotic systems rely heavily on vision and struggle with autonomous tactile exploration and object identification. We present TACTFUL, a vision-free tactile exploration framework that enables a multi-fingered robot to autonomously explore confined workspaces, discover objects through contact, and identify them via tactile reconstruction. Trained entirely on real hardware without simulation, our system learns a single policy that balances global workspace exploration with local surface refinement through a dynamic reward schedule. Our results demonstrate that tactile sensing, when paired with structured learning, can serve as an effective primary modality for object-level reasoning, achieving 77% success with 0.015 m average reconstruction error and outperforming baseline approaches on real-world objects.

#16 h=n/a

NoContactNoWorries: Estimating Contact through Vision and Proprioception for In-Hand Dexterous Manipulation

2026-06-23 cs.RO cs.AI 👤 Soham Patil

Soham Patil, Avirup Das, Sourabh Bhosale, Spandan Roy

Core Contributions

Replaces dedicated fingertip tactile hardware for binary contact sensing with a learned estimate from RGB-D vision and proprioception, inspired by how humans infer contact without touch sensors.
A transformer-based multimodal model fuses vision with the robot's body pose into a pseudo-tactile binary contact signal for in-hand object interactions.
A single contact-prediction model trained across multiple objects feeds downstream RL for in-hand reorientation and generalizes to novel objects.
Why it matters: sidesteps the cost, fragility, and integration burden of tactile hardware for binary contact; validated in simulation and on a real robot.

Show abstract

Perceiving physical contact is fundamental to dexterous manipulation. While robots often rely on dedicated hardware tactile sensors, humans exhibit a remarkable ability to infer contact by integrating visual information with an innate sense of their body's pose and movement. Inspired by this embodied perceptual skill, we investigate whether a robot can learn to infer contact from vision, an approach that also offers a scalable alternative to tactile hardware specifically for binary contact estimation, which faces practical challenges in cost, fragility, and integration. We present NoContactNoWorries, a transformer-based multimodal framework that fuses RGB-D vision with the robot's proprioception to infer binary contact states as a pseudo-tactile signal for hand-object interactions. We validate by training a single contact prediction model on multiple objects and show that the inferred contact signal supports downstream reinforcement learning agents for in-hand object reorientation, generalizing to novel objects. Experiments in both simulation and on a real-world robot validate our approach, highlighting the feasibility of inferring contact from vision and proprioception. Project Page: https://soham2560.github.io/no-contact-no-worries/

#21 h=n/a

PDS Joint: A Parametric Double-Spiral Joint Tailored for Dexterous Hands

2026-06-23 cs.RO cs.AR 👤 Haoyang Li

Haoyang Li, Yibo Wen, Yixiang Fan, Yiheng Xu, Yufeng Yue

Core Contributions

Designs a parametric double-spiral compliant joint that systematically shapes direction-dependent stiffness across flexion/extension, abduction/adduction, and pronation/supination for safer, more adaptable dexterous hands.
Uses Archimedean and logarithmic spiral templates per joint plus an asymmetry ratio to trade grasp stability against hyperextension resistance, revealing a non-monotonic dependence of lateral support on asymmetry.
Co-designs embedded inductive proprioception with a learning-based calibration pipeline (ArUco-tracked ground truth); a learned MLP cuts joint-state estimation error by 41.6% versus conventional curve fitting in the hardest abduction/adduction mode.
Demonstrated on an open-source dexterous hand grasping nine everyday objects and performing safe, contact-rich human-involved interactions.

Show abstract

Compliant joints can embed safety and adaptability into dexterous hands, but achieving large-stroke anthropomorphic motion while maintaining joint-specific, directiondependent stiffness and reliable proprioception remains challenging. This paper presents the PDS joint, a parametric doublespiral (PDS) compliant joint that enables systematic shaping of directional stiffness across multiple deformation modes, including flexion/extension, abduction/adduction, and pronation/supination. We instantiate the joint using Archimedean and logarithmic spiral templates for different hand joints and introduce an asymmetry ratio to tailor stiffness distributions for both grasp stability and hyperextension resistance. To make the joint practically usable under large deformation, we co-design embedded inductive proprioception and propose a learningbased calibration pipeline that maps raw inductive signals to joint states using ArUco-marker tracking. Experiments characterize the stiffness landscapes across geometric parameters and demonstrate a non-monotonic dependence of lateral support on asymmetry, indicating the importance of principled parameter tuning. For joint-state estimation in the most challenging abduction/adduction motion, a learned multilayer-perceptron (MLP) mapping reduces the error compared with conventional curve fitting by 41.6%. Finally, we integrate the proposed joints into an open-source dexterous hand as a demonstration platform, on which the hand grasps a set of nine everyday objects and performs safe, contact-rich human-involved interactions.

Legged Locomotion & Whole-Body Control

#14 h=n/a

FT-WBC: Learning Fault-Tolerant Whole-Body Control for Legged Loco-Manipulation

2026-06-23 cs.RO 👤 Yudong Zhong

Yudong Zhong, Pengfei Mai, Sikai Guo, Jiahang Cao, Zhihai Bi

Core Contributions

Tackles fault-tolerant loco-manipulation — keeping a legged manipulator stable and its arm reachable after actuator failures — a coupled problem prior locomotion-only fault-tolerance work left unaddressed.
Decoupled upper/lower-body policy architecture with a Fault Estimator that infers faulty joints from lower-body proprioceptive history and a Posture Adaptation Module that rewrites the arm policy's base-posture requests into safe, executable commands.
The fault-aware posture adaptation synthesizes compensatory gaits while preserving as much arm workspace as possible under weakening or locked failures.
Why it matters: improves survival rate and workspace in simulation and transfers zero-shot to a real legged manipulator.

Show abstract

Legged manipulators combine the mobility of legged platforms with the manipulation capability of robotic arms. However, arm-induced Center-of-Mass shifts and dynamic disturbances make the system more prone to instability under actuator failures, potentially leading to falls, task failures, or safety risks. Existing fault-tolerant control methods mainly focus on locomotion alone, leaving the coupled problem of whole-body stability and arm reachability in fault-tolerant loco-manipulation largely unaddressed. To bridge this gap, we propose FT-WBC, a fault-tolerant loco-manipulation framework for robust whole-body control of legged manipulators under actuator failures. FT-WBC adopts a decoupled upper- and lower-body policy architecture and introduces two key modules: a Fault Estimator (FE) and a Posture Adaptation Module (PAM). The FE predicts faulty joints from lower-body proprioceptive histories, while the PAM uses this fault information to adapt the base posture plan generated by the arm policy, converting potentially unstable posture requests into safe and executable base posture commands. Through this fault-aware posture adaptation mechanism, FT-WBC synthesizes compensatory gaits under actuator failures and preserves as much arm workspace as possible while maintaining whole-body stability. Simulation and real-world experiments show that FT-WBC significantly improves survival rate and workspace under weakening or locked failures, and transfers zero-shot to a real legged manipulator in the real world.

#22 h=n/a

SlipSense: Multimodal Sensing for Online Slip Detection in Legged Robots

2026-06-23 cs.RO 👤 Iris Szu-Yao Liu

Iris Szu-Yao Liu, Chien Chern Cheah, Meng Yee Michael Chuah

Core Contributions

Adds force-based early slip detection to quadrupeds via a custom lightweight sensorized foot, catching slips before the catastrophic instability that proprioception-only methods miss.
Combines a multimodal sensor-foot design with an LSTM that infers ground reaction forces and flags slip-indicative anomalies online, deployed blind (no vision) on a Unitree Go1 over slippery terrain.
Detects early-stage slips down to 24.1 ± 6.4 mm average displacement at 85.9% accuracy — a 3.3x finer detection resolution and 24% relative accuracy gain over a kinematic foot-velocity baseline.
Why it matters: lays a foundation for force-aware gait adaptation, letting future controllers estimate terrain friction and adjust constraints for stability.

Show abstract

Legged robots rely on accurate ground interaction awareness to traverse variable terrains, such as slippery surfaces. Existing slip detection methods often rely on kinematics and proprioception, which lack the sensitivity to detect early-stage slips that occur prior to catastrophic instability. Thus, this paper presents SlipSense, a novel framework for online force-based slip detection using a custom lightweight sensorized foot for quadrupeds to detect slip. The framework integrates a multimodal sensor design with a LSTM-based model to infer ground reaction forces and detect slip-indicative anomalies during locomotion. The proposed framework is deployed on a Unitree Go1 quadruped to demonstrate blind online slip detection over a slippery terrain. Our method detects early-stage slips down to an average displacement of 24.1 +/-6.4mm with an overall accuracy of 85.9%. This represents a 3.3-fold finer detection resolution and a 24% relative accuracy improvement over a standard kinematic baseline that uses foot velocity inferred through state estimation. The work in this paper serves as a foundation for force-aware gait adaptation in legged robotic locomotion, allowing future controllers to estimate terrain friction and adjust constraints, thus improving the overall stability of the system.

Perception, SLAM & Visual Localization

#2 h=n/a

Vision-Language Model Reasoning for Contextual Semantic Mapping in Intralogistics

2026-06-23 cs.RO 👤 Marvin Rüdt

Marvin Rüdt, Hao Pang, Constantin Enke, Zäzilia Seibold, Kai Furmans

Core Contributions

Adds an open-vocabulary semantic layer — object class and movability — on top of standard geometric SLAM maps, so warehouse robots grasp not just where obstacles are but whether they are likely to move.
Fuses SLAM geometry, SAM instance segmentation, instance clustering, and multi-view VLM reasoning queried zero-shot, requiring no task-specific training or predefined object taxonomy.
Reports 98.93% mIoU for semantic classification and 89.17% mAcc for movability estimation, and isolates VLM reasoning as the contextual-understanding bottleneck and clustering as the panoptic-quality limiter.
Practical payoff: context-aware filtering lets navigation ignore movable clutter, improving robustness in dynamic intralogistics environments.

Show abstract

Autonomous mobile robots operating in intralogistics environments rely on geometric maps for localization and navigation, but lack semantic understanding of objects and their contextual properties. We present a contextual semantic mapping pipeline that combines SLAM-based geometric mapping, SAM-based instance segmentation, instance clustering, and VLM multi-view reasoning to produce a contextual semantic map representation encoding geometric structure, object class, and object movability. By aggregating observations across multiple viewpoints and querying a VLM in a zero-shot, open-vocabulary setting, the pipeline infers contextual object properties--here demonstrated through movability--without requiring task-specific training or predefined object categories. We evaluate three VLMs under two prompting strategies and conduct a component-wise analysis of the pipeline. The proposed pipeline achieves 98.93 % mIoU for semantic classification and 89.17 % mAcc for object movability estimation. Component analysis identifies VLM reasoning as the primary bottleneck for contextual understanding and instance clustering as the main limitation for panoptic performance. The resulting semantic map supports context-aware filtering and robust navigation in dynamic intralogistics environments.

#3 h=n/a

Compact Object-Level Representations with Open-Vocabulary Understanding for Indoor Visual Relocalization

2026-06-23 cs.CV cs.RO 👤 Zhaopeng Cui

Zhaopeng Cui, Jiarui Hu, Jingbo Liu, Boming Zhao, Xiyue Guo

Core Contributions

Shifts indoor relocalization from low-level pixel matching to object-level reasoning, organizing semantics, layout, and geometry into a structured object map that exclusively drives camera pose estimation.
Leverages foundation models for open-vocabulary 2D-3D object matching, object-oriented reference frames as position priors, and a Distance-IoU-based frame-selection strategy to scale to large scenes.
Introduces a dual-path 2D Iterative Closest Pixel loss guided by object shape for stable, accurate pose optimization.
Why it matters: recovers interpretability (relocalization grounded in recognizable objects) while improving recall and accuracy across datasets — directly relevant to spatial and embodied AI.

Show abstract

Indoor visual relocalization plays a critical role in emerging spatial and embodied AI applications. However, prior research was predominantly devoted to low-level vision schemes, struggling to perceive scene semantics and compositions, which limits both interpretability and applicability. In this paper, we explore the issue of how to organize rich object information in a scene, including semantics, layout, and geometry, into a structured map representation, thereby utilizing object units exclusively to drive the camera relocalization task. To this end, we propose OpenReLoc, a camera relocalization system designed to provide scene understanding and accurate pose estimation capabilities. Leveraging recent foundation models, we first introduce a multi-modal mechanism to integrate open-vocabulary semantic knowledge for effective 2D-3D object matching. Additionally, we design object-oriented reference frames as position priors, paired with a reference frame selection strategy based on the Distance-IoU (DIOU), enabling extension to scalable scenes. Moreover, to ensure stable and accurate pose optimization, we also propose a dual-path 2D Iterative Closest Pixel loss guided by object shape. Experimental results demonstrate that OpenReLoc achieves superior relocalization recall and accuracy across various datasets. Our source code will be released upon acceptance.

#9 h=n/a

ArtiTwinSplat: Interactable Digital Twin Reconstruction via Gaussian Splatting from RGB-D videos

2026-06-23 cs.RO cs.CV 👤 Pranjal Mishra

Pranjal Mishra, René Zurbrügg, Max Wilder-Smith, Marco Hutter, Marc Pollefeys

Core Contributions

Automatically constructs articulated, photo-realistic digital twins of objects directly from RGB-D video — no CAD models, simulation assets, or manual annotation — attacking a major bottleneck in robot system integration.
Uses 3D Gaussian Splatting for geometric fidelity and photometric realism, coupled with an unsupervised articulation-discovery pipeline that recovers part structure and joint kinematics from observed motion alone.
Tracking and optimization stages yield stable, queryable twins supporting real-time rendering, viewpoint control, and interactive manipulation.
Why it matters: unlike simulation-confined prior work, it operates on real observations to produce twins immediately usable by downstream planning and learning, lowering the integration barrier for articulated-object manipulation.

Show abstract

Deploying robots in unstructured real-world environments needs accurate, interactive models of the objects. Constructing these models at scale remains a critical bottleneck for robotic system integration. We present ArtiTwinSplat, a framework that automatically constructs articulated, photo-realistic digital twins of objects directly from RGB-D videos, requiring no CAD models, simulation assets, or manual annotations. Our method is built on 3D Gaussian Splatting that preserve geometric fidelity and photometric realism, coupled with an unsupervised articulation discovery pipeline that recovers part structure and joint kinematics from observed motion alone. With tracking and optimization stages our method provides stable, queryable digital twins that support real-time rendering, viewpoint control, and interactive manipulation. Unlike prior methods confined to simulation, ArtiTwinSplat operates directly on real-world observations and produces twins that are immediately usable by downstream robot planning and learning systems. This method offers a practical, scalable pathway toward digital twin construction, lowering the integration barrier for articulated object manipulation in embodied AI and human-robot collaboration contexts.

#12 h=n/a

Decentralized Pose Graph Riemannian Optimization for Object-based Multi-Robot SLAM

2026-06-23 cs.RO 👤 Yixian Zhao

Yixian Zhao, Yan Huang, Yang Xu, Liang Li, Jinming Xu

Core Contributions

Removes a restrictive assumption in decentralized object-based multi-robot pose-graph optimization — that the communication graph matches the physical interaction topology — to support sparse, intermittent, time-varying communication.
Decouples the tightly coupled trajectory-and-object estimation via a consensus mechanism, then adds a distributed approximate-Newton scheme that exploits local second-order information directly on the SE(d) manifold to preserve geometric consistency.
Provides convergence guarantees to Riemannian first-order stationary points and a local condition-number analysis explaining why approximate second-order beats first-order Riemannian descent.
Why it matters: cuts iteration count and communication overhead without losing accuracy; validated on benchmarks, large-scale simulation, and real multi-robot experiments, robust to communication failures.

Show abstract

Pose graph optimization (PGO) is a key back-end component for state estimation in networked multi-robot simultaneous localization and mapping (SLAM). In object-based multi-robot SLAM, the problem becomes more tightly coupled because robots must jointly estimate both their trajectories and the poses of persistent objects observed by multiple agents. Existing decentralized solutions often assume that the communication graph closely matches the physical interaction topology, which is restrictive in realistic deployments where communication is sparse, intermittent, or time-varying. This paper presents a fully decentralized Riemannian optimization framework for object-based multi-robot PGO that decouples the coupled estimation problem via a consensus mechanism, enabling flexible communication topologies. To improve convergence under limited communication budgets, we further develop a distributed approximate-Newton scheme that exploits local second-order information while operating directly on the SE(d) manifold to preserve geometric consistency, and we establish the convergence to Riemannian first-order stationary points and provide a local condition-number analysis explaining the benefit of approximate second-order information over first-order Riemannian descent. The resulting method reduces iteration count and communication overhead without sacrificing estimation accuracy. Extensive evaluations on public benchmarks, large-scale simulations, and real-world multi-robot experiments demonstrate improved accuracy, runtime efficiency, scalability across network topologies, and robustness to communication failures.

#24 h=n/a

From Open Waters to Enclosed Cabins: ProteusVPR for Cross-Scene Visual Place Recognition in Maritime Perception and Cabin Inspection

2026-06-23 cs.CV cs.RO 👤 Zexi Chena

Zexi Chena, Zitai Huang, Qiwen Gu, Zhiqi Li, Shengli Dong

Core Contributions

Addresses cross-scene visual place recognition aboard ships, where robots cross sharply different domains — open decks (sparse texture, severe illumination changes) and enclosed cabins (repetitive, ambiguous structure) — that break VPR methods built for urban or indoor scenes.
Two-stage retrieve-then-refine: any standard VPR model performs initial retrieval, then a geometric-visual estimation network fuses the retrieved image with two preceding frames using geometric descriptors, a local affine coordinate system, and camera azimuth encoding for precise localization.
Releases XHZ, an 8K-panoramic ship-borne dataset with multi-floor cabins, deck-transition zones, and strict query-database separation for rigorous evaluation.
Quantitative: reduces mean localization error by over 60% on average across multiple VPR backbones in challenging maritime scenes.

Show abstract

Autonomous robotic inspection in maritime environments presents unique challenges for Visual Place Recognition (VPR) due to cross-scene perceptual shifts. Robots navigating ship-borne environments must transition between visually distinct domains: open decks with sparse textures and severe illumination changes, and enclosed cabins with repetitive structures and high visual ambiguity. Existing VPR methods, designed primarily for urban or indoor scenes, fail to generalize reliably across these starkly different scenarios. To address this, we propose ProteusVPR, a two-stage retrieval-refinement framework. The first stage employs any standard VPR model for initial image retrieval. The second stage introduces a geometric-visual estimation network that fuses the retrieved image with two temporally preceding frames, incorporating geometric descriptors, a local affine coordinate system, and camera azimuth encoding to achieve precise localization. To support this task, we introduce the XHZ dataset, an 8K-panoramic ship-borne dataset collected from an operational vessel, featuring multi-floor cabin structures, deck transition zones, and strict query-database separation for rigorous evaluation. Extensive experiments on the XHZ dataset demonstrate that ProteusVPR consistently improves the localization accuracy across multiple VPR backbones, reducing mean localization error by over 60\% on average and that ProteusVPR offers an effective and robust solution for precise visual localization in challenging, cross-scene maritime environments.

Navigation, Planning & World Models

#8 h=n/a

Optimization-based Safe Trajectory Planning for Autonomous Ground Vehicle in Multi-Floor Scenarios

2026-06-23 cs.RO 👤 Zishang Xiang

Zishang Xiang, Runda Zhang, Runqi Chai, Kaiyuan Chen, Senchun Chai

Core Contributions

Extends optimization-based trajectory planning to multi-floor scenarios, splitting the problem into task planning (selecting floor exits) and trajectory generation.
Task module uses generalized Voronoi diagrams plus multi-objective optimization to choose per-floor exits; the trajectory module uses a warm-started hierarchical framework for rapid convergence.
Introduces a correlation-constraint calculation that prunes obstacle constraints to keep optimization tractable in complex, cluttered environments.
Why it matters: addresses a rarely-handled structured navigation problem — vertical building transitions — with convergence-focused design, validated in simulation.

Show abstract

The development of trajectory planning strategies for autonomous ground vehicles (AGVs) represents a prevailing research interest within the domain of intelligent transportation systems. This paper introduces a trajectory planning framework tailored for multi-floor scenarios. The framework consists of two main modules: the task planning module and the trajectory planning module. The task planning module involves a strategic selection phase, where a task planning strategy based on generalized voronoi diagrams (GVD) and multi-objective algorithms is proposed to select the floor exits for each floor. The trajectory planning module utilizes optimization-based methods to generate high-quality trajectories, and a warm-started hierarchical planning framework is designed to ensure rapid convergence. Additionally, for handling complex obstacle constraints, a correlation constraint calculation method is designed for reducing obstacle constraints in trajectory planning. Finally, the feasibility and effectiveness of the proposed framework are verified through simulations.

#29 h=n/a

NavWM: A Unified Navigation World Model for Foresight-Driven Planning

2026-06-23 cs.RO cs.CV 👤 Yanghong Mei

Yanghong Mei, Longteng Guo, Ming-Ming Yu, Guiyu Zhao, Xingjian He

Core Contributions

Unifies latent world reasoning, multimodal action prediction, and controllable visual generation in a single navigation world model, instead of isolating perception, generation, and control.
Latent world tokens distill geometric and semantic priors for structural understanding; an anchor-based multimodal trajectory forecaster produces a diverse action space rather than a single deterministic path.
That diversity lets the generative world model act as a closed-loop planner, using visual foresight to evaluate and select the optimal path — countering myopic decisions and mode collapse.
Why it matters: reports SOTA across diverse robotics datasets in both high-fidelity future-state generation and zero-shot navigation success.

Show abstract

Conventional visual navigation policies often struggle with myopic decision-making and mode collapse in complex environments. While world models offer a promising alternative, existing paradigms typically isolate perception, generation, and control, failing to capture their shared spatio-temporal dynamics. In this paper, we propose NavWM, a unified navigation world model that seamlessly integrates latent world reasoning, multimodal action prediction, and controllable visual generation. At its core, NavWM leverages latent world tokens to distill geometric and semantic priors, endowing the agent with robust structural understanding. To overcome the limitations of deterministic policies, we introduce an anchor-based multimodal trajectory forecasting framework that generates a diverse action space. This inherent diversity explicitly empowers the generative world model to act as a robust closed-loop planner, utilizing visual foresight to evaluate and select the optimal path. Extensive experiments across diverse robotics datasets demonstrate that NavWM significantly advances the state-of-the-art, delivering remarkable improvements in both high-fidelity future state generation and zero-shot navigation success.

#30 h=n/a

DynaWM: Dynamics-Aware Distillation with World Model and Momentum Targets for Smooth Locomotion over Continuous Stairs

2026-06-23 cs.RO cs.AI 👤 Haidong Hou

Haidong Hou, Zhangguo Yu, Hengbo Qi, Jianlin Zhang

Core Contributions

Improves long-staircase traversal for bipedal-wheeled robots, where teacher-student frameworks suffer weakened dynamics-aware representations and incomplete terrain-geometry encoding.
Uses a world model as a regularizer to enforce forward-dynamics awareness and preserve full terrain geometry, plus a momentum target encoder that supplies consistent distillation targets and prevents dimensional collapse from non-stationary teacher updates.
PCA visualization and quantitative metrics show the encoder hierarchically captures terrain geometry with higher encoding capability.
Why it matters: yields superior terrain adaptability and motion smoothness, enabling real bipedal-wheeled hardware to climb diverse continuous stairs.

Show abstract

Recent advances in control have enabled bipedal-wheeled robots to traverse slopes and single-step obstacles, yet long staircase traversal remains challenging as current teacher-student frameworks suffer from weakened dynamics-aware representations and incomplete terrain geometry encoding. To bridge this gap, we propose DynaWM, a dynamics-aware representation learning framework. To enhance terrain encoding capability and enable transparent assessment, we introduce a world model as a regularizer to enforce forward-dynamics awareness, preserving comprehensive terrain geometry while facilitating hierarchical encoding visualization. To stabilize knowledge transfer, we employ a momentum target encoder to provide consistent distillation targets, preventing dimensional collapse from non-stationary teacher updates. Evaluation of the learned representations through Principal Component Analysis (PCA) visualization and quantitative metrics reveals that our encoder hierarchically captures terrain geometry with higher terrain encoding capability, leading to enhanced terrain adaptability and motion smoothness. Experimental results in simulation and real hardware demonstrate that our method achieves superior terrain adaptability and motion smoothness, enabling bipedal-wheeled robots to overcome diverse continuous stairs, as shown in Fig. 1.

Control, Coordination, HRI & Systems

#7 h=n/a

Parallel Dynamic Programming for Conic Linear Quadratic Control

2026-06-23 math.OC cs.RO 👤 Luyao Zhang

Luyao Zhang, Gabriel Bravo-Palacios, Brian Plancher, Sergio Grammatico

Core Contributions

Breaks the inherently serial nature of LQ and MPC solvers — which underuse modern multi-core CPUs — with a parallel-in-time divide-and-conquer approach.
Formulates the ADMM inner primal update as an LQ problem split along the time horizon, then derives a Riccati-recursion variant via dynamic programming to solve each time-segment subproblem in parallel.
Handles conic (constrained) optimal control, broadening applicability beyond unconstrained LQ.
Quantitative: numerical benchmarks on two real-world applications show as much as a 5x speedup over related approaches on multi-core CPU hardware.

Show abstract

Linear Quadratic (LQ) control problems are at the heart of linear control theory and Model Predictive Control (MPC). While performant, standard approaches to solving such problems are inherently serial, limiting real-time scalability despite the parallel computing power available on modern multi-core CPUs. Contributing to addressing this challenge and motivated by ``divide and conquer'' strategies, we present a parallel-in-time approach that solves computationally demanding conic optimal control problems through the use of the alternating direction method of multipliers (ADMM). In particular, we formulate the inner primal update of ADMM as an LQ problem and split the reformulated problem along the time horizon. This enables us to derive a variant of the Riccati recursion using dynamic programming to solve each subproblem in parallel. Numerical benchmarks on two real-world applications demonstrate as much as a 5x speedup compared to existing related approaches on multi-core CPU hardware.

#11 h=n/a

Explaining Failures of Cyber-Physical Systems with Actual Causality

2026-06-23 cs.RO 👤 Khen Elimelech

Khen Elimelech, Tom Yaacov, David A. Kelly, Hana Chockler, Moshe Y. Vardi

Core Contributions

Brings the formal framework of actual causality — previously applied only to simple systems like image classifiers — to explaining failures of complex cyber-physical systems such as self-driving cars.
Addresses the theoretical gaps needed for correct causal explanation derivation in the CPS domain, where black-box neural components make full pre-deployment verification impossible.
Contributes two practical, system-agnostic explanation-derivation algorithms that let users prioritize either explanation optimality or derivation efficiency.
Why it matters: post-failure causal explanations build trust and guide mitigation; demonstrated on a neural-network-controlled collision-avoiding autonomous car.

Show abstract

Modern autonomous Cyber-Physical Systems (CPSs), such as self-driving cars, face increasingly complex demands, and yet are expected to act reliably. The black-box nature often characterizing such systems, especially those relying on neural components, makes it impossible to fully verify the system behavior prior to deployment. Unfortunately, unexpected failures-when the system does not comply with its specification-are inevitable and may have catastrophic implications. To improve trust in the system and facilitate future mitigation after a failure occurs, it is important to try to derive an explanation for the unexpected system behavior. This paper introduces the novel concept of leveraging the framework of actual causality for CPS failure explanation. Up until now, this framework was only used to derive explanations in the context of simple systems, such as image classifiers. This paper addresses the theoretical gaps and provides the guidance needed to allow for correct explanation derivation in the CPS domain. Beyond the theoretical contribution, the paper presents two novel, practical, system-agnostic explanation derivation algorithms, allowing to prioritize either explanation optimality or derivation efficiency. The approach is demonstrated and evaluated in the context of a neural-network-controlled autonomous car, designed to avoid collisions.

#15 h=n/a

Varying Bundle Size Reactive Multi-Task Assignment using Selective Cost Estimation for Multi-Agent Systems

2026-06-23 cs.RO 👤 Niklas Dahlquist

Niklas Dahlquist, Shridhar Velhal, George Nikolakopoulos

Core Contributions

Makes combinatorial auction-based multi-robot task allocation tractable for real-time reactive use, where exponential bundle generation plus expensive path-planning cost estimation normally make it intractable.
Two-stage multi-fidelity bundle generation: a local search tree guided by a cheap heuristic (e.g., Euclidean distance) explores broadly, then high-fidelity path planning refines only the most promising candidates best-first.
A central coordinator solves a set-packing problem over the refined bids to ensure global feasibility and maximize overall utility.
Why it matters: enables reactive allocation with dynamic bundle sizes across environments without exposing agents' internal state or cost-estimation models.

Show abstract

This paper presents a scalable framework for multi-robot task allocation in complex environments where estimating task execution costs is computationally expensive. While combinatorial auction-based approaches offer reliable solutions, the exponential complexity of bundle generation typically renders them intractable for real-time reactive applications, particularly when accurate path planning is required for cost validation. We address this through a distributed, two-stage multi-fidelity bundle generation approach. Agents utilize a local search tree guided by a low-fidelity heuristic (such as euclidean distance) to rapidly explore the bundle space, applying high-fidelity path planning only to the most promising candidates in a best-first manner. These refined bids are then submitted to a central coordinator that solves a set packing problem to ensure global feasibility and maximize the overall utility. Simulation results in multiple environments demonstrate that the framework is able to improve the performance of reactive auction-based task allocation. Overall, the presented framework is shown to enable reactive task allocation with dynamic bundle sizes in multiple settings without exposing the agents' state and internal cost estimation models.

#18 h=n/a

Legible and Intuitive Multi-modal Robot State and Intent Communication Validated in Online and Real-world Studies

2026-06-23 cs.RO 👤 Tim Schreiter

Tim Schreiter, Jens V. Rüppel, Andrey Rudenko, Martin Magnusson, Achim J. Lilienthal

Core Contributions

Runs a systematic, large-scale validation of robot-to-human communication strategies for a mobile non-humanoid robot across standardized message types and both online and in-person settings.
Compares a low-expressive, unimodal LED strategy against a highly expressive, multimodal one (robotic gaze, gestures, voice) for signaling turning intent, attention requests, error, stuck status, and normal operation.
Finds strong evidence the multimodal strategy is more legible and intuitive, and crucially documents a virtual-to-real gap: legibility and interpretation confidence drop in real-world studies, especially for LED signaling.
Why it matters: quantifies that online HRI studies can overstate communication legibility, arguing for real-world validation before deployment.

Show abstract

Effective robot-to-human communication can increase transparency and trust, reduce uncertainty, and contribute to safer collaboration in shared workspaces. Designing and validating an effective robot communication strategy is challenging due to the varying and often limited communication modalities across robots, differences in how diverse recipients interpret messages, and the underexplored virtual-to-real gap in studies of communication legibility. We present a systematic, large-scale comparative validation of existing communication strategies for a mobile non-humanoid robot across message types and settings (online and in-person). Based on the prescribed message types in the existing standards for industrial robots, we realize and compare a low-expressive, unimodal LED-based strategy with a highly expressive, multimodal one that leverages robotic gaze, gestures, and voice. For each strategy, we analyze the communication of a turning intention, an attention request, error status, whether the robot is stuck, and whether it is functioning normally. We evaluate these strategies in replicated online and in-person experiments. We find strong evidence that highly expressive multimodal communication is perceived as more legible and intuitive than unimodal LED-based communication. Comparing the online and real-world study findings, we observe a notable decrease in overall legibility, particularly for signaling with LEDs. Similarly, confidence in message interpretation decreases during the real-world evaluation.

#20 h=n/a

Average Rankings Mask Per-Subject Optimality: A Friedman-Nemenyi Benchmark of EEG Motor-Imagery BCI Decoders

2026-06-23 cs.HC cs.AI cs.RO q-bio.NC 👤 Xavier Vasques

Xavier Vasques, Paul Barbaste, Olivier Oullier

Core Contributions

Stress-tests the common claim that one decoding pipeline (often a Riemannian or spatial method) is broadly best for EEG motor-imagery BCIs, testing the weakest version of that claim under the most favorable within-session, single-subject regime.
Massive scale: 1,056 decoding configurations and over 340,000 subject-level fits across three public datasets and two frequency bands, judged with Friedman omnibus, Nemenyi critical-difference, and Wilcoxon signed-rank statistics.
Covariance tangent-space and CSP are the strongest families, but their ordering is dataset-dependent and statistically indistinguishable on the largest cohort (Nemenyi p = 0.27); the single best pipeline is optimal for only 35% of participants.
Why it matters: matching pipeline to participant adds about seven accuracy points over the best fixed choice — a quantitative case for participant-aware model selection over a universal decoder.

Show abstract

Electroencephalography (EEG) is the dominant non-invasive modality for brain-computer interfaces (BCIs), yet reliable decoding of motor imagery is hampered by inter- and intra-individual variability. A recurring claim is that one decoding pipeline, most often a spatial or Riemannian method, is broadly preferable. We test the weakest version of that claim under the most favourable conditions. Using the Mother of All BCI Benchmarks (MOABB) framework, we evaluated 1,056 decoding configurations (feature extractor x scaler x classifier), >340,000 subject-level model fits, across three public left-versus-right motor-imagery datasets (PhysionetMI, 109 participants; Cho2017, 52; Zhou2016, 4) and two frequency bands (8-15 Hz, 8-30 Hz). Every model is fit and tested within a single session of a single participant, the easiest regime, giving every pipeline its best chance. We apply the statistics standard for multi-classifier comparison: Friedman omnibus tests, Nemenyi critical-difference analysis and Wilcoxon signed-rank tests with effect sizes. Covariance tangent-space projection (cov-tgsp) and Common Spatial Patterns (CSP) are the strongest families, but their ordering is dataset-dependent and, on the largest and most heterogeneous cohort (PhysionetMI), statistically indistinguishable (Nemenyi p = 0.27; Kendall's W = 0.11). At the individual level the single best pipeline is optimal for only 35% of PhysionetMI participants, and nonlinear descriptors are best for roughly one third; matching pipeline to participant adds about seven accuracy points over the best fixed choice. The ranking is not an artefact of dimensionality, and classifier and scaler choices are secondary to the feature representation. Even in the easiest regime, no single pipeline dominates: a lower bound on the personalization problem and a quantitative case for participant-aware model selection rather than a universal decoder.

#26 h=n/a

Importance of Intent-Sharing for V2X-based Maneuver Coordination

2026-06-23 cs.NI cs.RO 👤 Rafael Molina-Masegosa

Rafael Molina-Masegosa, Sergei S. Avedisov, Miguel Sepulcre, Javier Gozalvez, Yashar Z. Farid

Core Contributions

Quantifies how much directly sharing driving intentions — versus predicting other vehicles' trajectories on their behalf — improves maneuver coordination for connected and automated vehicles.
Compares intent-sharing against a kinematics-based trajectory-prediction method for coordinated highway lane changes, measuring the percentage of successful coordinations.
Finds substantial coordination improvements in two scenarios when vehicles have direct access to nearby vehicles' driving intentions.
Why it matters: argues intent-sharing should be built into the maneuver-coordination protocol rather than relying on behalf-of-others prediction.

Show abstract

This paper examines the critical role of intent-sharing in enabling effective maneuver coordination for connected and automated vehicles (CAVs). Successful maneuver coordinations require vehicles to accurately know other vehicles' driving intentions. Intent-sharing can be achieved by the remote vehicles directly communicating their plans with the ego vehicle, as opposed to the ego vehicle predicting the trajectory on the remote vehicles' behalf. In this paper, we investigate the potential of intent-sharing on maneuver coordination effectiveness by quantifying the percentage of successful coordinations. We analyze the potential of intent-sharing by comparing its effectiveness for coordinated lane changes in a highway scenario with the effectiveness of a trajectory prediction method based on current kinematic data. Our analysis demonstrates in two scenarios substantial improvements in maneuver coordination when CAVs have direct access to the nearby vehicles' driving intentions through intent sharing. These findings highlight the importance of including intent-sharing in the maneuver coordination protocol.

#27 h=n/a

The Evaluation Cost of Task Specialization in Evolutionary Multi-Robot Systems

2026-06-23 cs.RO 👤 Paolo Leopardi

Paolo Leopardi, Heiko Hamann, Jonas Kuckling, Tanja Katharina Kaiser

Core Contributions

Adds a cost-benefit lens to the known result that task specialists evolve more readily when subtask behaviors exist as building blocks — but a fixed evaluation budget must be split across subtasks, while a generalist spends the whole budget on one behavior.
Studies, in a physics-based foraging simulator, the total evaluation budget needed for evolved specialists to outperform generalists across multi-robot-system sizes.
Finds that as system size grows, a lower total evaluation budget suffices for specialists to beat generalists.
Why it matters: provides a quantitative budgeting guideline for when specialization pays off in evolutionary swarm design.

Show abstract

Task specialization can improve the efficiency of multi-robot systems (MRSs). Previous works have investigated the emergence of task-specialist robot controllers through evolutionary optimization and have argued that task specialization is more likely to evolve when subtask behaviors are readily available as building blocks. However, the available evaluation budget must be distributed across all subtasks, whereas a single generalist behavior can exploit the entire budget for its own optimization. We present a cost-benefit analysis of evolving task-specialist versus generalist behaviors in a foraging scenario here. In a physics-based robotics simulator, we study the total evaluation budget required to evolve task-specialist behaviors that outperform generalist behaviors across MRS sizes. We show that with increasing MRS size, a lower total evaluation budget is sufficient to evolve specialists that outperform generalists.

#28 h=n/a

Efficient Time-Domain Simulation of USV Motions in Short-Crested Irregular Waves Using an IRF-Based Framework

2026-06-23 physics.flu-dyn cs.RO 👤 Fei Duan

Fei Duan, Zihao Wang, Yaohua Zhou, Qing Xiao

Core Contributions

Replaces expensive regular-wave superposition with an impulse-response-function time-domain framework for predicting unmanned-surface-vehicle motion in short-crested irregular waves.
Transforms Froude-Krylov, diffraction, and radiation loads from the frequency domain to the time domain and evaluates responses via convolution-based force reconstruction, adding weak nonlinear restoring through instantaneous wetted-surface pressure integration.
Validated against model-test and full-scale USV measurements, with significant amplitudes, zero-crossing periods, and standard deviations agreeing well with data.
Practical guideline: a 30-degree directional-spectrum interval balances accuracy and cost — amplitudes are moderately directional-sensitive while periods are not — enabling efficient real-time seakeeping and control development.

Show abstract

Traditional time-domain prediction of vessel motions in irregular waves usually relies on superposing responses from many regular-wave components, which is computationally expensive for long-duration simulation and real-time applications. This issue is particularly relevant to unmanned surface vehicles (USVs), for which efficient and realistic motion prediction is needed for seakeeping assessment, simulation-based testing, and control-system development. This study applies an impulse response function (IRF)-based time-domain framework to predict vessel motions in short-crested irregular waves. Froude-Krylov, diffraction, and radiation loads are obtained from frequency-domain analysis and transformed into the time domain. Instantaneous responses are then evaluated directly through convolution-based force reconstruction, reducing the need for repeated regular-wave simulations. Weak nonlinear restoring effects are included by instantaneous wetted-surface pressure integration, and directional wave spectra are used to represent realistic sea states. The framework is validated against model-test measurements of an offshore supply vessel in long-crested beam irregular waves and full-scale measurements of a USV operating in real sea conditions. Predicted significant amplitudes, mean zero-crossing periods, standard deviations, and motion time histories agree well with measurements. The effect of directional-spectrum discretization is also examined. Results show that motion amplitudes are moderately sensitive to directional resolution, whereas motion periods are relatively insensitive. A 30 deg directional interval provides a practical balance between prediction accuracy and computational cost. The proposed framework offers an efficient tool for high-fidelity time-domain prediction of USV motions in realistic directional irregular seas.