🤖 Robotics arXiv Digest

Monday, July 6, 2026

📄 30 papers 📂 7 research areas Generated by Claude

🔭 Research Landscape

If today's 30 papers share a headline, it's that VLA and world-model architectures are converging on disentanglement over brute retraining. CamVLA (#1) splits camera-centric action generation from camera-perspective geometric grounding so a policy no longer needs known extrinsics; Cortex (#4) standardizes 32 canonical skill primitives so a VLM planner's output is actually executable by a VLA; InternVLA-A1.5 (#14) recasts future prediction as a compact latent-querying problem instead of pixel generation, preserving pretrained VLM semantics that prior unified models erode; DSWAM (#20) keeps a fast System-1 executor as the default and only wakes a System-2 planner when decomposition is useful; and CAC-VLA (#24) folds latent-action conditioning directly into the VLM rather than bolting on a separate reasoning module. The common move across all five is architectural: separate "what to do" from "how to move" and "where the camera is," and interface between them cheaply rather than fusing everything into one opaque backbone.

A second, closely related theme is world models diversifying past pixel-space prediction into explicit geometry and multi-paradigm control. Qantara (#16) trains a single JEPA checkpoint that serves latent planning, behavior cloning, and inverse dynamics without retraining, and its ablation showing planning success collapse from 90.1 to 53.3 when training-sampling doesn't match inference-time query patterns is a sharp reminder that training-inference alignment, not just model capacity, drives these results. GeoMoLa (#29) makes the same geometry-over-pixels bet explicitly, learning motion latents by predicting point-cloud evolution rather than reconstructing images, and its ablations confirm the geometric objective — not the latent architecture — is what drives performance. Deform360 (#3) turns this into an open empirical question at dataset scale: with 215+ hours of real visuotactile deformable-object data, it directly compares 2D pixel-space and 3D particle-space world models rather than assuming one is better, giving the field a shared testbed for a debate that's mostly been argued from small demos until now.

A third thread runs through the navigation and safety papers: robustness is being reframed from "perform well on a benchmark" to "know when your own signal is untrustworthy and act accordingly." Trajectory-Anchor Optimization (#26) explicitly targets the case where thermal VPR is confidently wrong — plausible but false loop closures with no drop in similarity score — and RCT-AD (#30) builds a per-frame reliability gate into BEV planning so degraded camera frames get reconstructed from trusted history rather than blindly fused. VLM-CASE (#9) does the analogous thing for driving control, coupling braking and steering through a friction budget that adapts to VLM-perceived weather and road conditions. But the Real-World Perturbation Testing paper (#18) is the useful check on all of this: its 72-perturbation, three-modality study finds that offline robustness metrics are only weakly predictive of real closed-loop failure for both camera and LiDAR pipelines — a reminder that reliability gates like #9 and #30 still need real-vehicle, not just simulated, validation before their safety claims can be trusted.

VLA & Foundation Models

Decoupling planning semantics from execution kinematics and camera geometry through lightweight, composable interfaces rather than monolithic retraining.

Robot Learning, World Models & Sim-to-Real

Pushing world models past pixel-space prediction into explicit geometry and multi-paradigm control, while chasing zero-shot transfer with less real-world data.

Navigation, Mapping & Localization

Hardening classical SLAM, VO, and scene-graph pipelines against real-world degeneracies: drift, occlusion, GNSS outages, and out-of-distribution false matches.

Autonomous Driving & Safety

Reliability-aware perception-to-control layers that gate action on scene confidence, tested against a sobering result on how poorly offline robustness predicts closed-loop failure.

#9 VLM-CASE
#18 ADS Perturbation Testing
#30 RCT-AD

Human-Robot Interaction & Social Robots

Designing robot behavior and touch around the needs of vulnerable or context-sensitive users, from PTSD-aware actuation to emergency evacuation support.

#7 Doorway Traversal
#12 Personalized Social Robots
#15 Trauma-Informed Touch

Task Planning & Multi-Agent Coordination

Coordinating variable team composition and long-horizon task graphs through game-theoretic and agentic-coding formulations rather than fixed policies.

#5 GaP
#17 Multi-Robot Teaming

Hardware, Control & Locomotion

Physical embodiment questions spanning neuromorphic tactile chips, long-tail humanoid balance recovery, and the aerodynamics of aerial contact.

#8 GelNeuro
#23 Athena-WBC
#28 Aerial Manipulation

VLA & Foundation Models

#1 h=N/A

From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model

2026-07-06 cs.CVcs.AIcs.LGcs.RO

Wenhao Li, Xueying Jiang, Quanhao Qian, Deli Zhao, Shijian Lu

Core Contributions

Argues the policy shouldn't need to know where the camera is: instead of requiring explicit extrinsics like prior view-robust VLAs, CamVLA predicts a camera-centric end-effector action plus a separate 6-DoF hand-eye matrix, then composes them with a deterministic geometric transform.
This factorization cleanly separates "how I should move" (pose-independent, camera-frame action) from "where I am looking from" (camera-perspective grounding), so the same action head generalizes across remounted or repositioned cameras without retraining.
Runs calibration-free, depth-free, and single-view — deployment needs only one monocular RGB frame, removing the extrinsics-calibration step that fragile prior view-robust VLAs depend on.
Shows consistent success-rate gains across diverse unseen viewpoints in both simulation and real-world robot data, evidence that the disentangled representation transfers rather than just memorizing training-time camera poses.

Show abstract

Real-world robot deployment rarely maintains the training-stage camera setup, where cameras often experience repositioning or remounting depending on actual scenarios. Existing view-robust Vision-Language-Action (VLA) policies tolerate such camera variations only when the camera extrinsics are explicitly provided, making them fragile and hard to use especially when view robustness is critical. We argue that the policy should not be told where the camera is, but rather figure it out by itself. To this end, we introduce Camera-Centric VLA (CamVLA), a new VLA model that decouples manipulation controls from camera geometry by predicting (i) a camera-centric end-effector action expressed in the local camera frame, and (ii) a 6-DoF hand-eye matrix relating cameras to the robot base. A deterministic geometric transformation composes the two predictions into a robot base-frame action. This disentangles how I should move in pose-independent camera-centric action generation from where I am looking from in camera-perspective geometric grounding. The resulting policy is calibration-free, depth-free, and single-view, requiring only a single monocular RGB image as the visual observation and task instruction at deployment. Evaluations in both simulation and real-world robot data show that CamVLA consistently improves success rates across diverse unseen viewpoints. Project page: https://alibaba-damo-academy.github.io/CamVLA/.

#2 h=N/A

LLM-as-a-Verifier: A General-Purpose Verification Framework

2026-07-06 cs.AIcs.CLcs.LGcs.MAcs.RO

Jacky Kwok, Shulu Li, Pranav Atreya, Yuejiang Liu, Yixing Jiang

Core Contributions

Reframes verification (judging whether a solution is correct) as its own scaling axis for LLM capability, distinct from pretraining and post-training compute — a framing that treats the verifier, not just the policy, as something to scale.
Departs from standard LM-judge scoring: instead of discrete scalar scores, it computes the expectation over scoring-token logit distributions to produce continuous, better-calibrated scores, and shows this granularity separates positive from negative solutions more cleanly.
Demonstrates that scaling repeated evaluation and criteria decomposition independently improve verification accuracy through variance and complexity reduction — i.e., verification quality itself has learnable scaling laws.
Directly relevant to robotics: achieves 87.4% on RoboRewardBench and shows the continuous verifier signal can serve as a dense reward for RL, improving sample efficiency of SAC and GRPO on robotics tasks — turning a general verification tool into a robotics reward source.

Show abstract

Scaling pre-training, post-training, and test-time compute have become the central paradigms for improving the capabilities of LLMs. In this work, we identify verification, the ability to determine the correctness of a solution, as a new scaling axis. To unlock this and demonstrate its effectiveness, we introduce LLM-as-a-Verifier, a general-purpose verification framework that provides fine-grained feedback for agentic tasks without requiring additional training. Unlike standard LM judges that prompt LLMs to produce discrete scores for candidate solutions, LLM-as-a-Verifier computes the expectation over the distribution of scoring token logits to generate continuous scores. This probabilistic formulation enables verification to scale along multiple dimensions: (1) score granularity, (2) repeated evaluation, and (3) criteria decomposition. In particular, we show that scaling the scoring granularity leads to better separation between positive and negative solutions, resulting in more calibrated comparisons. Moreover, scaling repeated evaluation and criteria decomposition consistently lead to additional gains in verification accuracy through variance and complexity reduction. We further introduce a cost-efficient ranking algorithm for selecting the best solution among candidates using the verifier's continuous scores. LLM-as-a-Verifier achieves state-of-the-art performance on Terminal-Bench V2 (86.5%), SWE-Bench Verified (78.2%), RoboRewardBench (87.4%), and MedAgentBench (73.3%). Beyond verification, the fine-grained signals from LLM-as-a-Verifier can also serve as a proxy for estimating task progress. We build an extension for Claude Code, enabling developers to monitor and improve their own agentic systems. Finally, we show that LLM-as-a-Verifier can provide dense feedback for RL, improving the sample efficiency of SAC and GRPO on robotics and mathematical reasoning benchmarks.

#4 h=N/A

Cortex: A Bidirectionally Aligned Embodied Agent Framework for Long-horizon Manipulation

2026-07-06 cs.ROcs.AIcs.CV

Jiaqi Peng, Xiqian Yu, Delin Feng, Yuqiang Yang, Wenzhe Cai

Core Contributions

Diagnoses a specific failure mode in hierarchical VLM-planner/VLA-executor stacks: high-level planning semantics and low-level execution kinematics don't align, so subtask plans the VLM emits aren't reliably executable by the VLA.
Fixes this by standardizing manipulation into 32 canonical skill primitives and injecting tractability principles (representative object attributes, improved trajectory reachability) directly into the data generation pipeline, rather than hoping the VLM learns tractable plans implicitly.
Scales via automatic annotation of 4,000+ hours of open-source video plus 30 hours of generated simulation data, and uses event-balanced sampling to specifically target planning ambiguity at subtask transitions.
Outperforms monolithic VLA baselines by 3.1% on Libero-long and 4.1% on RoboTwin, and — notably — completes unseen real-world long-horizon tasks like multi-stage chemistry experiments zero-shot by pairing the generalist planner with a fine-tuned VLA, something VLA fine-tuning alone can't achieve.

Show abstract

While recent Vision-Language-Action (VLA) models show promise toward generalist manipulation policies, they struggle with long-horizon tasks due to their Markovian nature-relying solely on current observations. Hierarchical dual-system methods address this but suffer from a gap between high-level planning semantics and low-level execution kinematics. We introduce Cortex, a bidirectionally aligned embodied agent framework with a customized planning interface that conveys executable and tractable subtask plans from high-level VLM to low-level VLA. Specifically, we standardize manipulation subtasks into 32 canonical skill primitives and inject tractability principles, such as representative object attributes and improved trajectory reachability, into the data generation pipeline. This enables automatic annotation of over 4k hours of open-source video data and generation of 30 hours of simulation data. We further devise an event-balanced sampling strategy to construct training data for fine-tuning the framework to better handle planning ambiguity during subtask transitions, enhanced by carefully designed harness engineering from task contexts to skill constraints during inference. Both open-loop VLM and closed-loop system evaluations demonstrate Cortex's efficacy, e.g., it outperforms monolithic baselines by 3.1% on Libero-long and 4.1% on RoboTwin. Notably, Cortex's generalist VLM enables zero-shot completion of unseen real-world long-horizon tasks, such as multi-stage chemistry experiments, by simply combining with a fine-tuned VLA-a capability infeasible through VLA fine-tuning alone.

#11 h=N/A

Green for Go, Red for No: Visual Grounding via Semantic Segmentation for VLA Navigation Policies

2026-07-06 cs.CVcs.RO

Adrian Szvoren, Dimitrios Kanoulas, Nilufer Tuptuk

Core Contributions

Provides the first empirical evaluation of visual grounding specifically for VLA navigation policies, addressing that VLA nav models remain susceptible to perceptual distractions and ambiguous scenes despite strong language-conditioned performance.
Uses SegFormer to segment traversable (green) vs. non-traversable (red) regions in real time, then compares observation-only grounding against joint observation-goal augmentation — isolating exactly where the grounding signal helps.
On OmniVLA with the Grand Tour dataset, grounding cuts mean waypoint error by 27-44% at the farthest waypoint, with larger gains for longer instructions than short ones, and essentially no improvement for image-goal tasks — a precise map of where grounding does and doesn't help.
Normalized error analysis reveals grounding mainly acts as a trajectory-length regularizer (30% shorter predicted paths) rather than improving per-unit-distance reasoning, and the authors are upfront that it cannot compensate for missing training signal on out-of-distribution instructions.

Show abstract

Vision-language-action (VLA) models enable robot navigation from natural language and visual goals, but remain susceptible to perceptual distractions and ambiguous scene interpretations. This paper presents the first empirical evaluation of visual grounding for VLA navigation policies. We propose a real-time segmentation-based grounding method that highlights traversable areas in green and non-traversable areas in red using SegFormer. Two variants are evaluated: observation-only segmentation and joint observation-goal augmentation. Using OmniVLA on the Grand Tour dataset, we show that visual grounding reduces the mean waypoint error by 27-44% at the farthest waypoint, depending on the instruction length. The benefits are greater for long instructions than for short instructions, and grounding provides little improvement for image goals. Normalized error analysis indicates that grounding primarily acts as a trajectory length regularizer, reducing the predicted path length by 30% without improving per-unit-distance reasoning. Our results indicate that visual grounding offers a simple, computationally inexpensive method to improve VLA navigation without model retraining, although it cannot compensate for missing training signals in out-of-distribution instructions.

#14 h=N/A

InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action for Compositional Generalization

2026-07-06 cs.RO

Haoxiang Ma, Junhao Cai, Xiaoxu Xu, Hao Li, Yuyin Yang

Core Contributions

Names a specific failure of unified VLM+action policies: existing designs erode the pretrained VLM's semantics, suffer interference among heterogeneous training objectives, and learn future prediction from scratch in pixel space, wasting the dynamics priors already baked into pretrained video generators.
Keeps a native VLM backbone training on VQA and subtask prediction (preserving semantics) and attaches a lightweight unified expert for continuous action, rather than overwriting the backbone with action-only objectives.
Recasts future prediction as latent-querying: a small set of learnable foresight tokens condenses task-relevant future under supervision from a frozen pretrained video generator, so the policy inherits world-model dynamics priors without ever learning pixel-level generation — and the video branch is discarded at inference to keep real-time control.
Pretrained on 1.2M robot episodes plus 3M multimodal samples, achieves best overall results across all six simulation benchmarks tested and shows the strongest compositional generalization on held-out instruction bindings in real-world tests — directly attributed to the preserved VLM semantics rather than to scale alone.

Show abstract

Unified models for robot manipulation aim to equip one policy with both the semantic priors of pretrained VLMs and the physical dynamics learned through future prediction. In practice, existing designs tend to erode the semantics of the pretrained backbone, suffer interference among heterogeneous objectives, and learn future prediction from scratch in pixel space, leaving the dynamics priors of pretrained video generators unexploited. We present InternVLA-A1.5, which builds the policy on a native VLM backbone that keeps training on VQA and subtask prediction, and attaches a lightweight unified expert for continuous action generation. Future prediction is recast as a latent-querying problem, where a small set of learnable foresight tokens condenses the task-relevant future into a compact latent code under the supervision of a frozen pretrained video generation model, so the policy inherits world-model dynamics priors without ever learning pixel-level generation. The video branch is discarded at inference, keeping real-time control. Pretrained on 1.2M robot episodes and 3M multimodal samples, InternVLA-A1.5 achieves the best overall results on all six simulation benchmarks. In the real world, the preserved semantics deliver the strongest compositional generalization on held-out instruction bindings, and the two designs together sustain long-horizon execution.

#20 h=N/A

DSWAM: A Dual-System World Action Foundation Model for Fine-Grained Robot Manipulation

2026-07-06 cs.ROcs.AI

Jian Zhu, Jianjun Zhang, Taiyi Su, Tianbin Liu, Zhangyuan Wang

Core Contributions

Notes a fair-comparison gap in the field: World Action Models (WAMs) and VLA policies are rarely compared head-to-head on real robots because systems differ in data, embodiment, and task protocol simultaneously, making claims of WAM vs. VLA superiority hard to trust.
DSWAM keeps a fast System 1 WAM executor as the default control path and only activates a System 2 VLM subtask planner when task decomposition is actually useful for complex multi-step household goals — avoiding the constant planning overhead a monolithic dual-system design would impose.
The System 1 executor is trained with action prediction and video co-training but predicts action chunks directly at inference without generating future video, keeping the dense video-supervision benefit of WAMs without paying its inference-time cost.
Adds practical real-robot engineering (TensorRT acceleration, asynchronous execution, real-time chunking) so policy queries never block control, and is evaluated specifically under the DeMaVLA deformable-manipulation setting with matched platform, pretraining data, and evaluation criteria against VLA baselines — the controlled comparison the field has been missing.

Show abstract

World Action Models (WAMs) provide a promising alternative to Vision-Language-Action (VLA) policies by using video-based world modeling as dense supervision for robot action learning. Existing WAMs excel at physically grounded execution, but typically lack the explicit language-level planning interface in VLM-based VLAs for decomposing coarse instructions. Such decomposition becomes important when household tasks involve complex multi-step goals, where coarse user commands need to be converted into sequences of fine-grained executable subtasks. Meanwhile, the field still lacks a fair real-robot comparison between VLA and WAM execution capabilities, since existing systems often differ in data, robot embodiments, and task protocols. To address both the decomposition gap and the need for a controlled WAM-VLA comparison, we introduce DSWAM, a Dual-System World Action Foundation Model for fine-grained robot manipulation. DSWAM keeps a System 1 WAM executor as the default control path and optionally activates a System 2 vision-language subtask planner only when task decomposition is useful. The planner predicts executable subtasks from short-term visual history and a global task prompt, while the WAM executor performs world-aware action generation for each instruction or subtask. The executor is trained with action prediction and video co-training, but inference directly predicts action chunks without explicit future video generation. To make this execution path practical on real robots, we further integrate TensorRT acceleration, asynchronous execution, and real-time chunking (RTC) so that policy queries do not block robot control. To provide a fair real-robot comparison with VLA policies, we build and evaluate DSWAM under the DeMaVLA real-world deformable manipulation setting with matched robot platform, pretraining data, post-training data, and evaluation criteria.

#24 h=N/A

CAC-VLA: Context-Gated Action Conditioning for Vision-Language-Action Models

2026-07-06 cs.RO

Yifu Xiong, Wenhao Yu, Jiaxuan Lin, Bojun Zou, Jiahao Li

Core Contributions

Identifies that VLA visual-language representations aren't explicitly optimized for action conditioning, leaving the action expert to bridge the gap between multimodal understanding and precise motor control on its own — a burden CAC-VLA tries to lift from the action expert.
Unlike recent action-reasoning methods that bolt on separate modules to generate explicit action plans or action-space reasoning signals, CAC-VLA trains the VLM itself to predict coarse-to-fine latent actions — structured codes encoded from future action segments — keeping the interface entirely within the VLM rather than adding an external framework.
A context gate adaptively calibrates how much influence the latent-action guidance has on the action expert, rather than injecting it uniformly, letting the model modulate reliance on latent-action signal per situation.
Reaches 98.3% average success on LIBERO and 89.5% on LIBERO-Plus, suggesting VLM-native, context-gated latent-action conditioning is a viable lighter-weight alternative to separate action-reasoning modules.

Show abstract

Vision-Language-Action (VLA) models have become a promising paradigm for generalist robot manipulation, where visual-language representations are used to condition continuous action generation. However, these representations are not explicitly optimized for action conditioning, leaving the action expert to bridge the gap between multimodal understanding and precise motor control. Recent action-reasoning methods introduce additional modules to generate explicit action plans or action-space reasoning signals, demonstrating the benefit of action-level guidance but often requiring separate action-generation frameworks. We propose CAC-VLA, a Context-Gated Action Conditioning framework that learns a lightweight latent-action interface directly within the VLM. Instead of generating executable trajectories, CAC-VLA trains the VLM to predict coarse-to-fine latent actions, which are structured representations encoded from future action segments, and adaptively leverages them to condition the action expert via a context gate. This enables VLM-native action conditioning while calibrating the influence of latent-action guidance on expert action generation. Experiments on LIBERO and LIBERO-Plus demonstrate the effectiveness of CAC-VLA, achieving 98.3% average success rate on LIBERO and 89.5% LIBERO-Plus, suggesting that context-gated latent-action conditioning is an effective interface for continuous expert control.

Robot Learning, World Models & Sim-to-Real

#3 h=N/A

Deform360: A Massive Multi-view Visuotactile Dataset for Deformable World Models

2026-07-06 cs.ROcs.CV

Hongyu Li, Wanjia Fu, Xiaoyan Cong, Zekun Li, Binghao Huang

Core Contributions

Targets a genuine open question in deformable-object world modeling — whether dynamics should be learned in 2D pixel space or 3D geometric space — that prior work couldn't resolve for lack of large, diverse real-world data, rather than proposing a new model itself.
Delivers scale: 198 objects, 1,980 interaction sequences, 215+ hours of data from 41 surround-view cameras plus bimanual tactile grippers, explicitly capturing both global object motion and localized contact-induced deformation together.
Introduces a markerless visuotactile 3D tracking pipeline to extract dense geometry and motion without physical markers, making large-scale deformable-object annotation tractable.
Its systematic 2D-vs-3D world model comparison surfaces a concrete trade-off between structural priors and scalability, giving the field a shared benchmark instead of isolated single-method claims.

Show abstract

Predicting object dynamics (i.e., world modeling) is a fundamental challenge for robotic manipulation, and modeling deformable objects presents a particularly difficult case due to their high-dimensional state spaces and complex material properties. While current world models approach this through two distinct paradigms: learning the dynamics over the 2D pixel space or more explicit 3D geometric space. A systematic understanding of their relative strengths and limitations remains elusive due to the lack of diverse, large-scale real-world data. To address this, we present Deform360, a large-scale visuotactile dataset featuring 198 daily-life objects, 1,980 interaction sequences, and over 215 hours of observations from 41 surround-view cameras and bimanual tactile grippers to capture both global motion and contact-induced local deformations. Leveraging a novel markerless visuotactile 3D tracking pipeline to extract dense geometry and motion, we systematically evaluate current state-of-the-art world models, comparing 2D video models against 3D particle models. Finally, we provide a preliminary demonstration indicating the real-world applicability of our dataset by performing robot planning tasks on deformable objects. Our analysis reveals key insights into the trade-offs between structural priors and scalability, providing a solid benchmark for future research in generalizable deformable object-centric world modeling. Project website: https://deform360.lhy.xyz

#16 h=N/A

Qantara: Bridge-Flow Training for Multi-Paradigm JEPA Control

2026-07-06 cs.LGcs.CVcs.RO

Ruslan Rakhimov, George Bredis, Yuriy Maksyuta, Daniil Gavrilov

Core Contributions

Points out that every existing JEPA-based world model for control commits at training time to one inference paradigm — either trajectory optimization in a learned dynamics model or direct behavior cloning — forcing the deployment-time choice to be made upfront rather than deferred.
Qantara's joint training objective pairs a Brownian-bridge interpolant between consecutive clean latents (state axis) with noise-to-data flow matching on the action axis, so a single checkpoint supports latent planning, behavior-cloning sampling, and inverse dynamics via a video-inverse composition, without retraining for each.
Shows the training-sampling distribution match matters concretely: concentrating mass on the edges of the (action-time, state-time) noise square (matching where inference actually queries the predictor) versus uniform interior sampling drops Push-T planning success from 90.1 to 53.3 — a large, specific ablation number showing why the training-inference alignment choice isn't cosmetic.
Sets new state-of-the-art on OGBench-Cube (+7.7 SR over DINO-WM, +19.7 over LeWM) and reaches 91.2 SR average on the LeWM control suite from the planning path alone, while the same weights' behavior-cloning path still reaches 82-83 SR on Push-T — evidence the multi-paradigm design doesn't sacrifice per-paradigm performance.

Show abstract

Joint-Embedding Predictive Architectures (JEPAs) underpin a growing family of latent world models for control from raw pixels, but every existing JEPA world model commits at training time to a single inference paradigm: either trajectory optimisation in a learned dynamics model, or direct behaviour cloning. A single checkpoint that serves both would defer this choice to inference, when deployment constraints (rollout cost, observation accessibility) determine which path wins. We present Qantara, an end-to-end JEPA whose joint training objective pairs a Brownian-bridge interpolant between consecutive clean latents on the state axis with noise-to-data flow matching on the action axis. The same checkpoint serves three inference paradigms without retraining: latent planning, behaviour-cloning action sampling, and inverse dynamics, which we query through a video-inverse composition that first predicts the next latent without action conditioning, then extracts the action. Training concentrates mass on the edges of the (action-time, state-time) noise square, where inference queries the predictor: replacing it with uniform interior sampling drops Push-T planning from 90.1 to 53.3 SR at matched compute. On the LeWM control suite, Qantara reaches a 91.2 SR three-train-seed average and sets new SOTA on OGBench-Cube (+7.7 SR over DINO-WM, +19.7 over LeWM). From the same weights, the behaviour-cloning and video-inverse paths reach 82-83 SR on Push-T and 71-73 SR on Cube. These results move JEPA world models from single-paradigm planners to multi-paradigm controllers.

#19 h=N/A

Closing the Reality Gap: Zero-Shot Sim-to-Real Deployment for Dexterous Force-Based Grasping and Manipulation

2026-07-06 cs.RO

Zhe Zhao, Zhibin Li, Yilin Ou, Mengshi Qi

Core Contributions

Tackles a specific reason dexterous multi-finger hands stay in simulation: contact-rich physics and imperfect actuation make sim-to-real control policy transfer unreliable, and most tactile sensing setups add hardware (dedicated torque sensors) that complicates deployment further.
Combines dense tactile feedback with joint torque sensing to explicitly regulate physical interaction, enabled by three specific engineering pieces: a fast tactile simulator computing distances between dense virtual tactile units and the object via parallel forward kinematics for high-rate, high-resolution touch signals; a current-to-torque calibration that eliminates the need for physical torque sensors by mapping motor current directly to joint torque; and actuator dynamics modeling with randomization to bridge non-ideal torque-speed effects.
Trains entirely in simulation with an asymmetric actor-critic PPO pipeline and deploys directly to a five-finger hand with no fine-tuning, demonstrating both controllable grasp-force tracking and in-hand object reorientation.
Claimed as the first demonstration of controllable grasping on a multi-finger dexterous hand trained entirely in simulation and transferred zero-shot to real hardware — a strong claim resting specifically on removing the torque-sensor hardware dependency, not just on tactile simulation fidelity.

Show abstract

Human-like dexterous hands with multiple fingers offer human-level manipulation capabilities but remain difficult to train the control policies that can deploy on real hardware due to contact-rich physics and imperfect actuation. We present a sim-to-real reinforcement learning method that leverages dense tactile feedback combined with joint torque sensing to explicitly regulate physical interactions. To enable effective sim-to-real transfer, we introduce (i) a computationally fast tactile simulation that computes distances between dense virtual tactile units and the object via parallel forward kinematics, providing high-rate, high-resolution touch signals needed by RL; (ii) a current-to-torque calibration that eliminates the need for torque sensors on dexterous hands by mapping motor current to joint torque; and (iii) actuator dynamics modeling with randomization to account for non-ideal torque-speed effects and bridge the actuation gaps. Using an asymmetric actor-critic PPO pipeline, we train policies entirely in simulation and deploy them directly to a five-finger hand. The resulting policies demonstrate two essential human-hand skills: (1) command-based controllable grasp force tracking and (2) reorientation of objects in the hand, both of which are robustly executed without fine-tuning on the robot. By combining tactile and torque in the observation space with scalable sensing and actuation modeling, our system provides a practical solution to achieve reliable dexterous manipulation. To our knowledge, this is the first demonstration of controllable grasping on a multi-finger dexterous hand trained entirely in simulation and transferred zero-shot on real hardware.

#21 h=N/A

PRISM: Personalized Robotic Dataset Generation via Image-based Scene and Motion Synthesis

2026-07-06 cs.RO

Dogyu Ko, Haneul Kim, Chanyoung Yeo, Dowoon Lee, Taeho Park

Core Contributions

Targets the gap between teleoperation data (well-aligned to the target environment but costly and unscalable) and pure simulation data (scalable but hard to make resemble a specific user's environment) by generating personalized datasets from just a single image and a language instruction.
Builds "digital cousin" scenes — semantically and geometrically aligned to the user's actual environment but diverse at the instance level — and synthesizes executable demonstrations without any human teleoperation, sidestepping both data-collection bottlenecks at once.
Policies trained on PRISM-generated data outperform baseline-generated-data policies on LIBERO and LIBERO-Plus, and reach up to 100% success on three real-world manipulation tasks — a striking number for a pipeline requiring only one input image.
Maintains stronger performance than baselines specifically when evaluated in environments that differ from training, suggesting the personalized-but-diverse scene generation approach generalizes better than either narrow teleop data or generic simulation data alone.

Show abstract

Recent advances in large-scale pretrained vision-language-action models have improved robot policy learning, but directly deploying such policies in user-specific environments remains challenging due to limited generalization, which inevitably requires collecting a dataset tailored to the target environment. Teleoperation yields well-aligned data but is costly and difficult to scale, whereas simulation scales easily but struggles to resemble the target environment and generate task-specific trajectories. To meet both simultaneously, we propose PRISM, an end-to-end pipeline that generates personalized robotic datasets from a single image and a natural-language instruction. PRISM constructs digital cousin scenes that are semantically and geometrically aligned with the user environment yet diverse at the instance level, and synthesizes executable demonstrations without human teleoperation. Extensive experiments show that policies trained on PRISM-generated datasets outperform those trained on baseline-generated datasets on LIBERO and LIBERO-Plus, achieve up to 100\% success rate on three real-world manipulation tasks, and maintain stronger performance when evaluated in environments that differ from those seen during training.

#27 h=N/A

Spatial Attention: Adapting Execution Horizons for Diffusion Policies via Observation Sensitivity

2026-07-06 cs.RO

Che-Sang Park, Junsu Ha, Jianlong Fu, Frank C. Park

Core Contributions

Points out that fixed-execution-horizon action-chunk sampling — the default in generative robot policies — forces a hard trade-off between responsiveness and computational cost, since every chunk runs for the same duration regardless of how sensitive that phase of the task actually is to disturbance.
Defines Spatial Attention as the expected squared norm of the gradient of the action log-likelihood with respect to the observation, giving a principled, computable measure of how sensitive the policy's action distribution is to observation changes at any given moment.
Shows theoretically that under a fixed chunk-sampling budget, the execution horizon that minimizes cumulative likelihood drop from disturbances should shrink as Spatial Attention rises — then operationalizes this by forecasting future Spatial Attention alongside the action chunk to dynamically assign shorter horizons to high-attention phases and longer horizons to low-attention phases.
On standard and perturbed tasks in both simulation and on a real robot, significantly improves success rates over fixed-horizon baselines while keeping the same average execution horizon — meaning the gain comes from smarter allocation of a fixed replanning budget, not from replanning more often overall.

Show abstract

Sampling action chunks via generative models has become a widely adopted methodology for robotic learning from demonstration. However, existing methods often struggle to balance responsiveness and computational cost because they execute each action chunk for a fixed execution horizon. In this paper, we adaptively adjust the execution horizon of sampled action chunks, balancing responsiveness and computational efficiency. We introduce Spatial Attention -- defined as the expected squared norm of the gradient of the action log-likelihood with respect to the observation -- which indicates the sensitivity of the policy's action distribution to variations in the observation. We show that, under a fixed budget of chunk samplings, the execution horizon that minimizes the cumulative likelihood drop induced by disturbances decreases as Spatial Attention increases. By forecasting future Spatial Attention values alongside the action chunk, our framework dynamically assigns shorter execution horizons to phases with high Spatial Attention, and longer horizons to phases with low Spatial Attention. Experiments on standard and perturbed tasks, in both simulation and on a real robot, show that our method significantly improves success rates over fixed-horizon baselines while maintaining the average execution horizon.

#29 h=N/A

Geometry-Aware Motion Latents for Learning Robust Manipulation Policies

2026-07-06 cs.ROcs.AI

Yunchao Zhang, Yijia Weng, Ruizhe Liu, Ming Hu, Leonidas Guibas

Core Contributions

Argues that learning motion latents from visual reconstruction alone captures appearance patterns rather than the actual physical motion needed for manipulation, since effective action abstraction requires understanding 3D geometric transformation, not just pixel change.
GeoMoLa learns discrete motion latent codes by predicting how point clouds evolve during manipulation, rather than reconstructing visual observations — a four-dimensional objective (3D geometry evolving through time) that structurally forces the latent to encode physical motion.
Achieves state-of-the-art performance using only single-view RGB-D input, while competing methods require multi-view reconstruction — a meaningfully lower sensing requirement for the same or better accuracy.
Ablations confirm geometric prediction specifically (not just the latent-code architecture) drives the performance gain, and the learned codes transfer to novel scenes with physically consistent transformations regardless of visual context, with real-world experiments confirming robustness in cluttered environments from minimal demonstrations.

Show abstract

Learning motion latents for robotic manipulation heavily relies on extracting motion patterns from visual sequences, yet effective action abstractions require understanding three-dimensional geometric transformations. Here, we introduce GeoMoLa (Geometry-Aware Motion Latents), which learns discrete motion latent codes by predicting how point clouds evolve during manipulation rather than reconstructing visual observations. This four-dimensional objective -- spatial geometry changing through time -- forces latent representations to encode actual physical motion rather than appearance patterns. GeoMoLa achieves state-of-the-art performance using only single-view RGB-D input, while existing methods require multi-view reconstruction, succeeding across diverse manipulation benchmarks. Our ablations reveal that geometric prediction is the key to driving performance, quantitatively validating that manipulation depends on spatial understanding. Furthermore, the learned codes exhibit effective motion abstraction: applying them to novel scenes produces physically consistent transformations regardless of visual context. Our real-world experiments also confirm this robustness capability, achieving robust manipulation with minimal demonstrations in cluttered environments where geometric reasoning determines success. Thus, we demonstrate that effective motion latents for robot control can better emerge from understanding motion through its three-dimensional effects rather than pixel-level patterns.

Navigation, Mapping & Localization

#6 h=N/A

Beyond Isolated Objects: Relationship-aware Open Vocabulary Scene Understanding via 3D Scene Graph Analysis

2026-07-06 cs.CVcs.RO

Xianhao Chen, Jiarui Hu, Yuanbo Yang, Xiyu Zhang, Tengyue Wang

Core Contributions

Identifies that existing open-vocabulary 3D scene understanding methods lift 2D language-aligned features into 3D but treat objects as context-independent, leaving relational cues (what's next to, on top of, inside what) unused for refining ambiguous segmentations.
Builds relational 3D scene graphs automatically from multi-view observations using VLM reasoning to infer relationships and prune geometrically implausible ones — no manual relationship annotation required, unlike earlier scene-graph approaches.
Introduces an Adaptive Gated Dual-Stream Contextual GAT that keeps dense geometric features and semantic CLIP embeddings in separate streams during edge-guided message passing, avoiding the feature interference that naive fusion causes, then adaptively recombines them.
Adds a hierarchical contrastive objective for instance-level consistency and category-level discrimination, and reports strong generalization across four benchmarks (ScanNetV2, ScanNet200, ScanNet++, Replica), indicating the relational signal transfers across dataset domains.

Show abstract

Open-vocabulary 3D scene understanding aims to segment 3D scenes beyond predefined categories by transferring semantic knowledge from vision-language models. Existing methods have advanced this task by lifting language-aligned 2D features into 3D, yet they often rely on context-independent semantic representations, leaving object relationships underexplored for contextual refinement. We propose RelGraphOV, a relationship-aware framework that uses 3D scene graphs to enhance open-vocabulary 3D understanding. Our method constructs relational scene graphs from multi-view observations by leveraging vision-language reasoning to infer object relationships and prune geometrically implausible connections, without manual relationship annotations. To aggregate relational context while avoiding feature interference, we introduce an Adaptive Gated Dual-Stream Contextual GAT that separates dense geometric features and semantic CLIP embeddings, performs edge-guided message passing, and adaptively fuses complementary semantics. A hierarchical contrastive objective further promotes instance-level consistency and category-level discrimination. Experiments on ScanNetV2, ScanNet200, ScanNet$++$, and Replica demonstrate strong performance and generalization ability. Project Page: https://cxavireh.github.io/relgraphov-projectpage

#10 h=N/A

Geometry-Aware Visual Odometry for Bronchoscopic Navigation via High-Gain Observer Fusion

2026-07-06 cs.RO

Mohammadreza Kasaei, Francis Xiatian Zhang, Feng Li, Farshid Alambeigi, Kevin Dhaliwal

Core Contributions

Addresses navigational bronchoscopy's dependence on pre-operative CT or external sensors — a hard constraint in critical care and resource-limited settings — by pursuing vision-only odometry instead, despite airway images being texture-poor and geometrically singular (tubular anatomy collapses parallax cues near the vanishing point).
Explicitly leverages vanishing-point cues from airway lumens: detected lumens are back-projected to 3D rays whose weighted fusion produces a stable forward heading even when parallax information is unavailable — a geometry-first strategy rather than relying on texture-based feature tracking that fails in airways.
Fuses this heading and looming-based velocity estimates with noisy VO outputs via a bespoke high-gain observer that enforces airway-following priors, specifically built to reject drift rather than just smooth it.
Validated on ex-vivo mechanically ventilated human lungs with electromagnetic tracking ground truth — a clinically realistic testbed — reducing absolute trajectory error by more than 50% versus ORB-SLAM2, LoFTR-VO, and DPVO, with the lowest relative pose error across all sequences.

Show abstract

Navigational bronchoscopy is critical for pulmonary interventions, yet current platforms depend heavily on pre-operative CT or external sensors, limiting their use in critical care and resource-constrained settings. Vision-only navigation offers a scalable alternative, but conventional visual odometry (VO) struggles with texture-poor airway images, specularities, and the vanishing-point singularities of tubular anatomy, leading to frequent tracking failures and drift. We present a geometry-aware VO framework that explicitly leverages vanishing-point cues from airway lumens. Detected lumens are back-projected to 3D rays, whose weighted fusion yields a stable forward heading even when parallax cues are absent. This heading, together with looming-based velocity estimates, is fused with noisy VO outputs using a bespoke high-gain observer that enforces airway-following priors and rejects drift. We validate the method on ex-vivo mechanically ventilated human lungs with electromagnetic tracking ground truth. Compared to state-of-the-art pipelines (ORB-SLAM2, LoFTR-VO, DPVO), our approach reduces absolute trajectory error by more than 50% and achieves the lowest relative pose error across all test sequences.

#13 h=N/A

ECO: Incremental Ego-Centric Octree Update for Point Streams

2026-07-06 cs.ROcs.GR

Jaemin Yu, Seongyoon Jeong, Kang-Wook Chon, Duksu Kim

Core Contributions

Identifies that standard global octree structures for mobile robot mapping suffer high latency and unbalanced tree growth under continuous point streams, since they keep growing rather than bounding themselves to what the robot actually needs.
Introduces the Ego-Centric Octree (ECO), which acts as a 3D sliding window bounded to the robot's immediate surroundings, and updates incrementally by categorizing regions into shift-out, shift-in, and overlap — eliminating redundant global coordinate transforms that dominate cost in naive incremental approaches.
On KITTI, reduces update times by up to 25.60% (24.87% average) versus full static reconstruction and up to 67.52% (54.60% average) versus a bounded incremental baseline — the comparison against an already-bounded baseline is the more telling number, since it isolates ECO's incremental-update gain specifically.
Downstream voxel-map generation runs up to 34.17% faster, and the bounded window naturally retains short-term temporal memory of moving objects, giving dynamic-scene context without unbounded cost growth.

Show abstract

Constructing octrees for mobile robots that process continuous point streams in real time poses significant computational and memory challenges. Standard global structures often suffer from high latency and unbalanced tree growth. We introduce the Ego-Centric Octree (ECO), a spatial data structure that acts as a 3D sliding window, dynamically bounding the mapping space to the robot's immediate surroundings. ECO uses an efficient incremental update algorithm that categorizes the environment into shift-out, shift-in, and overlap regions, eliminating redundant global coordinate transformations. Evaluations on the KITTI benchmark demonstrate that ECO reduces update times by up to 25.60% (24.87% on average) compared to full static reconstruction and by up to 67.52% (54.60% on average) compared to a bounded incremental baseline. Furthermore, ECO substantially lowers the total system latency of downstream tasks, running up to 34.17% faster than full reconstruction in voxel-map generation. In dynamic scenes, ECO naturally retains a short-term temporal memory of moving objects, providing useful temporal context while keeping update cost bounded and the tree balanced for real-time spatial perception.

#22 h=N/A

WinTA-GIL: Windowed Trajectory Alignment for GNSS-IMU-LiDAR Heading Refinement in Intermittent Signal Environments

2026-07-06 cs.RO

Kaixin Feng, Zhichao Wen, Zhaohong Liao, Xin Xia, You Li

Core Contributions

Addresses a blind spot in multi-source fusion localization: heading estimation lacks gravitational constraints and stays weakly observable, and most existing methods only perform a single initial alignment at startup, leaving heading unable to self-correct as drift accumulates over a long mission or after GNSS outages.
WinTA-GIL reframes heading estimation as a repeatable, trajectory-based consistency optimization: it registers high-precision local trajectories from LiDAR-Inertial Odometry against filtered GNSS observations within a temporal window, rather than relying on the one-time startup alignment conventional methods are stuck with.
Adds an adaptive re-estimation mechanism based on state discrimination that triggers heading corrections whenever needed, specifically to suppress inertial drift accumulated during GNSS-degraded stretches rather than waiting for the next clean GNSS fix.
Reports significant outperformance over state-of-the-art approaches on both open-source and self-collected datasets in both accuracy and robustness — the key claim being that heading correction is now an ongoing, triggerable process rather than a one-shot startup step.

Show abstract

Although multi-source fusion positioning systems have achieved significant progress, accurate and reliable heading estimation remains a critical challenge due to the lack of gravitational constraints and the inherent weak observability of heading in complex environments. Most existing methodologies are specifically tailored for the startup phase, relying on a singular initial alignment to establish the heading reference. Consequently, these approaches lack the adaptability required to refine heading estimates dynamically, which renders the system highly vulnerable to accumulated drift and observation noise during prolonged navigation or immediately following GNSS signal outages. To address these limitations, this paper proposes WinTA-GIL, a novel heading refinement framework that integrates information from Global Navigation Satellite System (GNSS), Inertial Measurement Unit (IMU), and Light Detection and Ranging (LiDAR) through a temporal window-based optimization strategy. Unlike conventional alignment methods restricted to the startup phase, WinTA-GIL leverages high-precision local trajectories from LiDAR-Inertial Odometry (LIO) to register against filtered GNSS observations. This approach transforms heading estimation into a repeatable, trajectory-based consistency optimization problem. In particular, an adaptive re-estimation mechanism based on state discrimination is incorporated to trigger heading corrections whenever necessary, thereby effectively suppressing the inertial drift accumulated during challenging conditions. Extensive experiments on both open-source and self-collected datasets demonstrate that WinTA-GIL significantly outperforms state-of-the-art approaches in both estimation accuracy and system robustness.

#25 h=N/A

SLAM: Structured and Localized Analytic Manifold Adaptation for Lifelong VPR

2026-07-06 cs.ROcs.CV

Kenta Tsukahara, Kanji Tanaka, Rai Hisada

Core Contributions

Targets catastrophic forgetting in lifelong Visual Place Recognition — the requirement to keep adapting to new environments without losing accuracy on previously learned ones — via a closed-form analytical approach rather than continual-learning fine-tuning.
Unifies three previously separate techniques into one closed-form analytical recursion: uncertainty-aware smoothing via the Unscented transform, topological space partitioning through a Gaussian Mixture Model, and H-infinity robust bound optimization — a single formulation rather than a pipeline of separately-tuned stages.
Ablations show the smoothing-plus-localized-mapping combination alone (without the H-infinity bound) already reaches 27.5% state-of-the-art nominal accuracy, while adding the H-infinity bound doesn't require splitting the architecture — it introduces a mathematically guaranteed minimax robust bound instead.
The practical payoff is a single regularization parameter that lets the system trade off nominal placement precision against worst-case disturbance attenuation, rather than needing separate architectures for the accuracy-optimized and robustness-optimized regimes.

Show abstract

Visual Place Recognition (VPR) in lifelong deployment requires continuous adaptation to new environments without catastrophic forgetting. In this paper, we propose SLAM, a Structured and Localized Analytic Manifold adaptation framework. Our framework elegantly unifies uncertainty-aware smoothing via Unscented transformation, topological space partitioning through a Gaussian Mixture Model (GMM), and $H_\infty$ robust bound optimization into a singular, unified closed-form analytical recursion. Exhaustive ablation studies demonstrate that while the synergistic combination of uncertainty smoothing and localized mapping (U+G configuration) achieves the state-of-the-art nominal accuracy of 27.5%, the full deployment of the $H_\infty$ bound does not require an architectural split; rather, it introduces a mathematically guaranteed minimax robust bound. This formulation enables the system to seamlessly modulate the intrinsic trade-off between nominal placement precision and worst-case disturbance attenuation through a single regularization parameter.

#26 h=N/A

Trajectory-Anchor Optimization for Overconfident Thermal Visual Place Recognition: Zero-Leakage OOD Auditing and Kidnapped-Robot Recovery

2026-07-06 cs.ROcs.CV

Zhiyuan Lu, Kanji Tanaka

Core Contributions

Names a specific and dangerous failure mode in foundation-model-based thermal VPR: under out-of-distribution or unmapped conditions, systems generate highly plausible but false loop-closure candidates without any drop in similarity score — the system is confidently wrong, not just uncertain.
Classical multi-hypothesis tracking (MHT) can mitigate this by tracking divergent trajectory beliefs but has exponential computational overhead that violates real-time constraints — TAO's whole design is compressing MHT-style multi-hypothesis reasoning into something real-time-feasible.
Compresses multi-view temporal verification of K=100 parallel hypotheses into a batched SE(2) Procrustes alignment problem, using tensor-level vectorization and single-invocation batched SVD to guarantee a strictly bounded O(KN) per-frame execution loop instead of MHT's dynamic tree expansion.
Under a strict zero-leakage evaluation protocol, shows a scale-dependent effect: within 5m, hallucinations are locally geometrically consistent enough to fool rigid alignment, but beyond 5m the 100 disparate hypotheses disperse across the global map, breaking the sliding-window co-visibility constraint and sharply escalating the optimization residual — giving TAO a distinct macroscopic convergence basin (10m) where it reliably isolates catastrophic topological breaks.

Show abstract

Modern thermal visual place recognition (TIR-VPR) frontends based on foundation models achieve remarkable closed-set retrieval but suffer from an overconfident forced-matching failure mode. Under out-of-distribution (OOD) or unmapped conditions, they generate highly plausible yet false loop candidates without a drop in similarity scores. While classical multi-hypothesis tracking (MHT) backends can mitigate these ambiguities by maintaining divergent trajectory beliefs, their exponential computational overhead violates real-time robotic constraints. To bridge this gap, we present Trajectory-Anchor Optimization (TAO). To counter the combinatorial challenge of evaluating parallel hypotheses (e.g., K=100), TAO compresses multi-view temporal verification into a batched SE(2) Procrustes alignment problem. By leveraging tensor-level vectorization and single-invocation batched SVD, this formulation bypasses the dynamic tree expansion of MHT, guaranteeing a strictly bounded per-frame execution loop of O(KN). Under a strict zero-leakage evaluation protocol, we show that while a passive geometric backend cannot mathematically separate metric localization errors from coherent hallucinations at a micro-scale (<5m) due to local visual ambiguities, TAO serves as an efficient fail-safe filter at a macro-scale. Within a 5m radius, hallucinations often possess a locally consistent geometry that deceives rigid alignment. However, beyond this threshold, the K=100 disparate hypotheses disperse spatially across the global map. This dispersion breaks the rigid temporal co-visibility constraint within the sliding window (N=20), causing the joint optimization residual to escalate sharply. Consequently, TAO establishes a distinct macroscopic convergence basin (10m) where multi-view geometric consistency reliably isolates catastrophic topological breaks and suppresses critical false acceptances.

Autonomous Driving & Safety

#9 h=N/A

VLM-CASE: Vision-Language Model Enabled Context-Adaptive Safety Envelopes for Anticipatory Safe Autonomous Driving

2026-07-06 cs.ROcs.CVeess.SY

Tianjia Yang, Ke Li, Ruwen Qin, Xianbiao Hu

Core Contributions

Targets the compounding failure mode where adverse weather degrades both what an autonomous vehicle can perceive and what it can physically do (grip, stopping distance), while current AD systems mostly react only after conditions have already changed.
A LoRA-finetuned VLM reads the front-camera image to report road surface and visibility conditions, which parametrizes a context-adaptive safety envelope (CASE) grounded in physical limits and responsibility-sensitive safety guarantees — coupling braking and steering through a shared friction budget rather than treating them independently.
Keeps the VLM asynchronous so it never blocks the real-time MPC control loop, addressing the classic tension between heavyweight scene reasoning and hard real-time control.
In closed-loop CARLA testing across varied weather, road-surface, and lighting conditions, VLM-CASE-MPC completes all trials and outperforms both a conventional MPC baseline and a state-of-the-art VLM-integrated controller, with ablations confirming the friction and visibility adaptations are complementary rather than redundant.

Show abstract

Adverse driving conditions, such as bad weather, remain a principal barrier to autonomous driving because they degrade two things at once: what the vehicle can perceive and what it can physically do. Human drivers cope by anticipation, reasoning about the scene and re-budgeting speed, following distance, and steering before grip or sight is lost, whereas current autonomous driving systems at best react after the fact. This paper proposes VLM-CASE, a framework that gives an autonomous vehicle this anticipatory capacity while keeping its motion bounded by a formal safety model at all times. A vision-language model (VLM), fine-tuned with low-rank adaptation (LoRA), reasons about the scene from the front-camera image and reports the road surface and visibility conditions. This output parametrizes a context-adaptive safety envelope (CASE), derived from physical limits and the guarantees of responsibility-sensitive safety, that couples braking and steering through a shared friction budget. A model predictive controller then drives freely within the envelope, while the VLM runs asynchronously so it never blocks the real-time control loop. We validate the framework in closed-loop CARLA simulation on tasks that demand both lateral and longitudinal control, across a range of weather, road-surface, and lighting conditions. The resulting controller, VLM-CASE-MPC, completes all trials, outperforming a conventional MPC baseline and a state-of-the-art VLM-integrated controller. Ablations confirm that the gains come from context adaptation, with the friction and visibility adaptations proving complementary. Furthermore, the framework is controller-agnostic and pairs with almost any low-level controller, offering a promising direction for safe autonomous driving. The dataset and supplementary materials for VLM-CASE are available at https://github.com/ytj254/VLM-CASE.

#18 h=N/A

Real-World Perturbation Testing of Autonomous Driving Systems

2026-07-06 cs.SEcs.RO

Stefano Carlo Lambertenghi, Matthias Weil, Andrea Stocco

Core Contributions

Directly interrogates whether perturbation-based robustness testing — mostly done offline or in simulation — actually predicts real-world autonomous driving failures, a validity question the field has largely assumed rather than tested at scale.
Runs 72 camera and LiDAR perturbations across three testing modalities (offline model-level analysis, hardware-in-the-loop, and closed-loop system-level testing on a full-scale AV), covering both an end-to-end vision model and a modular LiDAR perception-and-planning stack — a deliberately broad comparison to expose modality-specific gaps.
Finds a genuine disconnect: camera perturbations with limited offline impact can still cause unstable control and real-world failures, while LiDAR degradation is more consistent at the perception level but weakly predictive of system-level failure — meaning model-level metrics alone can't identify the most harmful perturbations for either sensor modality.
Also flags that real-time feasibility constrains what's even testable in the real world, and that robustness observations from recorded data don't consistently transfer to closed-loop behavior on physical vehicles — a cautionary result for anyone validating AD robustness purely offline, relevant to VLM-CASE and RCT-AD's reliability claims elsewhere in this batch.

Show abstract

Autonomous Driving Systems (ADS) must operate reliably under diverse conditions, yet representative data for rare or adverse scenarios is difficult to obtain. Perturbation-based testing is widely used to assess robustness, but most studies focus on offline datasets or simulation, leaving open questions about how such results translate to real-world driving. We present a large-scale study of 72 camera and LiDAR perturbations, evaluated across three testing modalities: offline model-level analysis, hardware-in-the-loop execution, and closed-loop system-level testing on a full-scale autonomous vehicle. The study covers both an end-to-end vision-based driving model and a modular LiDAR-based perception and planning stack. Our results reveal a clear gap between testing levels. For camera-based systems, perturbations with limited offline impact can still induce unstable control and failures in real-world driving. For LiDAR-based systems, degradation is more consistent at the perception level but weakly predictive of system-level failures. Across both modalities, model-level metrics alone are insufficient to identify the most harmful perturbations. We further show that real-time feasibility is a key constraint in real-world testing, and that robustness observations obtained from recorded data do not consistently transfer to closed-loop behavior on a physical vehicle, highlighting the importance of complementary real-world, system-level evaluation.

#30 h=N/A

A Reliable Context-Aware and Temporal Planning Framework for Autonomous Driving

2026-07-06 cs.ROcs.CV

Argho Dey, Yunfei Yin, Swachha Ray, Md Minhazul Islam, Zheng Yuan

Core Contributions

Targets a concrete safety problem in BEV-based autonomous driving: most temporal-fusion approaches aggregate camera features across frames without assessing whether each frame's observation is actually trustworthy, so occlusion, motion blur, or sensor noise silently destabilizes trajectory planning.
The Reliable Context Awareness module scores per-frame reliability and selectively retains trustworthy features through a quality-gated FILO memory, reconstructing degraded observations from reliable historical context — rather than either discarding degraded frames outright or blindly fusing them in.
A Temporal Trajectory Planner captures long-term dependencies and multi-agent interactions on top of this reliability-gated representation, while a joint detection-and-segmentation head injects semantic and motion cues into the shared BEV space to strengthen scene understanding.
On nuScenes, reports 61.5 Detection Score, 52.9 mAP, and 52.3 mIoU while maintaining real-time-suitable computational efficiency — concrete gains over recent end-to-end baselines in perception accuracy, motion prediction, and planning robustness simultaneously.

Show abstract

Safe operation of autonomous vehicles in dense urban traffic depends on perception and planning that remain reliable when onboard sensing is degraded. In real driving conditions, camera observations are frequently corrupted by occlusion, motion blur, illumination change, and sensor noise, and when such degraded observations are aggregated indiscriminately over time, trajectory planning becomes unstable and collision risk rises for both the ego vehicle and surrounding road users. Recent Bird's-Eye-View (BEV) approaches unify perception and planning through a shared spatial representation, but most fuse temporal information across frames without assessing the reliability of the underlying observations. We present a Reliable Context-Aware and Temporal Planning framework for Autonomous Driving (RCT-AD) that explicitly models feature quality and temporal consistency to support safer, more consistent planning. A Reliable Context Awareness module scores per-frame reliability and selectively retains trustworthy features through a quality-gated First-In-Last-Out (FILO) memory mechanism, reconstructing degraded observations from reliable historical context so that corrupted inputs do not destabilize the scene representation. A Temporal Trajectory Planner captures long-term dependencies and multi-agent interactions to produce smoother, safety-aware trajectories, while a joint detection-and-segmentation head injects semantic and motion cues into the shared BEV space to strengthen scene understanding. Experiments on the nuScenes autonomous driving benchmark show that RCT-AD improves perception accuracy, motion prediction, and planning robustness over recent end-to-end baselines, achieving 61.5 nuScenes Detection Score, 52.9 mean Average Precision, and 52.3 mean Intersection over Union, while maintaining competitive computational efficiency suitable for real-time deployment.

Human-Robot Interaction & Social Robots

#7 h=N/A

Socially-Aware Autonomous Doorway Traversal and Payload Delivery for Emergency Assistance

2026-07-06 cs.RO

Andrew Snowdy, Ananya Trivedi, Sarvesh Prajapati, Lorena Maria Genua, Taskin Padir

Core Contributions

Tackles a compound systems problem for emergency evacuation robots: opening ADA-compliant doors to keep exit paths clear while also locating and delivering rescue equipment, all while remaining socially aware of evacuees rather than obstructing them.
Uses a behavior tree as the core architecture specifically because it can select among high-level tasks based on environmental triggers and extend to new situations, rather than hard-coding a fixed task sequence — a deliberate choice for handling the branching decisions evacuation scenarios demand (yield to a person? walk through a held door? fetch equipment first?).
Evaluated with real breadth: 105 trials across five hardware and three simulation scenarios on the Toyota Human Support Robot, not a narrow single-scenario demo.
Completes 97 of 105 trials successfully (92%), a concrete reliability number for a socially embedded task mix that mixes navigation, manipulation, and human-aware yielding decisions in one pipeline.

Show abstract

In this work, we focus on the scenario of a robot-assisted emergency evacuation. We consider two capabilities relevant to such a setting. The first is opening doors ahead of the people being evacuated, so that their path toward an exit stays clear. The second is retrieving rescue equipment and delivering it to the emergency responders carrying out the evacuation. From a systems perspective, this involves several tasks at once. The robot must locate ADA-compliant door buttons and the rescue equipment it needs to retrieve. Additionally, it must remain aware of the people around it and adapt its behavior to them, so that it supports the evacuation rather than getting in the way. We address these demands with a behavior tree at the core of our framework. This structure is chosen for its ability to select high-level tasks based on environmental triggers, and to extend to new situations as they arise. We evaluate the system in 105 trials on the Toyota Human Support Robot, across five hardware and three simulation scenarios. These trials capture the decisions the robot must make in this setting: whether to press a door button, yield to a nearby person, walk through a door someone else is holding, or first retrieve rescue equipment before traversing the door. Overall, the system completes 97 of the 105 trials successfully. These results suggest our framework provides a practical basis for robotic assistance in broader emergency response tasks. Code and video demonstrations are available at https://github.com/AndrewSnowdy/hsr_mm_control.

#12 h=N/A

Toward Personalized Social Robots for Child Well-being: Data Requirement Principles from a Recommender-System Perspective

2026-07-06 cs.HCcs.RO

Jin Huang, Eric Nichols, Fethiye Irmak Dogan, Hatice Gunes

Core Contributions

Frames personalizing social robots for child well-being as a recommendation problem — choosing the best action per child — and argues the bottleneck isn't the model but the data required to instantiate a recommender-system framework in this domain.
Lays out why standard recommender data assumptions break for this population: a child's state shifts within and across visits so no fixed user profile holds, within-session feedback signals are weak and indirect, children are rarely seen more than once so identity-linking across sessions is hard, and care can't be randomized so all data is observationally biased.
Proposes four concrete data principles in response — an integrated profile, effectiveness signals, linkable coverage, and an exposure record logged at collection time — mapped explicitly to which recommender-system capability each principle unlocks.
Positions itself as actionable data-collection guidance rather than a new algorithm, useful for groups deploying social robots in clinical settings who need a checklist before personalization is even feasible.

Show abstract

Social robots are increasingly deployed in clinical settings to support the well-being of children, where effective support must be personalized to each child. Personalization, choosing the robot action best suited to each child, can be framed as a recommendation problem, and a recently proposed recommender-system framework for social robots offers a principled approach through user profiling, ranking, and responsible computing. Instantiating it, however, is blocked not by the model but by the data, which is hard to gather. A child's state shifts within and across visits, so no fixed description of the user holds. Within a session, the few signals of whether the robot's actions helped are weak and indirect. Across sessions, children are rarely seen more than once, and anonymization breaks the identity needed to link visits. Because care cannot be randomized, existing data is observational, biased toward whatever was already done. Each is a familiar recommender-system problem, and we propose four data principles in response: an integrated profile, effectiveness signals, linkable coverage, and an exposure record logged at collection time. We identify which of these principles each capability requires, and frame them as concrete guidelines for data collection.

#15 h=N/A

Designing Touch for Trauma-Informed Social Robots: A Design Space for Direct and Indirect Actuation

2026-07-06 cs.RO

Madeleine Rischer, Benedikt Bußmann

Core Contributions

Addresses a design gap in trauma-informed HRI: touch is a promising modality for grounding and emotional regulation in therapeutic settings, but PTSD-specific safety, transparency, autonomy, and trigger-avoidance requirements aren't yet built into how robot touch actuation is designed.
Distinguishes direct touch (physical robot-user contact) from indirect touch (mediated through wearables or smart textiles), treating them as structurally different design problems rather than variants of the same interaction.
Proposes a three-dimensional design space — Actuation Modality, Objective of the Actuation, and Intended Effect — and analyzes each dimension explicitly through trauma-informed-care principles rather than generic HRI comfort criteria.
Delivers a conceptual framework rather than a deployed system, intended as a foundation other groups can build trauma-sensitive touch interactions against, particularly for PTSD populations underserved by generic social-robot touch research.

Show abstract

Touch is a fundamental communication modality in human-robot interaction and may support grounding, emotional regulation, and stress reduction in therapeutic contexts. However, designing touch-based interactions for individuals with post-traumatic stress disorder (PTSD) requires careful consideration of trauma-informed care (TIC) principles, including safety, transparency, autonomy, and trigger avoidance. This paper investigates how touch actuation in social robots can be designed to align with trauma-informed care. We distinguish between direct touch, involving physical contact between a user and a robot, and indirect touch, mediated through artifacts such as wearables or smart textiles. Building on this distinction, we propose a design space comprising three dimensions: Actuation Modality, Objective of the Actuation, and Intended Effect of the Actuation. We analyze these dimensions through the lens of trauma-informed care and derive key design considerations for touch-based interventions. The resulting approach provides a conceptual foundation for developing trauma-sensitive social robots that support individuals with PTSD through touch-based interactions.

Task Planning & Multi-Agent Coordination

#5 h=N/A

GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks

2026-07-06 cs.ROcs.AIcs.CLcs.LG

Kaiyuan Chen, Shuangyu Xie, Letian Fu, Justin Yu, William Pacini

Core Contributions

Poses a pointed question: can agentic coding systems combine interpretable robot programming (TAMP/ROS-style) with the open-world adaptability of model-free policies, rather than treating them as competing paradigms?
GaP has a multi-agent harness generate directed computation graphs with perception/planning/control nodes from a Modular Open Robot Skill Library, then rehearses candidate graphs in a generated internal simulator to iteratively refine structure and parameters — search over programs, not just over policy weights.
Targets "Variational Automation": tasks with more object geometry/pose variation than fixed automation but that still demand industrial-grade reliability, a middle ground model-free policies often can't close.
Across 8 new VA benchmarks (4 simulated, 4 real-world), reports success rates significantly above baselines, suggesting graph-structured, simulation-rehearsed programs generalize better than end-to-end policies for this reliability-critical task class.

Show abstract

For robots to work reliably in commercial and industrial applications, can recent advances in agentic coding systems combine interpretable robot programming with the open-world adaptability of model-free policies? We focus on "Variational Automation" (VA), a class of tasks that have larger variations in object geometry and pose than fixed automation. Model-free policies often struggle to close the reliability gap for VA tasks, which must be executed persistently and reliably in commercial and industrial applications. Motivated by prior work on Task and Motion Planning (TAMP) and the Robot Operating System (ROS), we introduce Graph-as-Policy (GaP), a multi-agent coding harness that generates directed computation graphs with perception, planning, and control nodes from a Modular Open Robot Skill Library (MORSL). GaP then generates an internal simulation environment to rehearse task instances with different graphs in parallel to iteratively refine the graph structure and parameters to improve success rates and throughput. Evaluation with 8 new open VA task benchmarks, 4 in-simulation and 4 in real-world, suggests that GaP can achieve success rates that significantly outperform baselines. Details, code, and data can be found online: https://graph-robots.github.io/gap

#17 h=N/A

Multi-Robot Open Adaptive Teaming Across Unseen Environments, Partners, and Scales

2026-07-06 cs.ROcs.AI

Yang Li, Feng Xue, Fan Mo, Yunhao Liu, Jianhong Wang

Core Contributions

Formalizes multi-robot teaming under simultaneous, not isolated, uncertainty — unseen environments, unknown partners, and varying team sizes together — arguing that prior work's closed-world assumption of fixed teammates doesn't hold once any one of these three varies within an episode.
Proposes a hypergraphic-form game formulation to capture team-level cooperative relationships beyond pairwise interactions, explicitly distinguishing this game-theoretic strategic/payoff construct from graph neural network architectures that only model structural relations.
The Hypergraphic Open-ended Learning Algorithm (HOLA) progressively expands partner and environment diversity during training instead of optimizing against a fixed configuration, targeting robustness to compositions never seen during training rather than to a specific benchmark distribution.
Outperforms all baselines across all three adaptability dimensions on cooperative pursuit with multi-drone and multi-quadruped platforms, and — notably — transfers learned policies directly to Crazyflie and Zsibot L1 hardware with no fine-tuning, evidence the hypergraph formulation isn't just a simulation artifact.

Show abstract

Deploying robot teams in the real world requires simultaneous adaptation to unseen environments, unknown partners, and varying team sizes, yet existing approaches often address these challenges in isolation under the closed-world assumption of fixed teammates. We formalize this as open adaptive multi-robot teaming and propose a hypergraphic-form game formulation that captures team-level cooperative relationships beyond pairwise interactions, providing a principled foundation for coordination structure inference when team composition changes dynamically within episodes. Unlike graph neural network architectures, this is a game-theoretic construct for modeling strategic interactions and payoff structures among agents. Building on this formulation, we develop the Hypergraphic Open-ended Learning Algorithm (HOLA), which progressively expands partner and environment diversity during training rather than optimizing for fixed configurations. Evaluated on cooperative pursuit with multi-drone and multi-quadruped platforms, HOLA outperforms all baselines across all three adaptability dimensions. Learned policies transfer directly to physical hardware without fine-tuning, with successful deployments on Crazyflie and Zsibot L1 platforms confirming robust real-world coordination in novel environments with unseen teammates.

Hardware, Control & Locomotion

#8 h=N/A

GelNeuro: A Sensing-Computing Integrated Neuromorphic Tactile System for Texture Recognition

2026-07-06 cs.RO

Luoyang Bian, Xinpan Meng, Zhenghua Ma, Houcheng Li, Long Cheng

Core Contributions

Addresses a real system bottleneck in neuromorphic tactile sensing: most prior systems still route events through a host computer for readout or preprocessing before chip inference, undermining the low-latency/low-power promise of neuromorphic hardware in the first place.
GelNeuro pairs a GelSight Mini optical tactile front end directly with the Speck2f neuromorphic SoC, converting marker-motion-induced contact into DVS events processed entirely on-chip by a spiking CNN — sensor-to-chip with no host computer in the loop.
Introduces hardware-aware weight clamping specifically to prevent accuracy loss from 8-bit on-chip deployment, addressing a known quantization pain point for spiking networks.
Reports 96.3% accuracy on 15-class texture recognition within an 80ms window at just 19.6mW of board-level active power — over three orders of magnitude below conventional CPU/GPU baselines on the same task — and generalizes to unseen contact depths.

Show abstract

Neuromorphic visuo-tactile sensing offers a promising paradigm for low-latency and low-power robotic perception. However, existing systems still rely heavily on a host computer for event readout, preprocessing, or relaying prior to chip inference. This paper presents GelNeuro, a fully integrated sensing-computing visuo-tactile system that directly pairs a GelSight Mini-based optical tactile front end with the Speck2f neuromorphic system-on-chip (SoC). Contact-induced marker motions are captured as dynamic vision sensor (DVS) events and routed through the on-chip network to a spiking convolutional neural network (SCNN) classifier. To mitigate accuracy degradation during 8-bit deployment, a hardware-aware weight clamping strategy is introduced. Evaluated on a 15-class natural texture recognition task, hardware-in-the-loop testing on the physical chip achieves a 96.3% accuracy within an 80 ms inference window. Notably, the system consumes only 19.6 mW of board-level active power-over three orders of magnitude lower than conventional CPU/GPU baselines on the same benchmark. GelNeuro also exhibits robust generalization across unseen contact depths, demonstrating the viability of direct sensor-to-chip tactile recognition on edge neuromorphic hardware.

#23 h=N/A

Athena-WBC: Capability-Aligned Policy Experts for Long-Tail Humanoid Whole-Body Control

2026-07-06 cs.RO

Yuan Jiang, Ningyuan Zhang, Xicun Yang, Shidi Li, Yuzhi Jiang

Core Contributions

Challenges the standard fix for hard humanoid whole-body-control motions (sample harder clips more, isolate into subsets, assign specialized experts): even under targeted training, a residual set of feasible motions — especially high-dynamic transitions and balance-critical cases — stays unsolved.
Diagnoses the real cause as a capability mismatch: the default training recipe's effective capability doesn't match what certain motions demand, not just insufficient training exposure — reframing a data problem as an objective-design problem.
Athena-WBC uses capability-aligned experts: dynamic experts get a tracking-focused, constraint-aware objective that strips out conservative effort and temporal-control penalties while keeping physical feasibility constraints, while balance experts use a gravity curriculum to improve early-training survivability specifically for balance-critical clips.
Distills these privileged, motion-routed teachers into a single deployable controller via DAgger followed by RL fine-tuning, and on a full-size humanoid shows improved recovery of training-set long-tail motions and better held-out tracking than a strong SONIC-recipe baseline, using only a small number of experts rather than one expert per failure mode.

Show abstract

Large-scale humanoid motion-tracking controllers are commonly improved by reallocating training effort: difficult motions are sampled more often, isolated into smaller subsets, or assigned to specialized experts. We show that this view is incomplete. In strong whole-body-control baselines, a residual set of feasible training clips remains unsolved even under targeted training, especially for high-dynamic transitions and balance-critical motions. These failures arise not only from insufficient exposure, but from a mismatch between the motion demands and the effective capability induced by the default training recipe. We propose Athena-WBC, a compact teacher-student pipeline with capability-aligned policy experts for long-tail humanoid whole-body control. Dynamic experts use a tracking-focused, constraint-aware objective that removes conservative effort and temporal-control penalties while preserving physical feasibility constraints; balance experts use a gravity curriculum to improve early-training survivability. The resulting privileged teachers are motion-routed for DAgger distillation and then compressed into a single controller with deployable observations followed by RL fine-tuning. Experiments on a full-size humanoid show improved recovery of training-set long-tail motions and better held-out tracking than a strong SONIC-recipe baseline, using only a small number of experts.

#28 h=N/A

Aerial Manipulation: Contact, Medium Coupling, and the Geometry of Readiness

2026-07-06 cs.ROeess.SY

Antonio Franchi

Core Contributions

Argues against treating aerial manipulation as classical manipulation simply bolted onto a flying base — because flying agents stay aloft through continuous momentum and energy exchange with the surrounding medium, support, locomotion, stabilization, and task-directed interaction are intrinsically coupled rather than separable subsystems.
Proposes a medium-aware interpretation in which interaction can be mediated by contact, by the surrounding fluid, or both simultaneously — broadening the design space beyond pure contact mechanics to include aerodynamic coupling as a first-class interaction channel.
Organizes biological and robotic examples into a repertoire of interaction modes and a capability ladder, then develops an actuation-geometric viewpoint where redundancy induces task-equivalent fibers, along which internal motion can trade energy for active readiness, aerodynamic promptness, and passive medium coupling.
As a review, its contribution is conceptual rather than empirical: it explains why aerial manipulation stays harder than ground manipulation, why biological flyers still outperform robotic ones in this domain, and sketches how future platforms might not just command forces but also shape how the medium acts back on them.

Show abstract

Aerial robots are increasingly moving from remote observation toward physical interaction with objects, surfaces, structures, loads, and surrounding flows. This review argues that aerial manipulation cannot be understood as classical manipulation simply mounted on a flying base. Because flying agents remain aloft through continuous momentum and energy exchange with the surrounding medium, support, locomotion, stabilization, and task-directed interaction are intrinsically coupled. Building on broad views of manipulation as intentional environmental regulation through physical interaction, we propose a medium-aware interpretation of aerial manipulation in which interaction may be mediated by contact, by the surrounding fluid, or by both. The review organizes biological and robotic examples into a repertoire of interaction modes and a capability ladder, then develops an actuation-geometric viewpoint in which redundancy induces task-equivalent fibers. Internal motion along these fibers can trade energy for active readiness, aerodynamic promptness, and passive medium coupling. This perspective clarifies why aerial manipulation is difficult, why biological flyers remain broader than robotic systems, and how future platforms may command forces while also shaping how the medium acts back on them.