🤖 Robotics arXiv Digest

Tuesday, July 7, 2026

📄 30 papers 📂 6 research areas Generated by Claude

🔭 Research Landscape

If today's 30 papers share a headline, it's that VLA acceleration is shifting from bigger pretraining to cheap post-hoc interventions. OTQL (#1) fine-tunes flow policies with just 50-60 interaction episodes, pushing single-task success from 36% to 86% while cutting inference steps by 70%; ActionCache (#12) gets up to 34.43x speedup on GR00T-N1.6 with zero retraining at all, just a reusable cache of past denoising states; and SIEVE (#17) shows a policy can match full-data training using only 50% of demonstrations once redundant trajectories are pruned by compositional structure rather than scored individually. These three attack the same VLA cost problem — slow, data-hungry policies — from three different angles (RL fine-tuning, inference caching, data curation) without any of them retraining a foundation model from scratch, a notable contrast to LingBot-VLA 2.0 (#15), which takes the opposite bet and scales to 60,000 hours of cross-embodiment pretraining data.

A second, closely related thread is explicit geometric grounding returning to problems that had drifted toward flat 2D or latent representations. Lift3D-VLA (#14) aligns raw 3D point clouds with pretrained 2D positional embeddings so a VLA's vision encoder can reason about geometry and future dynamics jointly, gaining 10-11 points of success on MetaWorld/RLBench specifically under out-of-distribution perturbations. EAGOR (#2) makes the analogous move for omnidirectional navigation, replacing equirectangular-projection reasoning with a spherical-harmonic belief field to eliminate seam and latitude distortion. UniLM-Nav (#11) grounds affordance points and lifts them into a robot-centric frame for last-mile navigation. The common wager across all three: representations that respect the underlying geometry generalize better than ones that treat images as flat 2D arrays, and the OOD-specific gains in #14 are direct evidence for it.

A third thread, less about new capability and more about intellectual honesty, runs through this batch's diagnostic papers. "Why does Deep Learning Improve Visual SLAM?" (#9) isolates learned 2D data association and uncertainty — not the recurrent architecture — as the actual source of DL-SLAM's advantage. "Imagined Rollouts are Kinematic, Not Dynamic" (#24) shows a DreamerV3 world model's imagination error stays flat across a friction sweep even as the real policy's reward collapses through a gait-collapse boundary, meaning the model's imagined rollouts never "notice" the physics changing. The semantic-handoff paper (#18) quantifies how skills hitting 77-100% success in isolation still stall when chained from real prior-skill end states. And the fisheye calibration paper (#22) traces persistent calibration failures to a specific ill-conditioning mechanism rather than detector quality, lifting success from 68.1% to 99.3% with smarter frame selection alone. None of these four propose a new architecture; all four instead give the field a sharper, falsifiable account of why existing systems fail, which is arguably more useful right now than another benchmark-topping method.

VLA & Foundation Models

Post-training, caching, and structure-aware data selection are replacing bigger pretraining runs as the preferred lever for fixing and accelerating VLA policies.

World Models, Simulation & Data Generation

Neural simulators and generative world models as scalable substrates for training and data collection, alongside a sharp diagnosis of why long-horizon imagination still fails.

#3 Image2Sim
#4 RoboTALES
#20 RynnWorld-Teleop
#24 Kinematic World-Model Diagnosis

Navigation, SLAM & Perception

Geometric grounding (spherical, 3D, floor-plan-referenced) and root-cause failure analysis push classical navigation and mapping pipelines past their 2D and single-sensor assumptions.

Manipulation & Control

Structural fixes to sampling-based control and teleoperation — clustering, motion primitives, force-aware benchmarking — that target specific failure mechanisms rather than generic tuning.

Human-Robot Interaction & Assistive Robotics

From a deployable assistive wheelchair to an ethics framework and an acoustic data-physicalization platform, HRI work this batch spans concrete hardware and normative design guidance.

#6 Autonomous Wheelchair
#8 Responsible Personalisation
#23 AcoustoBots

Benchmarking, Validation & Interpretability

Infrastructure for trustworthy evaluation — scenario-scale validation, physically-grounded simulation, causal interpretability, and open-world hypothesis testing.

#21 Hypothesis-Driven Planning
#27 RoboVAST
#29 OrchardBench
#30 Driving the Wrong Way

VLA & Foundation Models

#1 h=50

Optimal Transport Q-Learning for Flow Policy Steering and Acceleration

2026-07-07 cs.RO 👤 S. Vijayakumar (h=50)

Andreas Sochopoulos, Esmeralda S. Whitammer, Nikolaos Tsagkas, João Moura, Michael Gienger

Core Contributions

RL post-training approach for flow/diffusion policies via advantage-weighted conditional optimal transport flow matching, explicitly avoiding the expensive distillation step that's become a default requirement for accelerating flow-based policies.
Needs only 50-60 interaction episodes to retrain, framing acceleration as a cheap post-training problem rather than a full retrain from a large offline dataset.
Quant: single-task success rises 36% to 86%, a pretrained VLA's success rises 38% to 76%, while inference steps per action drop 70% — fixing suboptimality and latency in the same pass rather than trading one for the other.
Suggests the robot's own rollout experience, not more human demonstrations, is what closes both the performance and speed gaps in flow-based VLA pipelines.

Show abstract

Diffusion and flow policies have recently demonstrated remarkable performance in robotic applications by accurately capturing multimodal robot trajectory distributions, especially in the context of vision language action (VLA) models. However, high quality policy performance also requires fast inference and high quality demonstrations, which are often hard to get. Lack of these leads to suboptimal policy behaviors and failure under distribution shifts. In this work we address the problem of fine-tuning and accelerating suboptimal flow-based policies using the robot's experience through RL post-training. We introduce Optimal Transport Q-Learning (OTQL), a new method for finetuning flow policies using advantage weighted conditional optimal transport flow matching. OTQL can finetune and accelerate flows with an interaction budget of 50-60 episodes while avoiding computationally expensive distillation in simulation and real-world robot tasks. Our results show that OTQL post-trains flow policies using the robot's own experience, increasing average success percentage of single-task policies from 36% to 86% and of a pre-trained VLA from 38% to 76% while reducing the number of inference steps per action generation by 70%.

#12 h=10

Training-Free Acceleration for Vision-Language-Action Models with Action Caching and Refinement

2026-07-07 cs.ROcs.CVcs.LG 👤 Daichi Fujiki (h=10)

Ryuji Oi, Hikari Otsuka, Kosuke Matsushima, Yuki Ichikawa, Masato Motomura

Core Contributions

Plug-and-play, training-free cache for flow-matching VLA action heads: stores intermediate denoising states keyed by compact multimodal context and reuses them across episodes or even different tasks to warm-start generation instead of denoising from scratch.
No retraining or architecture change required — directly deployable on existing flow-based VLA checkpoints, unlike distillation-based acceleration that needs retraining per model.
Quant: up to 11.75x speedup on π0.5 and 34.43x on GR00T-N1.6 while maintaining high success in a low-latency regime — the large gap between the two suggests cache hit-rate depends heavily on how the action head represents its denoising trajectory.
Directly complements #1: where OTQL reduces the number of required denoising steps via RL, ActionCache reduces the cost of each step via reuse — two compatible attacks on the same VLA inference bottleneck.

Show abstract

Vision-Language-Action (VLA) models have emerged as a promising approach for generalizable robotic manipulations. In particular, flow matching-based VLA models have shown remarkable success due to their capability to generate precise and smooth action sequences and capture multimodal distributions. However, the iterative denoising process in the action head acts as a major computational bottleneck, posing a critical challenge for real-time deployment. To address this challenge, we propose ActionCache, a plug-and-play external cache that opportunistically reuses past intermediate actions to warm-start generations from the vicinity of target actions, thereby drastically reducing the inference latency. Specifically, ActionCache stores the intermediate actions with compact multimodal keys, which enables retrieval from similar past contexts across different episodes or even different tasks. Experimental results in simulation and real-world environments demonstrate that ActionCache maintains high task success rates in a low-latency regime, achieving inference acceleration of up to $11.75\times$ and $34.43\times$ for representative flow-based VLA models, $π_{0.5}$ and GR00T-N1.6, respectively.

#14 h=9

Lift3D-VLA: Lifting VLA Models to 3D Geometry and Dynamics-Aware Manipulation

2026-07-07 cs.ROcs.CV 👤 Chenyang Gu (h=9)

Jiaming Liu, Qingpo Wuwu, Nuowei Han, Hao Chen, Zhuoyang Liu

Core Contributions

Directly addresses geometric information loss in existing 3D-VLA encoders by aligning raw point clouds with pretrained 2D positional embeddings, allowing point-cloud encoding inside the same vision encoder used for 2D pretraining rather than a bolted-on 3D branch.
Introduces Geometry-Centric Masked Autoencoding, a dual-objective pretext task that reconstructs the current point cloud and predicts its future geometric evolution — teaching the encoder structure and physical dynamics from one self-supervised signal.
Layer-wise temporal action modeling spreads action-chunk prediction across multiple LLM layers for temporal coherence, rather than predicting the whole chunk from a single final layer.
Quant: +10.8%/+11.1% mean success over the best prior VLA on MetaWorld/RLBench, +4pp over the strongest real-world baseline, with stronger OOD-perturbation generalization — geometric grounding specifically helps distribution shift, not just in-distribution accuracy.

Show abstract

Recently, Vision-Language-Action (VLA) models have demonstrated strong generalization across diverse tasks. However, effective robotic manipulation in physical environments fundamentally requires geometric understanding and spatial reasoning. While some VLA approaches attempt to incorporate 3D information, they are constrained by limited data availability and geometric information loss in current 3D encoding pipelines, and fail to jointly capture 3D geometry and temporally structured actions in dynamic environments. To address these limitations, we introduce Lift3D-VLA, a unified VLA framework that equips models with explicit 3D point cloud reasoning and enables temporally coherent action generation. First, building upon our previous work Lift3D, an enhanced 2D model-lifting strategy is proposed to geometrically align 3D points with pretrained 2D positional embeddings. This design enables direct point-cloud encoding within the VLA vision encoder while minimizing spatial information loss. Based on explicit 3D inputs, we propose Geometry-Centric Masked Autoencoding (GC-MAE), a dual-objective self-supervised framework that reconstructs the current point cloud while predicting its future geometric evolution. This formulation allows the 2D vision encoder to internalize both 3D structure and physical dynamics. To fully exploit 3D representations, we further design layer-wise temporal action modeling, which leverages multiple layers of the LLM to collaboratively predict action chunks, enabling temporally consistent predictions. Across 22 simulated tasks and 8 real-world manipulation tasks, Lift3D-VLA achieves 10.8% and 11.1% higher mean success rates on MetaWorld and RLBench than the best-performing prior VLA methods, and outperforms the strongest real-world baseline by 4 percentage points, while exhibiting stronger generalization to out-of-distribution perturbations.

#15 h=9

From Foundation to Application: Improving VLA Models in Practice

2026-07-07 cs.RO 👤 Shuailei Ma (h=9)

Wei Wu, Fangjing Wang, Fan Lu, He Sun, Shi Liu

Core Contributions

Scales pretraining to ~60,000 hours (50K robot trajectories across 20 embodiments plus 10K egocentric human videos) specifically to test whether cross-embodiment coverage, not just raw data volume, drives generalization.
Expands the action space beyond dual-arm platforms to heads, waists, mobile bases, and dexterous hands, pushing VLA scope from tabletop manipulation toward whole-body mobile manipulation.
Uses future prediction as an auxiliary proxy task, fed by separate video-representation and depth-estimation priors, rather than training temporal understanding purely from the action-prediction loss.
Demonstrates cross-embodiment long-horizon mobile manipulation across two distinct robot platforms on the GM-100 generalist benchmark — the payoff is transfer across embodiments, not just a benchmark score on one.

Show abstract

Despite recent progress of VLA foundation models, the disparity between laboratory conditions and real-world applications continues to impede their practical implementation. To bridge this gap, we present LingBot-VLA 2.0, which advances LingBot-VLA through improvements in three functional domains. (1) Generalization across tasks and embodiments. Compared to the previous version, we revamp the data processing pipeline and curate around 60,000 hours of data for pretraining, including 50,000 hours of robot trajectories spanning 20 robot configurations and 10,000 hours of egocentric human videos. (2) Expanded action space in addition to dual-arm hardware platforms. In particular, our system accommodates degrees of freedom for the heads, waists, mobile bases, and dexterous hands, thereby empowering the robots to tackle more complex tasks in practical scenarios. (3) Predictive dynamics modeling for improved temporal reasoning. Specifically, we formulate future prediction as a proxy task, facilitated by a video representation model for semantic priors and a depth estimation model for geometric cues. Evaluations on the GM-100 benchmark, conducted in a generalist setting, validate the beneficial impact of these proposed modifications. Furthermore, benefiting from the expanded pretraining data that covers whole-body degrees of freedom, LingBot-VLA-2.0 demonstrates strong cross-embodiment long-horizon mobile manipulation capability across the two robotic platforms.

#17 h=7

SIEVE: Structure-Aware Data Selection for Imitation Learning with VLA Models

2026-07-07 cs.RO 👤 Shijie Lian (h=7)

Changti Wu, Bin Yu, Zhaolong Shen, Shijie Lian, Xiaopeng Lin

Core Contributions

Reframes imitation-learning data selection around reusable motion primitives and their transition interfaces, rather than scoring whole trajectories or individual state-action pairs — capturing compositional structure that flat scoring methods miss.
Allocates a selection budget across composition patterns under a diminishing-returns objective, then picks medoid (central, stable) trajectories within each bucket — an explicit anti-redundancy mechanism rather than simple top-k scoring.
Quant: matches or surpasses full-data training using only 50% of demonstrations and 50% of training steps — redundant data isn't just wasteful, it can be actively pruned without hurting the policy.
Cross-cutting with #15's data-scaling story: SIEVE argues coverage of reusable structure, not raw hours, is the real scaling lever for VLA imitation learning.

Show abstract

Vision-Language-Action (VLA) models are typically trained by imitation learning on large-scale robot demonstration datasets, but more data does not necessarily yield better policies due to redundancy, noise, and uneven coverage. Existing data selection methods often assess demonstrations at either the trajectory or state-action level, missing the reusable structures that compose long-horizon behaviors. In this paper, we propose SIEVE, a structure-aware data selection method for VLA imitation learning. SIEVE views demonstrations as compositions of reusable primitives and transition interfaces. It first discovers visuo-motor primitives from segmented trajectories, then allocates selection budgets to composition patterns by maximizing reuse-aware structural exposure under diminishing returns. Finally, it selects medoid trajectories within each composition-pattern bucket to retain central, stable, and imitation-friendly demonstrations. Experiments across multiple datasets, benchmarks, and VLA models show that SIEVE consistently outperforms competitive data selection baselines. Notably, SIEVE can surpass full-data training while using only 50% of demonstrations and 50% of training steps, suggesting that reusable structure, captured through primitives and transitions, is an important signal for efficient VLA imitation learning.

#18 h=7

Diagnosing Semantic Handoff Failures in Agent-Orchestrated Vision-Language-Action Skill Composition

2026-07-07 cs.RO 👤 Weitao Zhou (h=7)

Ke Rui, Yushen Zuo, Jiawei Wang, Haoran Jia, Jinming Ma

Core Contributions

Distinguishes isolated skill competence (measured from clean, hand-designed skill-boundary states) from compositional robustness (measured from real chained states a prior skill actually produced) — a distinction most long-horizon VLA benchmarks collapse into a single end-to-end number.
Uses multi-view VLM verification as an agent orchestrator deciding advance/retry/replan between π0.5-based skill checkpoints, rather than assuming skills chain automatically once each is competent alone.
Quant: individual skills hit 77-100% success from clean snapshots, yet composed rollouts stall frequently from chained states — quantifying exactly how much the clean-boundary assumption inflates apparent competence.
Turns near-zero end-to-end success into categorized diagnostics (next-skill readiness, target grounding, low-level control failures) instead of one opaque failure rate, giving concrete targets for future skill-library design.

Show abstract

Long-horizon household tasks require robots to compose many language-conditioned skills, yet the boundary between consecutive skills is rarely explicit. A skill may satisfy its own postcondition while leaving the robot, objects, or camera views in a state from which the next skill cannot reliably start. We study this semantic handoff problem in BEHAVIOR-1K through an agent-orchestrated vision-language-action execution harness. The harness invokes $π_{0.5}$-based skill checkpoints trained from cleaned BEHAVIOR-1K demonstrations, assigns each skill typed arguments and a step budget, and uses multi-view vision-language model verification to decide whether execution should advance, retry, or replan. To separate isolated skill competence from long-horizon compositional robustness, we evaluate the same checkpoints under two initial-state distributions: clean skill-boundary snapshots and chained terminal states produced by previous skills. Selected navigation, grasping, placement, and door-opening skills achieve 77--100% success from clean snapshots under human-reviewed verification, yet composed rollouts still frequently stall from chained states. Execution traces attribute these failures to next-skill readiness, target grounding, and low-level control execution, revealing a substantial gap between single-skill success and reliable long-horizon task completion. These findings turn near-zero end-to-end task success into actionable diagnostics, showing that future VLA skill libraries must learn robustness to the messy chained-state distribution that clean demonstrations systematically underrepresent.

#19 h=7

FORGE: Towards Functional Tool-Use Generalization via Keypoint Trajectory Reasoning

2026-07-07 cs.ROcs.AIcs.CV 👤 Animesh Garg (h=7)

Chuhao Zhou, Liquan Wang, Shuxin Cao, Xiangyu Chen, Yuxuan Hu

Core Contributions

Formalizes "functional generalization" — using a novel tool to accomplish the same function as a trained one — as distinct from ordinary generalization, since visually/functionally similar tools still demand entirely different motor patterns.
Empirically compares affordance images, human video prompts, and 2D keypoint trajectories as intermediate representations, finding keypoint trajectories uniquely balance functional expressiveness against groundability into robot actions.
A two-stage decoupling — predict generalizable keypoint trajectories from action-free data, then ground them into robot actions from limited demonstrations — separates the data-hungry reasoning step from the data-cheap grounding step.
Quant: over 2x average success-rate improvement over SOTA on unseen tools across a seven-tool hitting-function benchmark, in both simulation and the real world.

Show abstract

While humans readily repurpose a book, a stone, or a shoe to drive a nail, robots trained on specific tools fail to transfer the same function to novel ones -- a gap we formalize as functional generalization. Such tools share a common functional intent that is visually recognizable, yet this perceptual similarity does not carry over to action space, where each tool demands an entirely different motor pattern. To bridge this gap, we explore intermediate representations including affordance images, human video prompts, and 2D keypoint trajectories, finding that keypoint trajectories best balance functional expressiveness and action groundability. Building on this, we propose FunctiOnal Reasoning and Grounded Execution (FORGE), a two-stage policy that decouples functional reasoning from action execution: predicting generalizable keypoint trajectories from action-free data, then grounding them into robot actions with limited demonstrations. On a seven-tool hitting-function benchmark, FORGE consistently outperforms state-of-the-art methods on unseen tools in both simulation and the real world, achieving over 2X improvement in average success rate.

World Models, Simulation & Data Generation

#3 h=35

Image2Sim: Scaling Embodied Navigation via Generative Neural Simulator

2026-07-07 cs.CVcs.RO 👤 Yinghao Xu (h=35)

Zihan Wang, Seungjun Lee, Yinghao Xu, Gim Hee Lee

Core Contributions

Decouples 3D spatial anchoring (feed-forward feature-Gaussian lifting) from photorealistic rendering (a separate one-step pixel-flow model) instead of one monolithic neural renderer — the split is what turns noisy, sparse Gaussian projections into clean panoramic RGB-D.
Functions as a fully automated data engine: converts ordinary video/image collections, not curated scans, into interactive scenes at a scale (~20K scenes, 10M+ navigation samples) manual capture can't approach.
Directly targets the classic scan-realism vs. synthetic-scalability tradeoff, arguing neural simulation from casual footage can deliver both rather than forcing a choice.
Reports zero-shot real-world transfer from policies trained purely in the neural simulation — the training substrate need not be a real or hand-built simulator at all.

Show abstract

Embodied navigation aims to build agents that interpret multimodal goals, reason in 3D space, and reach target destinations reliably in the real world. However, progress remains constrained by the lack of scalable, high-fidelity, and physically grounded interactive environments. Although real-world scanned datasets offer visual realism, they are limited by scale. In contrast, synthetic simulators scale more easily but often exhibit large sim-to-real gaps. We introduce Image2Sim, a real-time neural simulation framework that constructs high-quality interactive environments from posed RGB-D image sequences. The central idea is to decouple 3D spatial anchoring from photorealistic observation synthesis. For scene construction, Image2Sim uses a feed-forward feature Gaussian model that lifts posed RGB-D observations into a 3D feature-Gaussian representation in a single pass. For rendering, we propose a Geometry-Aware One-Step Pixel Flow model that transforms sparse and noisy Gaussian projections into high-quality panoramic RGB-D observations. Image2Sim also serves as a fully automated embodied data engine that generates high-fidelity observations, executable actions, and diverse navigation instructions at scale. It converts large collections of videos and images into nearly 20K interactive scenes and synthesizes more than 10 million navigation training samples. Navigation models trained entirely in these neural environments achieve strong improvements on major benchmarks and transfer effectively to real-world zero-shot settings. These results suggest that scalable neural simulation can serve as a practical training substrate for embodied navigation at scale.

#4 h=19

RoboTALES: Learning Reasoning-Guided Robot Policies via Task-Aligned Simulated Futures

2026-07-07 cs.RO 👤 Nicklas Hansen (h=19)

Hanan Gani, Tejal Kulkarni, Madhoolika Chodavarapu, Nicklas Hansen, Manmohan Chandraker

Core Contributions

Targets a specific failure of using video-generation models for policy learning: imagined futures drift from task intent and aren't reliably action-conditional, making raw generator rollouts unreliable as planning substrates.
Anchors generation with a hierarchical LLM planner that decomposes tasks into subgoals plus a VLM critic providing reward feedback — language-level structure constrains what would otherwise be unconstrained pixel generation.
Trains in a single stage where reasoning directly guides imagination, distinct from typical two-stage generate-then-distill pipelines.
Reports consistent gains over existing methods on RoboCasa/LIBERO10, with the largest margins specifically on long-horizon tasks — the reasoning anchor mainly pays off where compounding error is worst.

Show abstract

Pretrained video generative models are promising backbones for visuomotor control, but their imagined futures often drift from task intent and are not reliably action-conditional. As a result, these models can be difficult to use for planning or policy extraction. To address these limitations, we propose RoboTALES, a single-stage framework that learns task-aligned simulated futures and uses them to train robot policies. Our approach introduces two key innovations: (1) a hierarchical LLM-based planner that breaks complex tasks into a sequence of subgoals to guide the model's imagination; and (2) a VLM-based critic that evaluates these ``imagined'' futures and uses reward-based feedback to keep the model's internal representations focused on the goal. By anchoring the video generator in abstract reasoning, we produce temporally consistent rollouts and more coherent actions. We evaluate RoboTALES on diverse manipulation tasks from RoboCasa and LIBERO10, and show that our method consistently outperforms existing methods, especially in long-horizon tasks. Our code and models are publicly available at https://github.com/hananshafi/RoboTALES.

#20 h=6

RynnWorld-Teleop: An Action-Conditioned World Model for Digital Teleoperation

2026-07-07 cs.RO 👤 Biao Gong (h=6)

Haoyu Zhao, Xingyue Zhao, Hangyu Li, Biao Gong, Kehan Li

Core Contributions

Introduces "digital teleoperation": replaces the physical robot entirely with a generative world model during data collection, so an operator's hand-pose stream, not real hardware, drives synthesis of egocentric video — decoupling data collection from needing a functioning robot at all.
The recorded pose stream is embodiment-agnostic and retargetable to any target robot afterward, meaning one teleoperation session can seed training data for multiple different robot bodies.
Streaming autoregressive distillation compresses generation to single-pass inference, hitting 40+ FPS real-time on a single H100 — a substantial systems contribution, since naive video-diffusion generation would be far too slow for interactive teleoperation.
Reports policies trained purely on generated data achieve zero-shot sim2real transfer, and mixing generated data into real datasets consistently improves success — positioning this as a practical data-augmentation engine, not just a generation demo.

Show abstract

Scaling robot learning requires massive, diverse trajectory data, yet collection is currently bottlenecked by physical teleoperation, where every demonstration binds operator time to specific hardware and workspaces. We introduce digital teleoperation, a paradigm that decouples data collection from physical constraints by replacing the real robot with a generative world model. In this framework, an operator's hand-pose stream drives a robot-centric generative world model to synthesize high-fidelity egocentric videos from a single reference image. The recorded pose stream serves as an embodiment-agnostic action label transferable to any target robot via standard retargeting, yielding complete state-action trajectories for imitation learning independent of physical hardware. We instantiate this paradigm in RynnWorld-Teleop, a system that integrates depth-aware skeletal conditioning, progressive human-to-robot training on a video Diffusion Transformer, and streaming autoregressive distillation. This pipeline compresses the generative process into a single-pass inference, enabling 40+ FPS, real-time interactive generation on a single H100 GPU. Policies trained exclusively on RynnWorld-Teleop-generated data achieve effective zero-shot Sim2Real transfer across dexterous and diverse bimanual tasks. Moreover, augmenting real-world datasets with our digitally teleoperated data consistently improves success rates, demonstrating that RynnWorld-Teleop serves as a high-fidelity, scalable data engine for the next generation of robotic agents.

#24 h=5

Imagined Rollouts are Kinematic, Not Dynamic: A Diagnosis of Long-Horizon World-Model Failure

2026-07-07 cs.RO 👤 Sebastian Schmidt (h=5)

Finn Rasmus Schäfer, Korbinian Moller, Yuan Gao, Christian Oefinger, Sebastian Schmidt

Core Contributions

Reframes "compounding error" in long-horizon world models — the standard generic explanation for planning failure — into a specific, testable claim: models imagine kinematically (following a closed-form kinematic null) rather than dynamically (respecting real physical forces).
Introduces the imagined Kinematic-Consistency Error (iKCE), a per-step diagnostic comparing rollouts to that kinematic null, plus a perturbation protocol testing whether imagination responds when physics actually change regime.
Key finding: on a DreamerV3 checkpoint, imagined iKCE runs roughly 100x above matched real-physics rollouts, and stays statistically flat across a friction sweep even as the trained policy's actual reward collapses through a gait-collapse boundary — the world model doesn't notice the physics changing.
A targeted diagnostic contribution, not a new architecture: gives the field a specific, measurable symptom to test future world models against, rather than the vague "compounding error" framing.

Show abstract

Long-horizon failure in world models is conventionally attributed to compounding error, a generic framing that does not distinguish what kind of error compounds. We propose a kinematic-vs-dynamic reframing: world models tend to imagine kinematically rather than dynamically. We operationalize this as the imagined Kinematic-Consistency Error, a per-step diagnostic that measures how far a rollout departs from a closed-form kinematic null, paired with a perturbation protocol that tests whether iKCE responds when physical conditions cross a regime boundary. We instantiate the diagnostic on a released DreamerV3 checkpoint trained on DMC walker-walk, where imagined iKCE runs roughly two orders of magnitude above that of matched real-physics rollouts. Across a friction sweep that crosses the gait-collapse boundary, the model's iKCE stays statistically flat even as the trained policy's reward collapses through the same range, providing the kinematic-not-dynamic signature. The diagnostic distinguishes kinematic from dynamic imagination at horizons longer than the embodiment's gait period.

Navigation, SLAM & Perception

#2 h=39

EAGOR: Embodied Reasoning in Omni-direction

2026-07-07 cs.RO 👤 W. Yau (h=39)

Shriram Damodaran, Soumyaratna Debnath, Yan Wu, Wei-Yun Yau, Addison Lin Wang

Core Contributions

Reframes 360° directional reasoning as recursive Bayesian estimation directly on the sphere, rather than on the flattened equirectangular projection that most VLM-based omnidirectional systems inherit from perspective-image architectures.
Introduces a Spherical Harmonic Belief Field to propagate a continuous directional belief equivariantly under robot motion, eliminating ERP seam discontinuities and latitude distortion that break naive 2D-projection reasoning.
Training-free: works directly with existing VLM backbones without retraining, unlike task-specific navigation models built for one benchmark.
Quant: +34.4%/+45.6% relative gains on HOS/OSR-Bench, +14.6% navigation success, -17.7% steps, -24.5% angular error on a real legged robot — evidence the geometric fix, not more training, closes the gap.

Show abstract

Omni-directional (360°) cameras provide embodied agents with a holistic view of their surroundings, making them suited for directional reasoning in tasks such as navigation and object search. Existing Vision Language Models (VLMs) project 360° observations to 2D equirectangular projection (ERP) images and process them using architectures designed for perspective images. However, they ignore the spherical nature of 360° observations, where each pixel represents a viewing direction relative to the agent. Consequently, their direction estimates often become inconsistent under camera view transformations caused by agent motion. This limitation is particularly critical for map-free navigation, where the agent must continuously estimate the target direction in its egocentric frame. We propose EAGOR, a training-free, geometry-aware framework for embodied 360° directional reasoning. Instead of predicting target directions as ERP image coordinates, EAGOR formulates directional reasoning as recursive Bayesian estimation directly on the sphere. It maintains a continuous belief over target directions and propagates it equivariantly under agent motion without training the backbone VLMs. To achieve this, we introduce the Spherical Harmonic Belief Field (SH-BF), whose spherical harmonic representation provides a globally defined, rotation-aware basis for directional estimation on the spherical manifold. This formulation eliminates ERP seam discontinuities, latitude distortions, and interpolation errors. We evaluate EAGOR on two benchmark datasets and real-world experiments with a legged robot across directional reasoning tasks. EAGOR consistently outperforms existing methods, achieving average relative gains of +34.4% and +45.6% on HOS and OSR-Bench, respectively, while improving navigation success by +14.6%, reducing step count by 17.7%, and lowering mean angular error by 24.5%.

#5 h=18

Hilti-Trimble-Oxford Dataset: 360 Visual-Inertial Benchmark with Floor Plan Priors for SLAM and Localization

2026-07-07 cs.RO 👤 E. Szymańska (h=18)

Samuele Centanni, Yuhao Zhang, Yifu Tao, Julien Kindle, Frank Neuhaus

Core Contributions

Substitutes costly LiDAR-based mapping with consumer 360° cameras + IMU, aimed specifically at cost-prohibitive construction-site monitoring rather than general SLAM benchmarking.
Captures deployment stressors most SLAM benchmarks omit: 8 months, 7 floors, moving workers, fast motion, repetitive structure — conditions curated benchmark sequences don't stress-test.
Ground truth from a rigidly-mounted LiDAR-inertial rig gives trustworthy comparison targets despite the system under test using a cheaper sensor.
The open-challenge results are the real finding: 62 teams entered SLAM vs. only 22 for floor-plan-referenced localization, with localization errors notably higher — evidence floorplan-referenced localization is the less mature, harder problem.

Show abstract

Automated progress monitoring on construction sites is an active area of research and development. Robot and human-carried mapping systems have been developed to build 3D maps of building and infrastructure projects. While LiDAR-based mapping systems achieve high accuracy, the cost of LiDAR can be prohibitive. Consumer-grade cameras with wide field of view ("360 cameras") combined with embedded inertial measurement units (IMUs) provide a cost-effective alternative. To support change detection and progress monitoring, highly accurate visual Simultaneous Localization and Mapping (SLAM) and floor plan-referenced localization systems are required. In this paper we present a high-quality dataset collected at an active construction site, which captures realistic challenges such as variable lighting conditions, moving workers, fast motions, and repetitive structures. The dataset offers thirty visual-inertial sequences recorded across seven floors over an eight-month period of the construction project. Ground truth trajectories were collected using a high quality LiDAR-inertial SLAM system rigidly attached to the 360 camera. Additionally, we report the results of an open research challenge evaluating the best visual SLAM and localization systems from around the world. The Challenge attracted substantially higher participation in SLAM, with 62 teams compared to 22 in floor-plan-referenced localization, reflecting the broader maturity of SLAM methods. The higher errors in localization further highlight the difficulty of this task in construction and point to the need for continued research, which this dataset is intended to support. The dataset and the benchmark are publicly available at: https://hilti-trimble-challenge.com/dataset-2026.

#9 h=13

Why does Deep Learning Improve Visual SLAM?

2026-07-07 cs.CVcs.RO 👤 Giovanni Cioffi (h=13)

Giovanni Cioffi, Davide Scaramuzza

Core Contributions

Runs a controlled ablation to isolate which specific ingredient of deep-learning SLAM drives its advantage over classical geometric SLAM — learned 2D data association, learned uncertainty, or the recurrent architecture itself — a question usually assumed rather than tested.
Finding: the recurrent architecture is not what matters; learned 2D correspondence plus its uncertainty estimate is the actual source of the performance gain.
Practical implication: architecture novelty in recurrent SLAM backbones may be less valuable than improving the learned association/uncertainty modules directly, redirecting where engineering effort should go.
Complements #22 and #16 in this batch's broader theme of tracing SLAM robustness problems to a specific root cause rather than proposing another fusion pipeline.

Show abstract

Visual SLAM is a well-established technology utilized in a wide range of real-world applications. However, its performance still degrades under challenging visual conditions, such as low texture, severe motion blur, and poor illumination. Systems based on deep learning outperform classical geometry-based ones and achieve state-of-the-art results by combining learned 2D data association and uncertainty with differentiable geometric optimization in recurrent architectures. Still, it remains unclear exactly which components are fundamentally responsible for this success. In this paper, we ask: Is the superior performance of deep learning-based systems driven primarily by learned 2D data association, the combination of learned 2D data association and uncertainty, or the recurrent architecture itself? We investigate this question empirically by conducting a controlled study. Our findings reveal that the success of DL-based V-SLAM systems hinges on learned 2D data association and uncertainty rather than their recurrent architecture, underscoring the necessity of learning-based paradigms for the design of these components. Upon acceptance, the code will be released as open source.

#11 h=10

UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation

2026-07-07 cs.RO 👤 Lifeng Fan (h=10)

Zhuofan Zhang, Tianxu Wang, Guoxi Zhang, Yixiong Lin, Xilin Wang

Core Contributions

Targets "last-mile navigation" (reaching a manipulation-ready pose near a target, not just proximity) as a distinct zero-shot problem, decomposed into view selection, affordance grounding, and base-pose reasoning, all solved by one shared MLLM rather than task-specific modules.
Removes the manual pose annotation or per-task training that prior last-mile approaches required, extending the capability to open-vocabulary targets.
Quant: beats prior SOTA MoTo by 3.13 points on OVMM; ablations show all three pipeline components and the specific MLLM backbone both matter substantially, not just raw model scale.
Validated beyond simulation on a real Unitree B2 quadruped with a Z1 arm, connecting the zero-shot claim to an actual mobile-manipulation platform.

Show abstract

Mobile manipulation requires a robot to navigate to a target object or receptacle and then perform intended manipulation. However, reaching the vicinity of the target does not guarantee a manipulation-ready base pose, a problem known as last-mile navigation. Prior methods for last-mile navigation either rely on manual pose annotation or task-specific training, limiting their scalability to open-vocabulary settings with fine-grained spatial constraints. We propose UniLM-Nav, a unified framework for zero-shot open-vocabulary last-mile navigation. UniLM-Nav decomposes last-mile navigation into view selection, task-conditioned affordance grounding, and geometry-aware base-pose reasoning, all resolved with a shared multimodal large language model (MLLM) backend. Specifically, UniLM-Nav first selects a reference view that best captures the target object or receptacle from recently collected observations. It then grounds task-relevant affordance point in the selected view and lifts the result into the robot-centric coordinate frame. Finally, conditioned on the grounded affordance, task context, and robot geometry, it infers a manipulation-ready base pose for the robot. We evaluate UniLM-Nav on the OVMM benchmark, where it outperforms the previous state-of-the-art method, MoTo, by 3.13 percentage points. Analyses show that the components of our method are crucial to final performance, and that the choice of MLLM also has a substantial effect. We further deploy UniLM-Nav on a Unitree B2 quadruped robot with a 6-DoF Unitree Z1 manipulator, validating its applicability to real-world mobile manipulation tasks.

#16 h=9

APVI-SLAM: Real-Time Acoustic-Pressure-Visual-Inertial Localization and Photorealistic Mapping System in Complex Underwater Environment

2026-07-07 cs.RO 👤 Huajian Huang (h=9)

Hanwen Zhang, Yipeng Zhu, Xiaopeng Guo, Huajian Huang, Sai-Kit Yeung

Core Contributions

Targets underwater SLAM's core failure mode — vision degrading or failing outright — with a reliability-aware framework that dynamically reweights sensor estimators (DVL, pressure, vision) rather than fusing them with fixed weights.
A sliding-window freezing strategy specifically recovers from tracking-failure episodes, a targeted fix for intermittent visual failure rather than just averaging down noise.
Quadtree-guided mapping enables incremental 3D Gaussian reconstruction underwater despite the water medium's optical distortion, extending photorealistic Gaussian mapping — mostly demonstrated on land — to a much harder medium.
Contributes a new coral-reef multi-modal benchmark specifically because underwater mapping evaluation currently lacks one, addressing an infrastructure gap rather than only a method gap.

Show abstract

Extreme subsea environments often cause severe feature de-gradation and estimator divergence in underwater visual-inertial SLAM. Although sensors like Doppler Velocity Logs (DVL) and pressure gauges provide auxiliary constraints, robust multi-sensor fusion during intermittent visual failure remains challenging. To address this, we present APVI-SLAM, a real-time multi-sensor fusion SLAM system that achieves both accurate underwater localization and photorealistic mapping. Our approach introduces a reliability-aware localization framework that dynamically reweights sensor estimators and employs a sliding-window freezing strategy to recover from tracking failures, substantially enhancing system robustness. Furthermore, for high-fidelity scenes reconstruction, we propose an efficient quadtree-guided mapping module that facilitates incremental water-medium modeling and 3D Gaussian optimization. Recognizing the lack of benchmark for underwater mapping evaluation, we also contribute a coral reef surveying dataset with synchronized multi-modality data. Extensive experiments on public and our proposed benchmarks demonstrate that APVI-SLAM achieves state-of-the-art localization and reconstruction quality at real-time speeds.

#22 h=6

Observation Quality Matters: Robust Multi-Fisheye Calibration via Failure-Oriented Analysis

2026-07-07 cs.RO 👤 Shaojie Shen (h=6)

Peize Liu, Zhe Tong, Chen Feng, Shaojie Shen

Core Contributions

Runs a failure-oriented empirical analysis to find the real bottleneck in multi-fisheye calibration, ruling out detector recall and image-plane distribution imbalance in favor of a more specific cause: limited radial span couples focal scale with fisheye projection-shape parameters, producing ill-conditioned intrinsic initialization.
CO-Calib is a plug-in data-construction layer, not a new optimizer — it selects better calibration frames (initialization-friendly anchors, co-visible constraints, coverage-completion frames) without touching the existing calibration backend.
Quant: raises overall calibration success rate from 68.1% to 99.3%, a jump attributable purely to smarter frame and anchor selection rather than a new solver.
Reinforces this batch's diagnostic theme, alongside #9 and #24, by tracing failure to a specific, previously-unidentified root cause rather than proposing another incremental fusion improvement.

Show abstract

Reliable calibration of multi-fisheye camera systems remains challenging as rig size, camera arrangement diversity, and field of view increase. Existing pipelines can jointly optimize intrinsics, extrinsics, and target poses, but their success still depends heavily on empirical capture rules and the quality of the observations supplied to the solver. This paper studies this dependency through a failure-oriented analysis. We reveal that calibration failures are not sufficiently explained by detector recall loss or global image-plane distribution imbalance. Instead, the dominant failure factor lies in intrinsic initialization: observations with limited radial span couple focal scale with fisheye projection-shape parameters, producing ill-conditioned updates. Guided by this insight, we propose CO-Calib, a plug-in calibration-data construction framework that combines a robust learning-based target detector with an error-analysis-guided frame selector. CO-Calib constructs initialization-friendly anchors, co-visible multi-camera constraints, and coverage-completion frames without changing the existing calibration workflow or optimization backend. Extensive experiments on synthetic and real multi-fisheye systems demonstrate that CO-Calib improves the overall success rate from 68.1% to 99.3%, increases extrinsic accuracy, and augments real-world calibration stability. The source code will be made publicly available at https://github.com/HKUST-Aerial-Robotics/CO-Calib.

Manipulation & Control

#7 h=17

Learning to Throw Objects Safely in Multi-Obstacle Environments

2026-07-07 cs.ROcs.CVcs.LG 👤 H. Kasaei (h=17)

Mohammadreza Kasaei, Klemen Voncina, Hamidreza Kasaei

Core Contributions

Encodes obstacles and target as a compact potential field on a fixed-size grid instead of an explicit list of positions and sizes — this is what lets the RL policy generalize to arbitrary numbers and layouts of obstacles without per-scene retraining.
Initializes the policy from kinesthetic demonstrations before RL fine-tuning, comparing SAC, DDPG, and TD3 head-to-head; SAC comes out most consistent across scenarios.
Directly extends TossingBot-style throwing to cluttered scenes, closing a gap prior throwing work left unaddressed by assuming obstacle-free settings.
Quant: up to 90% success on real-robot cluttered-scene tests with unseen throwable objects — sim-to-real transfer holds even for obstacle configurations never seen in training.

Show abstract

Robotic throwing enables fast and efficient object placement beyond the robot's immediate workspace, but reliable throwing in cluttered environments remains underexplored. Existing approaches, such as TossingBot, learn throwing strategies from visual input but assume obstacle-free settings. In this paper, we address the problem of throwing objects into a target basket while avoiding obstacles placed randomly in the scene. We introduce a potential field state representation that compactly encodes both basket attraction and obstacle repulsion on a fixed-size grid, enabling reinforcement learning (RL) policies to generalize across arbitrary numbers and configurations of obstacles. The policy is initialized from kinesthetic demonstrations and optimized in simulation using three state-of-the-art RL algorithms (SAC, DDPG, TD3). Among these, SAC achieves the most consistent performance across scenarios. We compare the potential field representation against explicit state encodings and demonstrate that it achieves higher success rates and better scalability to unseen obstacle configurations. Real-robot experiments with unseen throwable objects confirm robust sim-to-real transfer, achieving up to $90\%$ success in cluttered scenes. These results demonstrate that PFR provides a practical and robust representation for safe and efficient robotic throwing in unstructured environments. A video showcasing our experiments is available at: https://youtu.be/ZZnJf8ua2dE

#10 h=12

ThorArena: Benchmarking Humanoid Physical Interaction with Human Motion-Force Demonstrations

2026-07-07 cs.RO 👤 A. Knoll (h=12)

Chenhao Yu, Hongwu Wang, Weitao Zhang, Youhao Hu, Jiachen Zhang

Core Contributions

Points out existing humanoid interaction benchmarks measure kinematic tracking — does the body follow the reference motion — but ignore whether the robot is handling the physical forces exerted on it, a blind spot ThorArena closes.
Introduces the Force-Aware Tracking Score, jointly scoring tracking accuracy, force robustness, control effort, and survival, replayed from real synchronized human motion-force capture across six contact-rich tasks.
Key claim: standard no-force evaluation hides substantial performance differences between control policies that only appear once external interaction forces are factored in — today's humanoid leaderboards may over-rank policies that are fragile to real contact.
Provides a standardized replay protocol so different whole-body controllers can be compared under identical recorded force profiles instead of ad hoc physical trials.

Show abstract

Humanoid robots are increasingly expected to perform contact-rich tasks that require not only accurate whole-body motion but also robust physical interaction with surrounding objects and humans. Although recent advances in humanoid motion imitation and whole-body control have achieved remarkable tracking performance, existing datasets and benchmarks primarily focus on kinematic motion while largely overlooking synchronized interaction forces. As a result, current evaluations fail to capture how external interaction forces affect tracking accuracy, stability, and control robustness. In this paper, we present ThorArena, a benchmark for evaluating force-aware humanoid interaction based on human demonstrations with synchronized motion and force measurements. We collect a real-world interaction dataset that simultaneously captures whole-body human motion and forces exerted by both hands across six representative physical interaction tasks. Based on these demonstrations, we propose force-aware evaluation metrics that jointly assess whole-body tracking accuracy, robustness under different force levels, control effort, and episode survival through the Force-Aware Tracking Score (FATS) and complementary diagnostic metrics. We further establish a unified benchmark protocol that replays recorded interaction forces in simulation and provides a standardized evaluation interface for different humanoid control policies. Experiments on representative whole-body control policies demonstrate that force-aware evaluation reveals substantial performance differences that remain largely hidden under conventional no-force evaluation. ThorArena provides a practical and reproducible framework for studying force-aware humanoid interaction and offers a new benchmark for evaluating contact-rich humanoid behaviors.

#13 h=10

MP-MPPI: A Motion Primitive Guided Sampling-Based Optimizer for Model Predictive Control

2026-07-07 cs.RO 👤 Eleni Kelasidi (h=10)

Marlon Mathisen, Aksel Vaaler, Olav Egeland, Eleni Kelasidi

Core Contributions

Injects hand-designed motion primitives into the MPPI sampling distribution alongside standard random perturbations, structuring the search rather than relying purely on stochastic sampling around the current control sequence.
Targets a known MPPI weakness, poor path-planning/global structure in sampling-based control, without abandoning MPPI's fast, reactive real-time properties.
Validated in a quadcopter obstacle-field simulation, showing improved control-space exploration while retaining real-time reactivity.
A lighter-weight structural fix to the same sampling-optimality problem CE-MPPI (#26) addresses via clustering — two different structural remedies for the same averaging/exploration failure in path-integral control.

Show abstract

This paper proposes a novel method that extends the Model Predictive Path Integral (MPPI) method with motion primitives for additional structured sampling, which enhances the convergence towards a globally optimal solution. By evaluating motion primitives and perturbed control sequences in a real-time sampling-based optimization loop, this work addresses the limitations of the path planning capabilities of sampling-based controllers. The algorithm is implemented on a quadcopter simulator and tested on an obstacle field navigation task. It is demonstrated that the proposed approach enhances exploration of the control space while maintaining the fast, reactive behavior required for real-time control.

#25 h=5

Prior-First, Condition-Second: Scalable and Controllable Hand Motion Completion

2026-07-07 cs.GRcs.RO 👤 Xuelin Chen (h=5)

Mingyi Shi, Xuelin Chen, Taku Komura

Core Contributions

Learns a generic body-hand kinematic prior from large-scale unlabeled motion data first, then adds semantic/text control via lightweight adapters injected at appropriate kinematic levels, rather than training one model end-to-end on scarce labeled body+hand+semantic data jointly.
This "prior-first, condition-second" order means the same frozen prior serves multiple control interfaces — attribute-based, text-driven — without relearning kinematic structure per interface.
Streaming autoregressive design generates coherent hand motion from body dynamics in real time, aimed explicitly at production animation pipelines rather than only offline generation.
Reports gains specifically in low-resource and cross-dataset settings compared to end-to-end conditioned baselines — the prior-first design is what buys robustness when labeled data is scarce, the realistic regime for hand-motion semantic labels.

Show abstract

Synthesizing hand motion that matches the full body motion and the semantic labels is a difficult task due to their high degrees of freedom and the lack of semantic labels. To cope with this issue, we propose a prior-first, condition-second framework for body-conditioned hand motion completion. Our framework first learns a generic body-hand kinematic prior from large-scale unstructured and unlabeled motion data, capturing the intrinsic coordination between global body dynamics and hand articulation. Semantic control is then introduced through lightweight adaptation on top of the frozen prior, avoiding the need to relearn kinematic structure for each control interface. Our framework centers on a streaming, autoregressive body-hand prior that generates coherent, kinematically consistent hand motion from body dynamics in real time, using structured kinematic modeling to maintain mechanical body-hand coupling. To enable practical controllability under limited supervision, we introduce semantically-layered adapters that inject conditioning signals at appropriate kinematic levels, supporting both self-supervised attribute control and weakly supervised text-driven control with only a few hours of labeled data. Extensive evaluations demonstrate that our framework improves kinematic plausibility, robustness, and controllability compared to end-to-end conditioned baselines, particularly in low-resource and cross-dataset settings. We further showcase real-time inference and an interactive authoring workflow, highlighting the applicability to production animation pipelines. Homepage: https://AIGAnimation.github.io/HandPrior/

#26 h=4

Clustering-Embedded Model Predictive Path Integral Control: Avoiding Averaging-Induced Failure and Enabling Efficient Cluster Selection for Dynamic Obstacles

2026-07-07 cs.RO 👤 Zidong Liu (h=4)

Zidong Liu, Kaixin Chang, Xu Chen

Core Contributions

Diagnoses a specific mechanism behind MPPI's failure in cluttered scenes: importance-weighted averaging blends incompatible rollouts (e.g., go-left and go-right around an obstacle) into a hesitant or colliding blended trajectory, rather than attributing failure to sampling variance generally.
Fixes this architecturally with DBSCAN clustering plus a geometric direction feature derived from collision points, isolating distinct feasible trajectory modes before they get averaged together.
Selection logic differs by context — minimum-cost mode in static scenes, but actively steering away from obstacle flux in dynamic scenes — treating static and dynamic obstacle avoidance as needing different selection criteria.
Quant: 48% reduction in time-to-goal and 12% shorter end-effector path on a real 6-DoF UR5e with CUDA-parallel rollouts in Isaac Gym — real hardware validation, not just simulation.

Show abstract

With the widespread availability of parallel computing hardware, sampling-based motion planning methods such as Model Predictive Path Integral (MPPI) control have become increasingly powerful for complex nonlinear systems in non-smooth task spaces. However, the sampling and forward-simulation pipeline in MPPI suffers from averaging-induced failure in cluttered environments, where the importance-weighted update averages incompatible rollouts and leads to hesitation or even collision when an obstacle lies directly ahead. This paper proposes Clustering-Embedded MPPI (CE-MPPI), a framework that architecturally resolves the averaging-induced failures inherent in standard MPPI within non-convex environments. Rather than simply mitigating interference, CE-MPPI redefines the control law by integrating a high-fidelity pruning and clustering stage. By leveraging density-based spatial clustering of applications with noise (DBSCAN) alongside a novel geometric direction feature that is extracted from collision-derived reference points, the system isolates feasible trajectory modes from the noise of infeasible rollouts. This is paired with an intelligent selection logic that optimizes for minimum cost in static scenes while actively steering opposite to obstacle flux in dynamic environments. Experiments in 2-D JAX-accelerated simulations show that CE-MPPI alleviates obstacle-front hesitation and avoids persistent coupling with moving obstacles in dynamic scenes. In particular, real-world tests on a 6-DoF UR5e manipulator with CUDA-parallel rollouts in Isaac Gym achieve a 48\% reduction in time-to-goal and a 12\% shorter end-effector path.

#28 h=4

DexTele: A Dual-Arm Dexterous Teleoperation System Based on Motion Retargeting and Adaptive Force Control

2026-07-07 cs.RO 👤 Zhaojie Ju (h=4)

Yuanchuan Lai, Qing Gao, Ziyan Liang, Xianfeng Cheng, Junjie Hu

Core Contributions

Targets two separate hard problems in dual-arm dexterous teleoperation simultaneously: cross-platform motion retargeting via a motion-graph encoder plus latent optimization, and compliant grasping force prediction via a VLM+MPC combination, rather than assuming fixed robot morphology and known objects.
The VLM component predicts required grasping force per target object, which MPC then optimizes online — using vision-language grounding to inform a low-level force-control parameter, not just high-level task planning.
Vision-based motion retargeting from human images, not motion-capture suits or gloves, lowers the hardware barrier for teleoperation, a practical deployment consideration.
Demonstrated generalization across multiple robot platforms — the cross-platform claim is validated empirically, not just architecturally implied.

Show abstract

In dual-arm dexterous teleoperation, cross-platform generalization of motion retargeting and interactivity of grasping are crucial. However, the heterogeneity of robotic architectures and the wide variety of grasping objects pose significant challenges to achieving precise motion retargeting and compliant grasping in dual-arm dexterous teleoperation. To address these challenges, a dual-arm dexterous teleoperation system (DexTele) is proposed based on motion retargeting and adaptive force control. First, a vision-based motion retargeting module is designed to generate preliminary robot motions from human images. In this module, a motion-graph encoder and latent optimization are proposed for precise and convenient cross-platform motion retargeting. Second, an adaptive grasping module is designed to achieve compliant grasping. This module combines a vision-language model (VLM) with model predictive control (MPC), allowing the system to predict the required grasping force for a target object and perform gradient-based online optimization. Finally, extensive experiments demonstrate that the DexTele achieves precise motion retargeting and compliant grasping with generalization across multiple robot platforms.

Human-Robot Interaction & Assistive Robotics

#6 h=18

Towards Real-World Applications with an Autonomous Powered Wheelchair

2026-07-07 cs.RO 👤 Antonio Paolillo (h=18)

Simone Arreghini, Alessandro Giusti, Alex Bordini, Enrico Ferrara, Giovanni Fulgoni

Core Contributions

Builds on a commercial self-balancing wheelchair (Genny Zero) rather than a research platform, so the integration story — RGB-D + LiDAR added onto existing body-shift control — is itself the contribution, aimed at deployability over novel algorithms.
Demonstrates two concrete assistive behaviors, hailing (calling the chair from a distance) and person-following, that map directly onto stated user needs rather than abstract navigation benchmarks.
Frames itself explicitly as a preliminary proof-of-concept and reports the technical gaps still blocking user-ready deployment — a more candid framing than claiming solved autonomy.
Complements the ethics-focused personalization paper (#8) in this batch with a concrete hardware case study of what deployable assistive autonomy actually looks like.

Show abstract

Wheelchair users call for assistive mobility systems that provide active support, adapt to dynamic environments, and are intuitive and user-friendly. However, powered wheelchairs typically still provide limited autonomy and lack effective integration with advanced perception and navigation capabilities, particularly in complex real-world environments. This paper presents a preliminary study toward autonomous powered wheelchairs for real-world assistive mobility. We introduce a proof-of-concept prototype that integrates autonomous perception, gesture-based interaction, and navigation on a commercially available self-balancing powered wheelchair. The proposed system builds upon Genny Zero, a commercial self-balancing wheelchair that enables hands-free and intuitive operation through body-weight shifting. To extend its capabilities toward autonomous operation, we integrate an RGB-D camera for human-aware perception and interaction, together with a LiDAR sensor for localization and navigation. We demonstrate the integrated system in two assistive applications: (i) hailing, allowing users to call the wheelchair from a distance; and (ii) people-following, where the wheelchair follows a person using leader-follower strategies, including a constrained indoor navigation example. The results highlight the potential of combining autonomous robotics with assistive mobility platforms, while also showing the feasibility of the proposed integration and identifying the main technical challenges that must be addressed before moving toward user-ready, accessible, and intelligent mobility solutions. A video demonstrating the experimental setup and results is available at: https://youtu.be/LVAix_Qx7bM.

#8 h=16

Responsible Personalisation: The Double-Edged Sword of Personalisation in Human-Robot Interaction

2026-07-07 cs.ROcs.AIcs.HC 👤 V. Charisi (h=16)

Antonio Andriella, Jauwairia Nasir, Andrea Rezzani, Alyssa Kubota, Dimitri Lacroix

Core Contributions

Argues existing responsible-AI personalization literature is too generic for HRI, since embodiment and social presence change how personalization risks manifest — a robot that "knows too much" reads differently than an app that does.
Structures risk analysis along two axes, lifecycle stage and interaction context (short vs. long-term, open vs. closed domain), rather than treating personalization risk as one undifferentiated hazard.
Names five concrete risk categories — autonomy erosion, biased user modelling, manipulation, dehumanisation, privacy violation — and traces how each shifts across those contexts.
A purely conceptual framework with no system or experiment, intended as design guidance and a research agenda rather than an evaluated method.

Show abstract

While personalisation is becoming a defining capability in human-robot interaction (HRI), the existing literature on responsible personalisation remains fragmented, offering isolated accounts of ethical risks without a structured understanding of how they emerge across interaction contexts. This gap is particularly critical in HRI, where robots' embodiment and social presence can amplify and reshape such risks or generate new types of risks. We present a lifecycle-based and context-sensitive framework for personalised HRI, grounded in an embodiment-aware perspective. The framework combines stages of the personalisation process with interaction characteristics (short-term vs. long-term, open-domain vs. closed-domain), enabling systematic analysis of how risks arise and evolve. Building on this, we conduct an integrative analysis of key ethical risks, including autonomy erosion, biased user modelling, manipulation, dehumanisation, and privacy violations, and examine how they manifest across contexts. We translate these insights into actionable design recommendations and outline open research challenges. By structuring both the design space and risk landscape of personalised HRI, this work provides a foundation for more systematic, transparent, and ethically grounded approaches to personalised robot behaviour.

#23 h=5

Embodied Human-Robot Interaction via Acoustics: A MARL Approach with AcoustoBots for Spatial Data Physicalization

2026-07-07 cs.RO 👤 Prateek Mittal (h=5)

Shiqi Liu, Narsimlu Kemsaram, Prateek Mittal, Pengyuan Wei, Sriram Subramanian

Core Contributions

Uses mobile robots carrying ultrasonic phased arrays to levitate and move a physical particle whose height encodes real urban data — population density, noise, traffic — physicalizing data as a moving, embodied object rather than a static or screen-based representation.
MADDPG-based multi-agent RL (centralized training, decentralized execution) handles collision-aware navigation while a separate high-rate acoustic controller maintains levitation trap stability during motion — decoupling "where to go" from "how to keep the particle floating" as two concurrently running control loops.
Demonstrates both single-robot traversal and dual-robot cooperative coverage on a scaled physical map, closing a full perception-display-action loop rather than a levitation demo in isolation.
Quant: 90%/80% task success for single/dual-robot regimes with low collision counts over repeated trials — a tangible entry in the "robot as communication medium" niche of HRI.

Show abstract

Traditional data physicalization is often static and disconnected from real environments, limiting its ability to convey embodied spatial dynamics and engage users. To address this limitation, we present AcoustoBots, a mobile acoustophoretic data-physicalization platform in which TurtleBot3 robots carry upward-facing 8 x 8 ultrasonic phased arrays. Each array levitates a particle whose height (1-10 cm) encodes a local urban scalar value, such as population density, noise, or traffic. A MARL (Multi-Agent Reinforcement Learning) policy based on the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm, with centralized training and decentralized execution, selects collision-aware navigation actions, while a high-rate Gerchberg-Saxton-Phased Array of Transducers (GS-PAT) acoustic controller maintains trap stability and updates array phases to achieve the commanded height during motion. This creates a closed perception-display-action loop. We evaluate single-robot city-to-city traversal and dual-robot cooperative coverage on a 4 m x 3 m scaled UK map using PhaseSpace-based localization for repeatable multi-robot trials. Results show stable in-motion levitation and consistent, location-dependent height rendering, with task success rates of 90% and 80% for the single and dual-robot regimes, respectively, over 10 trials per regime, and low collision counts. These findings support acoustophoretic levitation as a simple, glanceable, robot-mediated communication cue for embodied human-robot interaction in spatial analytics.

Benchmarking, Validation & Interpretability

#21 h=6

Hypothesis-driven Model Expansion under Uncertainty for Open-World Robot Planning

2026-07-07 cs.RO 👤 Anxing Xiao (h=6)

Anxing Xiao, Hanbo Zhang, Tianrun Hu, David Hsu

Core Contributions

Targets closed-world planning's core brittleness — pre-programmed knowledge bases failing on unexpected objects or actions — by having the robot generate, verify, and update hypotheses about its own world model rather than relying on a fixed ontology.
Integrates hypothesis verification directly into goal-reaching planning, not as a separate offline learning step, so the robot resolves world-model uncertainty while still making progress on the task.
Foundation models generate initial hypotheses over states and transitions; automated planning then produces action sequences that jointly test hypotheses and advance the task — coupling knowledge acquisition and task execution into one loop.
Iterative refinement from verification feedback corrects incorrect hypotheses mid-execution rather than requiring a full knowledge-base rebuild, validated in both simulated and real-world open-world settings.

Show abstract

We consider an open-world planning setting in which service robots must operate in unknown environments with incomplete knowledge of objects and actions. Traditional closed-world approaches with pre-programmed knowledge bases fail when robots encounter unexpected situations and tasks, posing a fundamental challenge for autonomous knowledge expansion in human environments. In this work, we propose an open-world planning framework that enables robots to automatically generate, verify, and update hypotheses about their abstract world models. Our key insight is to explicitly maintain uncertainty-aware knowledge expansion and integrate hypothesis verification into goal-reaching planning. The framework leverages foundation models to generate initial hypotheses over states and transitions, and applies automated planning to produce action sequences that jointly address hypothesis verification and task execution. Through iterative execution and refinement, the robot expands its knowledge by incorporating verification feedback from the foundation models when hypotheses prove incorrect. Extensive experiments in simulated and real-world environments demonstrate that our framework enables autonomous knowledge expansion and effective operation in open-world settings. These results indicate that integrating uncertainty-aware model expansion from robot foundation models with planning advances the practical deployment of household service robots.

#27 h=4

RoboVAST: Automated Scenario-Based Validation of Robots at Scale

2026-07-07 cs.RO 👤 Argentina Ortega (h=4)

Frederik Pasch, Samuel Wiest, Argentina Ortega, Nico Hochgeschwender

Core Contributions

Targets the reproducibility problem in robot validation directly: scenario selection today is manual and experience-driven, which RoboVAST replaces with a compositional, declarative scenario specification plus plugin-based generation and containerized execution.
Scale is the point: 5,480 scenario configurations, 100,000+ execution runs, 1,800+ simulated hours, 1,873 km traveled across five indoor maps — demonstrating the framework can run validation campaigns at a scale manual scenario design can't reach.
Twenty repetitions per configuration is a deliberate design choice enabling statistical separation of systematic failures from stochastic anomalies, addressing a subtle but common validation flaw where single-run failures get misattributed.
A purely infrastructure/methodology contribution rather than a new algorithm — its value is making other papers' navigation claims independently reproducible and scalable to test.

Show abstract

Validation of robotic systems critically depends on the operating conditions under which they are assessed. Scenario selection and variation are often manual, experience-driven, and difficult to scale, which harms reproducibility and weakens validation conclusions. We propose a scenario-based methodology that models scenarios compositionally and formalizes how these dimensions are varied, instantiated, executed, and interpreted. Building on this, we introduce RoboVAST, a framework that realizes declarative campaign specifications, plugin-based scenario generation, and scalable containerized execution with integrated result analysis. We demonstrate the approach with a navigation dataset comprising 5480 scenario configurations and over 100000 execution runs across five indoor maps with varied paths, sensor noise, software parameters, and obstacle settings, totaling more than 1800 hours of simulated operation and 1873 km traveled. Twenty repetitions per configuration allow us to distinguish systematic failures from stochastic anomalies.

#29 h=3

OrchardBench: A Physically-Grounded, GPU-Parallel Apple-Orchard Simulation Benchmark for Agricultural Robotics

2026-07-07 cs.ROcs.CV 👤 Humphrey Munn (h=3)

Humphrey Munn

Core Contributions

Builds trees as fully articulated, physically-parameterized bodies — Euler-Bernoulli beam-theory branches with a wood modulus of rupture, apples on tethers with literature-grounded detachment forces — bridging a gap between geometrically-detailed-but-inert graphics trees and physically-capable-but-tree-free GPU robot-learning simulators.
Every physical parameter traces to a published source, and per-environment domain randomization makes each parallel batched world a genuinely distinct tree, aimed at closing the agricultural sim-to-real gap through physical grounding rather than visual realism.
Runs many parallel environments at interactive rates on a single laptop GPU, making large-scale RL training for harvesting feasible without a server cluster.
Quant: the analytic baseline harvests only ~40% of detected fruit and ~1/8 of reachable fruit on a tree — an intentionally humbling baseline that leaves clear, well-defined headroom for future autonomy methods to be measured against.

Show abstract

Robotic tree-fruit harvesting is a flagship problem for agricultural automation, but progress is bottlenecked by the cost and irreproducibility of field experiments: an orchard is available only weeks a year, every tree is different, and a control error can permanently damage the crop or the plant. The tree models used in graphics and agronomy are geometrically detailed but physically inert, while the GPU-parallel simulators used in robot learning contain no plausible trees. We present OrchardBench, a physically-grounded, GPU-parallel simulation of apple-orchard trees on the Newton engine. Each tree is grown by a stochastic L-system and instantiated as a fully articulated body: branches are compliant torsional spring-dampers whose stiffness follows Euler-Bernoulli beam theory, they break at a wood modulus of rupture and fall as free hinges, and apples are independent bodies on stem tethers that detach at literature-grounded pull forces and load the branch when pulled. A moving, density-controllable foliage layer occludes the canopy as real leaves do. Every physical parameter is tied to a published source. Per-environment domain randomization makes each batched world a distinct tree, and a mobile manipulator with a wrist depth camera closes the loop with geometric fruit perception and an autonomous harvesting baseline. Careful engineering of the solver and the model lets OrchardBench run many parallel environments at interactive rates on a laptop GPU. We define the tasks and a metric suite spanning harvest completeness, throughput, and plant damage (with a per-canopy-zone breakdown), and report baseline results across foliage, fruit load, terrain, canopy zone, and parallelism. The analytic baseline succeeds on about 40% of the fruit it detects and harvests only about an eighth of the reachable fruit on a tree, leaving clear headroom for novel autonomy approaches.

#30 h=3

Driving the Wrong Way: Leveraging Interpretability in End2End Autonomous Driving Models

2026-07-07 cs.AIcs.CVcs.RO 👤 Franz Motzkus (h=3)

Franz Motzkus, Sebastian Bernhard

Core Contributions

Applies unsupervised dictionary learning as a post-hoc interpretability layer on top of an existing end-to-end driving model, decomposing its internal representation into semantically meaningful concepts rather than requiring an inherently interpretable architecture from the start.
Goes beyond passive interpretation to causal validation: demonstrates the extracted concepts causally influence the model's trajectory predictions, not just correlate with them.
Concept-level interventions let the authors manipulate and correct driving decisions directly, turning interpretability from a diagnostic tool into an active correction mechanism.
Reports measurable driving-performance improvements from these targeted interventions — evidence interpretability work can pay for itself in performance, not just transparency, for opaque end-to-end AD stacks.

Show abstract

The increasing adoption of end-to-end learning for autonomous driving introduces increased model complexity and opacity, raising the risk of learning undesired or erroneous behavior. In this work, we integrate unsupervised dictionary learning as a post hoc interpretability module within state-of-the-art driving models to decompose driving behavior into semantically meaningful concepts while demonstrating their causal influence on the model's driving decisions. We propose a stepwise framework for extracting and interpreting meaningful concepts from the end-to-end model and connecting them to the multifaceted model outputs, thereby revealing the underlying decision-making logic for the prediction of future trajectories. Furthermore, targeted interventions at the concept level allow us to manipulate and correct driving decisions, resulting in measurable improvements in overall driving performance. We thus demonstrate how interpretability can effectively be used to reduce model opacity, uncover erroneous behavior, and enable targeted mitigation, ultimately boosting model performance.