🤖 Robotics arXiv Digest

Wednesday, July 1, 2026

📄 30 papers 📂 7 research areas Generated by Claude

🔭 Research Landscape

If one word ties today's 30 papers together it is "world model", and the batch is unusually reflective about what the term should mean. Wei Zhang's tutorial (#9) tries to impose order, defining world models as action-conditioned predictive models and coining "world action models" as the bridge from predicted futures to control — and the rest of the day reads like a set of instantiations and stress tests of exactly that framing. ABot-M0.5 (#4) argues current World Action Models are structurally mismatched to mobile manipulation because they operate on coarse video chunks and entangle navigation with manipulation; the Structured 4D Latent Predictive Model (#16) attacks the same class from the geometry side, insisting that 2D video world models lack the 3D structure needed for physically consistent planning; and Path Planning in Physically Viable World Models (#29) pushes the idea outdoors, asking what-if terrain-change queries against a reconstruction rather than trusting a stale map. RoboWorld (#22) closes the loop by using video world models not to act but to evaluate policies, confronting head-on the two weaknesses — unreliable rollouts and slow inference — that the tutorial's optimism glosses over.

A second current is the field's growing habit of improving policies without touching their weights. Several of the strongest entries are test-time or adaptation-time interventions layered on frozen backbones: Domain Arithmetic (#12) adapts VLAs across camera-pose and robot-embodiment shifts through one-shot weight-vector arithmetic instead of collecting per-task demos; FAR (#18) turns each deployment failure into contrastive preference data so a robot stops repeating its own mistakes; and DART-VLN (#24) hardens navigation agents purely at inference with memory decay and an anti-loop penalty. AutoSpeed (#2) belongs to the same spirit, extracting stage-adaptive execution speed from demonstrations with no speed or stage labels. The throughline is a bet that generalist policies are bottlenecked less by parameters than by how state, structure, and the learning signal are marshalled at deployment — a theme that echoes recent weeks and again puts the most-cited manipulation authors (Haoqi Yuan's H-Tac tactile pre-training, #1; Wenchao Ding, #2) at the front of the ranking.

The third, quieter theme is honesty about uncertainty and safety in the estimation-and-control backbone, where much of today's methodological rigor lives. Three separate papers reach for conformal prediction to make guarantees distribution-free: robust operational-space control (#10) conformalizes disturbance bounds for redundant manipulators, and Conformalized Distance Fields (#27) conformalizes an entire predicted distance field at once rather than a scalar score, preserving spatial coherence as scenes get crowded. FastBridge (#15) makes the same move for aerial safety, replacing the fiction that commanded accelerations are instantaneous with an actuator-aware barrier enforced through full quadrotor dynamics, while GPU-parallel linearization bounds (#11) certify error for neural-network dynamics so learned models can enter real-time robust control. Autonomous driving forms its own reliability sub-plot — occlusion-aware planning via VLMs (#5), metric-agnostic forecasting that refuses to game benchmark metrics (#17), uncertainty-explicit V2X occupancy grids (#26), and a strategic guide (#28) that reframes dataset creation as a targeted intervention rather than default scale. Taken together the day reads as a field trying to make its most ambitious idea — predictive world models — actually trustworthy, whether through cleaner definitions, weight-free adaptation, or conformal safety.

VLA & Foundation Models

Vision-language-action policies and the systems around them: human-video tactile pre-training, unified mobility-manipulation world action models, weight-arithmetic adaptation, bimanual assembly, and fleet-scale serving.

#1 Human-Centric Transferable Tactile Pre-Training (H-Tac)
#4 ABot-M0.5: Mobility-and-Manipulation World Action Model
#12 Domain Arithmetic: One-Shot VLA Adaptation
#13 FurnitureVLA: Long-Horizon Bimanual Assembly
#20 ROSA: Robotics Foundation Model Serving System

Robot Learning, Rewards & Recovery

Learning signals and adaptation for manipulation: annotation-free stage-adaptive speed, demonstration-as-safety-envelope, VLM absolute/relative rewards, and failure-aware test-time retry.

#2 AutoSpeed: Stage-Adaptive Motion Speed Learning
#6 STT-LfD: Learning from Demonstration via Spatiotemporal Tubes
#7 VLM-AR3L: Absolute and Relative Rewards for RL
#18 FAR: Failure-Aware Retry for Test-Time Recovery

Navigation & Localization

Semantic and language-driven navigation: privacy-preserving depth-only 3D segmentation, VLM-based pose grounding, communication-enabled social navigation, and training-free VLN control.

#3 Privacy-Preserving Depth-Only 3D Semantic Segmentation
#21 Where Am I? Semantic Map Grounding via VLMs
#23 Robots Ask the Way: Communication-Enabled Social Nav
#24 DART-VLN: Test-Time Memory Decay & Anti-Loop

Autonomous Driving & Traffic Perception

Perception and planning for driving: VLM reasoning over planning-critical occlusions, metric-agnostic forecasting, V2X collective perception, and a strategic guide to dataset design.

#5 What's Hidden Matters: Occluded Agents via VLMs
#17 Towards Metric-Agnostic Trajectory Forecasting
#26 Beyond Line of Sight: V2X Collective Perception
#28 Creating Impactful Autonomous Driving Datasets

Multi-Robot & Aerial Systems

Coordination and field platforms: neural-accelerated collaborative capture, asynchronous distributed trajectory estimation, tilt-cued UAV tracking, and heterogeneous industrial fleet coordination.

#8 AMBUSH: Collaborative Capture with Neural Acceleration
#19 Asynchronous Distributed Trajectory Estimation
#25 Image-Domain Tilt Constrained UAV Tracking
#30 SCALE: Heterogeneous Multi-Robot Fleet Coordination

World Models & Predictive Simulation

Action-conditioned prediction and evaluation: a design-space tutorial, structured 4D latent prediction, neural simulators for policy evaluation, and physically viable planning models.

#9 From World Models to World Action Models (tutorial)
#16 Structured 4D Latent Predictive Model for Planning
#22 RoboWorld: Neural Simulators for Policy Evaluation
#29 Path Planning in Physically Viable World Models

Control, Safety & Teleoperation

Certified and disturbance-aware control: conformal operational-space bounds, GPU-parallel linearization error bounds, sensorless bilateral teleoperation, actuator-aware 3DGS safety filters, and conformalized distance fields.

#10 Robust Operational Space Control with Conformal Bounds
#11 GPU-Parallel Linearization Error Bounds
#14 Sensorless Four-Channel Bilateral Teleoperation
#15 FastBridge: Safety Filters on 3D Gaussian Splatting
#27 From Prediction Uncertainty to Conformalized Distance Fields

VLA & Foundation Models

#1 h=12

Human-Centric Transferable Tactile Pre-Training for Dexterous Robotic Manipulation

2026-07-01 cs.ROcs.CV 👤 Haoqi Yuan (h=12)

Chi Zhang, Penglin Cai, Ziheng Xi, Haoqi Yuan, Hao Luo

Core Contributions

Attacks the tactile-data bottleneck from the human side: builds H-Tac, a 160-hour egocentric human-video tactile-action dataset, sidestepping the small, narrow-contact robot datasets that cap tactile VLA performance.
Reframes tactile learning as human-to-robot transfer, pre-training contact representations from how people touch and manipulate rather than from dynamics-agnostic post-training on scarce robot tactile logs.
Targets the specific ceiling of tactile VLA: existing models bolt tactile on without modeling contact dynamics, whereas H-Tac aims to instill a dynamics-aware tactile prior that raises the downstream performance ceiling.
Positions egocentric human video as a cheap, scalable substitute for hardware-limited tactile collection — a bet that human demonstration data transfers usefully to robot hands.

Show abstract

As an essential modality for dexterous and contact-rich tasks, tactile sensing provides precise force feedback that cannot be reliably inferred from vision. However, limited by hardware and data collection systems, existing datasets with tactility remain small in scale and narrow in contact coverage. Meanwhile, Vision-Language-Action (VLA) models with tactile modality are constrained on dynamics-agnostic post-training, which limits the performance ceiling on downstream tasks. In this paper, we present H-Tac, a large-scale tactile-action dataset with 160-hour egocentric human videos containing rich and diverse contact interactions, together with a tactile pre-training framework that learns transferable, dynamics-aware tactile representations and transfers them to dexterous robotic manipulation.

#4 h=8

ABot-M0.5: Unified Mobility-and-Manipulation World Action Model

2026-07-01 cs.CVcs.RO 👤 Xinyuan Chang (h=8)

Ronghan Chen, Yandan Yang, Zuojin Tang, Dongjie Huo, Tong Lin

Core Contributions

Argues existing World Action Models are structurally mismatched to mobile manipulation: they operate on coarse video chunks and model navigation and manipulation as one entangled action stream, missing fine contact dynamics.
Unifies mobility and manipulation in a single world action model while disentangling the two action modes, directly countering the action-distribution conflicts that plague jointly-modeled navigation-plus-manipulation.
Fixes a training/inference mismatch where inverse dynamics is supervised in a way that does not match autoregressive rollout, a known source of compounding error at deployment.
Aims squarely at general-purpose robots that must both navigate and manipulate — a regime handled poorly by reactive VLAs (no world model) and by coarse video-chunk WAMs alike.

Show abstract

Mobile manipulation is a key capability for general-purpose robots, yet remains challenging for current embodied learning methods. VLA policies are typically reactive and lack explicit world modeling, while existing World Action Models (WAMs) are still poorly aligned with the structure of mobile manipulation: they operate on coarse video chunks, model entangled navigation-manipulation actions, and train inverse dynamics under supervision that does not match autoregressive inference. As a result, they often miss fine-grained contact dynamics, suffer from action-distribution conflicts, and accumulate error over long horizons. We present ABot-M0.5, a unified world action model for mobile manipulation that disentangles navigation and manipulation and aligns training with autoregressive inference.

#12 h=1

Domain Arithmetic: One-Shot VLA Adaptation under Environmental Shifts

2026-07-01 cs.ROcs.CVcs.LG 👤 Donghyun Shin (h=1)

Taewook Kang, Taeheon Kim, Donghyun Shin, Jonghyun Choi

Core Contributions

Attacks a common VLA failure: the same learned task breaks under environmental shifts such as a changed camera pose or a swap from a Panda to a UR5e arm.
Proposes one-shot adaptation via weight-vector arithmetic — adding a domain-specific information direction in weight space — instead of collecting multiple demonstrations per task in the target domain.
Analogy-based framing treats domain shift as a composable vector, so adaptation reuses structure rather than retraining, cutting the data-curation and training burden.
Aims to make VLA models portable across robots and viewpoints from minimal demonstrations, a practical lever for real deployment where re-collecting data is expensive.

Show abstract

Vision-Language-Action (VLA) models often fail to perform the same learned tasks under environmental shifts, such as changes in camera pose and shifts to a different but similar robot (e.g., from Panda to UR5e). Adapting these models to the shifted environment (i.e., target domain) often requires training on multiple demonstrations for each task, which are costly to collect. To reduce the burden of data curation and training, we propose an analogy-based method that adapts VLA models under environmental shifts through weight vector arithmetic with domain-specific information addition, named Domain Arithmetic. Our method enables one-shot adaptation to new domains by composing domain-specific weight directions, substantially reducing the demonstrations and training required.

#13 h=0

FurnitureVLA: Learning Long-Horizon Bimanual Furniture Assembly with Vision-Language-Action Model

2026-07-01 cs.ROcs.AI 👤 Chenyang Ma (h=0)

Chenyang Ma, Yue Yang, Radu Corcodel, Siddarth Jain, Andrew Wu

Core Contributions

First systematic study of real-scale bimanual furniture assembly with VLAs, moving past the toy-scale or single-arm settings that dominate prior assembly work.
Builds the full stack: a scalable simulation pipeline for expert-data generation and evaluation, plus a VR teleoperation system that lets a single operator provide bimanual demonstrations.
Handles extreme long horizons — up to 7 subtasks and 1550 control steps — with a progress-enhanced VLA that conditions on task progress to stay coherent across very long sequences.
Confronts the hardest axis of assembly, coordinated two-arm manipulation at real scale, that toy benchmarks conveniently sidestep.

Show abstract

Current work on robot furniture assembly mostly focuses on toy-scale settings or single-arm manipulation. We introduce FurnitureVLA, the first systematic study of real-scale bimanual furniture assembly using Vision-Language-Action models (VLAs). We formalize the task, develop a scalable simulation pipeline for expert data generation and evaluation, and build a VR teleoperation system for single-operator bimanual control to collect high-quality real-world demonstrations. To address extreme long-horizon assembly with up to 7 subtasks and 1550 control steps, we propose a progress-enhanced VLA, fine-tuned to condition on estimated task progress so that it remains coherent over very long action sequences.

#20 h=0

ROSA: A Robotics Foundation Model Serving System for Robot Factories

2026-07-01 cs.ROcs.DC 👤 Wenqi Jiang (h=0)

Wenqi Jiang, Jason Clemons, Rowland O'Flaherty, Hugo Hadfield, Alperen Degirmenci

Core Contributions

Rethinks robotics-foundation-model serving for factories, breaking the single-robot/single-model assumption that treats inference as an edge problem handled by an on-robot or dedicated nearby GPU.
Introduces shared GPU-pool serving so a fleet shares GPUs rather than dedicating hardware per robot, raising utilization across many robots and models simultaneously.
Designs around factory-scale objectives — throughput across a fleet — instead of minimizing a single robot's single-action latency, the traditional serving target.
A systems contribution to embodied-AI infrastructure, reframing RFM deployment as a datacenter-style serving problem for robot factories.

Show abstract

Robotics foundation models (RFMs) are making general-purpose robots increasingly practical for factory deployments. While RFM serving systems are central to this vision, existing systems are largely shaped by a single-robot, single-model assumption: inference is treated as an edge-computing problem handled by an on-robot or dedicated nearby GPU, and the serving objective is to minimize the latency of a single action model. In this paper, we propose ROSA, an RFM serving system for robot factories designed around three key principles. First, ROSA adopts shared GPU-pool serving, allowing a fleet of robots to share a pool of GPUs, improving utilization and throughput across many robots and models.

Robot Learning, Rewards & Recovery

#2 h=10

AutoSpeed: Annotation-Free Stage-Adaptive Motion Speed Learning for Robot Manipulation

2026-07-01 cs.RO 👤 Wenchao Ding (h=10)

Qingda Hu, Ziheng Qiu, Jieru Zhao, Zhongxue Gan, Wenchao Ding

Core Contributions

Challenges a hidden assumption in imitation-learned visuomotor policies: they copy expert execution speed and use a fixed prediction horizon, so easy and hard stages of a task run at the same pace.
Learns stage-adaptive motion speeds and prediction horizons — slowing for precise contact phases, accelerating through easy transit — to raise overall task throughput.
Key practicality: requires no speed or stage annotations, inferring stage difficulty from the demonstrations themselves, so it drops onto existing policies model-agnostically.
The payoff is faster task completion without retraining the base policy, treating temporal allocation as a free lever most IL policies leave on the table.

Show abstract

Different stages of manipulation tasks exhibit varying levels of difficulty, suggesting stage-dependent motion speeds and temporal prediction horizons. However, existing IL-based visuomotor policies typically imitate the execution speed of expert demonstrations and operate with a fixed temporal prediction horizon, limiting flexibility and overall task throughput. In this paper, we introduce AutoSpeed, a model-agnostic learning framework that enables existing visuomotor policies to predict trajectories with stage-adaptive motion speeds, without requiring speed or stage annotations. We treat future prediction horizons as adaptable and infer stage structure directly from demonstrations, yielding faster task completion while preserving success.

#6 h=6

Learning from Demonstration via Spatiotemporal Tubes for Unknown Euler-Lagrange Systems

2026-07-01 cs.ROeess.SY 👤 Ratnangshu Das (h=6)

Ratnangshu Das, Puneeth Shankar, Varuni Buereddy, Ravi Prakash, Pushpak Jagtap

Core Contributions

Reframes demonstrations not as a fixed reference to track but as a data-driven safety specification — an envelope of acceptable behavior rather than one trajectory to imitate.
Learns Spatiotemporal Tubes via heteroscedastic Gaussian Processes so the tube tightens where the task demands precision and widens where it tolerates slack, capturing time-varying intent.
Provides a closed-form feedback controller that enforces the learned tube while respecting actuator limits, for unknown Euler-Lagrange systems with no explicit dynamics model required.
Unifies motion learning and control in one framework, unlike decoupled LfD pipelines that first learn a path and then separately track it.

Show abstract

We present STT-LfD, a unified Learning from Demonstration (LfD) framework that integrates motion learning with control for unknown Euler-Lagrange systems. Unlike traditional decoupled approaches that track a fixed reference, the proposed method treats demonstrations as a data-driven safety specification. Using heteroscedastic Gaussian Processes, STT-LfD learns Spatiotemporal Tubes (STTs) as an intent envelope that capture time-varying precision requirements of a task. A closed-form feedback controller then enforces these learned constraints while respecting actuator limits, without requiring an explicit dynamics model of the system.

#7 h=5

VLM-AR3L: Vision-Language Models for Absolute and Relative Rewards in Reinforcement Learning

2026-07-01 cs.RO 👤 Min-Chun Hu (h=5)

Kuan-Chen Chen, Winston Chen, Wei-Fang Sun, Min-Chun Hu

Core Contributions

Tackles reward design in open-ended RL where goals are abstract and hard to quantify, using VLMs to supply the reward signal from visual observations plus a language goal.
Distinctive twist: learns both absolute rewards (a scalar value for a single state) and relative rewards (a preference between two states), rather than committing to one reward type.
Trains both reward models from VLM-generated preference labels, letting a pretrained VLM's judgment stand in for hand-engineered reward functions.
Targets tasks where scalar rewards alone are brittle, combining absolute grounding with relative comparisons for a more stable and informative learning signal.

Show abstract

Designing effective reward functions remains a major challenge in reinforcement learning (RL), particularly in open-ended environments where task goals are abstract and difficult to quantify. In this work, we present VLM-AR3L, a framework that leverages Vision-Language Models (VLMs) to provide both absolute and relative rewards for RL. VLM-AR3L interprets an agent's visual observations in the context of a natural language task goal, and learns both absolute and relative rewards from VLM-generated preference labels. The absolute reward model predicts scalar evaluations for individual states, while the relative reward model captures preferences between state pairs, together providing a richer supervisory signal.

#18 h=0

FAR: Failure-Aware Retry for Test-Time Recovery and Continual Policy Improvement

2026-07-01 cs.ROcs.AIcs.LG 👤 Haoran Hao (h=0)

Haoran Hao, Shahram Najam Syed, Jeffrey Ichnowski, Jeff Schneider

Core Contributions

Attacks naive retries that repeat the same mistake, and recovery methods that lean on human intervention, by letting robots learn from their own failures at test time.
Uses Failure-Contrastive Preference Adaptation, building preference data from failed attempts to steer the policy away from behaviors that just failed.
Adds lightweight test-time action adjustment so the robot adapts online and eventually completes the task autonomously, with no human in the loop.
Frames failure as a learning signal for continual policy improvement, turning each deployment failure into training data rather than a dead end.

Show abstract

Robot policies inevitably encounter failures when deployed in real environments. Naive retries often repeat the same mistakes, while many existing recovery methods rely on human intervention. In this paper, we propose Failure-Aware Retry (FAR), a framework that enables robots to learn from previous failures at test time, adapt their behavior accordingly, and eventually complete the task autonomously. FAR combines Failure-Contrastive Preference Adaptation, which constructs preference learning data from failures to steer the policy away from previously unsuccessful behaviors, with lightweight action adjustment at test time for continual improvement.

Navigation & Localization

#3 h=9

Privacy-Preserving Depth-Only Open-Vocabulary 3D Semantic Segmentation Via Uncertainty-Guided Test-Time Optimization

2026-07-01 cs.CVcs.RO 👤 Sicong Pan (h=9)

Xuying Huang, Sicong Pan, Maren Bennewitz

Core Contributions

Makes privacy a first-class constraint in open-vocabulary 3D segmentation: drops RGB entirely and uses only depth geometry, so no appearance-based, privacy-sensitive imagery is captured.
Confronts the hard consequence — depth alone makes open-vocab predictions highly uncertain — and turns that uncertainty into a guidance signal rather than treating it as a liability.
Uses uncertainty-guided test-time optimization to adapt predictions per-scene at inference, sharpening unreliable depth-only labels without any RGB supervision.
Opens a route to deploying 3D scene understanding in privacy-sensitive indoor settings (homes, hospitals) where continuous RGB capture is unacceptable.

Show abstract

Privacy-preserving perception is a critical requirement for deploying 3D scene understanding systems in real-world indoor environments, yet it remains underexplored in open-vocabulary 3D semantic segmentation. Existing methods typically rely on obtaining rich semantic cues from RGB images, which may expose privacy-sensitive visual information. Depth-only 3D geometry provides a privacy-preserving alternative, but the absence of appearance-based semantic cues makes open-vocabulary predictions highly uncertain and less reliable. Under this setting, we propose to convert uncertainty into a guidance signal for test-time optimization, refining depth-only open-vocabulary predictions without exposing appearance information.

#21 h=0

Where Am I? Semantic Map Grounding via Vision-Language Models for Multi-Modal Localization

2026-07-01 cs.RO 👤 Suraj Borate (h=0)

Suraj Borate, Aarav Shah, Madhu Vadali

Core Contributions

Reframes GPS-denied indoor localization as semantic reasoning rather than geometric estimation, mimicking how humans localize from object-level cues and labeled maps.
Asks a VLM to infer pose from three inputs at once — a front camera image, a polar LiDAR scan, and a top-down semantic grid map — fusing modalities inside one model.
Fine-tunes Qwen2.5-VL-7B with LoRA plus a lightweight regression head that predicts continuous (x, y, θ) directly from the final hidden state, bypassing text generation for precise numeric pose.
Demonstrates that VLMs can do metric localization, not just describe scenes, by grounding language-model reasoning in a semantic map.

Show abstract

We address robot localization in GPS-denied indoor environments by reframing it as a semantic reasoning task rather than a geometric estimation problem. Motivated by how humans localize using object-level cues and labeled maps, we ask whether a vision-language model, given a front camera image, a polar LiDAR scan, and a top-down semantic grid map, can infer the robot pose. We fine-tune Qwen2.5-VL-7B with LoRA and attach a lightweight regression head that predicts continuous pose coordinates (x, y, theta) directly from the final hidden state, bypassing text generation. Training uses a composite objective combining pose regression with semantic grounding.

#23 h=0

Robots Ask the Way: Communication-Enabled Social Navigation

2026-07-01 cs.RO 👤 Valentino Sacco (h=0)

Valentino Sacco, Luca Scofano, Indro Spinelli, Fabio Galasso

Core Contributions

Adds a proactive dimension to social navigation: instead of only reactive collision avoidance, robots actively request information from people to locate a target individual.
Defines Communication-enabled Social Navigation (CommNav), a new task in which the agent asks residents about recent sightings, locations, and movements to find someone.
Bridges the gap in current methods that adapt trajectories but have no mechanism to gather information through human-robot dialogue.
Aimed at assistive robots in multi-resident environments, where finding a specific person is far faster by asking than by exhaustive spatial search.

Show abstract

Assistive autonomous robots operating in multi-agent environments require efficient strategies to locate specific individuals among multiple residents. Current social navigation methods focus on reactive collision avoidance and trajectory adaptation, but lack mechanisms to proactively gather information through human-robot communication. We introduce Communication-enabled Social Navigation (CommNav). In this novel task, robotic agents actively seek assistance from residents to locate target individuals by requesting information about recent sightings, locations, and movements, integrating dialogue-driven cues into navigation decisions.

#24 h=0

DART-VLN: Test-Time Memory Decay and Anti-Loop Regularization for Discrete Vision-Language Navigation

2026-07-01 cs.ROcs.AI 👤 Shaoheng Zhang (h=0)

Shaoheng Zhang, Zhichen Li, Jie Mei

Core Contributions

Improves discrete vision-language navigation at test time only, with no retraining, targeting two failure modes of memory-based agents acting under partial observability.
Test-Time Memory Decay reweights stored memory at readout to suppress stale, redundant evidence — without rewriting the stored content itself.
Anti-Loop Regularization adds a lightweight next-hop penalty to curb inefficient local backtracking, a common cause of VLN agents getting stuck in loops.
Both are training-free control add-ons for frozen backbones, hardening existing VLN agents cheaply instead of retraining them.

Show abstract

Memory-based discrete vision-language navigation (VLN) agents must act under partial observability, yet even strong frozen backbones remain vulnerable at test time. Two common failure modes are stale historical evidence at memory readout and inefficient local backtracking during action selection. We present DART-VLN, a training-free test-time control framework for discrete VLN. DART-VLN combines Test-Time Memory Decay, a read-side memory reweighting rule that suppresses stale and redundant evidence without rewriting stored content, with Anti-Loop Regularization, a lightweight next-hop penalty that discourages repetitive backtracking.

Autonomous Driving & Traffic Perception

#5 h=8

What's Hidden Matters: Identifying Planning-Critical Occluded Agents using Vision-Language Models

2026-07-01 cs.ROcs.AIcs.CV 👤 Sangjae Bae (h=8)

Amirhosein Chahe, Tyler Naes, Jovin D'sa, Faizan M. Tariq, Sangjae Bae

Core Contributions

Rejects uniform conservatism toward occlusions in AV planning — which produces needlessly defensive driving — and instead asks which specific hidden agents actually matter to the plan.
Uses VLMs to reason about the particular occluded agents most critical to the ego trajectory, bridging perception and planning rather than blindly inferring hidden space.
Ranks occlusions by Planning KL-divergence (PKL), quantifying each hidden agent's impact on the planner so attention concentrates on planning-critical unknowns.
The upshot is less over-cautious behavior: the vehicle reasons about consequential hidden agents instead of treating every blind spot as equally dangerous.

Show abstract

Autonomous vehicles must safely navigate complex environments where planning-critical agents may be hidden from view. Current approaches often treat all occlusions with uniform conservatism, yielding needlessly defensive driving, or they infer hidden spaces without estimating the impact on the planner. This work bridges the critical gap between perception and planning by enabling Vision-Language Models (VLMs) to identify and reason about the specific hidden agents that are most critical to the ego-vehicle's trajectory. We introduce a novel framework that uses Planning KL-divergence (PKL), an importance measure that ranks occluded regions by their effect on the ego plan.

#17 h=0

Towards Metric-Agnostic Trajectory Forecasting

2026-07-01 cs.CVcs.RO 👤 Markus Knoche (h=0)

Markus Knoche, Daan de Geus, Bastian Leibe

Core Contributions

Exposes a benchmark-gaming problem: SOTA forecasters on Argoverse 2 and the Waymo Open Motion Dataset tailor their training objectives to each benchmark's metric, and those metrics reward conflicting behavior.
Proposes a paradigm shift — train with metric-agnostic probabilistic objectives and treat metric optimization as a separate downstream step applied to the predicted distribution.
Decouples what the model learns (a faithful trajectory distribution) from how it is scored, so a single model can serve multiple, conflicting metrics without retraining.
Reframes forecasting evaluation itself, arguing metric-specific training inflates leaderboard numbers without improving the underlying prediction.

Show abstract

Accurate trajectory forecasting of surrounding traffic participants is a core capability for autonomous driving, enabling vehicles to anticipate behavior and plan safe maneuvers. We observe that current state-of-the-art forecasting models on Argoverse 2 and the Waymo Open Motion Dataset tailor their training objectives to the different benchmark metrics. Because these metrics encourage conflicting behavior, we propose a paradigm change for trajectory forecasting: training models with metric-agnostic probabilistic objectives and treating metric optimization as a downstream task applied to the predicted distribution.

#26 h=0

Beyond Line of Sight: Hybrid Validation of V2X Collective Perception in Complex Scenarios

2026-07-01 cs.RO 👤 Markos Antonopoulos (h=0)

Markos Antonopoulos, Anastasia Bolovinou, Bill Roungas, Elena Daskalaki, Angelos Amditis

Core Contributions

Extends connected-vehicle perception beyond line of sight via V2X collective perception, fusing observations from multiple agents in complex traffic scenarios.
Uses a Bayesian fusion algorithm to merge heterogeneous sensor data into a shared probabilistic occupancy grid, where each cell carries both occupancy likelihood and uncertainty.
Emphasizes explainable, trustworthy situational awareness by keeping uncertainty explicit rather than collapsing to a hard occupancy decision.
Contributes a hybrid validation methodology bridging simulation and real-world testing, addressing the sim-to-real credibility gap for V2X systems.

Show abstract

This paper introduces a probabilistic framework and hybrid validation methodology for V2X-enabled Collective Perception (CP) in complex traffic scenarios. The proposed Bayesian fusion algorithm extends the perceptual horizon of connected and autonomous vehicles by integrating heterogeneous sensor observations from multiple agents into a shared probabilistic occupancy grid. Each cell of this grid encapsulates both occupancy likelihood and uncertainty, enabling explainable and trustworthy situational awareness beyond the ego vehicle's field of view. To bridge the gap between simulation and real-world deployment, we combine simulated and physical testing in a hybrid validation pipeline.

#28 h=0

Creating Impactful Autonomous Driving Datasets: A Strategic Guide from Research Gap to Benchmark

2026-07-01 cs.CVcs.AIcs.RO 👤 Richard Schwarzkopf (h=0)

Richard Schwarzkopf, Jonas Merkert, Frank Bieder, Annika Bätz, Alexander Blumberg

Core Contributions

Shifts autonomous-driving dataset literature from describing what datasets contain to prescribing how to design impactful ones, explicitly aimed at resource-constrained labs and startups.
Argues impactful dataset creation starts with a diagnosis: is the research question blocked by a data problem or by an evaluation problem?
Proposes selecting the minimal data operator(s) that closes the identified gap, recording new data only when nothing existing suffices, to avoid misallocating scarce resources.
A strategic guide rather than a dataset, reframing data collection as a targeted intervention instead of default large-scale gathering.

Show abstract

Well-designed autonomous driving datasets have fundamentally shaped research progress, yet existing literature primarily describes what datasets contain rather than how to strategically design impactful ones. This is especially limiting for small and medium-sized labs and startups that cannot afford to misallocate scarce resources. We argue that impactful dataset creation begins with a diagnosis: whether a research question is blocked by a data problem or an evaluation problem, and proceeds by selecting the minimal data operator(s) that closes the resulting gap, recording new data only when nothing existing suffices.

Multi-Robot & Aerial Systems

#8 h=3

AMBUSH: Collaborative Capture in Complex Environments with Neural Acceleration

2026-07-01 cs.RO 👤 Meng Guo (h=3)

Junfeng Chen, YinHang Luo, Xinyi Wang, Junrui Li, Meng Guo

Core Contributions

Brings collaborative target capture to cluttered, obstacle-dense environments, where prior analytical, geometric, and end-to-end RL methods largely assume obstacle-free or sparse-obstacle spaces.
Draws on the biological "ambush" strategy: rather than chasing, robots exploit environment structure to corner faster prey, turning obstacles into an asset instead of a hindrance.
Adds neural acceleration to make the strategy computationally tractable in complex maps, combining strategic reasoning with learned speedups.
Aimed at security, surveillance, and search-and-rescue, where dynamic targets must be captured amid real clutter rather than in open arenas.

Show abstract

Collaborative capture of dynamic targets is common in nature as an essential strategy for weaker species against the strong. Similar concepts have shown to be useful for numerous robotic applications, such as security and surveillance, search and rescue. However, most existing works focus on analytical and geometric solutions or end-to-end reinforcement learning methods, which are largely constrained to obstacle-free environments or scenarios with sparse, regularly distributed obstacles. This work tackles the problem from a unique perspective: the renowned strategy of "ambush" alone would suffice to capture faster targets by exploiting environmental structure, accelerated by a learned neural component.

#19 h=0

Technical Report: Asynchronous Distributed Trajectory Estimation of Multi-Robot Systems

2026-07-01 cs.RO 👤 (h=0)

Adam Pooley, Matthew Hale

Core Contributions

Addresses a gap in multi-robot distributed trajectory estimation: existing implementations assume synchronous communication and computation, which is unrealistic in the field.
Proposes an asynchronous block coordinate descent algorithm, letting agents update on their own schedules without lockstep coordination.
Derives an approximation of the maximum-a-posteriori estimation problem over a sliding window and shows the approximation introduces negligible error.
Concrete efficiency: eliminates up to 96.9% of communications, a large bandwidth saving for teams estimating each other's states over time.

Show abstract

Distributed trajectory estimation arises in many applications across robotics, but existing implementations typically do not consider asynchrony in agents' communications and computations. Therefore, we propose an asynchronous block coordinate descent algorithm for distributed trajectory estimation. We consider a team of agents that observes a team of robots and estimates their states over a sliding window. The agents solve an approximation of the maximum a posteriori estimation problem, which we derive. We show this approximation introduces negligible errors and eliminates up to 96.9% of communications compared to a synchronous baseline.

#25 h=0

Image-Domain Tilt Constrained Distributed Fusion for Maneuvering UAV Tracking with Multi-Camera Electro-Optical Observations

2026-07-01 eess.IVcs.ROeess.SP 👤 Minxing Sun (h=0)

Minxing Sun, Yao Mao

Core Contributions

Improves electro-optical tracking of small, maneuvering, intermittently-visible UAVs, where image and line-of-sight measurements constrain position but poorly constrain acceleration, so prediction lags during aggressive maneuvers.
Novel cue: uses the target rotorcraft's apparent roll and pitch (tilt) in the image as a low-level maneuver signal, since a banking drone visually betrays its intended acceleration.
Employs a weak-prior auto-labeling scheme and image-domain tilt constraints within a distributed multi-camera fusion framework.
Reduces short-horizon prediction lag during hard maneuvers by injecting visual tilt information the kinematic model alone misses.

Show abstract

Short-horizon prediction is essential for electro-optical UAV tracking, especially when the target is small, maneuvering, or intermittently observed. Image center, line-of-sight, and range measurements provide direct constraints on target position, but their constraints on acceleration are weak. As a result, prediction can lag during aggressive maneuvers. This paper proposes an image-domain tilt constrained distributed fusion method for maneuvering UAV tracking. The method uses the apparent roll and pitch of a rotorcraft target in the image as low-level maneuver cues, combined with a weak-prior auto-labeling scheme within a distributed multi-camera fusion framework.

#30 h=0

From Real-Time Planning to Reliable Execution: Scalable Coordination for Heterogeneous Multi-Robot Fleets in Industrial Environments

2026-07-01 cs.ROcs.MA 👤 Bo Cao (h=0)

Bo Cao, Zhe Liu, Hesheng Wang

Core Contributions

Coordinates dense, heterogeneous industrial robot fleets where real-time planning must simultaneously handle high robot density and differing motion capabilities.
Confronts execution reality: communication delays and disturbances make robots deviate from the temporal assumptions of planned paths, causing waiting and congestion propagation.
Presents SCALE, a reactive online coordination framework that bridges real-time planning and reliable execution, adapting when robots fall off schedule.
Targets scalable, reliable coordination in industrial settings, where a single delayed robot can cascade congestion across the whole fleet.

Show abstract

With the increasing deployment of heterogeneous robot fleets in industrial environments, efficient coordination remains a critical challenge. Real-time path planning must simultaneously accommodate high robot densities and heterogeneous motion capabilities, while communication delays, execution uncertainties, and other disturbances may cause robots to deviate from the temporal assumptions underlying planned paths. Such deviations can lead to excessive waiting and congestion propagation across the fleet. This paper presents SCALE, a reactive online coordination framework that enables real-time planning and reliable execution for large heterogeneous fleets.

World Models & Predictive Simulation

#9 h=2

From World Models to World Action Models: A Concise Tutorial for Robotics

2026-07-01 cs.ROcs.AIeess.SY 👤 Wei Zhang (h=2)

Xiaoxiong Zhang, Xiong Zeng, Wei Zhang

Core Contributions

A tutorial that imposes order on a term used inconsistently across communities, defining world models as action-conditioned predictive models of task-relevant future observations or states.
Offers a design-space taxonomy splitting observation-space versus state-space world models and comparing them on visual fidelity, spatial structure, physical interpretability, and control usability.
Introduces "world action models" as the bridge that connects predicted futures to executable control, unifying prediction and action under one framing.
Its value is conceptual clarity — a shared vocabulary and trade-off map rather than a new method — especially useful given how many papers in today's batch invoke "world models" with different meanings.

Show abstract

World models are increasingly used in embodied intelligence and generative simulation, yet their scope remains ambiguous across communities. This tutorial presents a design-space view of world models as action-conditioned predictive models that estimate the future evolution of task-relevant observations or states. We categorize existing methods into observation-space and state-space world models, comparing their trade-offs in visual fidelity, spatial structure, physical interpretability, and control usability. We further introduce world action models, which connect predicted futures with executable control, providing a unified vocabulary for the field.

#16 h=0

Structured 4D Latent Predictive Model for Robot Planning

2026-07-01 cs.ROcs.CV 👤 Zhiyi Li (h=0)

Zhiyi Li, Peilin Wu, Xiaoshen Han, Ruojin Cai, Yilun Du

Core Contributions

Argues prevailing video predictive models operate on 2D sequences and therefore lack the 3D geometric understanding needed for precise spatial reasoning and physical consistency in planning.
Predicts the evolution of a scene's 3D structure in a structured latent space, conditioned on observations and textual instructions, rather than forecasting raw pixels.
The structured 4D (3D + time) representation encodes the scene holistically, giving geometry-aware rollouts suited to robot planning.
Aims at long-horizon, spatially precise planning where 2D video world models produce physically inconsistent or geometrically implausible futures.

Show abstract

Video predictive models are emerging as a powerful paradigm in robotics, offering a promising path toward task generalization, long-horizon planning, and flexible decision-making. However, prevailing approaches often operate on 2D video sequences, inherently lacking the 3D geometric understanding necessary for precise spatial reasoning and physical consistency. We introduce a Structured 4D Latent Predictive Model, which predicts the evolution of a scene's 3D structure in a structured latent space conditioned on observations and textual instructions. Our representation encodes the scene holistically, enabling geometry-aware prediction for planning.

#22 h=0

RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation

2026-07-01 cs.RO 👤 Byeongguk Jeon (h=0)

Byeongguk Jeon, Seonghyeon Ye, JaeHyeok Doo, Sungdong Kim, Minjoon Seo

Core Contributions

Uses video world models to evaluate generalist policies without physical deployment, but confronts their two crippling weaknesses head-on: unreliable rollouts and slow inference.
Pairs a fast autoregressive video world model with task-progress-aware VLM scoring, so evaluation is both quick and grounded in whether the task is actually progressing.
Targets reliable long-horizon autoregressive rollouts — the regime where world-model error usually compounds and makes simulated evaluation untrustworthy.
Offers an automated evaluation pipeline as a scalable alternative to real-world policy testing, addressing throughput as well as fidelity.

Show abstract

Video world models are emerging as a scalable alternative for evaluating generalist robot policies, bypassing the physical constraints and engineering burdens of real-world deployment. However, evaluating policies with video world models remains challenging, as world-model errors can make generated rollouts unreliable and slow inference limits large-scale throughput. We introduce RoboWorld, an automated evaluation pipeline that pairs a fast autoregressive video world model with task-progress-aware vision-language model scoring, enabling reliable long-horizon rollouts at high throughput.

#29 h=0

Path Planning in Physically Viable World Models

2026-07-01 cs.RO 👤 Su Ann Low (h=0)

Su Ann Low, Cheng-Hsi Hsiao, Xingjian Li, Adam J. Thorpe, Ufuk Topcu

Core Contributions

Addresses robots planning from stale pre-deployment maps that assume terrain never changes, even when physical changes make old routes unsafe or unreachable at execution time.
Builds a physically viable world model that answers what-if queries about navigation under future terrain change, rather than trusting a frozen reconstruction.
Augments reconstructed scenes with physical reasoning so the planner can evaluate hypothetical terrain changes before committing to a route.
Aimed at unstructured outdoor missions on large or remote sites that cannot be remapped before every deployment.

Show abstract

Robots deployed in unstructured outdoor environments often plan from scene reconstructions collected before deployment because operators cannot remap large or remote sites before every mission. As a result, robots must make long-horizon planning decisions using stale maps that assume the terrain remains unchanged, even though physical changes to the environment may render previously feasible routes unsafe or unreachable at execution time. We present a physically viable world model for evaluating what-if queries for robot navigation under future terrain change. The system augments reconstructed scenes with physical reasoning to anticipate how the terrain may evolve.

Control, Safety & Teleoperation

#10 h=2

Robust Operational Space Control with Conformal Disturbance Bounds for Safe Redundant Manipulation

2026-07-01 cs.ROeess.SY 👤 Qin Lin (h=2)

Wenhua Liu, Fan Zhang, Qin Lin

Core Contributions

Targets redundant manipulators in constrained, human-interactive settings that need both accurate task-space tracking and rigorous safety guarantees under dynamic uncertainty.
Critiques two poles: classical computed-torque control degrades without accurate dynamics, while residual-learning approximations lack theoretical guarantees and rely on noisy full-state measurements.
Uses conformal prediction to build disturbance bounds, giving distribution-free guarantees rather than point estimates of the disturbance.
Delivers safe redundant manipulation with quantified uncertainty, positioning conformal bounds as the missing rigor between purely model-based and purely data-driven control.

Show abstract

Redundant robotic manipulators operating in constrained and human-interactive environments require accurate task-space tracking together with rigorous safety guarantees under dynamic uncertainties. Classical operational space computed torque controller (OSCTC) relies on accurate dynamic models and degrades in the presence of disturbances. In contrast, the data-driven paradigm of residual learning approximates disturbances as functions learned from full-state measurements, which are often noisy in practice, lack rigorous theoretical guarantees, and introduce additional design complexity. This paper proposes a conformal-prediction-based disturbance bound integrated into operational space control for distribution-free safety.

#11 h=1

GPU-Parallel Linearization Error Bounds for Real-Time Robust Optimal Control of Nonlinear and Neural Network Dynamics

2026-07-01 eess.SYcs.AIcs.LGcs.ROmath.OC 👤 Jeffrey Fang (h=1)

Jeffrey Fang, Keyi Shen, Anutam Srinivasan, Glen Chou

Core Contributions

Makes real-time robust optimal control of nonlinear and neural-network dynamics tractable by supplying sound linearization error bounds for the LTV approximations that make planning fast.
For analytic dynamics, introduces path-based Hessian bounds that are tighter than standard interval methods, reducing the conservatism of the guaranteed error.
For neural-network dynamics, derives certified bounds from NN-verifier-generated affine relaxations plus local Jacobian corrections, extending robust control to learned models.
Bounds are differentiable and GPU-parallel, so they slot into real-time planning while still guaranteeing robust constraint satisfaction.

Show abstract

This paper studies real-time robust optimal control for uncertain nonlinear systems, where linear time-varying (LTV) approximations make planning tractable but require sound linearization error bounds (LEBs) to guarantee robust constraint satisfaction. We develop tight, differentiable, GPU-parallel LEBs for LTV approximations of nonlinear and neural network (NN) dynamics. For analytic dynamics, we introduce path-based Hessian bounds that are tighter than standard interval methods. For NN dynamics, we derive certified LEBs using NN verifier-generated affine relaxations and local Jacobian corrections.

#14 h=0

Sensorless Four-Channel Control Architecture Using Inverse Dynamics Modeling for Human-Scale Bilateral Teleoperation

2026-07-01 cs.RO 👤 Amir Noohian (h=0)

Amir Noohian, Dylan Miller, Justin Valentine, Alan Lynch, Martin Jagersand

Core Contributions

Removes the force/torque sensors that four-channel bilateral teleoperation normally depends on, which are noisy, costly, and especially limiting at human scale.
Replaces them with inverse-dynamics modeling to reconstruct interaction forces, tackling the high inertia and modeling challenges of human-scale bilateral systems.
Validated on a customized WAM bilateral setup, showing the sensorless scheme outperforms conventional two- and four-channel and transparency-oriented controllers.
Makes transparent, high-fidelity teleoperation cheaper and more robust by trading hardware sensors for a dynamics model.

Show abstract

The four-channel teleoperation architecture is a well-established framework for achieving transparency in bilateral systems. However, its performance in human-scale teleoperation is limited by high inertia, modeling challenges, and reliance on noisy and costly force/torque sensors. This paper introduces a sensorless four-channel architecture based on inverse dynamics modeling. The controller is implemented and validated on a customized WAM bilateral teleoperation setup. Experiments demonstrate that the proposed approach outperforms conventional two- and four-channel schemes as well as transparency-oriented baselines.

#15 h=0

FastBridge: Closing the Model-Based Realization Gap in Safety Filters on 3D Gaussian Splatting for Fast Quadrotor Flight

2026-07-01 cs.RO 👤 Tscholl Dario (h=0)

Tscholl Dario, Nakka Yashwanth Kumar, Gunter Brian

Core Contributions

Closes a model-based realization gap in 3D Gaussian Splatting safety filters, which use single/double-integrator models that ignore actuator limits and assume commanded accelerations are realized instantaneously.
Introduces an actuator-aware, nonlinear safety filter enforced through the full quadrotor dynamics, so commanded accelerations are physically realizable even at high speed.
Builds on an analytic collision-cone barrier for 3DGS and derives a high-relative-degree collision constraint that respects the real vehicle dynamics.
Targets fast quadrotor flight under tight onboard compute, where the naive reduced-order safety assumption breaks exactly when speed makes collision avoidance hardest.

Show abstract

Fast quadrotor flight requires safe obstacle avoidance under tight onboard compute limits. While 3D Gaussian Splatting (3DGS) provides a continuous, geometry-aware scene representation for perception-driven navigation, existing 3DGS safety filters use reduced-order models such as single- and double-integrators that ignore actuator limits and assume commanded accelerations are realized instantaneously. Building on an analytic collision cone barrier for 3DGS, we introduce a nonlinear, actuator-aware safety filter enforced through the full quadrotor dynamics. We derive a high-relative-degree collision constraint that accounts for the true dynamics of the vehicle.

#27 h=0

From Prediction Uncertainty to Conformalized Distance Fields for Safe Motion Planning

2026-07-01 cs.ROeess.SY 👤 Jaeuk Shin (h=0)

Jaeuk Shin, Yoonseok Ra, Insoon Yang

Core Contributions

Improves safe motion planning under prediction uncertainty without sacrificing real-time performance, targeting a specific weakness in existing conformal approaches.
Prior methods conformalize a scalar score that aggregates per-obstacle prediction errors, losing spatial coherence and scaling poorly as scenes get crowded.
Instead conformalizes the entire predicted distance field at once — a functional conformal prediction that yields a field-level, distribution-free lower bound.
Safety follows uniformly: any trajectory satisfying the resulting constraint is certified safe, restoring spatial coherence that scales with scene density.

Show abstract

Safe motion planning in dynamic environments requires reasoning about the uncertainty in predicted obstacle motion without sacrificing real-time performance. Existing conformal approaches conformalize a scalar score that aggregates per-obstacle prediction errors, losing spatial coherence and scaling poorly with scene density. We instead conformalize the entire predicted distance field at once. This functional conformal prediction (FCP) framework yields a distribution-free, field-level lower bound, from which safety follows uniformly: any trajectory satisfying the resulting constraint is certified safe.