🤖 Robotics arXiv Digest

Monday, September 21, 2026

📄 30 papers 📂 8 research areas Generated by Claude

🔭 Research Landscape

Today's 30 papers split cleanly between a learning-heavy mainstream and a classical estimation-and-control backbone, and within the learning crowd the dominant move is to put explicit structure and memory back into otherwise opaque feed-forward policies. Chronos (#9) is the clearest statement of the thesis: it argues that VLA and generative imitation policies relying on the current observation take a 'Markovian shortcut' and elevates full observation history to the policy's latent state. SA-VLA (#26) makes the same complaint one level down, showing that a state-blind action tokenizer maps identical codes to wrong continuous actions, while ZR-0 (#16) injects dense Embodied Chain-of-Thought to align the shared high-level cognition across embodiments, SIR (#4) swaps opaque embeddings for sparsified scene graphs, and SPARK (#8) — a training-free neurosymbolic planner — more than doubles agentic and VLA baselines on LIBERO-PRO (43.7% vs 18.2%) precisely by spending compute on anchored perception rather than re-querying an LLM each turn. The recurring claim across these papers is that the binding constraint on generalist policies is structure, state, and memory — not more parameters or data.

A second strong current is manufacturing or transferring the supervision that policies need, especially across embodiment and sensor gaps. VLK (#10) reconstructs metric-scale scenes with Gaussian Splatting to synthesize vision-language-kinematics tuples no dataset provides; WARP (#7) and X-Morph (#22) both attack the human-to-robot embodiment gap, the former removing the action multi-modality that stops imitation from converging, the latter enforcing physical consistency when retargeting human motion onto quadrupeds and hexapods. STEAM (#19) extracts frame-level advantages from demonstrations with no reward labels, and ConCent (#23) pushes data efficiency to a single demonstration by making contact reproduction the central objective of sim-to-real. The tactile papers form their own version of this theme: Heterogeneous Tactile Transformer (#14) pools data across incompatible sensors toward a touch foundation model, and 'Seeing Touch from Motion' (#29) unifies image and motion-field tactile cues — together signaling that touch is following vision and language toward shared, transferable representations.

The third theme is a reminder that the field's estimation and control backbone is very much alive: the day's most-cited authors sit there, not in the VLA crowd. The top-ranked privacy-preserving cooperative localization paper (#1, h=35) recasts multi-robot localization as semidefinite programming over inscribed ellipsoids that exchanges only dual variables, and lateral platoon string stability (#2, h=33) extends a classical safety guarantee to the lateral axis as autonomy goes map-free. Around them sits a dense SLAM/perception cluster — closed-loop Gaussian SLAM with geometric feedback (MyGO-Splat #25), spherical multi-camera VIO (#30), test-and-check robust pose-graph optimization (#24), and point-supervised and panoramic tracking (#5, #28). Notably, MyGO-Splat's closed-loop geometry mirrors ReactiveBFM's (#3) closed-loop reactivity for humanoids: 'close the loop' is the day's quiet refrain, whether the feedback is geometric, physical, or cognitive. Taken together, the batch reads as a field converging on reliability — pursued through structured policies, cheaper transferable supervision, and feedback-driven estimation alike.

VLA & Foundation Models

Making vision-language-action policies force-aware, deployable on low-bit edge hardware, self-improving after deployment, and able to recover from their own failures.

#6 Opt2VLA: Force-Aware Vision-Language-Action for Cont...
#15 FoldQuantVLA: Native Low-Bit Quantization of Vision-...
#16 ME-Brain-1.0: Memory, Cognition and Action for Evolv...
#20 CARE: Experience-Guided Atomic Corrective Execution...

World Models & Generative Simulation

Predictive models of the future — as planners, as learned simulators, and as representations that should encode relational structure rather than pixels.

#2 Beyond Visual Quality: A Study of Test-Time Planning...
#14 Relationally Grounded Latent World Models for Autono...
#19 Uranus: Building the Next-Generation Simulation Infr...

Benchmarks & Diagnostic Evaluation

Protocols that expose what aggregate success rates hide: shortcut answers in LiDAR-LLMs, visual brittleness in VLAs/WAMs, H2R transfer scaling, and active-perception memory.

#1 Do LiDAR Language Models Really Understand Spatio-te...
#13 H2RBench: A Real-to-Sim Benchmark for Evaluating Hum...
#23 LIBERO-VPro: Benchmarking Closed-Loop Visual Robustn...
#30 ActiveArena: Benchmarking and Understanding Active P...

Tactile & Contact-Rich Manipulation

Touch as supervision, as world state, and as a pretrained modality — from distributed e-skins and single-point vibration sensors to grasp refinement and robotic guqin playing.

#9 TACIT: Tactile Contact Supervision for Spatial Atten...
#12 DexTacWAM: A Visuo-Tactile World-Action Model for De...
#17 GraspTune: Tactile-Driven Execution Refinement for R...
#18 Learning tactile perception from high-bandwidth sing...
#22 Tactile-JEPA: Topology-Aware Self-Supervised Represe...
#29 Phrase-Level Robotic Guqin Performance: Bimanual Mot...

Legged & Humanoid Skills

Generating quadruped skills from LLM-written reference motions and steering humanoid diffusion policies onboard in real time.

#3 MimicAgent: Quadruped Skills via Prompt-to-Trajector...
#26 PredActor: Predictive Action Diffusion for Steerable...

Multi-Robot & Human-Robot Collaboration

Coordination through language, bandwidth-aware consensus, shared factor graphs, pushing-only space transport, and help requests tuned to what humans actually know.

#4 Multi-Agent Transportation of Free-Flyers in Microgr...
#5 RoboTalk: Learning Multi-Robot Communication and Coo...
#7 Toward Human-in-the-Loop Robot Failure Recovery: Bri...
#11 AC-DC: Adaptive Communication for Scalable Dynamic A...
#25 CAST: Collision-Aware Assembly with Construction Rob...

Control, Planning & Safety

Classical guarantees for modern platforms: CBFs for hyper-redundant arms, switched adaptive aerial manipulation, time-optimal car paths with slip, and image-learned SE(3) navigation functions.

#10 Minimum Time Trajectories for a Car-Like Mobile Robo...
#21 SE(3) Neural Potential Fields for 6-DoF Trajectory P...
#24 Safety Control of a Hyper-redundant Robot via Adapti...
#27 A Switched Adaptive Control Framework for Aerial Man...

Perception & Field Robotics

Foundation-model perception adapted to messy field data — forest LiDAR across sensors and biomes, and depth from noisy analog FPV drone video.

#8 Toward a foundation model for forest point clouds
#28 AnalogDepth: Multi-view Geometry from FPV drones und...

VLA & Foundation Models

#6 h=48

Opt2VLA: Force-Aware Vision-Language-Action for Contact-Rich Humanoid Whole-Body Manipulation

2026-09-21 cs.RO 👤 Z. Kira (h=48)

Fukang Liu, Yipu Chen, Jaehwi Jang, Danfei Xu, Zsolt Kira

Core Contributions

Identifies a gap in humanoid VLAs: they output only geometric goals, yet contact tasks with similar motions can require very different forces, and vision degrades once contact occludes the scene.
Adds explicit continuous force references at the VLA-to-controller interface, tracked by task-specific RL whole-body controllers — force becomes a first-class action channel.
Scales supervision with whole-body trajectory optimization that produces dynamically feasible, contact-consistent force and torque labels instead of teleoperation data.
Enables language-conditioned force modulation (e.g., 'press harder') on real humanoid hardware, with torque supervision further improving force-tracking stability.

Show abstract

Humanoid robots are expected to perform diverse human-level tasks in daily environments, many of which require precise regulation of interaction forces. While recent vision-language-action (VLA) models have shown promise for semantic planning and visuomotor control, existing humanoid systems primarily represent actions through geometric motion goals and rely on whole-body controllers focused on motion tracking, with limited explicit reasoning or control of interaction forces. This limitation is particularly relevant in contact-rich tasks, where geometrically similar motions may require different force regimes depending on the task context and where visual observations may become unreliable after contact. In this work, we present Opt2VLA, a force-aware VLA framework that introduces explicit force commands at the VLA-to-control interface for humanoid whole-body manipulation. A single multi-task VLA policy jointly predicts both geometric motion goals and continuous contact-force references, which are tracked by task-specific reinforcement learning (RL)-based whole-body controllers. To provide scalable and physically grounded supervision, we generate dynamically feasible and contact-consistent training data via whole-body trajectory optimization (TO) with explicit force references. We evaluate Opt2VLA on three contact-rich humanoid tasks and show that explicit force conditioning enables more accurate and consistent force regulation than motion-only control, while physically grounded torque supervision from TO further improves force tracking accuracy and stability. Closed-loop evaluations further demonstrate language-conditioned force modulation with Opt2VLA in simulation and on humanoid hardware.

#15 h=17

FoldQuantVLA: Native Low-Bit Quantization of Vision-Language-Action Models via Consistent Folding

2026-09-21 cs.ROcs.AIeess.SY 👤 V. Ngo (h=17)

Hung T. Ho, Khanh D. Nguyen, Quang D. Nguyen, Thanh Q. Duong, Ngan Le

Core Contributions

Delivers true integer W4A4 VLA inference — not simulated quantization — with custom TensorRT plugins covering both the language backbone and the iterative action expert on Jetson AGX Orin and Ada GPUs.
Keeps one consistent activation representation across calibration, rounding and deployment (channel scaling + block Hadamard + dynamic per-token quantization), with no policy retraining.
Achieves 1.20–1.33× speedup over FP TensorRT on Orin and up to 1.52× on desktop across GR00T and π0.5 checkpoints.
A targeted mixed-precision recipe (8-bit attention-output and FFN down-projections) raises real-robot GR00T N1.7 success from 80.0% to 92.5% for only 1 ms extra latency — showing where precision actually matters.

Show abstract

Low-bit vision-language-action inference must reduce observation-to-action latency while preserving robot behavior. We present FoldQuantVLA, a post-training quantization framework that carries a consistent activation representation through calibration, weight rounding, and native integer execution. It combines channel scaling and block Hadamard transforms with dynamic per-token quantization, without policy retraining. Custom TensorRT plugins execute projections in both the language backbone and iterative action expert with four-bit weights and activations (W4A4) on Ada GPUs and Jetson AGX Orin. Evaluation spans LIBERO, SimplerEnv, and two robot platforms. Across three GR00T checkpoints and $π_{0.5}$, W4A4 achieves $1.20$ to $1.33\times$ speedups over floating-point TensorRT on Orin and $1.25$ to $1.52\times$ on desktop. Retaining language attention-output and feed-forward down projections at eight bits (W8A8) improves held-out action fidelity on all four checkpoints. Across four real-robot tasks, this configuration raises observed GR00T N1.7 success from $80.0\%$ with uniform W4A4 to $92.5\%$ over 80 trials per configuration, with a measured additional Orin latency of 1 ms.

#16 h=16

ME-Brain-1.0: Memory, Cognition and Action for Evolving Embodied Intelligence

2026-09-21 cs.RO 👤 Kun Zhan (h=16)

Wei He, Hengtao Li, Zhongrui Yu, Xuhan Zhu, Maokui He

Core Contributions

Moves embodied systems from train-and-freeze to deploy-and-evolve: experience from execution is consolidated and reused without retraining the model.
Three coupled modules — hierarchical Evolvable Memory, a Cognitive Core that distills experience into skills, and an Action Model that uses event-driven keyframes and local-world prediction to focus compute on decision-critical moments.
Reports gains across layers: +8.2/+9.6 points on embodied/agent benchmarks, 47.88% on RoboMME (+3.26), and on RoboDojo +9.12 points success over π0.5.
On a six-task real benchmark reaches 66.7% success, 11.7 points above DM0.5, suggesting memory-driven adaptation pays off on hardware.

Show abstract

Current embodied systems largely rely on pretrained capabilities that remain fixed after deployment, limiting their ability to learn from physical interaction. We introduce MachEmbodied-Brain (ME-Brain), a self-evolving embodied system organized around a closed loop of action execution, experience acquisition, experience evolution, and improved execution. Evolvable Memory consolidates multimodal trajectories into hierarchical, reusable experience; Cognitive Core transforms physical experience into transferable skills; and the Action Model combines event-driven keyframes, EventCell local-world prediction, and action-conditioned memory modulation to focus computation on decision-critical moments, regions, and historical evidence. Together, these modules shift embodied intelligence from train-and-freeze to deploy-and-evolve without model retraining. Cognitive Core outperforms the strongest comparison models by 8.2 and 9.6 points on embodied and agent benchmarks. The Action Model achieves 47.88% mean success on RoboMME, a 3.26-point improvement over the strongest baseline. On RoboDojo, it reaches a 21.51 mean Score and 16.03% success rate, exceeding $π_{0.5}$ by 10.10 and 9.12 points. On the six-task ME-RealBench, ME-Brain achieves a 69.5 mean Score and 66.7% success rate, outperforming DM0.5 by 12.8 and 11.7 points, respectively.

#20 h=11

CARE: Experience-Guided Atomic Corrective Execution for Vision-Language-Action Policies

2026-09-21 cs.RO 👤 Hu-Chuan Lu (h=11)

Junlan Xiao, Junwei Jiang, Zaibin Zhang, Yifan Wang, Zhongbo Zhang

Core Contributions

Generates recovery data from the policy's own real failures rather than hand-designed or random perturbations, so corrections target deviations that actually occur.
Models stage-conditioned post-failure deviation distributions and samples representative failure states plus corrective demonstrations from them.
At run time, 3D physical monitoring triggers small atomic adjustments or re-operations while keeping completed task progress, instead of restarting.
Gains +14.5 points in sim and +15.9 on real dual-arm tasks across VLA backbones, and introduces FSR-Bench for evaluating recovery from intermediate failure states.

Show abstract

Vision-Language-Action (VLA) policies achieve strong performance in robotic manipulation but remain brittle once execution deviates from nominal trajectories. We propose CARE (Corrective Atomic Robotic Execution), a framework that improves recovery by learning from failures encountered during execution. Instead of generating corrective data from manually designed or random perturbations, CARE collects failed rollouts, models stage-conditioned post-failure deviations, and uses the resulting empirical distributions to synthesize representative failure states and corrective demonstrations. At inference time, CARE combines stage-wise planning with physically grounded 3D monitoring to trigger atomic adjustments or re-operations while preserving task progress. We further introduce the Failure State Recovery Benchmark (FSR-Bench), which evaluates recovery from intermediate failure states under local deviations and structural anomalies. Experiments across multiple VLA backbones, simulation benchmarks, and real-world dual-arm tasks show consistent improvements, with average task-success gains of 14.5 points in simulation and 15.9 points in the real world. Code, models, and data are available at https://github.com/xiaojunlan/care

World Models & Generative Simulation

#2 h=111

Beyond Visual Quality: A Study of Test-Time Planning with World Action Models

2026-09-21 cs.RO 👤 Philip H. S. Torr (h=111)

Jianhao Yuan, Yu Yuan, Benjamin Ramtoula, Lukas Vierling, Paul Newman

Core Contributions

Asks whether a World Action Model's imagined futures are actually useful for choosing actions, instead of judging them by how realistic the video looks.
Quantifies the headroom: an oracle picking the best of several sampled actions from the same state lifts success from 68.9% to 79.2% — about 10 points of latent planning value.
Visual-quality, physical-consistency and task-progress selectors recover only a fraction of that gap; even a learned value adds little, meaning current WAMs generate consequential choices but don't make the consequences legible.
Counterfactual branching shows the opportunity is concentrated in a few decisive states and that more action diversity doesn't imply more outcome coverage — a guide for where test-time compute should be spent.

Show abstract

World action models generate actions together with visual predictions of their consequences. These paired outputs create the potential for planning by sampling multiple actions from one state, comparing their imagined outcomes, and choosing the action with the most promising predicted outcome. However, how to use imagined futures to guide action selection remains unclear. We examine this planning potential empirically. First, we estimate an oracle upper bound on selection by choosing the sampled candidate whose realised outcome is best. In a controlled same-state analysis, this choice raises success from 68.9% under uniform random selection to 79.2%. We then test selectors based on visual quality, physical consistency, and task progression as controlled interventions. Some tested selectors yield higher observed success, but the gains are uneven and the matched selectors leave much of the measured opportunity unrecovered. To investigate this gap, we examine whether sampled actions lead to different outcomes, whether these differences are visible in the predictions, and whether a score recognises them. Counterfactual branching from the same states shows that selection opportunity is concentrated in relatively few decisions in the initial candidate sets. Action spread and outcome coverage need not increase together. In a further evaluation across trajectory phases with complete action execution, the tested scores again recover little of the available improvement despite a small gain from learned value. These findings distinguish producing consequential action choices from recognising them in generated futures, motivating the evaluation of WAM predictions through their usefulness for decisions rather than visual quality alone.

#14 h=17

Relationally Grounded Latent World Models for Autonomous Driving

2026-09-21 cs.CVcs.RO 👤 A. Valada (h=17)

Fabian Schmidt, Markus Enzweiler, Abhinav Valada

Core Contributions

Tests whether traffic scene graphs can act as privileged, training-only supervision that makes a latent driving world model encode who-relates-to-whom explicitly.
Serializes actor-centric scene graphs from nuScenes annotations, embeds them with a frozen text model, and aligns the LAW latent to them; the branch is dropped at inference with zero added cost.
Cuts collision rate by 52% (0.456→0.217) and trajectory L2 by 5.9% vs a retrained LAW baseline, and beats caption-style semantic targets — structure, not just language, is what helps.

Show abstract

Latent world models learn predictive representations for autonomous driving, but the relational semantics these states preserve often remain implicit. We investigate whether traffic scene graphs can serve as privileged semantic supervision for latent world representations. Building on LAW, we construct actor-centric scene graphs from nuScenes 3D annotations, encode their serialized relational structure using a frozen text embedding model, and align the visual latent representations with this semantic target during training. We remove the supervision branch at inference, so it requires neither scene graphs nor 3D annotations and adds no test-time computation. On nuScenes, our method reduces average trajectory L2 error from 0.661 to 0.622 (5.9%) and collision rate from 0.456 to 0.217 (52.4%) relative to our retrained LAW baseline. It also outperforms an unstructured caption-style semantic target, supporting the benefit of explicit relational structure for latent world-model representation learning.

#19 h=11

Uranus: Building the Next-Generation Simulation Infrastructure for Embodied AI

2026-09-21 cs.ROcs.AI 👤 Wen-Kang Qin (h=11)

Wenkang Qin, Yukun Zhou, Noah Shen, Jisong Cai, Dongxiao Mao

Core Contributions

Builds a learned simulator conditioned on joint-position trajectories rather than a hand-constructed physics scene, aiming to remove the labor of asset and scene building.
Streams open-ended rollouts autoregressively — one latent frame (four RGB frames) per step with no fixed horizon — fed by future joint trajectories online.
Runs at 24 FPS after optimization with synchronized multi-view output across embodiments and camera setups, making interactive policy evaluation plausible.
Releases code and weights and explicitly documents OOD limitations, useful for the community weighing video-based simulators against physics engines.

Show abstract

Scalable simulation is essential for robot data generation, policy training, evaluation, and safe iteration, yet real-world interaction is costly and conventional simulators require labor-intensive construction. We present Uranus, a data-driven robot simulator built around a joint-trajectory-conditioned autoregressive diffusion model. Uranus offers three key capabilities: (1) streaming, open-ended rollout, which receives future joint-position trajectories online and autoregressively generates one latent frame per step, corresponding to four RGB frames, without a fixed horizon; (2) low-latency generation, achieving 24 FPS after inference optimization; and (3) scalable, extensible robot control, providing a unified interface for synchronized multi-view generation across diverse robot embodiments and camera configurations. We conduct comprehensive quantitative and qualitative evaluations on both in-distribution and out-of-distribution data, providing an objective assessment of Uranus and clearly identifying its current limitations. We release the code and model weights to empower the community with practical tools and insights.

Benchmarks & Diagnostic Evaluation

#1 h=140

Do LiDAR Language Models Really Understand Spatio-temporal Relationships?

2026-09-21 cs.CVcs.AIcs.RO 👤 L. van Gool (h=140)

Runyi Yang, Murat Akkoyun, Di Wen, Ruiping Liu, Yufan Chen

Core Contributions

Opens with an uncomfortable baseline: a constant 'always pick the same option' answer nearly matches the multiple-choice accuracy of B4DL-derived 4D LiDAR-LLMs, so existing accuracy numbers say little about spatio-temporal understanding.
Designs controls rather than just more questions — fixed-answer and candidate-content probes, and cross-scene pairs with identical prompts but opposite ground truth — to test whether a model actually discriminates physical relations.
Across 100k logged responses, temporal questions turn out solvable from candidate duration alone without seeing LiDAR, and both models miss every positive lateral-motion case — a concrete blind spot for driving safety.
Shows that an inference-time fix (temporal-shuffle contrastive decoding) mostly trades old errors for new ones, arguing the deficit is in the representation, not the decoding.

Show abstract

Recent 4D LiDAR language models aim to reason about objects and their evolving spatial relationships. Yet, in our evaluation, always selecting the same option nearly matches the multiple-choice accuracy of two B4DL-derived configurations. We introduce LiDAR-Hallu, a geometry-referenced benchmark and diagnostic protocol with 10,000 questions across 150 nuScenes scenes. It covers object existence, ego-relative position, distance ordering, relative motion, and temporal localization, with explicit rules for selecting objects, comparing times, and determining reference answers. Our protocol combines fixed-answer and candidate-content controls, cross-scene pairs with identical prompts but opposite reference answers, and relation-specific recall. Analysis of 100,000 recorded responses reveals failures hidden by aggregate accuracy. Candidate duration alone makes temporal answers predictable without observing LiDAR. On paired questions, the models frequently give the same answer to scenes requiring opposite answers. Relation-specific analysis further shows that both configurations miss every positive lateral-motion case across all tested conditions. Temporal-shuffle contrastive decoding provides little net improvement, as repairs are largely offset by new errors and the main failures persist. These results show that evaluating spatio-temporal reasoning requires testing whether models distinguish the queried physical relationships, rather than relying on individual-answer accuracy alone. The source code, checkpoints, and data are released at https://github.com/Awesome4D/4DMLLM_Hallucination_Bench.

#13 h=17

H2RBench: A Real-to-Sim Benchmark for Evaluating Human-to-Robot Transfer

2026-09-21 cs.RO 👤 Muhammad Zubair Irshad (h=17)

Chuyang Xiao, Haotian Zhan, Sriram Krishna, Peilin Meng, Muhammad Zubair Irshad

Core Contributions

Addresses the apples-to-oranges problem in human-to-robot transfer, where methods are reported on different tasks, scenes, objects and amounts of robot data.
Pairs real human video demos with simulated robot demos in a standardized Real2Sim protocol over four tasks, enabling controlled scaling studies of how much each method benefits from more human data.
Finds methods differ sharply in their ability to exploit additional human video — a key property that single-point comparisons miss.
Validates the proxy: sim rankings track real-robot results with Pearson r=0.89, Spearman ρ=0.85 and MMRV 0.06, so cheap sim evaluation can precede hardware trials.

Show abstract

Learning robot manipulation policies from human video demonstrations constitutes a promising avenue for scalable robot learning. However, comparing different human-to-robot (H2R) transfer methods remains challenging, as existing approaches are evaluated under different settings, including differing task suites, scene layouts, object instances, and amounts of robot supervision. To address this challenge, we present H2RBench, a Real2Sim benchmark for evaluating H2R transfer methods. H2RBench provides a standardized protocol built on real human video demonstrations and simulated robot demonstrations, and includes four manipulation tasks spanning diverse interaction requirements. We evaluate multiple representative H2R transfer methods, each adopting a different strategy for bridging the embodiment gap. Using H2RBench, we systematically characterize how each method scales with the amount of human demonstrations, revealing that methods differ substantially in their ability to leverage additional human data. We further show that simulation performance is broadly predictive of real-world robot performance, with an overall Pearson correlation of r = 0.89, Spearman correlation of \r{ho} = 0.85 and Mean Maximum Rank Violation (MMRV) of 0.06 across method-task configurations. These results establish H2RBench as a practical and scalable benchmark for comparative H2R evaluation prior to real-world deployment.

#23 h=10

LIBERO-VPro: Benchmarking Closed-Loop Visual Robustness of Robotic Foundation Models

2026-09-21 cs.ROcs.CV 👤 Anirudha Majumdar (h=10)

Huiqiong Li, Zhiting Mei, Anirudha Majumdar, Jingjing Chen, Yu-Gang Jiang

Core Contributions

Moves visual robustness testing into closed loop: perturbations occur during execution, including stale, missing or inconsistent camera streams that static corruption benchmarks never model.
Scale: 12 challenge categories, 3,296 task-condition cases, ~196k simulated episodes over three VLAs and three WAMs, plus 200 Franka rollouts.
Counter-intuitive finding: models tolerate heavy object occlusion but collapse when local interaction cues or familiar spatial priors are disturbed — they lean on layout memorization.
VLAs and WAMs show different robustness profiles, so 'which architecture is more robust' has no single answer.

Show abstract

Robotic foundation models achieve impressive performance on standard manipulation benchmarks, yet these evaluations typically assume clean, timely, and consistent visual observations throughout execution. We introduce LIBERO-VPro, a benchmark for systematically evaluating the closed-loop visual robustness of robotic foundation models by perturbing the visual evidence available during execution. LIBERO-VPro covers four complementary dimensions, including Visual Evidence Degradation, Camera Staleness, Visual Source Consistency, and Task-Relevant Scene Variation, spanning 12 challenge categories, 96 experimental settings, and 3,296 task-condition cases. We evaluate three vision-language-action models and three world-action models over approximately 196,000 simulated episodes, complemented by 200 real-world rollouts on a Franka Research 3. Our results reveal that strong nominal performance can mask substantial weaknesses in visual grounding and adaptation. Models often remain successful despite severe object-level occlusion, yet degrade sharply when local interaction cues are disrupted or familiar spatial priors are violated. They are also highly sensitive to stale or missing observations and struggle when changed task preconditions require behavioral adaptation. Finally, VLAs and WAMs exhibit distinct robustness profiles, showing that visual robustness is multi-dimensional and architecture-dependent. LIBERO-VPro provides a systematic diagnostic framework for developing robotic foundation models that can more reliably ground and adapt their actions under challenging visual conditions.

#30 h=9

ActiveArena: Benchmarking and Understanding Active Perception in Robotic Manipulation

2026-09-21 cs.ROcs.AI 👤 Enshen Zhou (h=9)

Yibo Li, Enshen Zhou, Rui Chen, Yanjun Ding, Mengzhen Liu

Core Contributions

Targets active perception — deciding where to look and what to remember — which passive-observation manipulation benchmarks can't measure.
Provides a controllable-viewpoint simulator and 35 tasks across 5 categories that require multi-round evidence gathering and memory, with ID/OOD splits over scenes, distractors and backgrounds.
A modular suite of 13 VLA configurations isolates memory writing, capacity, proprioception, subtask supervision and planning, revealing a large ID–OOD gap.
Planner-guided memory reaches near-best performance with only sparse memory, suggesting what to store matters more than storing more.

Show abstract

Active perception and manipulation are crucial for robots to interact with complex scenes. Existing benchmarks struggle to evaluate how robots effectively acquire and maintain information in memory in an active manner. To this end, we introduce ActiveArena-Sim, an active-perception simulator with controllable viewpoints and large-scale workspaces as the foundation. Built on this, we propose ActiveArena-Bench, which comprises 35 tasks across 5 fine-grained categories, covering visual exploration and interactive information acquisition. Each task is difficult to solve from passive observations alone, requiring multi-round evidence acquisition and memory-based reasoning. The benchmark provides rich memory annotations, standardized training data, and ID/OOD protocols featuring disjoint scenes, unseen distractor configurations, and novel backgrounds. Moreover, we present ActiveArena-VLA, a modular suite of 13 vision-language-action configurations for controlled studies of memory writing, memory capacity, proprioceptive state, subtask supervision, and high-level planning in active perception. Benchmark results reveal a substantial ID-OOD gap: uniform memory sampling, increased memory capacity under reliable write policies, proprioceptive inputs, and subtask supervision improve OOD generalization, while planner-guided memory management and decision-making achieve performance close to the best-performing configuration using only sparse memory. ActiveArena thus provides a unified testbed to develop and diagnose models for active perception and manipulation.

Tactile & Contact-Rich Manipulation

#9 h=30

TACIT: Tactile Contact Supervision for Spatial Attention in Dexterous Manipulation

2026-09-21 cs.RO 👤 K. Hashimoto (h=30)

Yanhou Lai, Fucai Zhu, Ruiqiang Wang, Koichi Hashimoto

Core Contributions

Uses tactile contacts measured during teleoperated demos as free supervision for where a policy should look, replacing human point annotation or VLM-derived spatial priors.
Contact events are projected as Gaussian targets onto earlier point clouds to train an attention head; the targets are used only in training, so inference cost is unchanged.
With just 10 demos per task, reaches 66.7% (ball placement) and 73.3% (peg insertion) vs 10–20% for input-matched visuotactile fusion and 20–43% for DP3.
Always reaches the object's approach region within 12 s, localizing the win to spatial generalization; an architecture-matched no-supervision control confirms the gain comes from the tactile signal, not added capacity.

Show abstract

Visuomotor policies trained from a few demonstrations may reproduce demonstrated trajectories without reliably following changes in object position. Existing approaches with explicit attention typically obtain spatial priors from human annotation or visual models. We introduce TACIT (tactile contact informs attention), which uses measured tactile contacts from teleoperated demonstrations to supervise spatial attention without additional point annotation. Gaussian targets over preceding camera point clouds supervise an attention head whose pooled output conditions a visuotactile diffusion policy. Targets are used only during training; tactile observations remain inputs at inference. In the primary real-robot benchmark, with ten demonstrations per task and five demonstrated placement regions, TACIT achieves 66.7% success on ball placement and 73.3% on peg insertion, compared with 10.0% and 20.0% for input-matched 3D visuotactile fusion and 20.0% and 43.3% for vision-only DP3. TACIT enters the 150 mm palm-to-object approach region within 12 seconds in all 30 trials per task; all remaining failures occur after arrival. Across three training seeds on real ball and simulated peg, TACIT outperforms input-matched fusion and an architecture-matched control without explicit attention supervision, supporting the contribution of supervision beyond branch capacity. Pre-contact and contact-time supervision show no consistent ordering. These results demonstrate that measured tactile contact provides effective spatial supervision for approach behavior from few demonstrations within the evaluated workspace.

#12 h=19

DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation

2026-09-21 cs.ROcs.AIcs.CV 👤 Ismini Lourentzou (h=19)

Haoran Yuan, Zekai Wang, Boning Shao, Haoran Lu, Trevor Darrell

Core Contributions

Extends World-Action Models beyond vision by making fingertip contact part of the predicted world state, not just an input — vision alone can't observe the contact dynamics that dexterous tasks hinge on.
A finger- and pose-aware tactile compressor fuses per-fingertip encodings into a latent injected into a video diffusion world model; it keeps 89.4% of contact recall while training 2.26× and inferring 1.29× faster.
On six contact-rich bimanual tasks with a 22-DoF platform it averages 70.6 vs 38.0 for the best baseline; removing tactile world modeling (while keeping the same tactile features) drops the mean from 74.7 to 26.6.
Adds touch to a pretrained video model in ~4 hours with ~100 demos per task and no tactile midtraining, preserving visual prediction within 0.5 dB — a cheap path to multimodal world models.

Show abstract

Dexterous manipulation depends on contact dynamics that are often only partially observable from vision. Recent World-Action Models (WAMs) couple predictive video world modeling with action generation, but remain largely vision-centric and therefore cannot directly model these contact dynamics. We present DexTacWAM, a visuo-tactile WAM that encodes each fingertip independently, aggregates the resulting features through a finger- and pose-aware tactile compressor, and injects the tactile latent into a video diffusion world model for joint visuo-tactile world modeling. Across six contact-rich dexterous manipulation tasks on a 22-DoF bimanual platform, DexTacWAM achieves the highest score on every task, averaging 70.6 versus 38.0 for the strongest baseline. Ablations attribute the gain to modeling contact evolution as part of the predicted world state rather than tactile conditioning alone: removing tactile world modeling reduces the four-task mean from 74.7 to 26.6 while keeping the same tactile features and action expert. After four hours of tactile-encoder adaptation with a frozen pretrained vision VAE, our continual vision-to-touch learning extends the pretrained video model to touch using roughly 100 demonstrations per task without tactile midtraining, while retaining visual prediction quality within 0.5 dB of vision-only counterparts. The compressor retains 89.4% of pre-fusion contact recall while enabling 2.26x faster training and 1.29x faster inference. Together, these results show that pretrained video priors can be extended to distributed multi-finger contact dynamics in a data- and compute-efficient manner.

#17 h=16

GraspTune: Tactile-Driven Execution Refinement for Robust Grasping

2026-09-21 cs.RO 👤 Daqiang Guo (h=16)

Juntao Li, Xingke Xia, Sichao Liu, Daqiang Guo

Core Contributions

Targets the execution stage of grasping — turning a visually plausible proposal into a stable physical grasp — which proposal-quality research largely ignores.
Learns control-facing contact semantics (contact change, contact risk, post-close readiness) from depth, tactile and history, conditioning a diffusion-pretrained residual policy refined with PPO.
Proposal-agnostic: adds +9.6 to +20.7 points of stable-grasp success on top of GraspNet, Contact-GraspNet, AnyGrasp and VGN across 60k+ simulated executions, and lifts unseen categories from 54.6% to 70.3%.
Transfers to a UR5e with Xense sensors without real fine-tuning, raising GraspNet from 71.0% to 84.3% over 1,000+ trials.

Show abstract

Visual grasp proposal generation has advanced rapidly, yet converting a selected proposal into a stable physical grasp remains a central execution-stage challenge. This paper introduces GraspTune, a tactile-driven execution-stage refinement framework that starts from a nominal proposal and applies bounded residual TCP motions during approach, contact formation, and final grasp execution. GraspTune learns control-facing contact semantics from local depth, tactile signals, state, and history using state-conditioned expert contact queries and multi-task supervision for contact change, contact risk, and post-close readiness. The representation conditions a diffusion-pretrained residual policy and is aligned with PPO for closed-loop execution. Across more than 60,000 simulated executions over 20 object categories, GraspTune establishes an execution-layer benefit across four proposal generators, raising stable grasp success by +19.22, +9.55, +12.45, and +20.70 percentage points for GraspNet, Contact-GraspNet, AnyGrasp, and VGN. A four-fold held-out category study raises unseen-object execution from 54.58% to 70.33%, showing category-disjoint generalization of contact correction. Across more than 1,000 real-robot trials on a UR5e setup with Xense fingertip sensors, GraspTune raises GraspNet execution from 71.0% to 84.3%, validating direct transfer without realworld policy fine-tuning. Together, these results turn visually plausible proposals into stable physical grasps for downstream contact-rich manipulation. A supplementary video is available at https://youtu.be/kcq7fSLNtzU.

#18 h=12

Learning tactile perception from high-bandwidth single-point sensing

2026-09-21 cs.ROcs.LG 👤 E. Virot (h=12)

Joseph Rigal, Emmanuel Virot, Caroline Pascal

Core Contributions

Challenges the push toward dense tactile arrays: a single high-bandwidth sensor (up to 100 kHz) can carry rich contact information through vibration dynamics.
Converts 1D tactile histories into time–frequency spectrograms so standard vision encoders and visuomotor pipelines can consume them unchanged.
Mounts sensors away from the contact surface but mechanically coupled to it, improving durability — a practical win for long-term dexterous deployment.
Demonstrates occluded manipulation from vibration alone and shows the representation works across accelerometers, force and strain sensors, including off-the-shelf hardware.

Show abstract

Tactile sensing is increasingly being incorporated into learning-based robotic manipulation, yet many existing approaches rely on spatially distributed sensors. Here we introduce SpectRobot, a framework that transforms single-point tactile signals into compact time-frequency spectrograms. These spectrograms encode high-bandwidth tactile histories as fixed-size image-like representations. They can be processed by standard vision encoders and integrated into learning pipelines originally developed for vision, while preserving temporal and frequency information unavailable to conventional cameras. Rather than increasing spatial density through arrays of tactile elements, SpectRobot exploits the rich dynamics contained in sparse, high-bandwidth single-point measurements. In our implementation, the sensors are mounted away from the contact surface while remaining mechanically coupled to it, reducing direct exposure to wear and potentially improving robustness in harsh environments and for long-term deployment on dexterous robots. Our experiments demonstrate that: (1) a robot can exploit single-point vibration signals to solve a visually occluded manipulation task; (2) temporal history strongly influences policy performance, while sensing bandwidth controls the spectral information available, with measurements extending to 100~kHz; and (3) the same representation can be used across different tactile sensing technologies mediated by acceleration, force, or strain. We further show that capabilities previously associated with research-grade instrumentation can be accessed using readily available, off-the-shelf hardware. We believe that broader access to high-bandwidth tactile sensing could facilitate the integration of contact dynamics into embodied learning systems and, for some tasks, offer an alternative or complement to increasing the spatial density of tactile sensing.

#22 h=10

Tactile-JEPA: Topology-Aware Self-Supervised Representation Learning for Distributed Tactile Sensors

2026-09-21 cs.ROcs.AI 👤 Andrey Sakhovskiy (h=10)

Elizaveta Kovtun, Matvey Konovalov, Andrey Sakhovskiy, Semen Budennyy

Core Contributions

Brings modern self-supervised pretraining to distributed electronic skins, whose sparse, irregular taxels make vision-style SSL a poor fit.
JEPA-style latent prediction of masked taxels, with masking guided by the sensor connectivity graph and applied at two scales to capture both local contacts and the global surface state.
Across magnetic and piezoresistive sensors and multiple embodiments, cuts force estimation error by 6.3% and in-hand orientation error by 20.8% over prior SOTA, with gains carrying into policy learning.
Argues the value of touch in robot learning hinges on encoder pretraining quality — positioning tactile encoders to follow the pretrained-vision-encoder playbook.

Show abstract

Tactile sensing is an essential modality for robots performing contact-rich, dexterous manipulation, particularly under visual occlusion. While pre-trained image encoders are standard in robot learning pipelines, tactile encoders are still commonly trained from scratch from raw, noisy signals, which might limit their expressivity. Existing self-supervised learning (SSL) approaches focus predominantly on vision-based tactile sensors, leaving distributed electronic skins largely unaddressed. These sensors, however, have a distinctive property: their sensing elements are sparse and irregularly arranged over the surface they cover, which makes direct reuse of visual SSL methods suboptimal. We present Tactile-JEPA, an efficient self-supervised pre-training method that uses the spatial arrangement of tactile sensors to learn topology-aware representations. Specifically, it is trained to predict the embeddings of masked sensing elements from the unmasked remainder, using the sensor connectivity graph to guide spatial masking. Our analysis shows that effective tactile representations require capturing both local contact details and the global state of the tactile surface, which we achieve through dual-scale masking. Across three diverse datasets spanning magnetic and piezoresistive sensors, different robot embodiments, and single- and paired-sensor configurations, Tactile-JEPA reduces force estimation error by 6.3% and in-hand orientation error by 20.8% over the prior state-of-the-art, with consistent gains in other downstream applications, including policy learning. Overall, our results demonstrate that the benefit of tactile sensing depends critically on the quality of encoder pre-training, a problem which Tactile-JEPA addresses directly. Code is available at https://github.com/E-Kovtun/tactile.

#29 h=9

Phrase-Level Robotic Guqin Performance: Bimanual Motion Planning and Audio-Tactile Interaction Monitoring

2026-09-21 cs.RO 👤 Tianwei Zhang (h=9)

Zhen Wang, Zhiheng Chen, Tianyuan Bao, Tianwei Zhang

Core Contributions

Uses the guqin as a demanding manipulation benchmark: millimetric string spacing, transient plucks and sustained harmonic contacts, all with an acoustic success criterion.
Frames performance as a hybrid discrete–continuous problem, planning finger assignment, configuration continuity and tight bimanual contact schedules across consecutive notes.
Closes the loop across three senses — vision for instrument localization, touch for harmonic contact, audio for plucking calibration.
Achieves 93.6% and 96.8% event correctness on a 25-event phrase on a real instrument.

Show abstract

Recent advances in humanoid robotics and embodied intelligence have enabled robots to perform increasingly complex manipulation tasks. However, musical instrument performance remains a formidable benchmark, demanding not only collision-free trajectory execution but also precise contact timing, asymmetric bimanual coordination, and target acoustic outcomes on physical instruments. The guqin, a seven-string fretless zither, presents unique manipulation challenges due to its millimetric string spacing, transient right-hand plucking, and sustained left-hand harmonic contacts. In this work, we present a physical heterogeneous dual-arm robotic system for phrase-level autonomous guqin performance. We formulate guqin playing as a hybrid discrete--continuous execution problem and develop a hierarchical planning framework that coordinates working finger assignment, configuration continuity, obstacle avoidance, and tight bimanual contact schedules across consecutive musical events. The system integrates vision-guided instrument localization, tactile-based harmonic contact monitoring, and auditory feedback-informed plucking parameter calibration. Real-world experiments on a 25-event phrase demonstrate that the system reliably executes coordinated open-string and seventh-hui harmonic sequences on a physical guqin, achieving 93.6% and 96.8% event correctness across repeated trials.

Legged & Humanoid Skills

#3 h=95

MimicAgent: Quadruped Skills via Prompt-to-Trajectory Generation

2026-09-21 cs.RO 👤 D. Ramanan (h=95)

Lucky Kant Nayak, Narayanan Palghat Parameswaran, Neehar Peri, Deva Ramanan

Core Contributions

Replaces reward shaping (and Eureka-style LLM reward writing, which the authors find fails to generalize across skills and morphologies) with a far easier target: have coding agents write reference motions.
Fills the missing ingredient for quadrupeds — they lack the mocap libraries that make example-guided RL work for humanoids — by synthesizing coarse reference trajectories from a text skill prompt.
With Claude Fable 5.1 inside the agentic harness, 87% of prompts yield semantically aligned trajectories that then train deployable imitation policies in sim and on hardware.
Reframes LLM-for-control from 'write the objective' to 'demonstrate the motion', which may scale better to exotic or dynamic skills.

Show abstract

We present MimicAgent, a prompt-to-trajectory generation framework for learning dynamic quadruped skills. Although reward shaping is extensively used when training quadruped policies, navigating the resulting reward landscape is notoriously difficult, requiring hours of "graduate student descent". Eureka attempts to automate reward design with LLMs, but we find that it struggles to generalize across diverse skills and morphologies. Our key observation is that it is far easier for a human - and by association, an LLM - to generate reference motions than to shape reward functions. Our hypothesis is motivated by the success of example-guided RL for humanoids, which exploits large-scale motion capture datasets as references for training locomotion policies. Unlike humanoids, quadrupeds lack such reference motion data. Towards this end, we propose MimicAgent, an agentic harness that, given a skill prompt, generates quadruped reference trajectories with coding agents. These coarse reference trajectories are then used to train example-guided RL policies that are deployable in simulation and in the real-world. Notably, we find that when prompting Claude Fable 5.1 within our agentic harness, 87% of prompts yield semantically aligned reference trajectories.

#26 h=9

PredActor: Predictive Action Diffusion for Steerable Onboard Humanoid Control

2026-09-21 cs.ROcs.LG 👤 Liang Ding (h=9)

Lei Ye, Haibo Gao, Yitang Li, Peng Xu, Zetong Jing

Core Contributions

Unifies three humanoid steering modes — text-conditioned behavior, test-time state objectives and direct action execution — in one diffusion policy that uses only proprioception.
Jointly denoises actions and an internal future-state trajectory, so classifier guidance can steer predicted states toward goals while only actions are executed — no separate tracker or privileged full-body state.
Reaches all 15 destination targets in sim and a text retrieval score of 0.580 vs 0.373 for conditional action diffusion, with similar disturbance survival.
Rolling denoising brings the full guided callback to 16.8 ms median / 19.4 ms p95 on a Jetson Orin NX, under the 20 ms control period, and runs on a Unitree G1.

Show abstract

Diffusion models offer flexible motion generation, but translating this flexibility into feedback-responsive humanoid control remains challenging. Hierarchical systems steer motion through references that may exceed a separate tracker's capabilities, leaving recovery and physical execution largely to the tracker. Action-only diffusion generates actions directly but lacks an explicit future-state trajectory for test-time motion objectives. Joint state-action diffusion provides this representation, yet representative controllers often depend on privileged full-body states, and support for learned behavior selection and test-time motion steering remains fragmented. We present PredActor, a predictive action diffusion policy that brings these complementary steering capabilities into one directly executed policy using proprioceptive observations. Conditioned on proprioceptive history and optional task context, PredActor jointly generates executable actions and an internal future-state trajectory. Classifier-free guidance strengthens text-conditioned behavior, while classifier guidance steers predicted states toward test-time objectives. Only actions are executed, without a separate motion-reference tracker or externally estimated full-body states as policy inputs. In simulation, PredActor reaches all 15 destination targets and achieves a text retrieval score of 0.580, compared with 0.373 for conditional action diffusion, with similar observed disturbance survival. To make this guided policy practical onboard, rolling denoising and computation-preserving runtime optimizations reduce the complete callback to 16.790 ms median and 19.383 ms p95 on a Jetson Orin NX, both below the 20 ms control period. We deploy PredActor on a Unitree G1; evaluations across simulation and physical hardware demonstrate text-conditioned motion, disturbance response, joystick control, and semantic interpolation.

Multi-Robot & Human-Robot Collaboration

#4 h=65

Multi-Agent Transportation of Free-Flyers in Microgravity Via Pushing Interaction Under Human-in-the-Loop Control

2026-09-21 cs.ROeess.SY 👤 D. Dimarogonas (h=65)

Gregorio Marchesini, Nicola De Carli, Sihyun Cho, Youngkyoung Kong, Elias Krantz

Core Contributions

Tackles transporting a passive free-flyer in microgravity using only unilateral pushes, where chasers must physically relocate on the target whenever the required force allocation changes.
Embeds CLF (tracking) and CBF (safety) constraints inside a mixed-integer thrust-allocation program, handling the combinatorial choice of which contact to use alongside continuous forces.
Makes the controller delay-aware so a human operator can supply the desired twist remotely while obstacle avoidance is still guaranteed despite intermittent control authority.
Relevant to on-orbit servicing and debris handling, where grasp points are often unavailable; currently validated in Gazebo only.

Show abstract

We propose a safety-critical framework for the cooperative transportation of passive targets in microgravity, where a team of chaser robots acts through unilateral pushing contacts to track a human-provided desired twist while ensuring safe target motion. The pushing-only nature of the interaction introduces sparse, configuration-dependent actuation constraints requiring chasers to physically relocate on the target body when the desired pushing allocation changes. To address these challenges, we formulate a delay-aware feedback control architecture leveraging Control Lyapunov Function (CLF) and Control Barrier Function (CBF) constraints within a mixed-integer thrust allocation program to enforce stability and safety of the target, respectively. The proposed framework enables reference tracking while guaranteeing obstacle avoidance with a circular obstacle despite intermittent control authority, providing a foundation for human-supervised cooperative transportation of free-flyers in space environments. The proposed framework is validated through Gazebo simulations.

#5 h=49

RoboTalk: Learning Multi-Robot Communication and Coordination from Multimodal Demonstrations

2026-09-21 cs.ROcs.AI 👤 S. Zilberstein (h=49)

Dorian Benhamou Goldfajn, Mason Nakamura, Saaduddin Mahmud, Justin Svegliato, Kyle H. Wray

Core Contributions

Targets small, on-device VLMs that must both talk and act in decentralized multi-robot manipulation under partial observability — a setting mostly addressed with large cloud LLMs.
Contributes a synthetic pipeline producing 7,950 multimodal trajectories over 53 kitchen tasks, with a leader-follower protocol, tool calls for perception/manipulation/navigation/communication, and rationale traces.
Fine-tuning lifts open models from ~2% to 77% success on held-out tasks, showing communication-plus-skill selection is learnable from demonstrations rather than requiring hand-designed protocols.

Show abstract

Multi-robot collaboration could enable more efficient and scalable solutions to complex robotic tasks, but collaboration under partial observability remains challenging. Natural-language communication offers a promising approach to coordinating robots under partial observability. However, in decentralized manipulation, jointly learning explicit inter-robot communication and skill-level action selection from multimodal demonstrations remains underexplored for small vision-language models (VLMs) intended for on-device deployment. To address this gap, we introduce RoboTalk, a synthetic data-generation pipeline and dataset of 7,950 multimodal trajectories spanning 53 mobile-manipulation kitchen tasks for training small VLMs to communicate and coordinate. The dataset includes a leader-follower planning protocol, tool calls (perception, manipulation, navigation, and communication), rationale traces, and diversified natural-language communication. Fine-tuning open-source models on our dataset can reach 77% success on novel held-out tasks, a significant improvement over the untuned open source models, which had a success rate of around ~2%.

#7 h=45

Toward Human-in-the-Loop Robot Failure Recovery: Bridging Communication Gaps in Human-Robot Collaboration

2026-09-21 cs.ROcs.HC 👤 Stefanie Tellex (h=45)

Promise Ekpo, Teju Vijay, Dhruv Mandalik, Tisha Jain, Arman Ibrayeva

Core Contributions

Challenges the single-listener assumption in inverse-semantics help requests: when a robot asks a bystander for help, the right wording depends on what that person already knows.
Introduces LD-HRI, a game and benchmark scoring requests by real human listener performance under controlled knowledge differences (446 human-written requests, 1,302 listener trials).
LLM-written requests help novices more than human-written ones, but a 16-point expert–novice gap persists — evidence that current generators don't adapt to audience knowledge.

Show abstract

Robots can recover from failures by asking bystanders for help, but effective human-in-the-loop recovery requires communication that accounts for differences in people's knowledge. Prior inverse-semantics work generates requests using a single listener model, leaving differences in listener knowledge untested. We introduce Listener Differences in Human-Robot Interaction (LD-HRI), a game, dataset, and benchmark that evaluates speakers through human listener performance. Our evaluation examines request properties, large language model (LLM) speakers, and inverse-semantics request-selection algorithms under controlled differences in listener information. The corpus contains 446 human-written requests and 1{,}302 listener trials. We additionally evaluated 24 frozen LLM-written requests with 70 human listeners across 560 trials. Novice success is descriptively higher with model-written requests across all four tasks, yet both request sources leave substantial expert--novice gaps, including 16 percentage points for LLM requests. LD-HRI makes these gaps measurable, providing a foundation for designing more robust communication in human-robot and human-agent interaction.

#11 h=20

AC-DC: Adaptive Communication for Scalable Dynamic Average Consensus in Multi-Robot Ergodic Search

2026-09-21 cs.RO 👤 Vasileios Tzoumas (h=20)

Robin Inho Kee, Begum Cannataro, Vasileios Tzoumas

Core Contributions

Makes dynamic average consensus practical under realistic radio limits by adapting who talks to whom, when, and which parts of the state are sent, without synchronized rounds.
Message size is independent of team size for a fixed state representation, so bandwidth doesn't explode as fleets grow.
In ergodic search with up to 80 robots, cuts uncertainty AUC by 27.5% while using 8.7× less traffic than decentralized baselines.
At 120 robots its payload (19.3 MB) nearly matches an idealized centralized ground station (19.2 MB) while remaining fully peer-to-peer.

Show abstract

We study scalable peer-to-peer dynamic average consensus (DC) for multi-robot systems under finite-range, finite-rate, and interference-constrained communication. We introduce Adaptive Communication for Dynamic Average Consensus (AC-DC), which jointly adapts Who communicates with whom, When, and over What parts of the consensus state, using local inputs and successfully received neighbor information. Each robot's consensus state estimates the current average of the robots' local inputs. AC-DC updates these estimates as local inputs change and averages the values exchanged between robot pairs. In AC-DC, robot pairs update without waiting for every robot to complete a communication round, and the selected-state messages carry consensus state coordinates independent of team size for a fixed state representation. We apply AC-DC to dynamic-priority multi-robot ergodic search: one consensus stream estimates team visitation for motion coordination, while the other fuses regional measurement information to update uncertainty maps and search targets. Across twelve settings with up to 80 robots and 20 paired trials per setting, AC-DC has the lowest mean (i) normalized covariance-trace area under the curve (AUC) and (ii) attempted modeled communication payload among the compared decentralized methods. Averaged across settings, AC-DC achieves paired AUC reductions of 27.5% relative to state-of-the-art baselines, with 8.7x less communication traffic. As the number of robots increases, we observe that AC-DC's communication payload approaches that of the ideal centralized baseline (one ground compute-station communicating directly with all robots): with 120 robots in a fixed 600 x 600 m scaling test, AC-DC uses 19.3 MB versus 19.2 MB for the ideal centralized baseline, while remaining peer-to-peer.

#25 h=9

CAST: Collision-Aware Assembly with Construction Robots using Simultaneous Trajectory Estimation and Planning

2026-09-21 cs.RO 👤 E. Bruun (h=9)

Karthik Shaji, Chisung Kim, John D'Amato, Edvard Bruun, Frank Dellaert

Core Contributions

Unifies state estimation and motion planning for multiple heavy construction robots in one factor graph, so plans continuously absorb measured, vibration-affected robot states.
Encodes inter-robot, object and workspace collisions plus learned cable constraints directly as factors, supporting workspaces that change as the structure is built.
Demonstrated on a timber post-and-lintel assembly with one arm gripping and another nailing — a step toward coordinated multi-robot construction outside the lab.

Show abstract

Multi-robot systems have shown increasing viability in construction due to their ability to execute high-precision actions while reducing human exposure to hazardous tasks. However, these environments have high-dimensional configuration spaces and possess substantial collision-avoidance constraints, which include other robots, assembly objects, and workspace boundaries. We utilize a single factor graph for trajectory estimation and planning that incorporates measured robot states together with explicit collision and learned cable constraints. This supports changing workspaces and enables synchronized, high-dimensional robot motion planning while accounting for the stiff, vibration-induced uncertainty of heavy robotic systems. We demonstrate the success of our framework on the construction of a post-and-lintel structure using one robot arm as a timber gripper, and a second robot as a nail-fastener.

Control, Planning & Safety

#10 h=29

Minimum Time Trajectories for a Car-Like Mobile Robot Moving with Rigid Wheels Under Non-Sliding Constraints

2026-09-21 cs.RO 👤 E. Rimon (h=29)

Joseph Ben-Asher, Elon Rimon, Leeor Ravina

Core Contributions

Extends classical time-optimal car-like path theory (12 kinematic primitives) by dropping the pure-rolling assumption and deriving non-sliding constraints from the vehicle dynamics.
Adds three new primitives that activate when wheel friction limits bind, with approximate analytic solutions — keeping the closed-form character that makes such results useful for planners.
Shows how realistic slip constraints reshape minimum-time maneuvers, relevant for fast autonomous vehicles where kinematic-only optimal paths may be physically infeasible.

Show abstract

This paper studies the minimum time trajectoriesvof a car-like mobile robot navigating in an obstacle free environment. The robot, with forward and backward speeds, is controlled by bounded front-wheels acceleration and limited front-wheels steering rate. The paper extends previous results which solved this problem for the kinematic car-like robot. However, the kinematic model assumes pure rolling at the wheels ground contacts. This assumption requires non-sliding constraints for the front and rear wheels that can only be handled by the robot dynamics. This paper formulates the non-sliding constraints based on the robot dynamics then augments the kinematic model time-optimal path primitives with three new path primitives associated with the non-sliding constraints. The three non-sliding path primitives together with the kinematic model twelve path primitives form the car-like robot time optimal trajectories. Approximate analytic solutions for the non-sliding path primitives are also provided. Examples study the time-optimal path primitives along representative maneuvers, illustrating how the non-sliding constraints influence the time optimal trajectories of the car-like robot.

#21 h=10

SE(3) Neural Potential Fields for 6-DoF Trajectory Planning Directly from Images Without Explicit 3D Reconstruction

2026-09-21 cs.ROcs.AI 👤 Vikas Dhiman (h=10)

Jeffrey Eiyike, Masoud Ataei, Elvis Gyaase, Vikas Dhiman

Core Contributions

Plans 6-DoF collision-free grasp approaches directly from posed RGB images, skipping explicit 3D reconstruction.
Supervises an SE(3) neural potential field with a navigation function (free-space geodesic distance) instead of image-only signals, removing the classic APF failures of obstacle grazing and local minima.
On a UR10, converges within 3 cm from every blocked start with all paths collision-free (vs 25% and 0% with image supervision alone), with clearance rising from <1 cm to ~8.7 cm.
Plans in ~2 s versus 67–133 s for RRT* on a reconstruction; the authors candidly attribute the gap to dense collision checking rather than planner complexity.

Show abstract

Reaching a 6-DoF grasp pose in clutter requires a collision-free trajectory, conventionally obtained by reconstructing the scene in 3D and planning inside that reconstruction, at the cost of its accuracy and compute. Potential fields learned directly from images remove that dependency but inherit the classical weakness of artificial potential fields: where attractive and repulsive gradients cancel, the descent grazes the obstacle instead of going around it, and can stall short of the goal. We present an SE(3) neural potential field learned from posed RGB images and supervised with a navigation function, the geodesic distance to the grasp through free space recovered from those same images during training, which removes both failures. On two tabletop scenes, from obstacle-blocked starts executed on a UR10, the field converges within 3 cm of the grasp from every start and every path it executes is collision-free against the ground-truth geometry, against 25% and 0% under image supervision alone; mean clearance rises from under a centimeter to 8.6-8.8 cm and arm-link contacts fall from 20.6-50.4% to 2.7-5.5% of executed configurations. Executed grasp success is 90.0% and 40.0% on the two scenes, the residual failures being refusals of the Cartesian executor rather than of the field. Planning takes about 2 s against 67-133 s for RRT* on a reconstruction of the same images, though under a common offline harness the two are comparable: the deployed margin is the cost of collision-checking a dense reconstruction, not planner complexity.

#24 h=10

Safety Control of a Hyper-redundant Robot via Adaptive Weighted Control Barrier Functions

2026-09-21 cs.RO 👤 Wen-Ci Xin (h=10)

Zijian Cai, Kiwan Wong, Wenci Xin, Wei Xiao, Daniela Rus

Core Contributions

Addresses a specific pain for slender hyper-redundant robots: uneven load along the body causes nonuniform tracking errors that standard CBF safety filters ignore.
Weighted CBFs distribute the safety correction unevenly along the body, cutting RMS tracking error by up to 59.6% (sim) and 87.7% (hardware) vs unweighted CBFs.
An adaptive weighting scheme calibrated on sim-to-real discrepancy further reduces error by 21.9%/8.5%, though it degrades when obstacles are very close.
In a cleaning coverage task, autonomous control matches or beats teleoperation while avoiding the frame collisions human operators caused.

Show abstract

Hyper-redundant robots are well suited for confined-space manipulation due to their high dexterity, but safe operation in cluttered environments remains challenging. In addition, their slender structures often lead to uneven load distributions and nonuniform tracking errors along the body. To address these issues, this work proposes a weighted control barrier functions (W-CBFs) framework that enforces safety constraints while reducing tracking errors caused by uneven loading. The proposed controller was first evaluated on a circular path-following task under different obstacle configurations. With fixed weights, compared to the non-weighted method, the maximum reduction in root-mean-square (RMS) tracking error was 59.6\% in simulation and 87.7\% in physical experiments. An adaptive weighting strategy was then investigated based on the discrepancy between simulated and experimental performance under different mapping functions. The RMS errors were further reduced by 21.9\% and 8.5\%, respectively, although the error increases when obstacles were located close to the robot body. Finally, the robot was evaluated in a cleaning task requiring coverage of a rectangular area and compared with manual teleoperation. Although the controller was not explicitly optimized for area coverage, the autonomous strategy achieved comparable or better coverage performance while avoiding collisions with the surrounding frame, whereas collisions occurred during manual operation.

#27 h=9

A Switched Adaptive Control Framework for Aerial Manipulators Under Dynamic Transitions

2026-09-21 cs.RO 👤 Simone Baldi (h=9)

Rishabh Dev Yadav, Saksham Gupta, Amitabh Sharma, Sarthak Mishra, Wei Pan

Core Contributions

Treats task phases of aerial manipulation (grasping, releasing, contact) as switched dynamics, since abrupt changes break controllers designed for a single smooth regime.
Adaptive control that needs no prior model of vehicle–arm coupling or state-dependent uncertainty — the terms that are hardest to identify on real aerial manipulators.
Provides a class of switching signals with guaranteed stability during transitions, turning 'when is it safe to switch' into a design rule; experiments beat state-of-the-art controllers.

Show abstract

Aerial manipulators represent the forefront of aerial robotics. Although potentially capable of complex interaction tasks, controlling aerial manipulators throughout the dynamic transitions occurring during task execution presents significant challenges. Abrupt or discontinuous changes in system dynamics generated by the transitions suggest the use of a switched approach, yet the available aerial manipulation methods are not designed for coping with switched regimes. In addition, most available methods fall short in coping with the tight couplings between the aerial vehicle and the manipulator, as well as in coping with the state-dependent uncertainties arising from the difficulty in modeling such couplings. We propose a switched-based adaptive control framework for aerial manipulators not relying on a priori knowledge of the vehicle-manipulator couplings and of state-dependent uncertainties. To guarantee stable manipulation despite changes in system dynamics, the framework provides a class of switching signals characterizing those transition phases for which the system is guaranteed to remain stable. Comparative experiments further validate the effectiveness of the proposed switched-based framework over the state of the art.

Perception & Field Robotics

#8 h=42

Toward a foundation model for forest point clouds

2026-09-21 cs.CVcs.RO 👤 R. Astrup (h=42)

Yuanwen Yue, Stefano Puliti, Damien Robert, Atakan Topaloğlu, Binbin Xiang

Core Contributions

First sets a strong supervised LitePT baseline that already reaches SOTA on forest semantic/instance segmentation, species classification and age regression, so pretraining gains are measured against a hard bar.
Curates an unlabelled corpus spanning airborne, UAV and mobile laser scanning across biomes, then compares scratch, supervised and self-supervised pretraining under varying label budgets.
Self-supervised pretraining speeds convergence and helps most when annotations are scarce, and transfers better across tasks than task-specific supervised pretraining.
Pinpoints tree instance discrimination — not semantics — as the remaining barrier to a general 3D forest foundation model, giving the community a clear next target.

Show abstract

Forest inventories increasingly rely on artificial intelligence (AI) models to derive forest attributes from large-scale 3D point clouds. Current models are typically specialized to a single task, sensor, and forest type, making adaptation expensive in terms of annotations, computation, and expertise. We ask whether a single pretrained model can instead learn transferable representations across diverse forest inventory settings. Inspired by recent developments in language modelling and computer vision, we take a step toward a foundation model (FM) for 3D forestry. Using LitePT as backbone, we first establish a strong supervised baseline that sets a new state of the art on forest semantic and instance segmentation, tree species classification, and age regression benchmarks. We then curate a large-scale unlabelled corpus spanning airborne, UAV, and mobile laser scanning across diverse forest ecosystems, and pretrain the same backbone using self-supervised learning. We systematically evaluate representation learning strategies by comparing training from scratch, supervised pretraining, and self-supervised pretraining across four representative forestry tasks, under varying annotation budgets. Compared with training from scratch, self-supervised pretraining accelerates model convergence and consistently improves performance when annotations are scarce. Compared with task-specific supervised pretraining, self-supervised pretraining yields more transferable representations across downstream forestry tasks. These findings identify the practical regime in which pretrained representations are most valuable and suggest that instance discrimination, rather than forest semantics, is the main remaining obstacle to a general-purpose 3D forest foundation model. Code and models are available at: https://github.com/prs-eth/ForPT.

#28 h=9

AnalogDepth: Multi-view Geometry from FPV drones under Analog Video Transmission

2026-09-21 cs.CVcs.RO 👤 Pedro F. Proença (h=9)

André Amorim, Pedro F. Proença

Core Contributions

Shows that analog FPV video noise — spatially structured and unlike AWGN — severely degrades Depth Anything 3, a blind spot for foundation-model perception on cheap, low-latency drones.
Builds a noise bank recorded from static real FPV captures and injects it during LoRA-based teacher-student distillation into DA3's DINOv2 backbone.
Real-noise injection beats PSD-matched Gaussian and AWGN synthesis on depth RMSE and Chamfer distance over six flight sequences, showing that noise structure — not just its spectrum — must be replicated.

Show abstract

Analog video transmission (VTX) remains widespread in FPV drones due to low latency, weight and low cost. However analog VTX suffers from complex spatially structured image degradation which differ fundamentally from digital image corruption (e.g. AWGN) used in standard training augmentation. This work shows that this type of noise severely degrades the accuracy of Depth Anything 3 (DA3), a state-of-the-art feed forward visual geometry foundation model. To address this gap, we present AnalogDepth, a parameter-efficient training pipeline that adapts DA3 to analog FPV imagery using student-teacher knowledge distillation with Low-Rank Adaptation (LoRA) injected into the DINOv2 backbone. Rather than synthesizing noise analytically, we build a noise bank from static FPV recordings under diverse conditions and compare real-noise injection against PSD-matched Gaussian synthesis and AWGN as baselines. Experiments on six real FPV flight sequences across three indoor scenes show that training with our noise bank consistently reduces per-frame depth RMSE and 3D reconstruction Chamfer distance compared to the pretrained DA3 baseline and both Gaussian noise variants. These results demonstrate that replicating the spatial structure of real analog transmission noise is critical for effective adaptation.