🤖 Robotics arXiv Digest

Wednesday, September 30, 2026

📄 30 papers 📂 7 research areas Generated by Claude

🔭 Research Landscape

Today's batch is dominated by world-action models (WAMs) moving from proof-of-concept to engineering object. Five papers treat the joint video-plus-action denoiser as a given and attack its cost, data or structure. Two come from the same group and attack inference from complementary angles: SteerQuant (#12) quantizes WAMs to 4 bits while routing numerical error toward streams that barely affect actions (≤0.8 pt LIBERO loss, 2.23× denoising speedup), and Sparse-WAM (#13) prunes future-frame tokens by action relevance rather than visual fidelity (≈2× speedup, training-free). The shared insight is that a WAM's imagined video is a means, not an end: compute and precision should follow what the action head actually reads. UniWAM (#16) and Ego4WAM (#24) push the other direction — what to feed the WAM — with 1.5M auto-generated mobile-manipulation episodes and a controlled study showing that human-robot alignment and task diversity matter more than raw hours of egocentric video, and that video-only pretraining remains useful without action labels.

A second theme is general-purpose LLM/VLM agents being dropped directly into embodied loops — and audited. ASENA (#11) and NavHarness (#14) both wrap navigation policies in agentic scaffolding (coding agents with persistent notes; goal/verify/memory sub-agents), and ASENA's 72%→98% R2R gain from ten passes of workspace evolution is a striking example of improvement without weight updates. But three benchmarks temper the enthusiasm: AssemblyWorld (#3) finds the best agent gets 81% of parts right yet only 59% of assemblies; LIBERO-Agent (#28) shows agents excel at perception and easy short-horizon steps but collapse on long-horizon composition; and Refusals That Bend (#22) shows a constitution-guarded planner's refusals flip on 20% of tasks when a single everyday object is added to the scene. Together they suggest the bottleneck is no longer high-level reasoning but precise geometry, cross-stage state, and robust safety alignment under ordinary environmental variation.

Third, several papers replace dense, uniform signals with structured, physically grounded ones. TacEx (#1) aims curiosity at the tactile channel instead of all model uncertainty; PRICE (#30) assigns per-chunk credit by matching physically corresponding states across rollouts instead of broadcasting one terminal reward; Matisse (#21) reads uncertainty from a generative 3D model's cross-attention to choose views of unseen geometry; and the multi-UAV estimator (#2) observes neighbours' tilt (thrust direction) to cut a structural ~300 ms acceleration delay. On the classical side, certification is a recurring note — PAC-NMPC (#7), residual-wrench margins for aerial contact (#18) and probabilistic matched-assistance bounds for exoskeletons (#8) all trade some nominal performance for guarantees that hold beyond the training distribution. GroundingPI and GroundAnything (#25, #26) make a related argument for perception: dedicated grounding pretraining, not borrowed VLM or video backbones, is what downstream VLAs and WAMs are missing.

World-Action Models & Efficient Action Generation

Making joint video-action models cheaper, better-fed and better structured, plus coarse-to-fine action experts.

#12 SteerQuant: Steering Quantization Error with Action-Guided...
#13 Sparse-WAM: Accelerating World Action Models via Action-Gu...
#16 UniWAM Technical Report: Unified Mobile Manipulation via M...
#24 Ego4WAM: What Matters When Scaling Egocentric Human Data f...
#20 Discrete Forcing: Infusing Discrete Guidance into Continuo...

Robot Learning, Adaptation & Credit Assignment

Exploration, memory-based skill reuse, sim-in-the-loop self-improvement and finer RL credit for manipulation policies.

#1 Tactile Curiosity Drives Robot Interaction
#15 Inline Memory Meets Reusable Skills: Memory-centric Framew...
#17 EmbodiRSI: Recursive Self-Improvement for Data-Efficient R...
#30 PRICE the Action Chunks: Physical Relational Credit Assign...
#10 Linear Recurrent Memory Suffices to Distil a World-Model P...

Agentic Embodied Systems & Benchmarks

General-purpose agents driving robots directly — and benchmarks probing where they break, including safety.

#3 AssemblyWorld: Rethinking 3D Assembly with General-Purpose...
#11 ASENA: Self-evolving Agents for Embodied Navigation
#14 NavHarness: Adaptive Goals for Agentic Vision-Language Nav...
#22 Refusals That Bend: Measuring and Predicting Task Malleabi...
#28 LIBERO-Agent: Evaluating General-Purpose Agents for Direct...

Grounding, Active Perception & Field Sensing

Grounding foundation models, generative-evidence active reconstruction, underwater sensing and bio-derived assets.

#25 GroundingPI: A Grounding Foundation Model towards Physical...
#26 GroundAnything: Reconciling Parallel Decoding with Precise...
#21 Matisse: Evidence-Space Reasoning for Active 3D Reconstruc...
#6 Active Mapping of Underwater Litter Using Camera-Sonar Fus...
#5 Video2SwimFish: An Automated Pipeline for Reconstructing C...

Social & Mobile Navigation

Safety-aware world models, legible motion around people, local-minimum escape and learned sampling for driving.

#4 Social-WM: Safety-Aware Latent World Models for Robot Soci...
#19 Rethinking Legibility in Social Robot Hallway Navigation:...
#29 Local-Minimum Escaper: Programmatic Subgoal Generation for...
#27 Sparse Planner: A Hybrid Planner for Efficient Sampling vi...

Aerial Robotics & Certified Control

Estimation for agile swarms and controllers with probabilistic or geometric guarantees, from UAVs to exoskeletons.

#2 Towards Agile Vision-Based Multi-UAV Flight: Revisiting St...
#7 RL-Guided PAC-NMPC for Probabilistically-Safe Perception-B...
#18 Residual Wrench Certification and Margin-Aware Control Syn...
#8 General Performance Guarantee for Human Torque Estimation-...

Multi-Robot Coordination

Task allocation and routing for fleets under partial observation and limited communication.

#9 Communication-Free Distributed Multi-Robot Task Allocation...
#23 HALO: Heterogeneous Allocation Via Localized Observations...

World-Action Models & Efficient Action Generation

#12

h=22

SteerQuant: Steering Quantization Error with Action-Guided Scaling in World-Action Models

📅 2026-09-30cs.RO👤 Zi-Cong Hong (h=22)

Yunhan Wang, Haodong Wang, Zhiming Liu, Zicong Hong, Qianli Liu

Core Contributions

Observes that equal numerical error in video, proprioception and action streams of a WAM has very unequal effects on actions, so accuracy-only quantization is the wrong objective.
Builds an action-impact map per stream and uses it to steer shared channel scaling, calibrated per stream and per denoising step — no duplicated weights or higher bit-widths.
Rudder, a custom 4-bit engine, fuses scaling and compensation into kernels so the method's overhead doesn't eat the speedup.
W4A8/W4A4 stays within 0.8 pts of full-precision LIBERO success with up to 2.23× denoising speedup; 1.35× end-to-end on a real dual-arm robot.

Show abstract

World-action models (WAMs) jointly generate future world states and actions through iterative denoising, using shared weights to process heterogeneous semantic streams of video, proprioceptive, and action tokens. Quantization reduces inference cost, but comparable numerical errors in different streams can have markedly different effects on final actions, making numerical accuracy alone insufficient for reliable control. We introduce SteerQuant, a 4-bit quantization framework for WAMs that steers errors toward computations with less influence on final actions. It maps how each stream's quantization errors affect final actions and uses this map to guide shared channel scaling. Activation scaling is further calibrated for each stream and denoising step to accommodate changes in activation ranges and action impact. This adapts quantization to different stream requirements without duplicating weights or increasing bit-widths for selected streams. To reduce the extra kernel launches and memory traffic introduced by scaling, we develop Rudder, a 4-bit inference engine for WAMs that fuses scaling and output compensation into low-bit kernels. Under W4A8 and W4A4, SteerQuant maintains mean LIBERO success within 0.8 percentage points of full precision, while delivering up to 2.23× denoising speedup over BF16 across three WAMs with reduced peak GPU memory usage. On a real dual-arm robot, W4A8 deployment achieves a 1.35× end-to-end inference speedup while maintaining average task success relative to BF16.

#13

h=22

Sparse-WAM: Accelerating World Action Models via Action-Guided Sparse Imagination

📅 2026-09-30cs.ROcs.AI👤 Zi-Cong Hong (h=22)

Xinling Xie, Haodong Wang, Jiazhi Mi, Zhiming Liu, Zicong Hong

Core Contributions

Unlike video-diffusion pruning that keeps visually salient tokens, Sparse-WAM keeps the future-frame tokens that the action tokens attend to.
Exploits the finding that action-to-future attention maps are highly stable across consecutive denoising steps, so token selections can be reused.
The Pilot engine minimises scoring and token-packing overhead, which naive sparse implementations lose to.
Training-free ≈2.0× (FastWAM-Joint, LIBERO) and 1.8× (Cosmos 3 Edge, RoboLab-120) speedups on an RTX 4090 with largely preserved success; complementary to SteerQuant (#12) from the same group.

Show abstract

World-action models (WAMs) leverage pretrained video models to improve generalization in robot control by jointly predicting future visual states and actions. This capability comes at a substantial inference cost, as dense future-frame tokens are repeatedly processed during denoising. Prior methods address this by token pruning that prioritizes visual fidelity to reduce denoising costs in video diffusion models. However, these methods do not use action relevance to determine which future-frame tokens to retain during joint denoising in WAMs. In this paper, we propose Sparse-WAM, a training-free framework for action-guided sparse imagination that selectively processes future-frame tokens to accelerate WAM inference. We observe substantial overlap in the spatial distribution of attention from action tokens to future-frame tokens (action-to-future attention) between consecutive denoising steps, despite continued updates to the future representations. Motivated by this, we develop Action-Guided Token Selection to retain frame-specific action-relevant regions together with cross-frame context. However, a naive implementation can incur attention-scoring and token-packing overhead that offsets the computational savings from pruning. We therefore introduce Pilot, an efficient engine that reduces sparse inference overhead through lightweight scoring and cross-step reuse of token selections. On LIBERO with FastWAM-Joint and RoboLab-120 with Cosmos 3 Edge, Sparse-WAM achieves inference speedups of approximately 2.0× and 1.8×, respectively, over dense eager inference on an NVIDIA RTX 4090, while largely preserving task performance.

#16

h=20

UniWAM Technical Report: Unified Mobile Manipulation via Mixed-Stream World-Action Modeling and Manipulation Anchor Pose Supervision

📅 2026-09-30cs.ROcs.CV👤 Ming-Cheng Li (h=20)

Wei Xue, Keliang Liu, Mingzhang Cui, Jinhua Xie, Jinjie Wei

Core Contributions

Unifies navigation-to-pose and manipulation in one mixed-stream WAM: shared backbone, separate action encoders/heads, trainable on independently sampled data.
Introduces Manipulation Anchor Pose supervision — where to stop and how to face — a target most mobile-manipulation pipelines leave implicit.
Auto-generates MAP-Data from 3D scenes: 1.5M episodes / 7,500 h with per-frame object boxes and image-plane MAP targets.
Beats the strongest baselines on MAP-Bench by 30.1% (position) and 44.0% (heading) error; leading results over 24 real tasks; code, data and benchmark released.

Show abstract

Mobile manipulation requires precise navigation to a manipulation-ready pose followed by reliable object interaction. These two stages differ in action spaces and visual requirements, which complicates unified policy learning. In addition, collecting diverse real-world navigation data with explicit manipulation-ready pose supervision remains costly and difficult to scale. We introduce UniWAM, a unified mixed-stream world-action model with separate action encoders and output heads for navigation and manipulation, sharing a common backbone. This design supports joint representation learning on independently sampled navigation and manipulation data. UniWAM supports independent inference for either stream and batch-parallel inference for both. We further introduce Manipulation Anchor Pose (MAP) supervision for where to stop and how to orient for manipulation. An automated pipeline constructs MAP-Data from large-scale 3D scenes, yielding over 1.5 million episodes and 7,500 hours. MAP-Data provides per-frame target-object bounding boxes and image-plane MAP coordinates as auxiliary navigation supervision. Together with projected end-effector trajectories for manipulation, these prediction targets provide stream-specific image-plane supervision for action learning from egocentric observations. With large-scale MAP-Data, UniWAM outperforms the strongest external baselines on our MAP-Bench by 30.1% in position error and 44.0% in heading error. Across 24 real-robot tasks, UniWAM achieves leading results in MAP navigation and mobile manipulation, with competitive manipulation performance. We have released code, data, and benchmark.

#24

h=15

Ego4WAM: What Matters When Scaling Egocentric Human Data for Robot Learning?

📅 2026-09-30cs.ROcs.CV👤 Liu Liu (h=15)

Zhihao Sun, Liu Liu, Xinjiang Wang, Haoyi Jiang, Wei Feng

Core Contributions

A controlled study with a fixed WAM backbone that disentangles which properties of egocentric human data actually help robots.
Human-robot aligned demonstrations give large out-of-distribution gains and reduce target-task robot data requirements.
Duration and task diversity shape different capabilities, so 'more hours' is not a sufficient scaling axis.
Video-only supervision without action labels remains a strong foundation for later video-action training; findings validated on real robots and RoboDojo.

Show abstract

Egocentric human data provides a scalable source of experience for robot learning, but varies substantially in human-robot alignment, behavioral coverage, and available supervision. Existing work shows favorable scaling with increasing human data, but it remains unclear which data properties drive downstream robot gains and how to use such data throughout the training pipeline. We present a systematic study of egocentric human data with different alignment and supervision under a unified world-action model framework. With the model backbone fixed, we disentangle the effects of human-robot alignment, data duration and task diversity, action supervision, and data usage strategies. We find that aligned human demonstrations substantially improve out-of-distribution generalization and reduce target-task robot data requirements; data duration and task diversity affect downstream capabilities differently; and video-only supervision remains effective without action labels, providing a strong foundation for subsequent video-action training. We validate these findings through closed-loop policy evaluation on both real robots and RoboDojo. Rather than treating data duration as the sole scaling axis, Ego4WAM shows how alignment, task diversity, available supervision, and usage strategy jointly shape the value of egocentric human data for robot learning.

#20

h=19

Discrete Forcing: Infusing Discrete Guidance into Continuous Denoising for Few-Step Action Experts

📅 2026-09-30cs.RO👤 Han Zhao (h=19)

Jingbo Wang, Wenxuan Song, Wenhao Yu, Han Zhao, Xi Wang

Core Contributions

Combines discrete action tokens (good coarse structure, low precision) and continuous flow matching (precise, but many steps) as an explicit coarse-to-fine process.
Predicted discrete tokens guide continuous refinement inside one shared diffusion-transformer backbone with specialised branches — same parameter count, one forward pass per branch.
Outperforms a parameter-matched continuous action expert in both accuracy and speed, with gains growing with capacity, and helps on high-precision and dynamic real-world tasks.

Show abstract

Efficient action generation in vision-language-action (VLA) models requires capturing both coarse action structure and fine-grained details. Discrete action tokens provide compact structural representations but sacrifice precision, while continuous action tokens offer high precision but often require multiple denoising steps. We introduce Discrete Forcing, a flow-matching framework that combines these representations through an explicit coarse-to-fine generation process. It first predicts discrete action tokens to establish a coarse action structure, then uses them to guide continuous action refinement. The discrete and continuous components share a common diffusion transformer backbone with specialized branches, maintaining a parameter count comparable to a conventional single-branch model while requiring only one forward pass per branch. Extensive evaluations across multiple benchmarks demonstrate improved performance and faster inference over a parameter-matched continuous action expert, with consistent performance gains as model capacity increases. Real-world experiments further demonstrate improvements on high-precision and dynamic manipulation tasks.

Robot Learning, Adaptation & Credit Assignment

h=46

Tactile Curiosity Drives Robot Interaction

📅 2026-09-30cs.ROcs.AI👤 Stelian Coros (h=46)

Klemens Iten, Alexander Proshkin, Bhavya Sukhija, Stelian Coros, Andreas Krause

Core Contributions

Reframes exploration for manipulation: instead of rewarding any epistemic uncertainty (which also rewards erratic free-space motion), TacEx decomposes model uncertainty by modality and chases only the tactile part.
Because touch is only informative near contact, the curiosity signal itself pulls the agent toward the regions where manipulation skills live — a cheap, task-agnostic prior with no reward engineering.
The reward-free exploration data is dense enough to train pick-and-place policies fully offline, turning exploration into a data-collection engine.
Shows that VLAs pretrained without tactile input still benefit substantially from tactile-curiosity post-training, suggesting touch can be retrofitted to existing foundation policies sample-efficiently.

Show abstract

Mastering robot manipulation skills via reinforcement learning (RL) remains largely sample-inefficient. The most common RL algorithms rely on random action sampling to discover new strategies, resulting in agents that allocate most of their training budget to motions in free space, away from the contacts from which manipulation skills emerge. Existing intrinsic motivation methods based on model disagreement or epistemic uncertainty improve on isotropic noise, but they can also reward uncertainty in functionally irrelevant transitions, such as erratic motions in free space. In this work, we argue that tactile feedback provides a natural signal for exploration, and introduce TacEx, a framework that incorporates touch into epistemic uncertainty-driven exploration by decomposing model uncertainty across sensory modalities and directing curiosity toward the tactile channel. By anchoring curiosity to the sense of touch, TacEx drives the robot to discover complex contact dynamics, learning to manipulate and grasp objects without task rewards or expert demonstrations during exploration. The interaction-dense dataset collected through this tactile-driven curiosity supports offline learning of downstream pick-and-place policies without additional environment interaction. We further use tactile-driven exploration to post-train vision-language-action (VLA) models. Although the VLAs are initially pre-trained without tactile feedback, post-training with TacEx substantially improves downstream performance while remaining highly sample-efficient.

#15

h=20

Inline Memory Meets Reusable Skills: Memory-centric Framework for Vision-Language-Action Model

📅 2026-09-30cs.CVcs.RO👤 Li-Qiang Nie (h=20)

Zaijing Li, Rui Shao, Bing Hu, Haoyu Zhang, Dongmei Jiang

Core Contributions

Recasts VLA adaptation as tuning an explicit query-skill memory rather than backbone weights, directly targeting cost and catastrophic forgetting.
Learnable memory tokens inserted in the VLA prefix produce control-aware query/skill representations inside the native action-conditioning path.
Query prototypes decide what to retrieve and skill values specify how to act, so new skills can be added with few parameter updates.
A Bridge-and-Adapt adapter with residual memory updates handles cross-domain shift; experiments cover in-domain, cross-domain and lifelong settings.

Show abstract

Vision-Language-Action (VLA) models have shown strong promise for general-purpose robotic manipulation, yet adapting them to new tasks and domains remains inefficient: existing methods often rely on parameter tuning, incurring substantial costs and risking catastrophic forgetting of previously learned tasks. To address this, we propose Optimus-R, a memory-centric VLA framework that formulates robotic adaptation as explicit query-skill memory tuning. Optimus-R introduces: (i) An Inline Memory Interface for skill extraction. It inserts learnable memory tokens into the VLA prefix stream, allowing the backbone to derive control-aware query and skill representations within the native action-conditioning pathway. (ii) A Query-Skill Memory Bank for skill learning. It externalizes skills into query prototypes for deciding what to retrieve and skill values for specifying how to act, supporting skill reuse and expansion with limited parameter updates. (iii) A lightweight Bridge-and-Adapt mechanism for skill updating. It aligns target-domain queries and skills with the existing memory space through a lightweight adapter and residual memory updates. Experiments on in-domain adaptation, cross-domain adaptation, and lifelong learning show that Optimus-R enables data-efficient skill learning while mitigating catastrophic forgetting.

#17

h=20

EmbodiRSI: Recursive Self-Improvement for Data-Efficient Robot Adaptation

📅 2026-09-30cs.RO👤 Ye Shi (h=20)

Haoran Lang, Haotao Lu, Shiyu Sang, Haoyang Luo, Guo Chen

Core Contributions

Closes a real-to-sim-to-real loop: builds a task-specific simulation of the deployment scene and uses it for repeated, failure-targeted policy improvement.
Collaborative Error Correction generates corrective trajectories from states the current policy actually reaches; Adaptive Data Collection steers demonstrations toward current weaknesses.
Simulation success rises 50.4%→83.5% over two rounds across 14 subtasks.
With 400 adaptive sim trajectories plus only 10 real ones per subtask, reaches 83.1% real success vs 75.0% for 200 real demos — a ~20× cut in real data.

Show abstract

Adapting robot manipulation policies to new tasks and environments remains highly data-intensive, while the data needed for further improvement depends on the policy's current capabilities and failure modes. We introduce EmbodiRSI, an agentic system for recursive self-improvement (RSI) in a real-to-sim-to-real setting, where task-specific simulations are constructed from target deployment scenarios and used as low-cost environments for iterative policy improvement before transfer back to the physical world. EmbodiRSI uses policy execution feedback to guide subsequent experience acquisition and policy updates. Two complementary mechanisms close this loop: Collaborative Error Correction generates agent-assisted corrective trajectories from policy-reached states, while Adaptive Data Collection directs expert demonstration generation toward the current policy's weaknesses. The task-specific simulation serves as a reusable workspace for policy warm-up, repeatable evaluation, failure diagnosis, and targeted data generation across successive RSI rounds. Across three tabletop environments and 14 subtasks, EmbodiRSI increases scene-balanced autonomous simulation success from 50.4% to 83.5% over two RSI updates. With 400 adaptive simulated trajectories and only ten real-world refinement trajectories per subtask, EmbodiRSI achieves 83.1% scene-balanced autonomous real-world success, compared with 75.0% for adaptation using 200 real-world demonstrations per subtask. These results demonstrate that feedback-driven recursive improvement in deployment-specific simulations can enable data-efficient adaptation of embodied policies to physical environments.

#30

h=15

PRICE the Action Chunks: Physical Relational Credit Assignment for Embodied Reinforcement Learning

📅 2026-09-30cs.RO👤 Haibao Yu (h=15)

Yangang Zou, Jiajun Lu, Weitao Zhou, Haibao Yu, Bozhou Zhang

Core Contributions

Outcome-based RL for VLAs gives every action chunk the same advantage, so useful early actions in a failed episode get penalised; PRICE fixes this without a learned critic or extra supervision.
Builds a physical relational graph that links rollouts reaching corresponding physical situations and pools their outcomes into success potentials at chunk boundaries.
Confidence-gated credit uses changes in these potentials to refine supervision, with a finite-sample directional bound connecting it to the terminal-success objective.
Improves success and learning speed over outcome-only baselines on LIBERO, RoboTwin 2.0 and real robots; continuation tests confirm credits track local progress.

Show abstract

Outcome-based reinforcement learning (RL) post-trains vision--language--action policies using terminal success signals, but assigns the same trajectory-level advantage to every action chunk. A failed episode can thus penalize useful early actions as if they caused the failure. Existing approaches seek finer-grained feedback through learned evaluators, adding task-specific supervision or additional model training. We explore, for the first time to our knowledge, whether physical relations across trajectories can provide action-chunk credit in embodied RL from terminal outcomes alone, without an auxiliary evaluator. The key insight is that rollouts reaching corresponding physical situations can serve as references for one another: their terminal outcomes provide evidence for assessing local progress. We introduce Physical Relations for Inferring Credit from Episodes(PRICE), with two components: (i) a physical relational graph that pools current and historical outcomes at corresponding chunk boundaries to estimate success potentials; and (ii) confidence-gated credit assignment that uses changes in these potentials to refine trajectory-level supervision. Our analysis connects oracle potential changes to the terminal-success objective and provides a finite-sample directional bound for outcome-independent evidence pools. Independent continuation tests show that PRICE's retained credits align with local progress, while experiments on LIBERO, RoboTwin 2.0, and real robots demonstrate improved task success over outcome-based baselines and faster learning.

#10

h=23

Linear Recurrent Memory Suffices to Distil a World-Model Policy for Robot Air Hockey

📅 2026-09-30cs.ROcs.LG👤 O. Obst (h=23)

F. Olivia Fan, Oliver Obst

Core Contributions

Tests a clean hypothesis: does memory-dependent control need nonlinear recurrence? Uses air-hockey defence with temporary puck-tracking loss, verifying that memory is necessary (resetting the DreamerV3 teacher's state hurts sharply).
Distils the teacher into 64-d policies with a diagonal linear recurrence plus optional rank-k nonlinear innovation; the k=0 model matches both GRU and teacher across all blackout lengths.
Adding nonlinear innovation rank gives no measurable gain, while the linear model is cheaper than a GRU — supporting SSM-style linear memory for embedded control.
Methodologically careful: five matched seeds and a held-out test split opened only after models were frozen, with explicit scope limits.

Show abstract

Does memory-dependent control need nonlinear recurrent dynamics? We study simulated air-hockey defence under temporary loss of puck tracking. A DreamerV3 teacher outperforms a memoryless policy under tracking loss, while resetting the teacher's recurrent state sharply reduces performance, which demonstrates that the task requires memory. We distil this teacher into compact recurrent policies with a 64 dimensional state, with a combination of a diagonal linear recurrence and an optional rank-k nonlinear innovation while retaining nonlinear observation encoders and action heads. Across five matched seeds, the purely linear recurrent model (k=0) matches both the GRU baseline and the teacher throughout the tested range of tracking loss. Increasing nonlinear innovation rank providing no measured benefits. This result is obtained on a fresh test split, which will be only opened after all models and analyses are frozen. The linear model requires fewer recurrent parameters and less computation than GRU, but performs comparably. These results suggest that, for this memory dependent control task, nonlinear representation learning around a simple linear memory mechanism can be sufficient, and that nonlinear recurrent dynamics are not necessarily required. These conclusions are limited to the simulated task, teacher, state dimension, and blackout horizon considered here, and to policies whose observation encoder and action head remain nonlinear.

Agentic Embodied Systems & Benchmarks

h=40

AssemblyWorld: Rethinking 3D Assembly with General-Purpose Agents

📅 2026-09-30cs.CVcs.RO👤 A. Cherian (h=40)

Jiahao Zhang, Yeying Fan, Moitreya Chatterjee, Suhas Lohit, Bernhard Egger

Core Contributions

Asks whether off-the-shelf agents can assemble 3D objects purely through visual interaction (rendered views, no mesh access), with geometric rather than visual-similarity scoring.
AssemblyWorldBench spans 100 tasks / 80 objects across furniture, industrial parts and fracture reassembly — three regimes that stress manuals, precision and shape matching differently.
Reveals a sharp gap between part-level and whole-object correctness: best system 80.9% part accuracy but only 59.4% complete assemblies, so small residual pose errors compound.
Open-source agents trail closed models in both execution reliability and accuracy; trajectory analysis shows agents revise structure but rarely fix fine positioning.

Show abstract

The task of 3D assembly requires translating an understanding of parts and their relationships into precise spatial arrangements. Can pretrained general-purpose agents assemble objects through visual interaction without additional assembly-specific fine-tuning? To investigate this question, we introduce AssemblyWorld, an interactive 3D environment in which agents inspect rendered views and manipulate supplied rigid parts, guided by images or assembly manuals when available. Agents perceive part geometry through 2D views rather than direct access to mesh vertices or faces, while their resulting assemblies are evaluated geometrically. Building on this environment, we construct AssemblyWorldBench, comprising 100 assembly tasks across 80 objects spanning furniture, industrial assembly, and fracture reassembly. Evaluating eight agent systems reveals substantial differences in their capabilities. The strongest system achieves 80.9% part accuracy but 59.4% complete-assembly success. The evaluated open-source systems lag substantially behind their stronger closed-source peers in both execution reliability and assembly accuracy. Analyses of visual references, interaction trajectories, and failures show how agents revise assemblies while leaving residual positioning errors. AssemblyWorld provides a common setting for both assessing the capabilities of interactive assembly agents and characterizing the gap between approximate structure recovery and precise reconstruction.

#11

h=22

ASENA: Self-evolving Agents for Embodied Navigation

📅 2026-09-30cs.RO👤 Jan Kautz (h=22)

An-Chieh Cheng, Isabella Liu, Edmund Bu, Johan Bjorck, Hongxu Yin

Core Contributions

Connects general coding agents to robot sensing, execution and a persistent workspace, so agents write programs, inspect outcomes, repair failures and reuse skills — all with frozen weights.
ASENA-VLN, a 4B monocular policy, is a strong standalone navigator (68.7% R2R, 70.2% RxR) and adds +11 pts when called as a tool by the coding agent.
Persistent workspace evolution over ten passes lifts recurring-task success from 72%→98% (R2R) and 65%→89% (RxR): a clear demonstration of experience accumulation without fine-tuning.
Real Unitree G1 demos combine search, inspection, spatial reasoning and synthesized gestures without a prior map.

Show abstract

We present ASENA, an embodied agent system that connects general-purpose coding agents to robot sensing, computation, supervised execution, and persistent experience. Agents can write and execute programs, inspect recorded outcomes, repair failures, and reuse notes and executable skills while keeping their model weights fixed. We further introduce ASENA-VLN, a 4B monocular navigation policy that serves as an optional tool within this programmable system. ASENA-VLN predicts body-frame trajectories for both extended routes and short-horizon behaviors using a shared vision-language decoder trained on route instructions, visual question answering, and a newly curated dataset of geometry-derived atomic navigation tasks. As a standalone policy, ASENA-VLN achieves state-of-the-art success rates of 68.7% on R2R and 70.2% on RxR. When integrated with a coding agent, learned navigation improves ASENA's success rate by 11 percentage points on both agentic benchmarks while reducing execution time. Through persistent workspace evolution and simulator feedback, ten passes over recurring 100-task subsets further improve success from 72% to 98% on R2R and from 65% to 89% on RxR. On embodied question answering, ASENA achieves state-of-the-art accuracy with fewer interaction steps. Finally, real-world demonstrations on a Unitree G1 combine search, visual inspection, spatial reasoning, and synthesized gestures without a pre-built map, illustrating how online programming extends robot behavior beyond route following and predefined skills.

#14

h=20

NavHarness: Adaptive Goals for Agentic Vision-Language Navigation

📅 2026-09-30cs.CVcs.RO👤 Li-Qiang Nie (h=20)

Haoxiang Shi, Zaijing Li, Muhe Ding, Xiang Deng, Yaowei Wang

Core Contributions

Splits agentic VLN into Goal, Verify, Memory and Visuomotor agents, so local actions are anchored to explicit adaptive subgoals rather than chosen step by step.
Goal-specific verification questions decide when a subgoal is done — and that boundary triggers memory compression, controlling context growth in long routes.
Evaluated on R2R-CE and RxR-CE across three backbones; real-world 83.3% success and 1.51 m error over eight routes × 3 trials.

Show abstract

Vision-Language Navigation (VLN) requires embodied agents to generate actions based on instructions and observations. General-purpose multimodal agents offer a promising basis for this task, but selecting plausible local actions does not ensure that execution remains consistent with the intended route, particularly in long-horizon tasks. Moreover, the accumulated interaction history increases the input required for subsequent decisions, resulting in a significant inference overhead. To this end, we introduce NavHarness, an Agentic VLN framework that includes a Goal Agent that sets adaptive goals for local actions, a Verify Agent that dynamically verifies whether a goal has been completed, a Memory Agent for multimodal context compression, and a Visuomotor Agent to execute adaptive goals. Specifically, the Goal Agent formulates adaptive goals based on the instruction, current observation, and execution history. Then the Visuomotor Agent executes navigation actions to achieve each goal, while the Verify Agent uses a goal-specific verification question to dynamically assess whether the observed outcomes satisfy the intended completion condition. Verified goal completion then marks a boundary for the Memory Agent to compress the corresponding multimodal interaction history while preserving information needed for subsequent navigation. We evaluate navigation on R2R-CE and RxR-CE, examine framework variants across three model backbones, and study context evolution during execution. For Real-World evaluation, NavHarness achieves 83.3% success and 1.51 m navigation error across eight challenging routes evaluated three times each.

#22

h=16

Refusals That Bend: Measuring and Predicting Task Malleability in Embodied VLM Planners

📅 2026-09-30cs.CRcs.RO👤 Muslum Ozgur Ozmen (h=16)

Leo Y. Lin, Mikhail Kuznetsov, Muslum Ozgur Ozmen, Z. Berkay Celik

Core Contributions

Asks a question existing red-teaming ignores: does a refusal survive an ordinary, non-adversarial change to the scene?
Adding a single everyday object flips 20.2% of 846 initially-refused tasks to compliance — and objects from a fixed generic list work about as well as task-specific ones.
Argues susceptibility ('malleability') is a property of the task, tied to how conspicuous the hazard is in instruction and scene.
Malleability can be predicted beforehand from a small open VLM's signals, finding vulnerable tasks 2.4× better than chance — suggesting per-task safety audits before deployment.

Show abstract

Embodied vision-language models (VLMs) are increasingly deployed as high-level planners for robots because they generalize across diverse environments. However, this requires their safety alignment to also hold in unseen environments. Existing red-teaming assumes an adversary who optimizes the prompt, the pixels, or text in the environment, and existing benchmarks ask whether a planner recognizes or mitigates a hazard in a fixed scene. Neither asks whether a refusal the planner has already given survives an ordinary change to the environment. We ask that question by placing a single everyday object into the environment, with no pixel, gradient, or prompt under adversarial control. On 846 tasks that a constitution-guarded planner initially refuses, we find 20.2% of tasks can be flipped to compliance by one or more objects, and the number of objects differs from one task to another. In addition, the object need not be chosen for the task, i.e., items drawn from a fixed list, with no knowledge of the environment or the instruction, bypass safety about as often as items proposed for the specific task. We qualitatively contrast the tasks bypassed most and least often and find that the distinction lies in how conspicuous the hazard is in the instruction and environment. Susceptibility to safety bypass is therefore a property of the task, which we call its malleability, and we show that it can be predicted before the target is ever queried. A composite of signals read from a small open-source VLM identifies malleable tasks 2.4× as often as picking at random. Everyday objects, whether placed by an adversary or introduced by ordinary rearrangement of the environment, are thus sufficient to overturn a refusal. Because susceptibility is determined by how a task is specified, we recommend assessing malleability per task prior to deployment.

#28

h=15

LIBERO-Agent: Evaluating General-Purpose Agents for Direct Embodied Manipulation

📅 2026-09-30cs.RO👤 Hou-Ze Xu (h=15)

Zijie Diao, Yitong Chen, Sicheng Xie, Tianyi Lu, Wujian Peng

Core Contributions

Unlike benchmarks where agents output Python programs or call high-level skills, agents here choose which observations to inspect, process them with their own tools and issue native actions.
Integrates 200 LIBERO tasks with a 30-task primary suite that separates perception, short-horizon execution and long-horizon composition.
Finds a steep reliability gap: strong on perception and easy short-horizon tasks, weak on hard short- and long-horizon ones; richer observations help, demonstrations help inconsistently.
GPT-6 Astra leads, mainly via mechanism interaction under sustained contact; remaining failures come from cross-stage interference and geometric error.

Show abstract

General-purpose agents can plan, use tools, and revise their behavior from feedback, but it remains unclear whether these capabilities transfer from digital environments to embodied manipulation. To investigate this question, we introduce LIBERO-Agent, an agent-native benchmark for evaluating these agents in robot manipulation tasks. Rather than asking agents to submit task-level Python control programs or operate through high-level robot skills, LIBERO-Agent provides an interactive robotic environment where agents can select which observations to inspect, process them with their own tools, and issue native action commands. LIBERO-Agent integrates 200 tasks into a common interaction framework and provides a 30-task primary suite that separates perception, short-horizon execution, and long-horizon composition. Results reveal a pronounced reliability gap: while agents perform well on perception and easy short-horizon tasks, their performance degrades substantially on hard short-horizon and long-horizon tasks. Richer observations improve short-horizon manipulation, while demonstration benefits depend on the agent and format. Among these agents, GPT-6 Astra achieves the strongest overall performance. Further analysis shows its major advantage lies in mechanism interaction, especially when sustained physical contact is needed, while its remaining failures stem from cross-stage interference and geometric errors.

Grounding, Active Perception & Field Sensing

#25

h=15

GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives

📅 2026-09-30cs.CVcs.AIcs.LGcs.RO👤 Yi-Xiao Ge (h=15)

Qize Yu, Lianrui Fan, Boyu Chen, Jiaqi Liang, Xini Ding

Core Contributions

Argues VLAs and WAMs inherit weak grounding from general VLM/video backbones and fails in clutter and on tiny objects; trains a dedicated 4B grounding model outputting points and boxes as quantized tokens.
Pretraining + SFT + GRPO reinforcement learning; 73.68% average across 34 benchmarks, above the much larger GPT-6 Astra (71.54%).
As a drop-in visual backbone it beats all tested backbones in four OOD RoboTwin 2.0 settings (up to +24.8% relative) and, with 50% of RoboCasa-GR1 demos, outperforms baselines trained on 75%.
Data-recipe analysis finds dense grounding helps both manipulation and driving, and OCR acts as a catalyst for perceptual learning.

Show abstract

Precise grounding matters. It specifies which object is the target and where that object is, even in clutter and for tiny objects, and it has to be fast enough for closed-loop control. Yet vision-language-action (VLA) and world-action models (WAMs) take perception from general-purpose vision-language and video-generation backbones, which still fail in these settings. We introduce GroundingPI, a 4B grounding foundation model that generates points and boxes as quantized coordinates in a shared vocabulary. Training combines multimodal and spatial pretraining, supervised fine-tuning, and reinforcement learning with GRPO, using supervision from public datasets and dedicated data engines. Against 44 baselines across 34 grounding benchmarks spanning 11 perceptual capabilities, GroundingPI establishes a new state of the art, averaging 73.68%, above the larger GPT-6 Astra (71.54%). As a downstream visual backbone, GroundingPI improves performance on robotic manipulation and autonomous driving. On RoboTwin 2.0, it outperforms every mainstream backbone we evaluate in all four out-of-distribution settings, by up to 24.8% relative to the strongest backbone. On RoboCasa-GR1, GroundingPI trained with 50% of the demonstrations outperforms those baselines trained with 75%. On nuScenes, used as the visual backbone, GroundingPI attains an average open-loop L2 error of 0.296 m. We systematically analyze GroundingPI's pretraining in scale and data composition. Downstream autonomous driving and robotic manipulation improve as the pretraining is scaled. Analyzing the data recipe across these 11 perceptual capabilities shows dense grounding's substantial benefits for both, and OCR's potential as a catalyst for perceptual learning. These results support grounding as a perceptual foundation, and dedicated perceptual pretraining as a promising direction for foundation models of physical intelligence.

#26

h=15

GroundAnything: Reconciling Parallel Decoding with Precise Visual Grounding at Flash Speed

📅 2026-09-30cs.CVcs.AIcs.LGcs.RO👤 Shi-Long Liu (h=15)

Qize Yu, Lianrui Fan, Bowen Ping, Xini Ding, Zetian Song

Core Contributions

Treats grounding as order-free evidence extraction and switches from autoregressive to blockwise bidirectional diffusion decoding so boxes and relations are refined in parallel.
Converts an AR model to diffusion with joint objectives, then SFT and GRPO; the AR variant reaches 72.42% over 30 benchmarks, competitive with GPT-6 Astra.
Entropy-guided diffusion decoding averages 61.75% vs 53.32% for the fast LocateAnything baseline at similar scale.
Self-speculative mode gives a 4.51× speedup for a 0.74-pt COCO F1mIoU drop — important for latency-bound robot perception. Companion to GroundingPI (#25).

Show abstract

Autoregressive (AR) grounding models serialize spatial predictions, introducing sequential latency and imposing a causal order on output tokens. We view grounding as visual evidence extraction: objects, locations, and spatial relations are jointly constrained by the image and query, yet their dependencies do not imply an intrinsic left-to-right generation order. This distinction makes bidirectional diffusion a natural fit, allowing spatial hypotheses to emerge in parallel and be jointly refined through iterative denoising. We introduce GroundAnything, a 4B-parameter grounding foundation model that reconciles fast parallel decoding with precise localization through blockwise denoising. Training combines grounding pretraining from public datasets and dedicated data engines, direct AR-to-diffusion conversion with joint AR and diffusion objectives, supervised fine-tuning, and GRPO-based reinforcement post-training. Across 30 grounding benchmarks, our autoregressive variant, GroundAnything-VLM, establishes a new overall state of the art among similarly sized models at 72.42%, remaining competitive with GPT-6 Astra (71.35%). With entropy-guided decoding, GroundAnything also surpasses the prior state of the art at this scale, averaging 61.75% versus 53.32% for the fast MTP-based LocateAnything model. We further explore decoding strategies, showing that an optional self-speculative mode achieves a 4.51× speedup over the AR counterpart with a 0.74 percentage-point drop in COCO F1mIoU. Infrastructure experiments show that progressive inference optimizations translate parallel decoding into practical speedups. These support efficient visual grounding in latency-sensitive real-world systems.

#21

h=19

Matisse: Evidence-Space Reasoning for Active 3D Reconstruction

📅 2026-09-30cs.CVcs.RO👤 Rajat Talak (h=19)

Xihang Yu, Kaichen Zhou, Lorenzo Shaikewitz, Clément Jambon, Xiao Zhan

Core Contributions

Unlike uncertainty estimates computed over already-observed geometry, Matisse reads evidential uncertainty from a pretrained generative 3D model's cross-attention, so it can reason about unseen structure.
One Evidential Information Gain criterion drives both next-view selection and keyframe retention, unifying two usually separate problems.
Propagates uncertainty through intermediate latents, avoiding full reconstruction while planning, and handles multi-object scenes with occlusion-aware balancing.
Chamfer distance −12.7% (GSO30), −3.8% (YCB-V), −9.2% (Replica); 1.50× faster; matches Stream3D using only 14% of the views.

Show abstract

How can a 3D reconstruction system acquire and retain useful information to understand the geometry of a scene from partial views under a limited computation budget? Existing active view acquisition methods typically estimate uncertainty over observed or instantiated geometry, limiting their ability to reason about unseen structure, while long-horizon reconstruction methods often retain redundant observations. We introduce Matisse, a training-free framework that unifies active reconstruction and keyframe selection by leveraging evidence provided by a pretrained generative 3D model. Matisse estimates Evidential Uncertainty from cross-attention evidence associated with 3D latent tokens and derives an Evidential Information Gain to guide both view acquisition and keyframe selection based on the expected reduction in posterior entropy. Matisse supports multi-object scenes through occlusion-aware, object-balanced aggregation and propagates uncertainty through intermediate latents to avoid full reconstruction during planning. Matisse reduces Chamfer distance by 12.7%, 3.8%, and 9.2% on GSO30, YCB-V, and Replica, respectively, relative to the best baseline on each dataset, and achieves a 1.50× end-to-end speedup over the best active reconstruction baseline on GSO30 with the same reconstruction backend. In the GSO30 keyframe selection experiment for long-horizon reconstruction, Matisse achieves comparable Chamfer distance using 14% of the input views compared with Stream3D.

h=26

Active Mapping of Underwater Litter Using Camera-Sonar Fusion

📅 2026-09-30cs.RO👤 L. Buşoniu (h=26)

David Rete, Patrick Boros, Lucian Busoniu

Core Contributions

Fuses forward-looking sonar and camera into one Bayesian occupancy map, so each sensor covers the other's weakness (turbidity vs. noise/low resolution).
Each sensor gets empirically fitted range- and bearing-dependent detection and false-alarm tables, making the fusion principled rather than heuristic weighting.
Next-best-view selection balances voxel-entropy exploration against exploitation of likely debris, replacing fixed lawnmower surveys.
In a realistic underwater simulator, active mapping finds litter faster than lawnmower coverage, and dual-sensor beats either sensor alone.

Show abstract

Marine litter is a growing threat to the underwater ecosystem, driving demand for autonomous survey methods that can locate debris efficiently over large areas. Existing survey methods typically follow predefined paths or operate with a single sensing modality, typically a camera (with image quality suffering in poor-visibility conditions) or sonar (usually noisy and low-resolution). We present an active mapping framework in which a forward-looking sonar and a camera both feed into a shared Bayesian occupancy map, and an optimization problem is solved at each step to decide on the next best view. Candidate viewpoints are scored by a two-term utility that balances exploration of uncertain regions via voxel entropy against exploitation of likely objects. Each sensor is characterized by range- and bearing-dependent detection and false-alarm probability tables determined from data. We evaluate the approach in a realistic underwater simulator, demonstrating that active mapping finds objects faster than a lawnmower coverage pattern, and that the dual-sensor approach works better than using either of the individual sensors.

h=36

Video2SwimFish: An Automated Pipeline for Reconstructing Controllable Fish Models and Biological Locomotion from Real Fish Videos

📅 2026-09-30cs.RO👤 I. Fuller (h=36)

Hangong Chen, Linfeng Cheng, Tahsin Zaman Jilan, Ian Fuller, Lee Caesar

Core Contributions

Automates the creation of controllable, individual-specific fish assets from multi-view video — mesh, VLM-designed articulation and a swimming policy per animal.
The Biological Locomotion Manifold, extracted from observed midline curvature, bounds the action space to real fish motion, so learned policies swim like that particular fish.
Releases 120 individuals across 6 species with paired videos, assets and policies, and a benchmark scoring fidelity (trajectory shape, curvature, tail-beat frequency), not just task success.
Finds that the best-performing method on task completion is not the most faithful — exposing a tension that matters for bio-inspired robotics and realistic underwater sim (e.g. a BlueROV capture case study).

Show abstract

We present Video2SwimFish, an automated pipeline and benchmark for building controllable fish assets from real-fish videos for underwater embodied AI. Given synchronized multi-view videos of an individual fish, the pipeline reconstructs a metrically scaled deformable mesh from a VLM-selected canonical frame, generates internal articulation adapted to that individual's morphology through a VLM actor-critic loop, and extracts a Biological Locomotion Manifold (BLM) from the fish's observed midline curvature. The BLM provides a low-dimensional action space bounded by real-fish motion, enabling an individual swimming policy to be learned for each reconstructed fish. We release two paired datasets: synchronized top- and front-view recordings of 120 individual fish across 6 species, and the controllable assets and individual swimming policies derived from them. Because every asset is tied to the animal it came from, the dataset supports a benchmark that evaluates locomotion learning not only on task success but on fidelity to that individual in trajectory shape, body curvature, and tail-beat frequency, across trajectory following, reward-free swimming behavior transfer from video, and a downstream case study in which a simulated BlueROV underwater robot captures one of the assets. We find that task success and locomotion fidelity do not necessarily improve together: the method achieving the highest task completion is not the method achieving the highest locomotion fidelity, and we identify faithful reproduction of individual animal locomotion as an open challenge for the community. Project website: https://hangongchen.github.io/video2swimfish-web/.

Social & Mobile Navigation

h=37

Social-WM: Safety-Aware Latent World Models for Robot Social Navigation

📅 2026-09-30cs.RO👤 Mooi Choo Choo Chuah (h=37)

Zhihao Zheng, Mooi Choo Chuah

Core Contributions

Key observation: in social navigation the commanded action and the realized action diverge exactly when a pedestrian or obstacle constrains motion — that gap is itself a safety signal.
Trains a latent world model on actual observed futures plus a 'realizable' inverse-dynamics head that maps transitions to the action really executed, not the one commanded.
At planning time, imagined rollouts are scored by nominal-vs-realizable discrepancy, flagging unsafe actions before execution without pedestrian tracking or privileged human state.
63.77% success with human collisions down to 21.67% on Social-HM3D, and zero-shot transfer to Social-MP3D, all without online RL; dynamics are goal-agnostic so position- and image-goals share one model.

Show abstract

Safe social navigation requires a robot to anticipate not only the future consequences of its actions, but also whether a nominal action can actually be executed under surrounding physical and social constraints. We present Social-WM, an efficient latent world-model planning framework trained from egocentric RGB video sequences. Our key observation is that social-navigation experience contains a systematic discrepancy between the nominal action and the realizable action: a nominal forward action may be fully executed in free space, but needs to be constrained when heading towards a pedestrian or obstacle. Social-WM learns these safety-relevant consequences directly through action-conditioned future prediction, where the target is the actual observed future following each command. We further introduce a realizable inverse-dynamics objective that associates observed latent transitions with the action actually realized rather than the nominal one. At deployment, candidate actions are imagined through the latent world model, and the inverse dynamics model estimates their realizability; nominal--realizable discrepancy then provides a safety signal before execution. The learned dynamics and realizability model remain goal-independent and support both position- and image-goal navigation. On Social-HM3D, Social-WM achieves 63.77% success while reducing human collisions to 21.67%, and maintains strong performance under zero-shot transfer to Social-MP3D, without explicit pedestrian tracking, privileged human state, or online reinforcement learning.

#19

h=19

Rethinking Legibility in Social Robot Hallway Navigation: Impact of Intent Representation and Human Distraction

📅 2026-09-30cs.ROcs.HC👤 Christoforos Mavrogiannis (h=19)

Pranav Goyal, Andrew Stratton, Christoforos Mavrogiannis

Core Contributions

Questions whether legibility formulations built for static observers fit hallway navigation, where people move and are often distracted.
Two controlled user studies (N=45 each) within a shared MPPI framework compare passing-side vs destination-based legibility and attentive vs distracted pedestrians.
Adaptive passing-side legibility yields smoother human motion and is rated more competent and less demanding — intent should be expressed at the interaction level, not the goal level.
Behavioural benefits persist under distraction even when subjective ratings don't show it, a caution for HRI studies relying only on questionnaires.

Show abstract

We focus on legible robot motion generation in social navigation settings. Legibility in human-robot interaction (HRI) is often described as the property of robot motion that enables an observer to confidently infer the robot's intent. While mature frameworks exist for generating legible motion in front of static observers, social robot navigation presents a new challenge: the robot must clearly convey its intent while ensuring human safety in dynamic pedestrian environments where human attention is often divided. With the goal of enabling robots to generate legible motion in dynamic and constrained spaces, we investigate how the choice of representation and the level of human attention shape navigation performance and human impressions. Focusing on the ubiquitous and demanding scenario of hallway navigation, we conduct two controlled user studies involving alternative legibility formulations implemented within a shared model predictive control framework. Study 1 (N = 45) investigates the role of intent representation, showing that passing-side legibility, particularly when adaptively updated, leads to smoother human motion and is perceived as more competent and less mentally and physically demanding than destination-based and non-legible baselines. Study 2 (N = 45) examines the effect of pedestrian attention, demonstrating that legible motion allows for smooth human motion even under distraction, even if this is not consistently reflected in subjective ratings. Together, these findings suggest that effective legible motion in social robot navigation benefits from interaction-level intent representations that support coordination, with some effects persisting even when human attention is divided. Code is available at https://github.com/fluentrobotics/Legible_MPPI.

#29

h=15

Local-Minimum Escaper: Programmatic Subgoal Generation for Robust Navigation in Unknown Environments

📅 2026-09-30cs.RO👤 Si-Wei Cheng (h=15)

Yin Gu, Xinming Zhang, Shanze Wang, Siwei Cheng, Wei Zhang

Core Contributions

Makes local-minimum escape explicit: rather than hoping a DRL policy learns recovery through reward shaping, LME generates and selects subgoals programmatically.
Interpretable heuristics score candidate subgoals on surrounding obstacle geometry and location safety, using only local observations.
Planner-agnostic and training-free — it upgrades different local planners' ability to escape traps, and works on both differential-drive and quadruped robots in sim and real.

Show abstract

Mapless navigation in unknown and partially observable environments remains challenging for mobile robots, particularly when local minima prevent the robot from making progress toward its goal. Existing local navigation methods often lack an explicit mechanism for escaping such situations, while deep reinforcement learning (DRL) approaches typically learn recovery behaviors implicitly through reward design and policy optimization. In this work, we propose LME (Local-Minimum Escaper), a programmatic hierarchical framework that explicitly generates and reasons subgoals to guide robots out of local-minimum regions. LME operates solely on local observations and selects candidate subgoals using interpretable heuristic criteria that account for both surrounding obstacle geometry and candidate-location safety. A local planner then generates low-level motion commands toward the selected subgoal. This design enables LME to handle environments both with and without local minima within a unified framework, while remaining independent of the underlying local planner and requiring no additional training. Extensive experiments in simulated and real-world environments demonstrate that LME provides robust navigation performance and generalizes to challenging unseen scenarios. Furthermore, the generated subgoals can be used to guide different local planners, substantially improving their ability to escape local minima. Successful deployments on both differential-drive and quadruped robots further demonstrate the practical applicability and generality of the proposed framework.

#27

h=15

Sparse Planner: A Hybrid Planner for Efficient Sampling via a Conditional Variational Autoencoder

📅 2026-09-30cs.RO👤 Richard Membarth (h=15)

Wenguang Xu, Giovanni Lucente, Karem Mohamed, Richard Membarth

Core Contributions

Sampling-based planners need dense sampling for good trajectories, causing heavy and unpredictable runtimes in complex traffic.
A conditional VAE learns which trajectory parameters are good given scene context and samples there directly, reducing density instead of speeding up evaluation.
Lower trajectory cost than FISS+ with one-eighth the sampling density, better distance-keeping around obstacles, and more stable runtime — useful for real-time safety budgets.

Show abstract

Trajectory planning is a core component of autonomous driving systems, where real-time performance and solution quality directly affect safety and reliability. Sample-Based Motion Planning (SBMP) is widely adopted for its ability to approximate near-optimal solutions through parameter space sampling. However, achieving high-quality trajectories typically requires dense sampling, leading to substantial computational overhead and significant runtime variability in complex traffic scenarios. To address this limitation, we propose a Sparse Planner (SP) that improves sampling efficiency by learning the conditional relationship between scene context and effective trajectory parameters using a Conditional Variational Autoencoder (CVAE). By modeling the structure of high-quality sampling distributions, SP directly generates cost-effective samples in the parameter space, significantly reducing the required sampling density while preserving solution quality. Experimental results show that SP achieves lower trajectory cost than the state-of-the-art FISS+ planner while using only one-eighth of the sampling density. In addition, SP demonstrates improved distance-keeping capability in obstacle-rich scenarios and maintains reduced and more stable runtime characteristics, indicating enhanced computational efficiency and predictable runtime behavior.

Aerial Robotics & Certified Control

h=44

Towards Agile Vision-Based Multi-UAV Flight: Revisiting State Estimation

📅 2026-09-30cs.RO👤 M. Saska (h=44)

Michal Pliska, Matouš Vrba, Ondřej Víta, Martin Jiroušek, Viktor Walter

Core Contributions

Identifies a structural limit of position-only neighbour tracking: velocity and acceleration are inferred from accumulated displacement, adding a constant ~300 ms delay regardless of how aggressive the flight is.
Exploits the physics of multirotors — tilt reveals thrust direction, i.e. acceleration, before displacement accumulates — by fusing detector-provided tilt into a new linear thrust-constraining Kalman filter.
Benchmark of 4 position-only vs 5 pose-aware estimators on two real and one photorealistic dataset (3–21 m/s²): pose-aware estimation cuts velocity/acceleration error by 40% / 57%.
Closed-loop NMPC leader-follower: position-only estimation cannot even sustain stable hovering, while the tilt-aware filter tracks lateral manoeuvres above 2 g — near the camera frame-rate limit.

Show abstract

Agile multi-UAV flight requires accurate and low-latency onboard estimation of the kinematic states of neighboring UAVs for collision avoidance, motion coordination, etc. Most vision-based approaches rely on position-only measurements, inferring velocity and acceleration indirectly from displacement. We show that this introduces a fixed structural delay in the estimation of higher-order states, which limits the achievable agility. To address this, we propose to integrate tilt measurements, provided by a state-of-the-art visual detector, which inform about the thrust direction of co-planar multirotor UAVs. We benchmark four position-only and five pose-aware estimators, including a novel formulation of a linear thrust-constraining Kalman filter, on two real-world and one high-fidelity photorealistic simulated dataset over different levels of agility (3-21 m/s^2). In our setup, pose-aware estimation consistently reduces the average velocity and acceleration estimation errors by 40% and 57% across the three datasets with the proposed KF formulation outperforming the other estimators. Position-only filters exhibit a constant ~300 ms delay in acceleration step response independent of agility, whereas the tilt-constrained estimators operate near the physical response limit given by the camera frame-rate by observing the change in thrust direction before the displacement accumulates. In a closed-loop leader-follower simulated experiment with NMPC control, position-only estimation of the leader's state fails to facilitate stable hovering of the follower, while the proposed estimator enables tracking of lateral maneuvers exceeding 2g of acceleration.

h=26

RL-Guided PAC-NMPC for Probabilistically-Safe Perception-Based Navigation in Unknown Environments

📅 2026-09-30cs.RO👤 Marin Kobilarov (h=26)

Adam Polevoy, Dillon Capalongo, Katherine Tang, Mark Gonzales, Marin Kobilarov

Core Contributions

Combines RL's long-horizon competence with stochastic MPC's guarantees: RL trains probabilistic actor-critic and sensor-prediction models that PAC-NMPC then uses inside its sampling-based optimisation.
Hard constraints give finite-time statistical bounds on collision probability and on value-function improvement, so the short-horizon MPC inherits the RL policy's long-horizon behaviour.
Scales to high-dimensional dynamics and large perception inputs — a regime where earlier certified planners struggle.
Hardware results on an agile fixed-wing UAV doing vision-based navigation in unknown environments show improved safety over the raw RL policy.

Show abstract

In this paper, we present an approach for combining stochastic nonlinear model predictive control (SNMPC) and reinforcement learning (RL) to enable probabilistically-safe perception-based navigation in unknown environments. Our method first uses RL to train probabilistic actor-critic and sensor prediction models. We then leverage these probabilistic models in a sampling-based SNMPC framework known as Probably Approximately Correct (PAC)-NMPC, which uses hard constraints to enforce finite-time statistical guarantees on the probability of collision and value function improvement. By ensuring that our finite-horizon SNMPC policies decrease the value function in expectation, we can approach the long-horizon performance of the RL approach while satisfying probabilistic safety constraints. Through simulation experiments, we show that our approach can improve the safety of perception-based RL navigation policies and scale to high dimensional systems with large sensor input spaces and complex nonlinear dynamics. We also demonstrate our approach through hardware experiments, showing improved performance for vision-based navigation with an agile fixed-wing aerial vehicle in unknown environments.

#18

h=20

Residual Wrench Certification and Margin-Aware Control Synthesis for Aerial Physical Interaction

📅 2026-09-30cs.ROeess.SY👤 Abhinav Sinha (h=20)

Abhimanyu Khadga, Abhinav Sinha, Shashi Ranjan Kumar

Core Contributions

Certifies how much wrench authority a multirotor retains after hover and contact loads, using convex geometry to compute signed margins for ball and ellipsoidal reserves.
Command projection onto a tightened feasible set preserves that reserve during contact; the reserve is linked to gain synthesis with certified local tracking, exponential convergence and robust ultimate boundedness.
Enables morphology comparison: at equal thrust, an optimised octarotor beats a hexarotor on margin, while a variable-tilt quadrotor admits a higher normalised push limit for wall contact.
Beyond the certified boundary, simulations show exactly the predicted failure modes (rotor saturation vs. tilt-servo limits).

Show abstract

We develop a task-relative framework for certifying residual wrench authority after hover and contact loading in multirotors with bounded actuators. Using convex geometry, we derive signed margins for prescribed convex reserves, including Euclidean balls and weighted ellipsoids. We obtain computable reserve certificates from actuator-interiority bounds to support slack-maximizing allocation. To preserve the required reserve, we propose command projection onto a tightened feasible set. We then connect the available reserve to structured gain synthesis and certify a local tracking region that respects actuator limits. Within this region, we establish nominal exponential convergence and robust ultimate boundedness. For the prescribed task and morphology family, we show that the optimized octarotor attains a larger margin than the optimized hexarotor at equal total thrust. We evaluate the proposed framework in closed-loop simulations of sustained rigid-wall contact using a fixed-geometry octarotor and a variable-tilt quadrotor with matched installed thrust. The quadrotor's local certificate admits a higher normalized push limit for the prescribed task family. In tests beyond the realizability boundaries, we observe the predicted loss of authority through rotor-thrust limits for the octarotor and tilt-servo limits for the quadrotor.

h=25

General Performance Guarantee for Human Torque Estimation-Based Task-Agnostic Assistive Exoskeleton Control

📅 2026-09-30cs.ROcs.LGeess.SY👤 B. Berret (h=25)

Duy Hoang, Bastien Berret, Olivier Bruneau, Laurent Fribourg

Core Contributions

Formalises 'matched assistance' — the robot's torque actually helps the human's movement — and designs the interaction torque to guarantee a lower bound on its probability.
The guarantee holds over the full estimated-torque error distribution, including unseen tasks, addressing the core weakness of task-agnostic exoskeletons that rely on imperfect torque estimators.
Validated on the ABLE upper-limb exoskeleton across multiple tasks: theoretical bounds are confirmed experimentally, movements stay smooth and human effort drops.

Show abstract

Accurate human torque estimation is crucial for enabling task-agnostic control in robotic exoskeleton systems. However, estimation errors may cause mismatches between the robot assistance and the human intention, degrading controllability and task performance. In this paper, we address this issue by formally defining matched assistance as scenarios in which the robot positively contributes to human movement. Based on this definition, we develop a theoretical framework to design the robot's desired interaction torque that guarantees a lower bound on the matched assistance probability. Importantly, the proposed guarantee holds over the entire torque distribution, including unseen data beyond the training tasks. This provides our method with strong reliability and generalization, both of which are critical for effective exoskeleton control. The proposed strategy is implemented on the ABLE upper-limb exoskeleton and evaluated in a multi-task setup. Experimental results validate the theoretical guarantees and demonstrate that the proposed strategy achieves effective general performance across several tasks, guaranteeing movement smoothness while reducing human physical effort.

Multi-Robot Coordination

h=23

Communication-Free Distributed Multi-Robot Task Allocation under Partial Observations Using Labeled Multi-Bernoulli Filtering

📅 2026-09-30cs.ROeess.SY👤 A. Kamimura (h=23)

Takumi Ito, Akiya Kamimura

Core Contributions

Removes inter-robot communication entirely: each robot tracks neighbours with a Labeled Multi-Bernoulli filter and runs its own greedy auction on the estimates.
Continuous re-estimation and re-allocation during execution lets robots converge on consistent assignments even when initial beliefs disagree.
Monte Carlo simulations show effective cooperative allocation that is robust to clutter and observation noise — relevant for comm-denied or bandwidth-limited deployments.

Show abstract

This paper proposes a communication-free multi-robot task allocation framework based solely on local observations. In this study, tasks are defined as reaching target locations. Each robot estimates the positions of neighboring robots using a Labeled Multi-Bernoulli (LMB) filter and independently assigns tasks through a greedy auction-based strategy. By continuously updating state estimates and reallocating tasks during execution, the proposed method enables decentralized coordination without explicit communication. Monte Carlo simulations demonstrate that the proposed method enables effective cooperative task allocation without inter-robot communication while remaining robust to measurement clutter and observation uncertainty.

#23

h=16

HALO: Heterogeneous Allocation Via Localized Observations for the Vehicle Routing Problem

📅 2026-09-30cs.ROcs.MA👤 Or D. Dantsker (h=16)

Andrew Meighan, Hyungsub Kim, Or Dantsker

Core Contributions

Targets decentralised fleets with limited observation and communication, a setting where centralised VRP solvers for ~1,000 tasks are not deployable.
Splits VRP into allocation and routing; allocation uses a heterogeneous GNN whose message passing separately learns spatial distribution and robot-task compatibility.
On partially observable online VRP, nearly matches an all-knowing offline variant and beats heuristics; on standard static VRP it outperforms specialised architectures by up to 14.06% with the fastest runtimes.

Show abstract

Scalable robotic fleets have become increasingly popular for various applications such as package delivery, warehouse management, and military operations. Prior fleet control algorithms solve centralized routing problems with up to 1,000 tasks in controlled environments, yet they fail to consider realistic constraints such as limited observation and communication ranges typical of decentralized fleets. Thus, deploying existing fleet control algorithms into real-world settings is currently infeasible. To tackle this, we propose Heterogeneous Allocation via Localized Observations (HALO) to solve the Vehicle Routing Problem (VRP). HALO is a hybrid method that splits the VRP into allocation and routing portions to provide onboard, real-time solutions to robots in dynamic environments. During the allocation phase, HALO utilizes a heterogeneous graph neural network framework with unique message passing layers to explicitly separate the learning of spatial distributions and task-to-robot compatibility. Evaluation results on a partially observable, online variant of the VRP show HALO significantly outperforms the heuristic baseline while maintaining similar solution quality to an all-knowing offline variant of HALO. While HALO is explicitly designed for partially observable environments, it imposes no strict upper bound on the observation space allowing us to test HALO on the traditional static, single-depot VRP. Here, HALO outperforms state-of-the-art architectures strictly optimized for the static variant of the VRP by up to 14.06%. Throughout all testing, this framework maintains the quickest execution times which emphasizes its potential for large-scale, real-time deployment.