🤖 Robotics arXiv Digest

Tuesday, October 13, 2026

📄 30 papers 📂 6 research areas Generated by Claude

Semantic Scholar h-index lookup returned no matches for this batch; papers are listed in fetch order.

🔭 Research Landscape

Today's batch is dominated by world-action models (WAMs) becoming general-purpose robot backbones. GreenWAM (#19) is the most ambitious: one model trained across 20 robot families in a shared anatomical action space, serving both as a top WorldArena world model and as a competitive policy on RoboTwin, LIBERO, and CALVIN. UniW(A)M (#22) takes a complementary route, using a mixture-of-experts to share dynamics between world modelling and several WAM interfaces, then routing per action chunk (88.3% vs 76.7% real-world success). LoopWAM (#2) shows WAMs can learn from the full quality spectrum of deployment rollouts, including failures, raising RoboChallenge success from 23.7% to 43.3% over three rounds. ForceWAM (#8) and ILA (#7) bring force and tactile signals into the action/future representation itself. Yet the evaluation papers urge caution: WorldOracle (#14) is the first benchmark that checks world-model rollouts against paired real executions, and #28 shows a learned grid world model (FloWM) can be beaten by a 14 KB Kalman tracker on out-of-view object memory, failing badly on motions outside its built-in velocities.

The second theme is self-improving agentic robotics, appearing in at least five papers. ChunkAct (#1) borrows action chunking from VLAs to cut LLM calls 2.9× while improving success; EmbodiedEvo (#5) and SEAR (#17) let coding agents accumulate reusable Skills/Lessons, and both find that procedural, generic routines transfer while task-specific solutions do not. SEAR adds rigorous controls asking whether agents are really learning or just re-rolling, finding that on easy tasks re-rolling does as well. The survey on Physical RSI (#12) frames the goal formally and concludes that no current work yet achieves verified recursive gains. On the policy-learning side, EqC (#18) uses energy-landscape consistency to decide when to ask for help, and ACCRUE (#30) provides continual off-policy RL with near-zero forgetting, cutting real-robot training time by 31–69%. Read with yesterday's D-World, a clear direction is forming: frozen foundation models plus external, inspectable memory that improves through experience.

Third, data and robustness are increasingly measured, not assumed. LIBERO-RIG (#11) shows nine VLAs drop from 0.93 to 0.51 success under hardware/setup perturbations (camera bumps, control rate, gripper pads) that visual augmentation cannot fix. AXIS-Bench (#4) systematically studies how demonstration quality should enter VLA pretraining, finding quality conditioning beats filtering. Human-data pipelines (EgoExplorer #3, EgoMatrix #15, Steertuning #20) compete on fidelity and efficiency, with Steertuning reaching 71% across 12 new home environments from 5 human demos and 2 minutes of fine-tuning. RoboFQE (#21) offers a rollout-free way to rank policies from offline data, sidestepping the compounding errors that WorldOracle-style evaluations expose in video world models.

World-Action Models & Predictive Control

Unified world/action foundation models, deployment-driven improvement, counterfactual recovery, and generative planning.

#19 GreenWAM
#22 UniW(A)M
#2 LoopWAM
#27 ForkCast
#24 Model-Based Online Decision Making via Generative…
#28 Grid or Nodes? Probing Object Memory in World Models…

Self-Improving & Agentic Robotics

LLM agents that write code, chunk actions, evolve skills, and policies that learn continually or know when to ask for help.

#1 ChunkAct
#5 EmbodiedEvo
#17 SEAR
#12 Who Improves the Robot? Toward Physical Recursive…
#6 AutoAgent0
#18 EqC
#30 ACCRUE

Contact-Rich & Tactile Manipulation

Encoding force and contact evolution into actions, prompts, world models, and simulators.

#7 Interaction Latents as Actions for Contact-Rich Manipulation
#8 ForceWAM
#10 Contact Trajectory Prompting
#13 RoboSimStudio

Human Data & VLA Training Recipes

Egocentric capture, human-to-robot data engines, few-shot steering, and quality-aware pretraining.

#3 EgoExplorer
#15 EgoMatrix
#20 Steertuning Policies from a Few Human Videos
#4 What Matters in Quality-Aware VLA Pretraining: A…

Benchmarks & Policy Evaluation

Hardware robustness, world-model fidelity against real executions, and offline policy evaluation.

#11 LIBERO-RIG
#14 WorldOracle
#21 RoboFQE

Perception, Multi-Agent & HRI

Radar occupancy, incompleteness-aware perception, heterogeneous team planning, MARL diversity, part assembly, and voice-cloning HRI.

#23 RadarGSOcc
#25 Perception Beyond the Visible
#9 Collaborative Planning for UAV-UGV Teams under…
#16 Optimizing Team Behavior via Extremum-seeking Control in…
#29 LLM2R
#26 Known-Voice Cloning and the Limits of Personality…

World-Action Models & Predictive Control

#19

h=n/a

GreenWAM: A World Foundation Model for Robotics

📅 2026-10-13cs.RO👤 Gennady Sidorov (h=n/a)

Gennady Sidorov, Valerii Kornilov, Denis Grankin, Polina Fedotova, Aleksey Postnikov

Core Contributions

A single world foundation model serving as both world model and policy across 20 robot families, via a shared anatomical action space in common joint coordinates.
Joint training in visual, action-conditioned, and world-action modes; policies split an action expert off the generator.
Leads WorldArena 1.0 on trajectory, depth, and 3D accuracy; ranks near the top on RoboTwin 2.0 and gets the best zero-shot LIBERO-plus result among compared methods.
On a real humanoid it beats policies adapted from π0.5 and Cosmos 3 Edge, and can be distilled to a single-step real-time policy.

Show abstract

A robot needs a world model that predicts how its actions will change the scene and a policy that chooses those actions. Models are usually built and evaluated for only one of these roles, and every robot records its actions in its own format. We ask whether one robot world model can serve both roles across different bodies, and what makes this work. We present GreenWAM, a world foundation model for robotics, together with the recipe that produces it: (1) a shared anatomical action space that describes 20 robot families, from bimanual arms to dexterous humanoids, in common joint coordinates; (2) joint training on these families in visual, action-conditioned and world-action modes; and (3) policies that split an action expert off the generator. As a world model, GreenWAM attains top-tier trajectory, depth and 3D accuracy on WorldArena 1.0, ahead of every world-action model and video generator in our comparison. As a policy, it scores among the top entries of the RoboTwin 2.0 leaderboard, reaches the highest zero-shot LIBERO-plus success among the compared methods and is competitive with dedicated vision-language-action and world-action policies on CALVIN and LIBERO. We further adapt GreenWAM to a real humanoid, where it succeeds more often than policies adapted from $π_{0.5}$ and from Cosmos 3 Edge, and distill it into a single-step policy for real-time control. Controlled experiments yield three findings: robot specialization improves world prediction and control together; an action expert helps when split off at the start of adaptation; and keeping world-model modes during policy adaptation preserves robustness to scene changes.

#22

h=n/a

UniW(A)M: A Unified Mixture-of-Experts Framework for World and World-Action Models

📅 2026-10-13cs.ROcs.AIcs.CVcs.LG👤 Jingbo Xu (h=n/a)

Jingbo Xu, Jianjie Fang, Zhuohang Li, Zihao Song, Zhaolu Wang

Core Contributions

Unifies world models and multiple world-action model variants in one diffusion-Transformer MoE, so prediction and control share learned dynamics.
Shared experts hold common dynamics while reusable capability experts give each interface its own conditional pathway.
A plug-in router selects the best WAM interface per execution chunk with the backbone frozen, raising OOD success from 63.3% to 76.7%.
88.3% real-world success vs 76.7% for the strongest baseline, and co-training with world modelling improves every action interface.

Show abstract

World models (WMs) and world-action models (WAMs) focus on future world prediction and executable action generation, respectively. Although these models learn shared physical dynamics and interaction semantics from embodied trajectories, they are typically trained in isolation, which restricts knowledge reuse between prediction and control. The central challenge is therefore to develop a unified learning framework that shares these dynamics while preserving the distinct modeling capabilities required by each paradigm. We propose UniW(A)M, a mixture-of-experts world-(action) model built upon diffusion Transformers that unifies action-conditioned world modeling and diverse WAM variants. Shared experts capture common dynamics, while atomic capability experts are reused across interfaces so that each interface keeps its own conditional pathway. At deployment, a plug-in adaptive interface router (PAIR) learns from a small subset of target-task chunk scenarios to select a suitable WAM interface for each execution chunk, balancing task success and execution efficiency while keeping the backbone frozen. We evaluate UniW(A)M on LIBERO, RoboTwin 2.0, and three real-world manipulation tasks, where it achieves 88.3% average real-world success versus 76.7% for the strongest baseline. On WorldArena, it leads action-conditioned world modeling on all three tasks. AC-WM co-training contributes to action generation on every action interface and PAIR raises OOD success from 63.3% to 76.7%.

h=n/a

LoopWAM: Closed-Loop Generalist Policy Improvement from Deployment Rollouts

📅 2026-10-13cs.ROcs.CV👤 Wenxuan Guo (h=n/a)

Wenxuan Guo, Yingping Liang, Zhaoguo Wang, Ziyuan Li, Angyuan Ma

Core Contributions

Learns from all deployment rollouts, successes, suboptimal runs, and failures, from itself and other policies, rather than filtering to expert-quality data.
Source-dependent noise sampling unifies action learning and action-conditioned world modelling; quality steering keeps video supervision from failures while down-weighting their actions.
Three deploy-and-train rounds lift RoboChallenge Table30-V2 success from 23.7% to 43.3%; shirt folding rises from 13.2% to 55.6% in two rounds.
Also reaches 94.5% on RoboTwin 2.0 and 49.5% on RoboCasa365 without extra embodied pretraining, showing failures are useful training signal for world models.

Show abstract

Scaling expert demonstrations alone does not enable generalist robot policies to improve from deployment. Real-world rollouts contain successes, suboptimal behavior, and failures: imitating every recorded action can reinforce poor behavior, while filtering out failures discards evidence of physical interactions and their consequences. We introduce \textbf{LoopWAM}, a closed-loop world-action model that learns directly from this full quality spectrum, using rollouts from both itself and other deployed policies. Source-dependent noise sampling unifies behavior learning and action-conditioned world modeling, while quality steering and quality-weighted supervision retain full video supervision across qualities and regulate action imitation. Video and action tokens interact throughout parallel denoising, allowing learned consequences to inform action generation directly. On RoboChallenge Table30-V2, three deployment-and-training rounds raise success from 23.67\% to 43.33\%, a 19.7-percentage-point gain, with a final score of 56.80. LoopWAM also achieves 94.5\% average success on RoboTwin~2.0 and 49.5\% overall success on RoboCasa365 without additional embodied pretraining, obtaining the best overall results among the compared methods across all three benchmarks. On challenging shirt folding task, two rounds of deployment-driven improvement after single-task post-training raise success from 13.20\% to 55.56\%. These results demonstrate closed-loop improvement from mixed-quality experience under both multi-task learning and single-task post-training. \href{https://loop-wam.github.io/}{\textcolor{orange}{Project Page}}

#27

h=n/a

ForkCast: Predictive Recovery for Vision-Language-Action Policies via Counterfactual Futures

📅 2026-10-13cs.RO👤 Chenru Wen (h=n/a)

Chenru Wen, Huajian Zeng, Haokai Ding, Jian Kang, Anqing Duan

Core Contributions

Decides whether and how to intervene in a frozen VLA by comparing predicted futures of three options: continue (Commit), move the camera (Reobserve), or replan the arm (Replan).
A candidate-conditioned world model plus a task-progress head replaces separate failure detectors and hand-set thresholds.
Raises pooled success under injected occlusion and slip from 13.3% (frozen π0.5) to 30.0% on RoboTwin 2.0.
Transfers to two other VLA backbones without retraining and improves real-robot success after adaptation.

Show abstract

Vision-language-action (VLA) policies perform manipulation tasks using camera images and language instructions, but object slip or visual occlusion can disrupt execution. Recovery is typically learned during policy training or delegated to a monitor that triggers a scripted repair after detecting a failure. We introduce ForkCast, a predictive recovery framework that compares counterfactual futures to determine whether and how to intervene in a frozen VLA policy. At each decision point, ForkCast evaluates three candidates: executing the policy's nominal action chunk (Commit), adjusting the camera viewpoint (Reobserve), and replanning the arm trajectory (Replan). A candidate-conditioned world model forecasts the latent outcome of each candidate at a common future horizon, and a shared task-progress head scores the predicted outcomes. By evaluating the policy's own continuation alongside the repair candidates, a single comparison based on task progress determines both whether and how to intervene, without a failure detector, an intervention threshold, or updates to the VLA backbone. On five simulated RoboTwin 2.0 tasks with injected occlusion and object slip, ForkCast increases pooled success from 13.3% for the frozen $π_{0.5}$ policy to 30.0% and transfers to two additional VLA backbones without updating ForkCast. After adaptation on real-robot candidate rollouts, ForkCast also improves success over the base policy under both clean and visibility-limited conditions. Code will be released to benefit the research community.

#24

h=n/a

Model-Based Online Decision Making via Generative Trajectory Planning

📅 2026-10-13cs.LGcs.AIcs.ROeess.SY👤 Haldun Balim (h=n/a)

Haldun Balim, Yilun Du, Na Li

Core Contributions

Uses a diffusion model as a proposal distribution over action sequences for model-based planning, not as the policy itself.
Samples candidate sequences from the prior and refines them with learned dynamics, reward, and value models, avoiding high-dimensional optimization from scratch.
Matches or exceeds model-free, diffusion-policy, and model-based baselines on continuous control and long-horizon manipulation.
Stays stable in regimes where strong baselines degrade, offering a cleaner way to combine generative models with planning.

Show abstract

Online decision-making faces a tension between efficient inference and long-horizon reasoning. Value-based methods rely on short-horizon bootstrapping that limits the propagation of long-term information, while trajectory optimization methods plan explicitly but must solve a high-dimensional optimization problem at every decision step. We propose Generative Trajectory Planning (GTP), a model-based framework that addresses this tension by learning a generative model as a proposal distribution over action sequences. Instead of optimizing sequences from scratch, GTP samples candidate sequences from a diffusion prior and refines them using learned dynamics, reward, and value models, enabling efficient trajectory-level reasoning. Across continuous control and long-horizon manipulation tasks, GTP matches or exceeds state-of-the-art model-free, diffusion-policy, and model-based baselines, and remains stable in regimes where strong baselines degrade. Our results suggest that generative models are best understood not as policies, but as structured proposal distributions for planning, offering a new perspective on how to integrate generative modeling with model-based decision-making.

#28

h=n/a

Grid or Nodes? Probing Object Memory in World Models Under Partial Observability

📅 2026-10-13cs.ROcs.CVcs.LG👤 Justin McMillen (h=n/a)

Justin McMillen, Yasin Yilmaz

Core Contributions

Compares a learned grid world model (FloWM) with an oracle-fed Kalman tracker for remembering objects that leave the camera view.
Shows that probe choice alone moves FloWM's out-of-view recall from 66% to 89%, a caution for world-model memory evaluations.
The tracker matches FloWM out of view using 14 KB vs 5.6 MB, and keeps 93% on diagonal motion where FloWM falls to 11% even after fine-tuning.
Argues for object-centric node memory, while noting a learned version must still replace the oracle and handle noise, long gaps, and bounces.

Show abstract

A robot's world model must remember objects that leave the camera's view. FloWM, a learned world model, keeps them in a spatial grid of features that it moves with five built-in velocities. A node memory instead keeps each object's position and velocity as numbers. We compare them in FloWM's own block world, with an oracle-fed Kalman tracker as the node memory. How the grid is read changes its score: a single-cell probe finds 66% of out-of-view blocks, while FloWM's own probe 89%. FloWM also scores 8 points higher in view although no frame is shown, which we attribute to its write-back of its own predictions. Out of view, the tracker reaches FloWM's range in 14KB against 5.6MB, with no probe. On diagonal motion, outside its five velocities, FloWM finds 11% of blocks even after fine-tuning, while the unchanged tracker keeps 93%. Faster blocks defeat both until one more allowed speed takes the tracker to 94%, where FloWM's built-in shifts would need 25 copies of its grid. A learned node memory must still replace the oracles and handle velocity under noise, long gaps, and the bounce.

Self-Improving & Agentic Robotics

h=n/a

ChunkAct: Action Chunking for Agentic Robotics

📅 2026-10-13cs.ROcs.AIcs.CLcs.CV👤 Bingxin Xu (h=n/a)

Bingxin Xu, Wenrui Bao, Xinxin Liu, Jiawei Zhou, Yuzhang Shang

Core Contributions

Observes that once an LLM agent picks a sub-goal, the following primitives are routine, so reasoning over the full history at every primitive is wasted compute.
Introduces action chunking for agentic robotics: one LLM call launches a chain of primitives that runs autonomously and returns control only on failure.
Offline compilation of memory into chunks makes test-time execution memory-free, so LLM cost scales with decisions rather than primitives.
On LIBERO-Pro: +5.0 pts success with 2.9× fewer LLM calls and 4.4× fewer input tokens than the prior agentic SoTA.

Show abstract

Agentic robotics places a large language model (LLM) agent in the robot control loop, where it composes perception, motion, and vision-language-action (VLA) tools under the guidance of a memory and thereby generalizes to new tasks without retraining. This generality comes at a steep price in efficiency. Every primitive requires a round of LLM reasoning over the full interaction history, so the cost of an episode grows with the number of primitives. We find this reasoning largely redundant: once a sub-goal is chosen, its primitives follow a routine already prescribed by memory. We present ChunkAct, which introduces action chunking to agentic robotics. Online, a single call launches a chunk of chained primitives, which executes autonomously and returns control only upon failure. Offline, ChunkAct compiles the memory into these chunks, so execution is memory-free at test time and the planner is reserved for decisions that require reasoning. LLM computation therefore scales with the number of decisions rather than the number of primitives. On LIBERO-Pro, ChunkAct improves task success by 5.0 points over the state of the art in agentic robotics while requiring 2.9x fewer LLM calls, 4.4x fewer input tokens, and 2.3x fewer output tokens.

h=n/a

EmbodiedEvo: Self-improving Coding Agents for Physical Intelligence

📅 2026-10-13cs.RO👤 Wenyi Li (h=n/a)

Wenyi Li, Yizhe Chi, Youjie Zheng, Chunyu Liu, Xiaoqiu Wang

Core Contributions

Treats code as both the policy and the agent's workspace, extending SWE-agent-style inspect/edit/execute loops to physical tasks.
The agent evolves reusable Skills and Lessons on a single Playground scene, then freezes this harness for unseen scenes.
Sets a new best on RoboDojo-Sim (score 44.48, 37.8% success); Skills/Lessons match raw-history performance at 53% lower execution cost.
Experience transfers down the model scale: lessons from a stronger model raise weaker executors' success by 5.5–8.9 pts.

Show abstract

Bringing general-purpose agents into the physical world calls for an executable representation of physical intelligence. Compared with vision-language-action (VLA) models, which map observations and language instructions directly to actions, code makes control logic explicit and reusable. Code can serve not only as a policy representation but also as the workspace in which the agent solves a task. Rather than following a prescribed workflow of planning, execution, and verification, the agent can autonomously inspect the state, revise and execute code, and use feedback to choose the next step. The success of this SWE-style loop in digital environments motivates its extension to physical tasks. Yet physical execution remains prone to recurring failures. To retain and refine experience for later scenes, we introduce EmbodiedEvo, where a coding agent evolves reusable Skills and Lessons through feedback on a single Playground scene, then freezes the resulting harness for diverse new scenes. On RoboDojo-Sim, EmbodiedEvo achieves an average score of 44.48 and a success rate of 37.82\%, surpassing all officially verified results. Paired tests show the largest gains on memory-dependent tasks and successful reuse of coordinated procedures, while fine contact control remains difficult. Experience from stronger models also improves weaker executors: Astra's experience raises Sol and Luna's success rates by 8.86 and 5.50 percentage points over no experience. Compared with raw interaction history, Skills and Lessons achieve comparable task performance at 53.0\% lower execution cost.

#17

h=n/a

SEAR: Self-Evolving Agentic Robotics

📅 2026-10-13cs.ROcs.AIcs.CL👤 Xinyi Wang (h=n/a)

Xinyi Wang, Bangzheng Li, Hanyu Wang, Lin Zhang, Andrea Yaoyun Cui

Core Contributions

Asks whether self-evolving robot agents truly learn or just get more attempts, adding controls for retries, task familiarity, privileged state, budget, and luck.
Covers three modes (code-as-policy, agent-as-policy, auto-research VLA fine-tuning) across 7 frontier LLMs, 330 sim and 5 real tasks, and over four years of agent runtime.
Learning curves depend on budget and task difficulty: on easy tasks re-rolling short attempts does as well as evolving.
Generic control code transfers across tasks and simulators while task-specific solutions do not; a text-only model keeps pace with a comparable multimodal one.

Show abstract

Frontier language-model agents can now improve robots on their own: they write control code, test it, and keep what works as skills and memories for the next task. We introduce SEAR (Self-Evolving Agentic Robotics), a framework and benchmark built around two questions. Is the agent really learning, or just trying more often? And does what it learns still help on new tasks, in a new simulator, and on a real robot? The framework runs an agent in three modes: it writes code that becomes the policy (code-as-policy), acts live in the control loop (agent-as-policy), or works as a researcher that fine-tunes a vision-language-action model (auto-research). The benchmark adds controls that separate retained experience from repeated attempts, familiar tasks, privileged simulator state, larger budgets, and luck. Across seven frontier language models, 330 simulated and five physical tasks, and over four years of agent runtime, we highlight three findings. First, the learning curve of a robotic agent depends both on the overall time budget and the task difficulty relative to model capability. Harder tasks make longer sessions worthwhile; on easier tasks, re-rolling short attempts does just as well. Second, control code and shared routines transfer across tasks and simulators, while task-specific solutions do not. On a real robot, we observe a performance gap between agent-designed practice and novel human-designed scenes. Third, different models evolve more effectively in different ways. Some do best acting live, others writing code first. And a text-only model keeps pace with a comparable multimodal one: perception helps with error assessment, but agentic reasoning and tool use can matter more. We release the benchmark and codebase to support future research.

#12

h=n/a

Who Improves the Robot? Toward Physical Recursive Self-Improvement

📅 2026-10-13cs.AIcs.LGcs.RO👤 Hai Zhou (h=n/a)

Hai Zhou, Fan Yang, Zeyi Chen, Lejun Zhang, Chenyang Fu

Core Contributions

Defines Physical Recursive Self-Improvement: physical experience updates both a robot's capabilities and its improvement procedure.
Proposes an L0–L5 ladder separating capability vs meta-level updates and within- vs across-task gains, reserving L5 for independently verified recursive gain.
Concludes that no surveyed work yet demonstrates Physical RSI as defined, a useful reality check amid self-improvement claims.
Sets research priorities: reliable feedback, causal attribution, safety preservation, and benchmarks.

Show abstract

Robots are increasingly autonomous in task execution, yet their improvement remains largely human-driven. This study investigates Physical Recursive Self-Improvement (Physical RSI), in which physical experience guides lasting updates to both a robot's capabilities and its improvement procedure, making subsequent improvement more effective across tasks. Physical RSI faces challenges from noisy observations, continuous dynamics, costly trials, and irreversible consequences. We present an L0-L5 framework that differentiates capability-level from meta-level updates and within-task from across-task gains, reserving L5 for independently verified recursive gain. We study existing methods around harness adaptation and model training, alongside safety and control mechanisms that govern execution and successive updates. Our analysis reveals that, despite substantial progress in self-improvement, the surveyed literature does not yet demonstrate Physical RSI as defined by our framework. We identify research priorities in reliable feedback, causal attribution, safety preservation, and benchmarking, outlining a path toward robots that autonomously improve how they learn while ensuring safety. The GitHub repository for Physical RSI is available at https://github.com/SAIL-Research-Lab/Physical-RSI.

h=n/a

AutoAgent0: An Agentic Runtime for Safe Closed-Loop Driving

📅 2026-10-13cs.RO👤 Jiageng Liu (h=n/a)

Jiageng Liu, Seth Z. Zhao, Zewei Zhou, Alejandro Huerta, Aidan Bayer-Calvert

Core Contributions

Frames closed-loop driving safety as a test-time execution problem rather than a training problem.
Wraps existing end-to-end driving experts with an independent detection module, a rule-based trajectory verifier, and an agentic recovery planner.
When nominal trajectories fail verification, the runtime composes alternatives from action primitives and refines them using verifier feedback.
Improves closed-loop robustness on Fail2Drive and NavSafe-∞ across multiple experts without retraining any of them.

Show abstract

Safe autonomous driving requires more than accurate planning at a single frame: it demands reliable closed-loop execution in dynamic environments. Although end-to-end driving policies have advanced rapidly, the accumulated closed-loop errors, especially in unseen scenarios, can push vehicles into states from which the same policy struggles to recover. We address the challenge by treating closed-loop driving safety as a test-time execution problem. We introduce AutoAgent0, an agentic runtime and harness framework that augments existing driving experts with complementary detection skills, rule-based action verification, and feedback-driven recovery. To address inaccurate scene perception, an independent detection module constructs scene context using complementary perception skills, and a rule-based verifier checks proposed trajectories against this context before execution. When the nominal trajectories are unreliable, the orchestrator of the runtime activates Recovery Planning, where an agentic loop constructs alternative action proposals from action primitives and iteratively refines them using verification feedback. This design combines independent scene assessment with feasibility checks and online recovery, addressing both perception errors and the limited recovery capabilities of individual experts. Evaluations on Fail2Drive and NavSafe-$\infty$ demonstrate improved closed-loop robustness over single-expert execution in long-tail and safety-critical scenarios. These results demonstrate that runtime supervision can improve closed-loop robustness across multiple driving experts without retraining the underlying policies.

#18

h=n/a

EqC: Equilibrium Energy Control for Introspective Lifelong Robot Learning

📅 2026-10-13cs.RO👤 Haodong Yang (h=n/a)

Haodong Yang, Runhan Huang, Bowen Zhang, Iofai Lei, Zunzhe Zhang

Core Contributions

Gives generative policies an explicit energy landscape so they can tell when their own behavior is no longer well supported.
Action refinement is cast as equilibrium-seeking optimization; the geometry of refinement provides a competence signal.
Energy-Gated DAgger triggers expert takeover only on persistent energy inconsistency, collecting recovery data near the competence boundary.
Works with both Diffusion Transformer and flow-matching VLA policies, progressively expanding autonomy in real bimanual tasks.

Show abstract

Long-term robot autonomy depends on more than effective action generation: a policy must recognize when its behavior is no longer sufficiently supported and use assistance to improve. We introduce Equilibrium Energy Control (EqC), a framework for introspective lifelong robot learning built around an explicit conditional energy landscape. EqC casts generative action refinement as equilibrium-seeking optimization, while the geometry of the refinement process reveals whether the generated behavior remains consistent with the learned landscape. This competence signal enables Energy-Gated DAgger, where persistent energy-consistency violations trigger expert takeover and the resulting recovery trajectories are aggregated for subsequent policy updates. EqC integrates naturally into both Diffusion Transformer policies and flow-matching-based vision-language-action (VLA) policies. Across simulation and real-world bimanual manipulation, EqC matches or improves control performance and robustness while enabling timely intervention near the policy's competence boundary. A real-world study further shows how these interventions progressively expand the range of situations handled autonomously, providing a practical path from static generative controllers toward self-improving robot systems.

#30

h=n/a

ACCRUE: Continual Off-Policy Reinforcement Learning for Robot Foundation Models

📅 2026-10-13cs.ROcs.AIcs.LG👤 Changyeon Kim (h=n/a)

Changyeon Kim, Kimin Lee, Yashraj Narang, Yijie Guo

Core Contributions

Continual off-policy RL on top of robot foundation models that both learns new tasks efficiently and retains old ones.
Carries lightweight actor and critic networks across tasks, reusing value knowledge, while distilling the actor toward cached targets on a few retained episodes.
Fixed visual features keep the inherited learner and replay compatible even when the base policy is updated.
Near-zero forgetting on contact-rich assembly sequences and 31.0–68.5% less real-robot training time.

Show abstract

Reinforcement learning (RL) post-training improves robot foundation models (RFMs) beyond their demonstrations, but real-world deployment demands more: learning new tasks efficiently while retaining previously acquired behaviors. Existing continual RL methods on top of VLAs mitigate forgetting but rely on on-policy updates with limited forward transfer, whereas off-policy methods improve single-task sample efficiency without addressing learner persistence across tasks. We propose ACCRUE, a continual off-policy RL framework that carries lightweight actor and critic networks across tasks, allowing each stage to reuse behavioral and value knowledge acquired through prior RL. The inherited critic adapts only on current-task transitions and guides current-task actor learning, while prior behavior is preserved by distilling the actor toward cached targets on a small number of retained episodes. Fixed visual features keep the inherited learner and its replay compatible with base-policy updates. Across contact-rich assembly sequences in simulation and on real-robot hardware, ACCRUE accelerates subsequent-task learning with near-zero forgetting and significantly reduces real-robot training time by 31.0\% to 68.5\%.

Contact-Rich & Tactile Manipulation

h=n/a

Interaction Latents as Actions for Contact-Rich Manipulation

📅 2026-10-13cs.RO👤 Yuhang Zheng (h=n/a)

Yuhang Zheng, Yujie Zang, Yuchen Liu, Yupeng Zheng, Zhennan Jiang

Core Contributions

Makes actions themselves encode how contact and force evolve, instead of treating tactile signals only as inputs or auxiliary targets.
Learns decodable interaction latents via visually conditioned joint action-tactile reconstruction, used as flow-matching targets for VLAs, WAMs, or from-scratch policies.
The same latent space supports offline improvement via expert-consistency rewards and correction of low-reward predictions.
Improves real-world success by 21.0–36.5 pts over tactile-conditioned direct-action baselines across three policy families.

Show abstract

Contact-rich manipulation requires robots to coordinate not only where to move, but also how contact and force should evolve throughout an interaction. Yet most robot policies represent actions primarily as motion trajectories, using tactile signals as inputs or auxiliary prediction targets without explicitly encoding contact dynamics in the action representation. We propose ILA (Interaction Latents as Actions), which uses decodable interaction latents that jointly capture the evolution of motion, contact, and force application as policy action units, providing a shared representation for action generation, evaluation, and offline improvement. We learn these latents through visually conditioned joint action-tactile reconstruction with a time-conditioned implicit decoder. Using them as flow-matching generation targets supports post-training of pretrained vision-language-action models (VLAs) and world action models (WAMs), as well as training flow-matching policies from scratch. A shared decoder maps generated latents to executable actions. For offline improvement, we learn a tactile-conditioned expert reference model from interaction latents encoded from expert demonstrations. The resulting expert-consistency rewards guide weighted learning from successful rollouts, while observation-compatible expert references guide correction of low-reward predictions from failed rollouts. Across three policy families, ILA improves success rates averaged over four real-world tasks by 21.0-36.5 percentage points relative to tactile-conditioned direct-action baselines. Ablations highlight the benefit of encoding contact evolution in action targets beyond tactile conditioning, while the same latent space supports expert-guided offline improvement. Project website: https://mrsecant.github.io/ILA/.

h=n/a

ForceWAM: Decoupling Future and Response Learning from Demo and Play Data for Contact-Rich Manipulation

📅 2026-10-13cs.RO👤 Zijun Xu (h=n/a)

Zijun Xu, Wendi Chen, Xinkai Wang, Jiachen Zou, Hongjin Chen

Core Contributions

Separates two kinds of knowledge: what a task outcome should look like (from demos) and how contact responds to local actions (from task-agnostic play).
A world model predicts future video and 6-axis force/torque jointly; an inverse dynamics model trained on both demos and play converts those futures into actions.
Between world-model updates, the IDM reacts to fresh wrench feedback, giving reactive force control.
85–95% success on sliding, box flipping, and ring placement; adding play data adds 15–45 pts.

Show abstract

Expert demonstrations show how to complete contact-rich tasks, but provide limited coverage of how interaction forces respond to different actions. Task-agnostic play provides broader experience of contact dynamics without specifying task goals. To fully leverage these two types of data, we introduce ForceWAM, a force world action model that decouples knowledge of task-compatible interaction outcomes from knowledge of how local actions change contact. A world model learns from successful demonstrations to jointly predict future video and six-axis force/torque trajectories. Together, these futures describe the expected evolution of motion and contact during the task. A separate inverse dynamics model (IDM) learns from both task and play, mapping current observations and recorded visual-wrench futures to their associated robot actions. Play data thus provide knowledge of force responses for goal-directed execution. At deployment, the IDM reactively translates generated futures into actions, reusing them with fresh wrench feedback between world-model updates. Across real-robot force-regulated sliding, box flipping with and without disturbances, and ring placement, ForceWAM achieves $85$-$95\%$ mean success, surpassing world-modeling and reactive baselines. Our experiments further show that adding task-agnostic play data improves success by $15$-$45$ percentage points, demonstrating its value for learning contact-rich execution. Project page and code are available at https://force-wam.github.io/.

#10

h=n/a

Contact Trajectory Prompting: In-Context Transfer of Contact-Rich Behaviors from a Single Demonstration

📅 2026-10-13cs.RO👤 Xian Nie (h=n/a)

Xian Nie, Yujie Zang, Yuhang Zheng, Yupeng Zheng, Songen Gu

Core Contributions

Shows that motion alone cannot specify contact-rich behaviors that look the same but need different force intensity or direction.
Encodes a single demonstration as an ordered contact-trajectory prompt; the policy retrieves relevant parts using live sensor feedback.
Reaches 78.7% success on unseen objects across wiping, rotation, and insertion, +13.0 pts over the strongest baseline.
Swapping only the reference demo changes wiping force level or cap-rotation direction, extending in-context learning to physical interaction.

Show abstract

In-context robot learning enables robots to acquire new behaviors from demonstrations without test-time parameter updates. However, visual and kinematic specifications alone may not distinguish contact-rich behaviors that share similar motions but require different interaction intensities or patterns. We propose Contact Trajectory Prompting (CTP), a framework for one-shot transfer of contact-rich manipulation behaviors. CTP encodes motion and contact information from a single demonstration into an ordered contact trajectory prompt that captures their temporal evolution and specifies the desired physical interaction behavior. During execution, the policy uses online sensory feedback to retrieve interaction-relevant information from the prompt, conditioning closed-loop action generation on the demonstrated contact behavior. Across real-world wiping, rotation, and insertion tasks on objects excluded from training, CTP achieves 78.7% overall success, outperforming the strongest evaluated baseline by 13.0 percentage points. Changing only the reference demonstration enables CTP to follow contact specifications for both intensity and direction, including an intermediate wiping force level and clockwise or counterclockwise cap rotation. These results demonstrate that contact trajectories serve as effective behavioral specifications, extending in-context robot learning beyond motion imitation to physical interaction transfer. Project website: https://ctp-robotics.github.io/project_page.

#13

h=n/a

RoboSimStudio: An Interactive Toolkit for Rapid Real2Sim of Robot Manipulation Tasks

📅 2026-10-13cs.ROcs.AIcs.GRcs.LG👤 Kaiyuan Chen (h=n/a)

Kaiyuan Chen, Shuangyu Xie, Uksang Yoo, Xinyue Zhu, Cael Magner

Core Contributions

Makes the hidden choices in physics simulation (solver, contact model, stiffness, friction) explicit and editable for real2sim tuning.
Designed for both humans and coding agents to iterate on simulation based on observed behavior.
Built on the open Newton simulator and demonstrated with visual evaluation, force feedback, and agentic Real2Sim2Real loops.
Lowers the cost of getting contact-accurate simulation for manipulation, a frequent bottleneck for sim-trained policies.

Show abstract

It is difficult to accurately model the physics of the interaction between a robot and the objects it manipulates, including contact mechanics, coupled contact dynamics, deformations, stiffness and friction. We present RoboSimStudio (RSS), an interactive toolkit that makes choices of solver, contact model, and physical parameters explicit and editable, allowing both humans and coding agents to iteratively refine simulations based on observed physical behavior. We implement RSS with the open Newton physics simulation system and describe experiments with human-guided real2sim using (1) visual evaluation, (2) force-feedback and (3) agentic Real2Sim2Real. Code, video and additional information can be found on the project website.

Human Data & VLA Training Recipes

h=n/a

EgoExplorer: A Vision-Motion-Touch Observatory for Human Loco-Manipulation In Situ

📅 2026-10-13cs.ROcs.CV👤 Yunsong Zhou (h=n/a)

Yunsong Zhou, Xuekun Jiang, Haiqi Shen, Hangxu Liu, Chaojian Shi

Core Contributions

A 100-hour wearable dataset that aligns egocentric video, whole-body motion, finger articulation, and touch for professional loco-manipulation in real settings.
Calibrate-Complete-Constrain pipeline recovers full-body motion from sparse sensors by separating mounting offsets from articulation and adding temporal and geometric constraints.
Cuts body-point error by 71.9% versus the strongest baseline, with better hand stability and contact estimation.
Fills a gap for humanoid learning, where existing datasets capture only part of the locomotion-posture-interaction coupling.

Show abstract

Learning humanoid loco-manipulation from human demonstrations rests on observing how locomotion, posture, and object interaction evolve together as tasks unfold. Scaling collection across diverse environments, in turn, favors lightweight, portable capture systems. Yet, existing datasets provide only partial views of this coordination. Thus, we introduce EgoExplorer, a 100-hour vision-motion-touch observatory of human loco-manipulation built with a wearable system. With an emphasis on professional activities, its continuous recordings align egocentric video with reconstructed whole-body motion, fine-grained finger articulation, and tactile observations, accompanied by step-level descriptions. To achieve this coverage with sparse body sensing, we develop the Calibrate-Complete-Constrain pipeline. Concretely, reusable calibration fixes participant geometry and sensor attachments before motion inference to disentangle mounting offsets from articulation. A temporal prior then completes unobserved joint motion, while layered constraints promote measurement agreement, temporal continuity, and geometric consistency, properties that motion plausibility alone does not guarantee. Against tracker-fitted references, the method reduces body-point error by 71.9% relative to the strongest baseline; matched evaluations also show gains in hand stability and contact estimation. Preliminary studies explore multimodal generation and conditioning alongside humanoid retargeting with whole-body control in simulation.

#15

h=n/a

EgoMatrix: Physically Faithful Dexterous Robot Demonstrations from Egocentric Human Videos

📅 2026-10-13cs.RO👤 Ziheng Wang (h=n/a)

Ziheng Wang, Yanming Shao, Ting Mao, Kaihan Chen, Yiqun Wang

Core Contributions

Turns monocular egocentric human videos into bimanual dexterous robot trajectories that replay on real robots, plus paired rendered robot observation videos.
Prioritizes relative hand-object geometry and contact over metric scene reconstruction, then uses retargeting and RL to reproduce tasks in simulation.
Reaches 54.8% end-to-end simulated success vs 14.5% for the strongest baseline, with SoTA reconstruction on most public-benchmark metrics.
Re-renders trajectories in diverse backgrounds to expand data while keeping physical consistency.

Show abstract

Egocentric human videos offer a scalable alternative to robot teleoperation for collecting dexterous manipulation data. However, two challenges limit their use as reliable training data: accurate hand-object interaction (HOI) reconstruction from monocular RGB and high-quality robot demonstrations, especially visual observations. We present EgoMatrix, an end-to-end data engine that converts monocular egocentric human videos into bimanual dexterous trajectories and paired robot observation videos, with the trajectories directly replayable on real robots. It reconstructs HOI with emphasis on relative geometry and contact rather than exact metric scene recovery, then uses retargeting and reinforcement learning (RL) to reproduce the demonstrated tasks in simulation. We jointly render robot hands and objects from these simulated trajectories to produce physically faithful observations synchronized with state and control records. Rendering the same trajectories in diverse, task-compatible backgrounds expands paired training data while preserving robot-object motion and contact. Across TACO, OakInk2, and self-collected data, EgoMatrix achieves 54.84% end-to-end success in simulation versus 14.52% for the strongest baseline. Reconstruction reaches state-of-the-art performance on most metrics of the two public benchmarks. Qualitative results show more faithful hand-object interactions than prior methods, while feature-space analysis shows closer alignment with real-robot observations than GPT-6 Astra. Our project page is available at https://spikew726.github.io/EgoMatrix/.

#20

h=n/a

Steertuning Policies from a Few Human Videos

📅 2026-10-13cs.ROcs.AI👤 Billy Yan (h=n/a)

Billy Yan, Isaac Tu, Kelly Lee, Blake Chang, Kenny Ma

Core Contributions

Learns new tasks in new environments from 5 human demonstrations in under 5 minutes end-to-end.
Pretrains a 400M diffusion policy on 100 hours of densely annotated egocentric human interaction data for background and object robustness.
Fine-tunes for only 2 minutes, then steers reverse diffusion toward the demo whose observation best matches the deployment scene.
Achieves 71% success across 12 tasks in 12 new environments, including real home kitchens.

Show abstract

Current recipes for robot learning require massive amounts of robot demonstrations to obtain robust and adaptive policies. In this work, we introduce Steertuning, a system that enables robots to learn new tasks in new environments from 5 human demonstrations in under 5 Steertuning consists of two parts. (1) We pretrain a 400M-parameter diffusion policy on our egocentric human dataset, which consists of 100 hours of densely annotated random interactions. This pretraining makes Steertuning robust to background differences and enables spatial and object generalization when deployed on new tasks. (2) Given 5 demonstrations of a new task, we finetune our model for only 2 minutes (2000 training updates) and steer the policy's actions during reverse sampling using the demonstration whose observation is closest to that of the deployment environment. Together with a carefully designed data processing pipeline for embodiment transfer, this enables Steertuning to achieve a 71% success rate across 12 tasks in 12 new environments, ranging from lab settings to kitchens in new homes. The entire process, from data collection for a new task to robot policy deployment, takes less than 5 minutes. Our full dataset, model weights, codebase, and robot videos will be made available at our website: https://steertuning.github.io.

h=n/a

What Matters in Quality-Aware VLA Pretraining: A Benchmark and Empirical Study

📅 2026-10-13cs.ROcs.AIcs.LG👤 Mingxuan Yan (h=n/a)

Mingxuan Yan, Litian Gong, Dihong Huang, Zhixu Li, Tianyu Zhang

Core Contributions

A controlled study of how demonstration quality should be used in VLA pretraining, matching data and compute across weighting, filtering, and conditioning.
Finds approach-aware episode-level quality conditioning beats plain BC, while filtering works better with chunk-level labels (with model-dependent reversals).
Fine-tuning with a constant 'good' tag after conditioned pretraining helps whether or not tags are used at inference.
AXIS-Bench enables relabelling quality on shared trajectories; simulation pretraining lifts real-world success from 9.6% to 31.7%.

Show abstract

How should demonstration quality be used during vision-language-action (VLA) pretraining to improve performance after target-task fine-tuning? We compare weighting, filtering, and conditioning across reward definitions and label granularities using 5,000 naturally suboptimal human demonstrations and 24 target tasks. Within each VLA, we match demonstrations and training budgets. Additional comparisons examine pretraining data size and the roles of quality tags during fine-tuning and inference. Across three VLAs, we find that approach-aware episode conditioning improves mean success over plain behavioral cloning; conditioning and filtering favor episode and chunk labels, respectively, on average, with model-dependent reversals; and constant-tag fine-tuning improves mean success after conditioned pretraining with either tagged or untagged inference. We introduce AXIS-Bench to support these controlled comparisons. Built on crowdsourced simulation demonstrations, it provides recorded object and contact states for recomputing quality labels on shared trajectories. Replayable trajectories support workspace-aligned rendering and user-defined real-world target tasks. In our real-world experiment, simulation pretraining raises success from 9.6\% to 31.7\%. Project page: https://axis-bench.github.io/.

Benchmarks & Policy Evaluation

#11

h=n/a

LIBERO-RIG: A Hardware-Setup Robustness Benchmark for Vision-Language-Action Models

📅 2026-10-13cs.ROcs.AI👤 Andrea Yaoyun Cui (h=n/a)

Andrea Yaoyun Cui, Hanyu Wang, Xinyi Wang, Bangzheng Li, Lin Zhang

Core Contributions

Isolates a neglected source of failure: the robot's own hardware and setup (camera remounts, calibration drift, control rate, gripper wear) with tasks held fixed.
11 perturbation axes × 5 severities = 210 conditions, 500 episodes each, with proper statistical testing across nine VLAs.
Pooled success falls from 0.932 to 0.767 at medium and 0.509 at highest severity.
Training on visual augmentations does not help with faults that leave images unchanged (e.g., actuation gain), pointing to a gap in current robustness recipes.

Show abstract

Robot policies are expected to generalize across deployment environments, including variations in the robot itself: cameras are bumped and remounted, calibrations drift, controllers run at different rates, and end-effector pads wear down. We present LIBERO-RIG, a benchmark that perturbs only the robot's hardware and setup while holding every LIBERO task fixed. The benchmark spans eleven perturbation axes, including camera mounting and optics, camera-to-arm calibration, actuation gain, control rate, proprioceptive noise, base pose, table height, and gripper pads. Each axis has five severity levels, yielding 210 conditions across the four LIBERO suites. We evaluate nine recent vision-language-action models using 500 episodes per condition and substantiate each finding with rigorous statistical tests. Pooled success falls from 0.932 to 0.767 at the middle severity and 0.509 at the highest severity. Performance degradation varies substantially across perturbation axes and action interfaces, and training on visual perturbations does not mitigate faults that leave camera images unchanged. We release the benchmark together with manifests specifying the fixed perturbation parameters for each episode.

#14

h=n/a

WorldOracle: Benchmarking Action-Conditioned World Models against Real-World Executions

📅 2026-10-13cs.ROcs.CV👤 Haoyi Duan (h=n/a)

Haoyi Duan, Liliang Chen, Nan Wang, Yang Zhang, Yuhang Lu

Core Contributions

First benchmark that scores action-conditioned world models against paired real-world executions given identical start frames and action sequences.
Four metrics cover physical state correctness, interaction spatial alignment, key-region fidelity, and multi-view action coherence, going beyond visual quality.
Evaluation episodes include both successes and failures, testing whether models predict the consequences of bad actions.
Evaluation of seven ACWMs reveals distinct failure modes, providing a reference for using world models as simulators or evaluators.

Show abstract

We introduce WorldOracle, a benchmark specifically designed to evaluate action-conditioned world models (ACWMs) against paired real-world executions. Given the same initial observation and low-level action sequence, WorldOracle asks whether a predicted rollout reproduces what happens in the real world, including physical consequences and task-state transitions. We evaluate models along four complementary dimensions: physical state correctness, interaction spatial alignment, key interaction area fidelity, and multi-view action coherence. To support this evaluation, we curate real-world training and evaluation suites that cover different interaction dynamics and embodiments. The evaluation suite contains long, multi-segment episodes with both successful and failed executions, enabling evaluation across varied action outcomes and behavior distributions. Experiments on seven representative ACWMs reveal distinct model strengths and failure modes. WorldOracle provides a systematic foundation for measuring and advancing action-conditioned world modeling for manipulation. We will release the datasets, evaluation code, and leaderboard.

#21

h=n/a

RoboFQE: Rollout-Free Evaluation of Visual Robot Policies from Offline Trajectories

📅 2026-10-13cs.RO👤 Yunfu Deng (h=n/a)

Yunfu Deng, Sangwoo Shin, Brahma S. Pavse, Josiah P. Hanna

Core Contributions

Evaluates visual robot policies from offline trajectories, without real, simulated, or world-model rollouts.
Combines fitted Q-evaluation with a pretrained vision foundation model and learned terminal-success labelling, avoiding behavior-policy density estimation.
Preserves policy rankings on long-horizon tasks where video world-model evaluators degrade from compounding errors.
Pretrained encoders matter most on long horizons; learned success labels remove the need for ground-truth rewards with small accuracy loss.

Show abstract

Evaluating robot policies before deployment typically requires producing new trajectories through real-world execution, simulators, or learned world models. These rollout-based approaches impose infrastructure requirements; simulator and world-model evaluators additionally face prediction errors that can compound over the evaluation horizon. In this paper, we study off-policy evaluation as an alternative to rollout-based robot-policy evaluation methods. Specifically, we introduce RoboFQE, an evaluator that estimates the discounted expected return of a target policy from offline robot trajectories without estimating a behavior-policy density, combining fitted Q-evaluation with a pretrained vision foundation model and learned terminal success annotation to bridge the gap between standard OPE benchmarks and continuous-control visual manipulation. Controlled experiments across simulated and real-world manipulation tasks show that RoboFQE produces accurate discounted-return estimates and preserves policy rankings even on long-horizon manipulation tasks where video world-model evaluators are degraded by autoregressive prediction errors. Ablation experiments show that pretrained vision encoders substantially outperform from-scratch alternatives, particularly on long-horizon tasks, and that learned terminal success annotation enables evaluation without ground-truth reward labels with only slight reduction of accuracy.

Perception, Multi-Agent & HRI

#23

h=n/a

RadarGSOcc: Doppler-Aware 3D Semantic Occupancy Prediction Using 4D Radar Tensor with Geometric-Semantic Feature Extraction

📅 2026-10-13cs.CVcs.AIcs.RO👤 Christofel Rio Goenawan (h=n/a)

Christofel Rio Goenawan, Dong-Hee Paek, Seung-Hyun Kong

Core Contributions

Processes raw 4D radar tensors for 3D semantic occupancy, separating geometric and semantic (including Doppler) feature paths.
Introduces K-Radar 3DSemOcc, the first large-scale radar-tensor occupancy benchmark.
25.5 weighted mIoU vs 21.9 for RadarOcc (+17% relative) with fewer parameters and less compute, outperforming camera-based methods.
Proposes an EBKL proxy metric for adverse weather where per-frame labels are unavailable, supporting radar as a reliable all-weather modality.

Show abstract

3D semantic occupancy prediction is crucial for autonomous driving, but camera-based methods degrade in low-light conditions and LiDAR-based methods are sensitive to adverse weather. 4D radar is more robust, yet its sparsity and the high cost of radar tensor processing limit its use. We propose Radar Geometric-Semantic Occupancy Prediction (RadarGSOcc), an efficient framework that directly processes 4D radar tensors by separately extracting geometric and semantic features, including Doppler information, with a deformable attention-based geometric refinement module as its key component. We also introduce the K-Radar 3DSemOcc benchmark, the first large-scale benchmark for 3D semantic occupancy prediction using 4D radar tensors, built on the K-Radar dataset with refined LiDAR-based annotations. RadarGSOcc predicts 15 classes within a 51.2m range and achieves 38.2% IoU, 7.6% mIoU, and 25.5% weighted mIoU, outperforming RadarOcc by 17% relative (25.5 vs. 21.9) in weighted mIoU while using fewer parameters and lower computation. It substantially improves truck prediction while trading off car accuracy, outperforms camera-based approaches, and narrows the gap with LiDAR-based methods. To assess robustness without per-frame ground truth in adverse weather, we further propose a novel proxy metric, Environment-Based KL Divergence (EBKL), under which 4D radar proves a reliable and efficient modality for 3D semantic occupancy prediction. The dataset and code will be publicly available.

#25

h=n/a

Perception Beyond the Visible: Incompleteness-Aware Scene Understanding for Robot Autonomy

📅 2026-10-13cs.ROcs.CV👤 Rohit Mohan (h=n/a)

Rohit Mohan

Core Contributions

A research vision organizing perception around three kinds of incompleteness: missing evidence (occlusion), missing knowledge (novel objects), and unreliable sensors.
Connects amodal perception, semantic occupancy forecasting, open-world understanding, label-efficient learning, and uncertainty-aware fusion under one framework.
Proposes belief-centric 4D scene representations that jointly model geometry, semantics, motion, and uncertainty.
Argues for active sensing and safe adaptation so robots reduce uncertainty through interaction.

Show abstract

Reliable robot autonomy requires reasoning about the world from partial and noisy observations. However, many perception systems model only visible scene regions, rely on fixed semantic categories, and assume reliable sensor measurements. These assumptions break under occlusion, novel object encounters, and sensor degradation. In this paper, I present a research vision for incompleteness-aware scene understanding. I organize this vision around three fundamental challenges: incomplete evidence, incomplete knowledge, and incomplete sensor reliability. I discuss my research contributions toward addressing these challenges through amodal perception, semantic occupancy forecasting, open-world scene understanding, label-efficient learning, and uncertainty-guided multi-modal fusion. Building on these contributions, I outline future research directions toward belief-centric 4D scene representations. These representations aim to jointly model geometry, semantics, motion, and uncertainty. I also discuss active sensing and safe adaptation strategies that allow robots to reduce uncertainty through interaction. The goal is to develop calibrated and actionable scene representations that support reliable autonomy in dynamic, open-world environments.

h=n/a

Collaborative Planning for UAV-UGV Teams under Operational Constraints via Masked Error-bounded Neuro-Symbolic Diffusion

📅 2026-10-13cs.RO👤 Zihao Deng (h=n/a)

Zihao Deng, Qianhuang Li, Maggie Wigness, John Rogers, Donghyun Kim

Core Contributions

Plans UAV-UGV teams where ground vehicles act as mobile chargers, under energy and road-network constraints.
Formulates planning as conditional masked diffusion and reweights action distributions with a logic-derived admissibility indicator, enforcing non-differentiable constraints at execution.
Replaces entropy-guided unmasking with error-bounded scheduling over the constraint-aware distribution, improving plan optimality.
Validated in simulation, on real-world maps, and on physical robot teams.

Show abstract

Collaborative planning under operational constraints is essential for heterogeneous robot teams in large-scale missions. Unmanned Aerial Vehicles (UAVs) provide rapid area coverage but are limited by onboard energy, while Unmanned Ground Vehicles (UGVs) can serve as mobile recharging stations yet are constrained by road-network traversability. Effective collaboration must enforce operational constraints during execution while enabling parallel, coordinated actions. Masked Diffusion Models (MDMs) have shown promise through token unmasking but have two key limitations: entropy-guided scheduling assumes distributional alignment and reliance on differentiable constraints for diffusion-time optimization. We propose Masked Error-bounded Neuro-Symbolic Diffusion (MEND), which formulates collaborative planning as a conditional masked generative process to generate context-aware action sequences for UAV-UGV teams. MEND introduces a neuro-symbolic diffusion paradigm that constrains conditional masked diffusion policies at execution by reweighting the learned action distribution with a logic-induced admissibility indicator, thereby enforcing compliance with operational constraints. MEND introduces an error-bounded inference scheduling at execution that constructs the unmasking schedule by selecting positions with minimal predictive entropy over the constraint-aware action distribution, enhancing collaborative planning optimality under operational constraints. Extensive experimental results in simulation, on real-world maps, and with physical robot teams demonstrate that MEND advances UAV-UGV collaborative planning under operational constraints.

#16

h=n/a

Optimizing Team Behavior via Extremum-seeking Control in Multi-agent Reinforcement Learning

📅 2026-10-13cs.MAcs.AIcs.ROeess.SY👤 Svar Rajankumar Patel (h=n/a)

Svar Rajankumar Patel, Kshitij Jerath

Core Contributions

Tackles the costly tuning of behavioral-diversity targets in cooperative MARL by adapting the target online.
Two timescales: Diversity Control regulates diversity to a setpoint, while an Extremum-Seeking Controller adjusts the setpoint from mean returns alone.
Automatically settles on low diversity when homogeneous policies suit the task and higher diversity when specialization is needed.
Matches or exceeds grid-searched best targets across six configurations, showing classical feedback control can replace hyperparameter search.

Show abstract

In cooperative Multi-Agent Reinforcement Learning (MARL), agents must learn to coordinate their actions to achieve shared objectives. A key indicator of effective coordination is behavioral diversity, i.e., the agents' potential to enact distinct yet complementary control policies given the same set of observations. Although behavioral diversity can enable role specialization and richer joint behaviors, its appropriate level is highly context- and task-dependent and often costly to tune. We propose Adaptive Diversity Control (ADiCo), a two-timescale framework that adapts behavioral diversity online during training. On a fast timescale, Diversity Control (DiCo) regulates system diversity toward a target value. On a slower timescale, an Extremum Seeking Controller (ESC) adjusts this target using only mean episodic returns, without requiring an explicit model or analytical gradient of the diversity-performance landscape. We evaluate ADiCo on six configurations across two cooperative MARL tasks, where it maintains task-appropriate diversity levels: low diversity when homogeneous policies are beneficial and higher diversity when heterogeneous policies are required. Through this adaptation, ADiCo matches or exceeds the mean performance of DiCo (best grid-searched diversity target) and Unconstrained DiCo (diversity allowed to emerge without target regulation). These results demonstrate that incorporating classical feedback control into MARL training enables effective coordination by providing a systematic alternative to task-specific manual selection of the diversity target.

#29

h=n/a

LLM2R: From Large Language Models to 3D Reassemblers

📅 2026-10-13cs.CVcs.AIcs.RO👤 Zhouhan Zhong (h=n/a)

Zhouhan Zhong, Qiuhong Shen, Xingyi Yang, Jinyi Xu, Xinchao Wang

Core Contributions

Recasts 3D part assembly as sequence modelling with a pretrained decoder-only LLM adapted by LoRA, predicting all part poses in one non-autoregressive pass.
Learnable summary tokens appended after all parts work around causal attention, letting each part's pose head see the whole assembly at constant token cost.
Leads PartNet and PartNeXt metrics and gives the lowest translation error and fastest inference on Breaking Bad.
The advantage grows with part count: +33.5 pts over GARF at 100 parts, and over 16 pts better zero-shot on unseen categories.

Show abstract

3D part assembly aims to reconstruct a complete shape from a set of unassembled parts by predicting a rigid SE(3) transformation for each part. Because the placement of a part is rarely determined by its own geometry, the task demands reasoning over the entire part set, and existing methods, which rely on pairwise relations or iterative generation and verification, scale poorly to objects with many parts. We present LLM2R, which casts part assembly as sequence modeling with a pretrained decoder-only large language model (LLM): a Point-BERT encoder and a Q-Former compress each part into a block of geometric tokens, the blocks of all parts are concatenated, and a causal LLM adapted with Low-Rank Adaptation (LoRA) processes the sequence in one forward pass. Causal attention, however, lets the hidden states of a part observe only the parts that precede it. LLM2R therefore appends a block of learnable summary tokens after the last part: these tokens follow every part, so their states aggregate the whole assembly at a constant token cost, and a part-aware pose head fuses each part's states with them before predicting its rotation and translation. One architecture thus handles semantic and geometric assembly and predicts all poses in a single non-autoregressive pass. LLM2R achieves the highest part accuracy and success rate on all three PartNet categories, the best rotation error, translation error, and part accuracy on PartNeXt, and the lowest translation error and fastest inference on Breaking Bad, where only a fracture-specialized method is more accurate. Its advantage grows with the number of parts: at 100 parts it exceeds GARF by 33.5 points in part accuracy on Breaking Bad and Assembler by 29.1 points on PartNet, and it transfers zero-shot to unseen categories with more than 16 points higher part accuracy than the strongest of three baselines.

#26

h=n/a

Known-Voice Cloning and the Limits of Personality Transfer for Character Design

📅 2026-10-13cs.CLcs.AIcs.HCcs.RO👤 Johanna Magdalena Kuch (h=n/a)

Johanna Magdalena Kuch, Maud van Lier, Daksitha Senel Withanage Don, Patrick Gebhard, Tanja Schneeberger

Core Contributions

Tests whether a cloned familiar voice transfers the speaker's personality impressions to a virtual character, a concern for manipulative social robots.
Two studies: a lecturer's cloned voice with their students, and online content creators with their communities.
Personality transfer was limited and selective; recognizable voices activated expectations but did not reconstruct the full person impression.
Perceived artificiality and voice-appearance fit dominated judgments, tempering fears that voice cloning alone makes characters seem like capable social actors.

Show abstract

Recent advances in zero-shot voice cloning systems make it possible to reproduce a recognizable voice from a short recording and attach it to arbitrary virtual characters. Prior work suggests that familiar vocal identity can carry over associated personality impressions to new targets such as robots or virtual characters. However, these works have mainly examined familiarity and personalization effects rather than directly testing whether a cloned familiar voice systematically transfers personality impressions to an embodied virtual character. Such transference raises the concern that cloned voices could be used to manipulate people into engaging with virtual characters as capable social actors. Across two studies, we investigated whether personality impressions transfer when a virtual character speaks with the cloned voice of a familiar person. Study 1 examined this question in a real-world university setting using the cloned voice of a lecturer familiar to the participating students. Study 2 examined the same question online using voices of social media content creators and members of their communities. Drawing on the concepts of identity activation, trait transfer, and person reconstruction, we show that evidence of personality transfer was limited in both studies. Observed effects were selective and did not generalize across traits or study contexts. In the online study in particular, recognizable voice identity cues activated expectations about the speaker but did not reconstruct the broader person impression. Qualitative responses further indicated that judgments were frequently shaped by perceived artificiality and by whether voice and appearance fit together. Our findings moderate common concerns that cloned voices straightforwardly manipulate people into perceiving artificial characters as capable social actors.