🤖 Robotics arXiv Digest

Monday, October 12, 2026

📄 30 papers 📂 6 research areas Generated by Claude

Semantic Scholar h-index lookup returned no matches for this batch; papers are listed in fetch order.

🔭 Research Landscape

The most provocative result in today's batch is #6: a frozen frontier VLM (GPT-6-Astra) used directly as the policy scores 92.1% on the memory-dependent RoboMME benchmark, beating the reported human-with-oracle-skills score (90.5%) and a hierarchical system that pairs the same model with a trained low-level policy (79.1%). It sits alongside a cluster of papers that push general-purpose language models deeper into the stack without fine-tuning them: D-World (#19) keeps the LLM frozen and instead makes the world model explicit and self-correcting as an executable digital twin; #21 shows an LLM coding agent can be steered from imperative defaults to declarative reactive behaviors with expert-written guidelines; ProAgent (#27) asks the model to decide when to act unprompted. The counterweight comes from the evaluation papers: the Habitat long-horizon benchmark (#8) finds simple ReAct beats VLM-TAMP and Tru-POMDP yet tops out at 45% success, and #30 finds that asking LLM agents to predict a partner's next action lowers success in 15 of 16 configurations. Language models are increasingly capable controllers, but explicit reasoning about other agents or uncertainty is still shallow.

A second theme is skepticism toward cheap proxies in imitation learning. The audit in #3 is a model of careful science: published smoothness metrics for demonstration curation (SAL, TED) mostly encode episode length, and curating by TED actually cuts downstream success by a third (0.240 vs 0.360). Two other papers locate the value of supervision in subtle places: #10 shows that human interventions arrive late and proposes learning a cost from where and when experts intervene, while FOMO-Dex (#18) shows that raw teleop commands are worse imitation targets than force-corrected ones. VAQ (#14) makes a related point about representation: action quantization should preserve state reachability (and therefore value), not reconstruction error. Together they argue that what you train on matters more than the policy class, a useful corrective to scale-only narratives.

Third, formal guarantees are being reshaped to fit real robot objectives. Four control papers extend safety filters beyond forward invariance: recurrence/liveness via backup reachability CBFs (#7), infinite-horizon reach-avoid certificates for stochastic hybrid contact systems (#9, collisions 68→0 in ball dribbling), joint-feasibility margins that anticipate when multiple HOCBF constraints will exhaust bounded acceleration (#25), and anytime-valid conformal target retention for multi-agent interception (#23). Meanwhile, generative policies are being steered at inference time rather than retrained: MOSAIC (#24) composes frozen diffusion priors under new constraints, and PAF (#11) adds local anchors to SE(3)-equivariant policies. The emerging direction is a stack where pretrained priors (diffusion models, VLMs) supply behavior and lightweight, verifiable layers supply constraint satisfaction.

Frontier Models & Agentic Embodied Reasoning

Using frozen LLMs/VLMs as policies, planners, programmers, and world-model editors, and stress-testing their reasoning.

#6 Can a Frontier VLM Serve as a Robot Policy for…
#19 D-World
#21 Declarative Reactive Robot Behaviors from Imperative…
#27 Proactive Physical AI through Continuous Observation and…
#17 Generative Flow on Symbolic Abstractions for Embodiment…
#8 Evaluating Long-Horizon Robot Planning under Uncertainty
#20 Interactive Navigation Beyond Reachability: Benchmark…
#30 Observe, Share, or Predict? Disentangling Partner…

Generative Policies & Action Representations

Latent-action policies, diffusion steering, equivariance, and action spaces designed for learning and search.

#26 UniLAP
#24 MOSAIC
#14 Value-Preserving Action Quantization
#11 Phase-Conditioned Affordance Fields for…
#4 MIRROR
#2 Act4R

Demonstration Quality & Contact-Rich Data

What makes demonstration data useful: curation audits, delayed interventions, force-aware teleop, and real2sim augmentation.

#3 Smoothness Ranks Skill, Not Success
#10 Improving Policy Learning from Delayed Interventions
#18 Force-Modulated Dexterity
#16 DexNinja

Safety-Certified & Optimal Control

Barrier functions, reach-avoid certificates, and NMPC with guarantees beyond simple forward invariance.

#7 Filtering for Liveness
#9 Reach-Avoid Certificates for Contact-based Stochastic…
#25 Joint-Feasibility Margin Feedback for Bounded-Input…
#22 Task-Space Motion-Force Nonlinear Model Predictive…
#23 Multi-Agent Deferred-Decision Trajectory Optimization…

Navigation, Perception & Mapping

Perception-aligned flight, humanoid corridors, dynamic semantic maps, physics-aware SLAM, and driving encoders.

#5 ClearNav
#15 LiDAR-based Humanoid Navigation in Dynamic, Non-Convex…
#12 ReForesight
#13 Physical SLAM
#29 ConDenseDrive

Aerospace & Air-Traffic Autonomy

Learning-based guidance for spacecraft rendezvous and generative modelling of multi-aircraft traffic.

#1 From Orbit Matching to Docking
#28 DiTSG

Frontier Models & Agentic Embodied Reasoning

h=n/a

Can a Frontier VLM Serve as a Robot Policy for Memory-Dependent Manipulation?

📅 2026-10-12cs.RO👤 Jack B. Jedlicki (h=n/a)

Jack B. Jedlicki, Amish Sethi, Heng Yang

Core Contributions

A frozen GPT-6-Astra, with no RoboMME training, outputs every end-effector waypoint and gripper command while keeping the full episode in context.
Reaches 92.1% on 800 test episodes, above the human-with-oracle-skills score (90.5%), the best leaderboard entry (60.8%), and a hierarchical variant using the same model plus a trained policy (79.1%).
The gap versus the hierarchical system suggests that handing control to a separately trained low-level policy can hurt when memory and timing matter.
Remaining 63 failures cluster in two tasks (premature completion, mistimed button presses), giving a concrete map of where frontier models still slip.

Show abstract

We use a frozen frontier vision-language model, GPT-6-Astra, directly as the robot policy on RoboMME, a benchmark of 16 memory-dependent manipulation tasks spanning temporal, spatial, object, and procedural memory. The model keeps the whole episode in context and outputs every end-effector waypoint and gripper command, with no RoboMME-specific training. On the 800 test episodes it succeeds in 92.1%, above the previously reported human score of 90.5%, which was obtained with an oracle skill interface, and far ahead of the best leaderboard entry (60.8%) and of a hierarchical system that pairs the same model with a RoboMME-trained policy (79.1%). It matches or exceeds the human score on 13 of the 16 tasks. Its 63 failures concentrate in two tasks and are mostly premature completions and mistimed or unintended button presses. These results blur the usual division between long-horizon planning, memory and end-effector control, and raise the question of how much of this stack a single general-purpose model can take over in embodied systems. Project website: https://embodied-astra.github.io. Code, prompts and all episode logs: https://github.com/jackbjed/embodied-astra-robomme.

#19

h=n/a

D-World: An Explicit World-Language-Action Model with Recursive Self-Improvement under Dual Control

📅 2026-10-12cs.ROcs.AI👤 Xuan Wen (h=n/a)

Xuan Wen, Baichao Wang

Core Contributions

Makes the robot's world model explicit: an LLM perception agent writes the scene as an executable digital twin that can be inspected and corrected.
After each action chunk, named physical parameters are revised against measurements and only accepted if replay residuals do not grow, a verifiable form of 'recursive self-improvement' with a frozen LLM.
Frames probing actions as dual control and shows that cheap local revision lowers the threshold at which exploration pays off.
On a dual-arm robot, a single revision cut a bread-depth residual from 22.4 mm to 1.5 mm, avoiding full scene reconstruction.

Show abstract

Frontier large language models (LLMs) now reason about space well enough to program robots, yet the world models behind robot policies are implicit, held in network weights or predicted video, and cannot be inspected or corrected directly. We present D-World, an explicit world-language-action model built on the Sense, Plan and Act decomposition. A perception agent writes the scene as an executable digital twin using generated vision and geometry code. A planning agent searches for actions in simulation and, after each executed action chunk, revises the twin's object states and named physical parameters against what the robot measured. A parameter revision is accepted only if replaying the latest chunk does not increase any measured residual beyond its tolerance, and every chunk is checked in the updated twin before it runs. Structural mismatches return to perception for reconstruction. We call this measurement-grounded cycle of correction and revision recursive self-improvement (RSI), in which the LLM stays frozen while the world it acts in improves. We model the choice of actions that both advance the task and expose model error as a dual control problem and show that the cost of incorporating evidence into the twin sets the threshold at which probing pays, so cheap local revision counteracts the turn-off of cautious controllers. Each episode runs a real-to-sim-to-real loop, and its recorded twins can be refitted and re-rendered offline into higher-fidelity environments for post-training. Because perception and planning communicate only through the twin and language, either can be replaced by a learned model, and since inference dominates the loop, the loop speeds up with faster models. On a dual-arm robot, towel and toaster tasks show local revisions, such as one reducing the bread's rendered-depth residual from 22.4 to 1.5 mm, that let execution continue without reconstruction.

#21

h=n/a

Declarative Reactive Robot Behaviors from Imperative Language Models

📅 2026-10-12cs.RO👤 Vincent Berenz (h=n/a)

Vincent Berenz, Tasha Kim, Bernhard Schölkopf

Core Contributions

Targets open-ended behaviors (patrolling, greeting) rather than terminating tasks, which suit declarative reactive programs that LLMs rarely see in training.
Shows that a reusable set of expert-written guidelines redirects an LLM coding agent from imperative habits to correct reactive Playful scripts.
The agent co-designs Python sensing/acting modules and the declarative behavior logic starting from a generic robot API.
Guidelines generalize across robots and behaviors, supporting both one-shot generation and conversational refinement.

Show abstract

Large Language Models (LLMs) are predominantly trained on imperative programming languages, and existing work on LLM-based robot programming reflects this: it focuses on goal-directed tasks expressed as plans. However, many robotic applications require not tasks but behaviors--open-ended, continuous patterns of sensory-motor engagement with no termination condition (e.g., greeting visitors, patrolling an area). Such behaviors are better served by declarative and reactive programming, on which LLMs have virtually no training. We investigate whether this gap can be bridged. Using Playful, a reactive scripting language, we show that an LLM-based coding agent can generate correct declarative and reactive programs for robotic behaviors. These programs are produced from under-specified natural language descriptions, given suitable context and a set of reusable programming guidelines. The guidelines encode design principles that redirect the agent from imperative defaults toward proper reactive patterns. Developed once by an expert, they generalize across robots and behaviors. Starting from only the generic robot operating system API, the agent jointly designs both the imperative component (Python modules for sensing, acting, and evaluating) and the declarative component (the reactive script that composes them into behavior logic). The declarative nature of reactive programming supports both incremental refinement through conversational turns and one-shot generation of complete behaviors.

#27

h=n/a

Proactive Physical AI through Continuous Observation and Timely Intervention

📅 2026-10-12cs.CVcs.AIcs.RO👤 Zhenhua Yang (h=n/a)

Zhenhua Yang, Jiafeng Liang, Xingyu Zhou, Yi Jiang, Xiayu Cao

Core Contributions

Defines Proactive Physical AI: the system must notice, without instruction, when a situation warrants action and act at the right moment.
Separates a streaming timing model (whether/when to intervene) from a response model (what to say or which device capability to invoke).
An automated pipeline converts ordinary videos into supervision for intervention timing, enabling scale.
Achieves more accurate timing with fewer false triggers than the backbone baseline, and works on a real robot.

Show abstract

Physical AI is commonly formulated around an explicitly specified task: given a goal or instruction, a system perceives, reasons, and acts. In many real-world settings, however, important tasks emerge from continuously changing world states before a person explicitly formulates them. We study this complementary problem as Proactive Physical AI, where an AI system continuously observes the physical world, identifies situations that warrant intervention, and initiates appropriate actions at the right time. We develop ProAgent, an end-to-end system built around two components: a Jev-style proactive timing model (dubbed StreamingJev) that determines whether and when intervention is needed, and a Response Model that generates responses for people or invokes available device capabilities for physical execution. To support learning at scale, we develop an automated data pipeline that converts videos into supervision for proactive timing. Experiments demonstrate strong proactive performance under both explicit objectives and standing roles, combining more accurate intervention timing with fewer false triggers than the source-backbone baseline. Real-robot evaluations further demonstrate ProAgent's ability to identify emerging needs and translate them into appropriate physical actions.

#17

h=n/a

Generative Flow on Symbolic Abstractions for Embodiment Capability Discovery

📅 2026-10-12cs.ROcs.AIcs.LGcs.SC👤 Youwei Yu (h=n/a)

Youwei Yu, Lantao Liu

Core Contributions

Generates training tasks grounded in what a specific robot can actually do, instead of LLM-sampled tasks that are only semantically plausible.
A GFlowNet builds symbolic task skeletons; each is compiled into a simulator and rewarded by low-level learning success, steering generation to the capability frontier.
Discovers diverse, executable tasks across heterogeneous robots, including non-obvious learnable behaviors.
Distilling discovered tasks into one language-conditioned policy improves unseen long-horizon tasks, offering an automatic-curriculum alternative to hand design.

Show abstract

Achieving complex, long-horizon robot behavior remains challenging because successful execution depends on skills grounded in a robot's physical affordances. Existing training-task distributions are often handcrafted or sampled from large language models (LLMs), relying on semantic plausibility rather than embodiment-specific feasibility. This mismatch can produce curricula that are too difficult, redundant, or blind to non-obvious but learnable behaviors. We propose Generative Flow on Symbolic Abstractions (FOSA), a framework for discovering embodiment-specific capabilities by learning a grounded task generator. FOSA parameterizes an abstract policy as a Generative Flow Network that constructs symbolic task skeletons. Each skeleton is compiled into a simulation environment, where low-level policy-learning performance provides the reward signal for the abstract policy. Across heterogeneous robot platforms, FOSA discovers diverse and executable tasks near the robot's empirical capability frontier. After distilling the discovered task policies into a single language-conditioned policy, FOSA improves performance on unseen complex and long-horizon tasks. The code and demonstration videos are available at https://www.youwei-yu.com/fosa.

h=n/a

Evaluating Long-Horizon Robot Planning under Uncertainty

📅 2026-10-12cs.RO👤 Harsh Muriki (h=n/a)

Harsh Muriki, Gayathri Aravindan, Yuchen Zheng, Sonia Chernova

Core Contributions

Provides a controlled benchmark (7 household tasks, 90 task-environment-memory combinations, 1,350 episodes in Habitat) that varies how wrong the robot's initial memory is.
The simplest planner, ReAct, wins (45.3% success, 71.7% completion) over VLM-TAMP and Tru-POMDP, questioning the payoff of heavier structured planners.
Counter-intuitively, accurate initial memory does not always produce the best results, indicating planners use prior knowledge inconsistently.
Rule-based trace analysis surfaces recurring placement, precondition, and redundant-action errors that aggregate success rates hide.

Show abstract

Household robots must successfully navigate im-perfect circumstances, both in their internal models and the external world. We introduce a benchmark of seven long-horizon household tasks in Habitat that vary initial robot memory and environment configurations. We evaluate adaptations of ReAct, VLM-TAMP, and Tru-POMDP within a shared execution framework across 90 tasks-environment-memory combinations, totaling 1,350 episodes. Our evaluation measures task success, partial completion, and action efficiency, along with rule-based analysis of execution traces. ReAct achieves the highest overall task success (45.3%) and completion (71.7%), while having accurate initial memory does not consistently yield the highest performance for every planner. We identify recurring placement errors, precondition errors, and redundant actions through trace analysis. These findings highlight differences in how the evaluated planners use initial knowledge and respond to uncertainty.

#20

h=n/a

Interactive Navigation Beyond Reachability: Benchmark and Agentic Framework

📅 2026-10-12cs.ROcs.AI👤 Delun Li (h=n/a)

Delun Li, Xueyao Wei, Rui Chen, Yifei Wang, Junzhi Yu

Core Contributions

Extends ObjectGoal navigation to cases where the agent must change the world (open doors, open containers) to reach or reveal the target.
MultINav-Bench (3K episodes) annotates whether interaction is required or unnecessary, and scores navigation success, interaction effectiveness, and cost separately.
MultINav-Agent maintains a state-conditioned interaction graph and uses a VLM to choose interactions by expected change in connectivity and visibility.
Consistently outperforms adapted baselines across channel, container, and mixed tasks.

Show abstract

Existing embodied navigation tasks commonly assume that traversability and object states remain fixed during an episode. Yet reaching a goal may require opening a door to access a room and then opening a container to reveal a hidden target. We formulate interactive navigation beyond reachability, unifying channel-induced changes in spatial connectivity, container-induced changes in target visibility, and their sequential dependencies. We introduce MultINav-Bench, a 3K-episode ObjectGoal benchmark in MolmoSpaces spanning channel, container, and mixed tasks, with interaction-necessity annotations covering required and unnecessary episodes. Its data generation pipeline automates task construction across multiple task types. The evaluation protocol separately measures navigation success, interaction effectiveness, and cost. We further propose MultINav-Agent, a three-stage agentic framework built on a persistent, state-conditioned interaction graph. Semantic interactive perception grounds entities and interaction states; a vision-language model selects task-relevant actions based on expected changes in connectivity and target observability; and post-interaction verification updates the graph to guide subsequent navigation, exploration, or interaction. Experiments on MultINav-Bench show that MultINav-Agent consistently outperforms adapted baselines across channel, container, and mixed tasks.

#30

h=n/a

Observe, Share, or Predict? Disentangling Partner Information in Embodied Multi-Agent Collaboration

📅 2026-10-12cs.AIcs.RO👤 Adrián Szlatincsán (h=n/a)

Adrián Szlatincsán, Filippo Ziliotto, Bruno Lepri, Luciano Serafini, Massimiliano Luca

Core Contributions

Isolates three forms of partner information for LLM agents without dialogue: observing the partner's state (OAS), sharing object knowledge (KE), and predicting the partner's next action (OANA).
OAS improves task quality and reduces wasted effort; KE mainly improves efficiency (up to 25% fewer steps).
Explicitly predicting the partner's next action lowers success in 15 of 16 settings, suggesting current LLM 'theory of mind' is shallow.
Coordination gains come from reacting to shared context rather than anticipation, with clear asymmetries between robot and simulated human roles.

Show abstract

LLMs are used as high-level controllers for embodied agents, but it remains unclear whether they support collaborative reasoning, such as anticipating a collaborator's actions in real-time teamwork without dialogue. We ask if observing the partner's state, exchanging object knowledge, and predicting the partner's next action, shape task outcomes and coordination when agents can share percepts but never intentions, plans, or task-level messages, and whether the two agents occupy symmetric roles. Using the Habitat-based PARTNR benchmark, we introduce a Decoupled Partial Observability regime that enforces strictly individual perception, and we augment decentralized zero-shot ReAct planners with Other Agent's State (OAS) observation, object Knowledge Exchange (KE) between co-located agents, and Other Agent's Next Action (OANA) prediction, evaluated on the full 2 x 2 x 2 grid with four planners on 1,000 validation tasks. OAS observation improves task quality, reducing extraneous effort for every planner, whereas KE primarily improves execution efficiency, cutting simulation steps by up to 25% with little change in success. The two are largely redundant for strong planners but complementary for the weakest, for which KE without the OAS is the worst configuration. OANA prediction lowers the success rate in 15 of 16 model x context pairs while raising explicit partner mentions to 55-100%, and its accuracy gains lie mainly in the coarse action type, partly because agents recognize the action the partner is already performing. The robot retrieves objects sooner and completes its tasks earlier than the simulated human. Shared context reduces total task time by shortening the remaining duration that the human works alone. Also, coordination benefits arise from responding to shared context, especially the OAS, rather than explicit anticipation, yielding shallow, planner-dependent partner modeling.

Generative Policies & Action Representations

#26

h=n/a

UniLAP: A Unified Latent Action Policy for Cross-Embodiment Manipulation

📅 2026-10-12cs.ROcs.CV👤 Lei Yao (h=n/a)

Lei Yao, Qiao Feng, Mutian Tong, Qihao Yun, Lap-Pui Chau

Core Contributions

A continuous latent-action policy that avoids the granularity bottleneck of discrete latents and does not need predicted future frames at inference.
Distills a latent-action-model teacher with a flow-matching objective, keeping multimodality while producing directly executable latents.
Label-free motion targets decouple latents from appearance changes, yielding an embodiment-agnostic manifold that adapts quickly to new platforms.
At 1.1B parameters, beats latent-action and VLA baselines on hand-pose forecasting, single-arm, humanoid, and real-world tasks.

Show abstract

Latent Action Models (LAMs) can capture manipulation priors from action-free videos, but discrete latents usually impose a granularity bottleneck, limiting their ability to represent fine-grained action. Recent continuous alternatives mitigate this, but they are typically used as proxies for world modeling and require a further planning step to produce executable actions. We thus introduce UniLAP, a unified continuous latent-action policy that models the latent space via flow matching and bypasses reliance on future frames during inference. By distilling a LAM teacher with a flow-adaptive objective, UniLAP inherits the teacher's action representation while retaining the multimodal distribution of feasible behaviors. To decouple latent representations from task-irrelevant appearance, we extract a label-free motion target to isolate relevant kinematic cues from spurious visual dynamics. With the established embodiment-agnostic latent manifold, UniLAP can be easily adapted to novel platforms. Pretrained on multi-source videos at a compact 1.1B parameter scale, UniLAP outperforms state-of-the-art latent-action and Vision-Language-Action (VLA) baselines across hand pose forecasting, single-arm and humanoid manipulation, and real-world deployment.

#24

h=n/a

MOSAIC: Mode- and Support-Aware Inference-Time Control of Robot Diffusion Models

📅 2026-10-12cs.RO👤 Sabrina Bodmer (h=n/a)

Sabrina Bodmer, Rahel Rickenbach, René Zurbrügg, Melanie N. Zeilinger, Colin Jones

Core Contributions

Gradient-free inference-time steering of pretrained diffusion policies that avoids both pushing samples off the learned support and collapsing modes.
Treats initial noise and every reverse-step perturbation as a joint input, adapting its distribution with importance weights computed on fully denoised outputs.
Mode-wise updates prevent averaging between modes into infeasible actions; adaptive penalties balance feasibility against staying in-distribution.
Can coordinate multiple independently trained diffusion models through shared costs, e.g., collaborative box manipulation without a joint model.

Show abstract

Generative policies such as diffusion models provide expressive data-driven priors for robot behavior, yet the situations a robot faces at execution time rarely match its training data and impose constraints and objectives the policy never learned. Existing inference-time guidance methods either rely on gradients that can push samples away from the learned prior, or on sampling procedures that can be inefficient, collapse modes, or average them into out-of-distribution predictions. We introduce MOSAIC, a mode- and support-aware framework for gradient-free inference-time control of robotic diffusion models. MOSAIC treats the initial noise and all stochastic reverse-step perturbations as a joint input to the pretrained denoising process and adapts a distribution over this input using importance weights computed from fully denoised outputs. Physical feasibility and consistency with the learned support are balanced using adaptive penalties, while mode-wise importance-weighted updates preserve modes and avoid destructive averaging. The same formulation can jointly steer multiple independently trained diffusion models through shared coupling costs, enabling coordinated behavior without requiring a joint model. We evaluate MOSAIC on navigation, dexterous grasping, motion planning with unseen geometry, and collaborative box manipulation with independently trained robot priors. Across these settings, MOSAIC adapts and composes frozen diffusion priors under new deployment constraints without retraining.

#14

h=n/a

Value-Preserving Action Quantization

📅 2026-10-12cs.ROcs.LG👤 Max Muchen Sun (h=n/a)

Max Muchen Sun, Jueun Kwon, Benjamin Riviere, Todd Murphey

Core Contributions

Proves that the value error from action quantization is tightly bounded by the reconstruction error of state reachability, not of the actions themselves.
VAQ therefore quantizes actions to preserve their effects on future states, making discrete search (MCTS) valid for continuous robot control.
VAQ-MCTS matches strong BC baselines and is markedly more data-efficient and robust to heterogeneous demonstrations, with zero-shot adaptation to new environments.
Solves all 30 real-world pushing trials using only 10 minutes of demonstrations.

Show abstract

Search-based policy learning, such as Monte Carlo tree search (MCTS), offers the flexibility to trade runtime computation for data efficiency and adaptation to novel environments. For robot learning, however, applying search requires discretizing the continuous action space and associated dynamics, typically through action quantization. While reconstruction-based quantization is sufficient for representing the continuous action space alone, policy learning additionally requires the quantized actions to preserve their effects on future states and values. We show that such requirement is not necessarily captured by action reconstruction error. Instead, we prove that the error in the value function resulting from action quantization is tightly bounded by the reconstruction error of state reachability under the quantized actions. Based on these results, we propose value-preserving action quantization (VAQ), which quantizes actions by preserving state reachability. We further combine VAQ with MCTS (VAQ-MCTS) as a value-preserving, search-based policy learning framework for behavioral cloning. Using a wide range of object pushing tasks as a controlled benchmark, we evaluate VAQ-MCTS against state-of-the-art behavioral cloning methods. VAQ-MCTS matches or exceeds their performance under standard settings, while substantially improving data efficiency and robustness to heterogeneous demonstrations, while enabling zero-shot adaptation to novel environments. Hardware experiments further demonstrate the effectiveness of VAQ-MCTS, completing all 30 real-world object pushing trials with only 10 minutes of demonstration data.

#11

h=n/a

Phase-Conditioned Affordance Fields for SE(3)-Equivariant Visuomotor Policies

📅 2026-10-12cs.ROcs.AI👤 Xinzhe Chen (h=n/a)

Xinzhe Chen, Liqi Huang, Sihua Ren, Haowen Sun, Mingyang Li

Core Contributions

Breaks the 'global rigid transform' assumption of SE(3)-equivariant policies, which fails when the task-relevant object changes across task phases.
Learns a phase-conditioned per-point affordance score; its weighted centroid becomes a moving canonical anchor replacing the fixed gripper frame.
Adds local inductive bias while keeping global equivariance, consistently improving MimicGen success under varied spatial randomization.
Maintains its lead on real multi-object tasks under lighting changes.

Show abstract

SE(3)-equivariance provides a useful geometric inductive bias for visuomotor policies, encoding that actions covary with rigid transformations of the observation. However, existing SE(3)-equivariant visuomotor policies build this bias on a global equivariance assumption that treats the entire observation as governed by a single rigid transformation, which breaks down in real-world manipulation tasks where the task-relevant region changes across different objects. We address this by learning a phase-conditioned affordance field (PAF) over the observed point cloud, a per-point score that the SE(3)-equivariant policy uses in two complementary ways. Its score-weighted centroid serves as a learned, phase-conditioned anchor that canonicalizes the observation and the action around the active region, replacing the fixed gripper-frame reference, and the score itself is appended as a per-point feature. This introduces a local geometric inductive bias, while preserving the policy's global SE(3) equivariance. We validate on multiple MimicGen tasks under varying spatial randomization, and PAF consistently improves the success rate of the underlying policy. On multi-object real-robot tasks, PAF outperforms baselines and keeps its lead under lighting variations.

h=n/a

MIRROR: Reflecting Hidden Symmetries from Offline Robot Interaction Data

📅 2026-10-12cs.RO👤 Shiqi Liu (h=n/a)

Shiqi Liu, Yihang Yao, Peide Huang, Mengdi Xu, Ding Zhao

Core Contributions

Learns how a known symmetry group acts jointly on pixel observations and actions from offline data, removing the need for hand-specified, viewpoint-dependent image transforms.
Adversarial tuple matching over transition tuples plus group-axiom regularization yields a 61% lower compatibility error than an adapted LaLiGAN.
Using the learned transforms for frame-averaged BC lifts mean Robosuite success from 80.5% to 85.1% (up to +9.7 pts per task).
Policies trained in simulation on heightmaps/segmentation masks transfer zero-shot to a real Kinova GEN3.

Show abstract

Symmetry-aware robot policy learning requires coupled observation-action transformations. For pixel observations, even simple physical symmetries can induce complex, viewpoint-dependent image transformations that are difficult to specify by hand. Existing symmetry-aware methods often rely on representations with predefined transformations, requiring preprocessing tailored to the sensing setup and task. We introduce MIRROR, a framework that learns how a known symmetry group operates jointly on observations and actions in partially observable Markov decision processes (POMDPs) from offline interaction data. MIRROR couples these transformations through adversarial tuple matching over one-step transition tuples, while explicit group-consistency regularization penalizes violations of the group-operation axioms. On four Robosuite tabletop manipulation tasks, MIRROR achieves the lowest aggregate error on three of four symmetry-quality metrics, including a 61% reduction in compatibility error (which measures consistency under transformation composition) relative to an adapted LaLiGAN baseline. Using the learned transformations in frame-averaged behavior cloning (BC) yields higher mean success rates than vanilla BC on all four tasks, increasing average success from 80.48% to 85.12%, with per-task gains up to 9.66 percentage points. We also repeat symmetry discovery and policy training in simulation with heightmap and segmentation-mask observations, then deploy the resulting policy zero-shot on a real Kinova GEN3 arm. Project page: https://pomdp-mirror.github.io.

h=n/a

Act4R: Action-conditioned 4D reconstruction for kinematics-free control

📅 2026-10-12cs.RO👤 Zhaoliang Zhang (h=n/a)

Zhaoliang Zhang, Chenhao Yu, Zhimin Shao, Xijun Liu, Cheng Peng

Core Contributions

Replaces explicit kinematic models with a learned action-to-3D-scene-flow mapping built on a 3D foundation model, so a new robot can be controlled without CAD, URDF, or joint sensors.
Spatial-Action Encoder plus a spatial alignment loss ties action signals to future point maps; inverse control is done by optimizing actions to match a desired flow.
Outperforms prior SoTA on scene-flow prediction and transfers to closed-loop inverse control in simulation across several embodiments.
Points toward 'kinematics-free' control of soft, cheap, or poorly modelled robots from camera observations alone.

Show abstract

Precise control of movement across various robotic embodiments is a longstanding challenge. Modeling complex robotic structures requires significant time, dedicated sensors, and manual labeling. Recent advancements in 4D representation learning offer an interesting alternative: can we leverage 4D features lifted from 2D observations to densely map a robot's spatial movement based on the action signal? To this end, we propose Act4R, a framework that builds on 3D foundation model to predict a robot's 3D scene flow with any given action. Act4R introduces a Spatial-Action Encoder, which encodes the current 2D robot state and its action signals into embeddings that represent future robot state in 3D. Then, both the current and future spatial embeddings are decoded into point maps to estimate action-conditioned scene flow. To effectively learn the mapping between action and future state, Act4R introduces spatial-action fusion in its transformer-based encoder, and a spatial alignment loss that guides the encoder for future flow prediction. After training on some samples of a given embodiment, Act4R can perform inverse control by optimizing the signal to achieve the desired scene flow. We evaluate Act4R on a public real-world dataset and synthetic data collected in simulator. Experiments on various robotic platforms show that Act4R outperforms the previous SoTA in scene flow prediction. Our method also performs better in inverse control, both on the action accuracy metric and in closed-loop control in simulation.

Demonstration Quality & Contact-Rich Data

h=n/a

Smoothness Ranks Skill, Not Success: An Audit of Published Trajectory-Smoothness Curation Metrics Under Episode-Length Controls

📅 2026-10-12cs.RO👤 Jaiveer Bassi (h=n/a)

Jaiveer Bassi

Core Contributions

A rigorous negative result: the Spectral Arc Length curation metric (AUROC 0.952) is matched by episode length alone (0.957) and drops to 0.628 once duration is controlled.
Curating demonstrations by the contact-aware TED metric cuts downstream BC success by about a third (0.240 vs 0.360, p=0.0001), worse than random selection.
Explains the failure mechanistically: successful manipulation involves high-deviation motion through contact, whereas failures look like smooth low-amplitude hovering.
Shows that curation conclusions flip between GMM-BC and diffusion policies, warning against policy-agnostic data-quality claims.

Show abstract

Trajectory smoothness has been proposed as a cheap, training-free signal for curating imitation-learning demonstrations, most recently by rinse, which introduces Spectral Arc Length (SAL) and a contact-aware Trajectory-Envelope Distance (TED). We test whether these published metrics survive episode-length controls on real robomimic data. On can/mh, SAL separates better from worse operators with AUROC 0.952, but episode length alone achieves 0.957, and inside a duration-matched band SAL falls to 0.628 while raw mean jerk, the baseline rinse argues against, rises to 0.896. The confound replicates on square/mh. On can/mg, where every episode is exactly 150 steps, SAL is at chance (0.454) and TED is inverted (0.376) against task success, while jerk reaches 0.706. Training GMM behavior-cloning policies on each metric's curated subset (10 seeds), curating by TED costs roughly a third of downstream success relative to random selection of the same size (0.240 vs. 0.360, p=0.0001, surviving Bonferroni correction), while SAL and jerk are indistinguishable from not curating. Under a diffusion policy TED still ranks worst, though its deficit is not resolvable at five seeds, and jerk becomes significantly better than both baselines, so the curation decision does not transfer unchanged across the two policy configurations we tested. Inspecting trajectories shows why TED inverts: successful manipulation requires sustained high-deviation motion through contact, while failure is low-amplitude hovering that a smoothed envelope follows closely.

#10

h=n/a

Improving Policy Learning from Delayed Interventions

📅 2026-10-12cs.RO👤 Xiru Zhu (h=n/a)

Xiru Zhu, David Meger, Gregory Dudek

Core Contributions

Identifies an overlooked flaw in interactive imitation learning: experts intervene late, so intervention data under-represents error-avoidance behavior.
Shows empirically that delayed interventions are not useful 'noise'; they are strictly less informative than timely ones.
LfDI learns a cost function from where/when interventions occur plus coarse delay information, then infers supervision for states with no expert labels.
Outperforms delay-unaware baselines in simulation, relevant to any human-in-the-loop data collection setup.

Show abstract

In this work, we explore the idea of expert intervention delays in human-supervised robotic systems for imitation learning system. Previous works in imitation learning have primarily examined the impact of suboptimal experts, where the expert policy makes mistakes, but assumed expert interventions occur immediately after errors are made. Although delayed expert intervention can be viewed as a form of noise that might improve policy robustness, our results show that learning from delayed expert interventions is overall less effective than learning from non-delayed expert interventions; delayed expert interventions are less effective at demonstrating error avoidance behaviors. To address this, we propose Learning from Delayed Intervention (LfDI), which learns a cost function based on where and when the expert intervenes, combined with limited information about the extent of intervention delays provided by the expert. This cost function is then used to infer additional supervision for states without expert data, guiding the policy to avoid states that would likely trigger interventions. We evaluate our method, Learning from Delayed Intervention (LfDI), on simulated environments and show that it outperforms other methods that do not account for intervention delays.

#18

h=n/a

Force-Modulated Dexterity: Unified Compliance for Contact-Rich Manipulation

📅 2026-10-12cs.ROcs.AI👤 Ying Yuan (h=n/a)

Ying Yuan, David Held, Jitendra Malik, Haozhi Qi

Core Contributions

Lets teleoperators get force-aware behavior without haptic feedback by applying force-driven corrections only along task-relevant directions.
Expresses commands in coordinates defined by task geometry and contact structure, so compliance is unified across very different tasks (card picking, cutting, screwdriving, nut-and-bolt).
Reduces excessive contact forces vs stiff control, and operators collect more successful demos per hour after brief practice.
Imitating the corrected actions (not raw operator commands) yields better policies, e.g., more rotation achieved in screwdriving.

Show abstract

Teleoperation is widely used to collect high-quality demonstrations for robot learning. Dexterous manipulation, however, requires coordinated control of motion and contact forces. Operators can specify motion, but struggle to reproduce the subtle force adjustments of human manipulation without feeling the robot's contacts. We introduce Force-Modulated Dexterity (FOMO-Dex), a unified compliance formulation for contact-rich manipulation. FOMO-Dex expresses nominal actions in coordinates defined by task geometry and contact structure. It applies force-driven motion corrections along selected directions while leaving the remaining command components unchanged. Across card picking, paper cutting, screwdriving, and nut-and-bolt assembly, FOMO-Dex reduces excessive contact forces compared with stiff control. A user study shows that operators collect more successful demonstration data per hour after brief practice. For policy learning, imitating the corrected actions improves card-picking success over imitating raw operator commands. Policies trained on FOMO-Dex demonstrations also achieve more rotation in nut-and-bolt manipulation and screwdriving than those trained on demonstrations collected with the stiff controller. Videos are shown on https://yingyuan0414.github.io/fomo-dex/ .

#16

h=n/a

DexNinja: Learning Robust Dexterous Cutting Policy with a Real-to-Sim-to-Real Data Engine

📅 2026-10-12cs.ROcs.CV👤 Haozhe Lou (h=n/a)

Haozhe Lou, Runyi Yang, Wanqi Zhong, Caiyun Liu, Yicheng Liu

Core Contributions

Tackles dexterous food cutting, where real data is destructive and sim-to-real gaps are severe due to topology change.
Real2sim2real pipeline: reconstruct objects from captures, randomize physically meaningful material parameters within category constraints, and synthesize episodes in a custom differentiable tactile-cutting simulator.
With tens of real demos plus hundreds of augmented episodes, success exceeds 60%, and augmentation adds over 20 pts over real-only training.
Policies generalize to unseen object shapes and sizes.

Show abstract

Cutting a piece of food looks effortless in human hands. For a robot, however, every cut is a high-contact interaction with a deformable, elasto-plastic object whose behavior changes with pose, contact mode, and cutting stage. Collecting real-world data is expensive and wasteful: each trial typically contaminates or damages the food, so obtaining scalable real world data for dexterous cutting is expensive and difficult. Simulation is an appealing alternative, but a large sim-to-real gap exists in both physics and perception. This gap is especially severe for topology-changing interactions like cutting, where small errors in contact, friction, or material and elasticity responses quickly compound. We propose \textsc{DexNinja}, a real2sim2real framework that turns a handful of real demonstrations into a dense and realistic training distribution. \textsc{DexNinja} (i) reconstructs object instances from real captures, (ii) randomizes physically meaningful parameters under category-level constraints, and (iii) produces augmented dexterous manipulation episodes using a custom differentiable simulator that couples robot dynamics, tactile contact, and deformable cutting. We evaluate \textsc{DexNinja} on a dexterous food-slicing task and show that (1) with tens of real demonstrations and hundreds of augmented simulator data, the success rate could reach more than 60\%. (2) the augmented data consistently improves success rates when only using real data with more than 20\% via sim-to-real, and (3) the resulting policies generalize to out-of-distribution objects with varying shapes and sizes.

Safety-Certified & Optimal Control

h=n/a

Filtering for Liveness: Recurrence via Backup Reachability Control Barrier Functions

📅 2026-10-12eess.SYcs.RO👤 David E. J. van Wijk (h=n/a)

David E. J. van Wijk, Pio Ong, Aaron D. Ames, Joel W. Burdick

Core Contributions

Introduces a safety-filter analogue for liveness: guarantees the system revisits a target set within a maximum time, rather than never leaving it.
Combines a backup reachability CBF (always able to return in time) with a timing trigger that only invokes a reaching controller when the nominal controller would miss the deadline.
Enables objectives that require temporary excursions (e.g., traversing between disjoint regions) that standard CBF invariance would forbid.
Demonstrated on drone racing with a recurring beacon-visibility requirement.

Show abstract

Liveness requires a dynamical system to repeatedly accomplish a desired objective over time. In this work, we consider a particular form of liveness known as recurrence, which requires the closed-loop trajectory to regularly revisit a prescribed set, with no more than a specified amount of time between visits. Inspired by the control barrier function (CBF) safety-filter paradigm, we develop a controller-filtering framework that makes adjustments to a nominal controller only when necessary to guarantee recurrence. Unlike conventional CBF methods, which enforce set forward invariance, our framework allows the trajectory to leave the set while guaranteeing its timely return. To achieve this, our framework combines two components: a backup reachability CBF that preserves the ability to return to the prescribed set within the allowable time, and a timing-based mechanism that invokes a reaching controller only when the nominal behavior would otherwise fail to meet the return-time requirement. The resulting framework guarantees recurrence while preserving the nominal behavior when possible, enabling control objectives that require temporary excursions from the set, such as traversal between disjoint regions. Our method is successfully utilized in a drone racing example with a recurrence constraint on beacon visibility.

h=n/a

Reach-Avoid Certificates for Contact-based Stochastic Hybrid Systems

📅 2026-10-12eess.SYcs.ROmath.PR👤 Tianyi Xia (h=n/a)

Tianyi Xia, Ryan K. Cosner

Core Contributions

First infinite-horizon reach-avoid certificates for stochastic hybrid systems where control acts only through the reset kernel at guard-triggered jumps (i.e., contact events).
Conditions only need to hold on a sublevel corridor, and a pointwise safety filter turns verification into control for arbitrary nominal controllers.
Proves non-Zeno execution and bounds expected settling time and number of resets; certificate search is an LP over any smooth dictionary.
Certified region covers 93.6% of the ground truth; in a dribbling-ball task collisions fall from 68 to 0 over 200 episodes.

Show abstract

This work studies certified reach-avoid control of stochastic hybrid systems in which a stochastic differential equation governs continuous flow and a controlled stochastic kernel governs guard-triggered resets. The input appears only in the reset kernel, where it determines the post-jump distribution. Certificates for such systems have so far assumed spontaneous or controller-chosen jump times, a finite horizon, or discrete-time dynamics; we certify, over the infinite horizon, that the target is reached before the unsafe set with at least a prescribed probability. First, we introduce reach-avoid contact certificates, whose flow and jump conditions are imposed only on a sublevel corridor, and prove that any such certificate with a compatible reset policy meets the specification; a pointwise safety filter extends this guarantee from verification to control by constraining arbitrary nominal controllers. Next, we prove that executions are non-Zeno and bound the expected settling time and reset count. We frame certificate search as a linear program over any twice-differentiable dictionary followed by independent verification. We illustrate our approach via comparisons where the certified region is a subset covering 93.6\% of the ground truth and where our certificate-based safety filter reaches the goal while reducing collisions from 68 to zero across 200 episodes of a dribbling ball problem.

#25

h=n/a

Joint-Feasibility Margin Feedback for Bounded-Input Second-Order Control Barrier Functions

📅 2026-10-12cs.RO👤 Siyuan Yang (h=n/a)

Siyuan Yang, Shaoming He

Core Contributions

Defines a signed joint-feasibility margin whose sign exactly tells whether multiple second-order CBF constraints still admit a common bounded input.
The margin has physical acceleration units and a dual form that identifies which constraints conflict.
A one-step lookahead over nine candidate commands keeps the margin positive, preventing the hard HOCBF-QP from becoming infeasible.
In 20 paired trials, it intervenes before infeasibility in critical tasks while leaving the nominal command untouched in easy ones.

Show abstract

Multiple second-order control barrier function constraints can exhaust their common control authority under bounded acceleration, even when the current state remains feasible. We define a signed local joint-feasibility margin whose sign exactly identifies whether the current barrier half-spaces intersect the admissible input set. For unit-scaled distance constraints, it has acceleration units and a support-function dual that identifies weighted constraint conflicts. We predict the margin one sample ahead for nine task-directed candidates, selecting either the least-deviating candidate that attains a desired margin or the candidate with the largest predicted margin before solving the same hard HOCBF quadratic program. A controlled one-step case shows that, at the state predicted under the hard-QP command, two constraints remain individually feasible but become jointly infeasible, whereas the margin-feedback controller retains a positive next margin. Separately, across 20 paired initial conditions, the margin-feedback controller retains the hard-QP command in a low-pressure task and acts at positive margin in a critical task before the hard-QP problem becomes infeasible; its trajectories remain feasible at that time and complete the task. Source code is publicly available. https://github.com/xirhxq/cbf/tree/robio2026-preventive-regulation

#22

h=n/a

Task-Space Motion-Force Nonlinear Model Predictive Control for Precise and Compliant Control under Uncertainties

📅 2026-10-12cs.RO👤 Quang-Nam Nguyen (h=n/a)

Quang-Nam Nguyen, Ludovic Righetti

Core Contributions

A single task-space NMPC formulation that plans joint motions under motion and force requirements and transfers across platforms without per-task OCP redesign.
Analyzes the controller's intrinsic task-space impedance, proving partial contraction and differential passivity under unmodelled interactions.
This makes it possible to close the force-feedback loop inside the planning layer, a long-standing difficulty for force-feedback NMPC.
Validated on hardware for hybrid motion-force control on a redundant arm and compliant quadruped locomotion, retaining hard constraints and real-time rates.

Show abstract

In Nonlinear Model Predictive Control (NMPC), the control law is implicitly determined by the solution of an Optimal Control Problem (OCP). Consequently, OCP design is often tailored to specific platforms and tasks, demanding extensive expert knowledge and tuning effort. To address this, we propose a unified NMPC framework that directly plans and controls the robot joints subject to task-space motion and force requirements and is capable of seamlessly generalizing across multiple robotic platforms and diverse tasks. Furthermore, optimization-based methods often treat interactions as rigid contacts, making the torque solution sensitive to the modeling errors of the robot dynamics and the environment. We study the intrinsic task-space impedance of our proposed controller, its partial contraction for differential stability and differential passivity under unmodeled interactions. As a result, our framework enables closing the force-feedback loop in the high-level planning layer to achieve precise motion-force tracking performance, addressing the long-standing challenge of force-feedback NMPC. Finally, we validate our framework on physical hardware, demonstrating hybrid motion-force control on a redundant manipulator and compliant locomotion on a quadruped, while preserving core NMPC advantages such as hard constraint satisfaction, optimal redundancy exploitation, and real-time planning under unforeseen interactions.

#23

h=n/a

Multi-Agent Deferred-Decision Trajectory Optimization with Anytime-Valid Target Retention for Moving Candidates

📅 2026-10-12eess.SYcs.LGcs.RO👤 Farbod Siahkali (h=n/a)

Farbod Siahkali, Mengxue Hou, Vijay Gupta

Core Contributions

Handles multi-agent interception when the true target among several moving candidates is unknown and its terminal location is uncertain.
Combines conformal terminal-location prediction, sequential candidate rejection, capacity-constrained assignment, and candidate-contingent deferred-decision planning.
Provides anytime-valid target-retention and fixed-time coverage guarantees, plus a bound on verified-plan-but-capture-fails probability.
Captures 79% of targets under a strict control budget (85% relaxed) across 100 paired scenarios.

Show abstract

We study multi-agent terminal interception when one of several moving candidates is the unknown target and both target identity and candidate terminal locations are uncertain. We develop a receding-horizon deferred-decision trajectory optimization (DDTO) controller that combines conformal terminal-location prediction, sequential candidate rejection, capacity-constrained assignment, and candidate-contingent planning. Under exchangeability conditions for terminal and identity residuals, with predictable querying, we establish fixed-time terminal-location coverage and anytime-valid target retention. At the final replan, when one control step remains, we bound the joint probability that a tightened-set DDTO plan passes nonlinear rollout verification but capture fails. The closed-loop controller applies fallback guidance to any agent without a verified DDTO plan and, across 100 paired scenarios, captures 79% of targets under the strict control-effort budget and 85% when that budget is relaxed.

Navigation, Perception & Mapping

h=n/a

ClearNav: Perception-Aligned Quadrotor Flight Using Differential-Physics

📅 2026-10-12cs.ROcs.AIcs.LG👤 Aditya Azad (h=n/a)

Aditya Azad, Arthur Huang, Guanrui Li

Core Contributions

Pinpoints a single mechanism behind most quadrotor collisions in clutter: moving in directions the onboard camera cannot see (heading misalignment and lateral excursions).
Fixes it with three simple reward terms (forward-looking heading, lateral-velocity penalty, proximity speed penalty) trained via differentiable physics.
Across 2,304 paired held-out flights, collisions drop 65% versus SoTA with +6.8 pts success at up to 5 m/s; the proximity term alone accounts for a 70% reduction at matched speeds.
Honest reporting: gains are significant on static scenes but not at suite level with moving obstacles.

Show abstract

Autonomous quadrotor flight in cluttered environment is an key capability for tasks like inspection and search. Navigation policies trained in simulation and transferred to hardware can fly at high speed in open space, but they can still collide as obstacle clearance shrinks. We attribute most of these collisions to a single mechanism, namely motion in directions the onboard camera does not observe. Heading that is not aligned with the direction of travel and lateral excursion during obstacle avoidance both carry the vehicle toward obstacles outside its field of view. We introduce ClearNav, a navigation policy trained with two reward terms that constrain motion to the observed direction, a forward-looking heading target and a penalty on lateral velocity. A third term penalizes speed near obstacles so that an obstacle entering the field of view remains avoidable. In simulation, across 18 held-out scenarios with 2304 paired flights per policy unseen in training, ClearNav improves success by 6.8 points, cuts the collision rate by 65% relative to the state of the arts, and flies at maximum 5m/s. For the proximity term alone under matched commanded speeds, the collision reduction still holds at 70%. The effect is significant on static scenes and consistent in direction on moving obstacles, though not significant there at the suite level, showing that safer flight in clutter is possible even on the limited sensors and compute of small drones. We also validate our navigation policy in a real-world cluttered environment, showing that it enables the quadrotor to navigate through the cluttered environment successfully and safely.

#15

h=n/a

LiDAR-based Humanoid Navigation in Dynamic, Non-Convex Environments via Spatio-Temporal Safe Corridors

📅 2026-10-12cs.RO👤 Shubham S. Kumbhar (h=n/a)

Shubham S. Kumbhar, Panagiotis Artemiadis

Core Contributions

Builds spatio-temporal safe corridors directly from LiDAR particle maps, preserving non-convex free space that convex obstacle bounds throw away.
Couples corridors to LIP footstep planning and whole-body control so a humanoid can move around and even inside moving non-convex structures.
96% success on Digit in a 24×24 m arena; iterative polytope generation clearly beats single-iteration baselines in narrow passages.
Robust to clutter, sensor noise, control delay, and pushes.

Show abstract

Navigating humanoid robots through dynamic, non-convex environments requires tightly coupling onboard perception with footstep planning and dynamic locomotion control. Existing methods frequently discard complex free space through simplified convex obstacle bounds and fail to account for real-time dynamic occupancy predictions. This paper presents a real-time framework for humanoid navigation that directly constructs spatio-temporal safe corridors from LiDAR-based particle representations and couples them with Linear Inverted Pendulum (LIP)-based planning and Whole Body Control (WBC) to enable navigation around and within moving non-convex structures without restrictive geometric assumptions. Simulations on the Digit humanoid yield a 96% success rate in navigating around and inside moving non-convex obstacles in a 24 by 24 m^2 arena. Ablation studies confirm that the proposed iterative polytope generation significantly outperforms single-iteration baselines in narrow corridors, while evaluations demonstrate robustness to clutter, sensor noise, control delays, and force perturbations.

#12

h=n/a

ReForesight: Occlusion-Robust Re-Identification and Future Occupancy Forecasting for Dynamic Semantic Mapping

📅 2026-10-12cs.RO👤 Didar Seyidov (h=n/a)

Didar Seyidov, Faith Ogunwoye, Hajira Amjad, Danil Tolkachev, Yuri Sarkisov

Core Contributions

Adds instance-level future occupancy to semantic maps by moving each tracked object's own points to its forecast pose ('ghost' clouds), avoiding a learned occupancy decoder.
A tiny 36.6k-parameter MLP with a Kalman fallback cuts 3 s final displacement error from 0.728 m to 0.662 m, beating a GRU on every CODa sequence.
Runs at 0.04 ms per instance on one CPU core, making it practical for low-cost platforms.
Open-source ROS 2 toolkit with occlusion-robust re-ID, field-tested on an outdoor cleaning robot.

Show abstract

Robots sharing walkways need to know where people will be, yet semantic occupancy maps show only where they are. We present ReForesight, an open-source ROS 2 toolkit that adds instance-level future occupancy and re-identification after occlusion to a particle-based semantic mapping pipeline. A 36.6k-parameter MLP predicts each tracked person or vehicle up to 3 s ahead, guarded by a Kalman fallback and a stationary gate, and the object's own points are moved to the forecast pose, forming a "ghost" point cloud in the map frame without an occupancy decoder. An optional tracker re-associates objects after occlusion using keypoint descriptors the pipeline already computes and a metric 3D gate. On four held-out CODa sequences, the MLP lowers the mean 3 s final displacement error from 0.728 m (Kalman) to 0.662 m and beats a similar-size GRU (0.692 m) on every sequence, at 0.04 ms per instance on one CPU core. Code, all models, and a ROS-free demo are released; field demonstrations use an outdoor cleaning robot.

#13

h=n/a

Physical SLAM: Physics-aware Simultaneous Localisation and Mapping

📅 2026-10-12cs.RO👤 Guangming Wang (h=n/a)

Guangming Wang, Zhishuo Liu, Haibing Wu, Wanli Ma, Yixiong Jing

Core Contributions

Extends SLAM from 'where things are' to 'how deformable things will respond', coupling dynamic Gaussians to a spring-mass physics model fitted online.
Virtual observation forces gently pull simulated nodes toward depth points instead of overwriting them, keeping physics and observations consistent.
Improves reconstruction/prediction PSNR over PhysTwin by 10.4%/11.0% and reduces camera tracking error by 20.6% vs 4DGS-SLAM on TUM RGB-D.
The resulting map supports action-conditioned simulation, bridging mapping and model-based manipulation of deformables.

Show abstract

To reconstruct a predictive 4D representation of the physical world from monocular video captured by a moving camera, we jointly reconstruct dynamic Gaussian radiance fields and infer their underlying physical dynamics. This paper proposes \emph{Physical SLAM}, a physics-aware method for constructing 4D Gaussian fields of deformable objects and predicting their future state when subjected to external actions, given a sequence of RGB-D images. Our method separates the scene into a static background Gaussian field for camera localization and a dynamic object Gaussian field driven by a spring--mass physical model of elastic deformation, inertia, and damping. To integrate visual observations with physical dynamics, we introduce virtual observation forces that gradually pull simulated surface nodes toward observed depth points, avoiding abrupt replacement of simulated node positions. Meanwhile, the physical parameters of the spring--mass model are updated from simulation errors aggregated over the full reconstruction observation window, enabling the reconstructed dynamics to align with observations. To couple appearance, geometry, and physics, we further design a physics-aware Gaussian transformation and surface alignment strategy, where dynamic Gaussians are transformed along the deforming physical surface of the spring--mass model and constrained to remain aligned with the surface during rendering-based optimization. The resulting representation supports observed-frame rendering and action-conditioned simulation, extending SLAM maps to describe how deformable objects respond to manipulation. On the PhysTwin benchmark, our method improves reconstruction and prediction PSNR over PhysTwin by 10.4\% and 11.0\%, respectively. On the TUM RGB-D benchmark, it reduces average camera tracking error by 20.6\%, improves SSIM by 3.6\%, and reduces LPIPS by 8.3\% compared with 4DGS-SLAM.

#29

h=n/a

ConDenseDrive: Knowledge-Dense, Token-Sparse Visual Encoder for End-to-End Autonomous Driving

📅 2026-10-12cs.CVcs.RO👤 Shengzhen Li (h=n/a)

Shengzhen Li, Yongkang Li, Hao Gao, Chenxi Wu, Yajing Sun

Core Contributions

Argues the visual encoder, not the planner, is the bottleneck in end-to-end driving, and redesigns it to be knowledge-dense and token-sparse.
Distills complementary visual, semantic, and geometric knowledge from three foundation models, then folds spatiotemporal context into register tokens.
Register-guided token compression gives a 2.0× speedup while improving planning.
Reaches 94.5 PDMS (NAVSIM v1), 90.8 EPDMS (v2), and 37.4 HD-Score zero-shot on HUGSIM.

Show abstract

End-to-end driving requires rich spatiotemporal representations to support scene understanding and trajectory planning under strict latency constraints. However, most systems initialize their visual encoders from generic image pretraining and densely process all patch tokens for each camera view and input frame, limiting both representation quality and inference efficiency. To address these challenges, we present ConDenseDrive, a knowledge-dense and token-sparse visual encoder for end-to-end autonomous driving. Our three-stage framework first makes the encoder knowledge-dense by distilling complementary visual, semantic, and geometric knowledge from three vision foundation models. It then adapts the distilled encoder to multi-frame end-to-end driving and integrates spatiotemporal context into register tokens for planning. Finally, it makes the encoder token-sparse by applying register-guided token compression, reducing redundant patch computation and yielding a 2.0$\times$ speedup while improving planning performance. ConDenseDrive achieves 94.5 PDMS on NAVSIM v1 and 90.8 EPDMS on NAVSIM v2, and reaches 37.4 HD-Score in zero-shot evaluation on HUGSIM. Extensive experiments and qualitative analyses demonstrate that visual encoder design provides an effective route to reliable and efficient end-to-end driving.

Aerospace & Air-Traffic Autonomy

h=n/a

From Orbit Matching to Docking: A Hierarchical Reinforcement-Learning Approach to Multiphase Rendezvous Trajectory Design

📅 2026-10-12cs.ROcs.LGeess.SY👤 Mariko Storey-Matsutani (h=n/a)

Mariko Storey-Matsutani, Anatoly Zlotnik, Lee Holguin, Zigfried Hampel-Arias, Richard Linares

Core Contributions

Covers the entire rendezvous sequence (orbit matching → far/mid/close approach → docking) in one RL system, where prior RL work handled only isolated phases or hand-coded switching.
A learned high-level manager picks among five phase-specialist policies from the spacecraft state, so phase transitions are learned rather than triggered by fixed thresholds.
Compared directly against a sequential-invocation heuristic, giving evidence on whether learned switching actually yields smoother transitions in a multi-regime dynamics problem.
Relevant beyond space: it is a template for any task whose dynamics and constraints change sharply across stages.

Show abstract

As the complexity of space missions increases, autonomous spacecraft guidance must accommodate transitions between mission phases with distinct dynamics, objectives, and constraints. Rendezvous and proximity operations (RPO) are particularly challenging because they span multiple operating regimes, from orbital-scale matching and far-range approach to close-range maneuvering and terminal docking. Existing reinforcement-learning approaches generally address only a subset of this sequence or rely on manually defined switching conditions between controllers. This study presents a hierarchical reinforcement-learning framework for autonomous, three-degree-of-freedom RPO that spans the complete trajectory from orbit matching to docking. Five specialized policies are trained for orbit matching, far-range approach, mid-range approach, close-range approach, and docking. A learned high-level manager policy then selects among these phase-specific controllers based on the evolving spacecraft state. The hierarchical approach is evaluated over the full end-to-end rendezvous sequence and compared with a fixed sequential-invocation heuristic to assess trajectory performance and transition behavior. Training and inference metrics are also analyzed for the individual phase policies and the complete hierarchical system.

#28

h=n/a

DiTSG: Context-Aware Air Traffic Scene Generation with Multi-Agent Diffusion Transformers

📅 2026-10-12cs.MAcs.LGcs.RO👤 Seokbin Yoon (h=n/a)

Seokbin Yoon, Kyungmin Kim, Taejung Kwon, Keumjin Lee, Peng Wei

Core Contributions

Learns realistic air-traffic scenes including the effects of controller interventions (speed advisories, vectoring) that rule-based simulators miss.
Uses stacked multi-agent inverted-Transformer layers as a diffusion backbone to generate all aircraft trajectories jointly, conditioned on required arrival times.
Generated scenes reproduce statistical properties and inter-aircraft interactions of recorded traffic.
Useful for capacity studies of how many aircraft a controller can safely manage.

Show abstract

Understanding how many aircraft, and which traffic configurations, can be safely and efficiently managed by air traffic controllers within a given airspace is fundamental to air traffic management. Air traffic controllers continuously modify aircraft trajectories through interventions such as speed advisories and radar vectoring, and the patterns of these interventions are difficult to capture effectively using existing air traffic simulators. To address this limitation, we propose Diffusion Transformers for Traffic Scene Generation (DiTSG), which learns to generate air traffic scenes from recorded multi-aircraft trajectories. DiTSG employs stacked multi-agent inverted Transformer layers as the denoising backbone of a diffusion transformer, jointly generates the trajectories of multiple aircraft within a traffic scene, and conditions the generation on agent-specific required times of arrival. The resulting trajectories generated by DiTSG preserve realistic inter-aircraft interactions observed in actual air traffic operations. Experiments show that the DiTSG generates diverse traffic scenes that reproduce the statistical properties of recorded traffic while preserving realistic inter-aircraft interactions.