🤖 Robotics arXiv Digest

Wednesday, September 23, 2026

📄 30 papers 📂 8 research areas Generated by Claude

🔭 Research Landscape

Today's clearest thread is a re-examination of what generative priors actually contribute to control. NowWAM (#12) finds that denoising the current frame works as well as predicting the future, halving tokens and gaining 6.1 points on LIBERO-Plus — undercutting the premise that video-prediction foresight is what makes world-action models useful. Frozen Flows Forget (#5) reaches a complementary diagnosis: latent world models on frozen features silently lose object motion, and pixel error rewards that failure. Against that skepticism, three papers show world models earning their keep when grounded in the right state: PointCast (#7) unifies rigid, articulated and deformable dynamics as tracked points and plans inside MPC; the insertion world model of #20 gets 56% zero-shot on unseen parts versus 7% model-free; and LiMA (#2) makes slow imagination usable by running it asynchronously beside a fast controller.

A second theme is making pretrained VLAs work beyond single-shot, stateless episodes. ARMS (#3) and MemBodied (#21) both bolt compact memory onto π-family backbones rather than growing the context window — ARMS for always-on dual-arm streams with an agent-causal self-history, MemBodied with an episode anchor of the initial scene (7.8× over stateless on RMBench). LiMA and ARMS independently land on asynchrony as the design principle: slow context/intent modules must never block fast action. On the data and fine-tuning side, TANDEM (#15) reduces human teleop to on-demand 'magic operators' inside TAMP (0%→60% with 20 demos per task), #28 dissects advantage-guided RL post-training into separately testable stages, and #30 distills task experts via velocity fields — all aimed at getting more from limited robot data.

Finally, several papers push back on end-to-end learning by assigning each component the job it does best. #24 shows VLMs describe but cannot measure, delegating geometry to depth; VGM-VS (#16) drops a pretrained geometry model into classical PBVS for sub-millimetre insertion; LEAP-CBF (#10) and the STL tube-reward work (#6) use RL to construct formal safety or specification structures rather than replace them; and GLASS (#25) and Amplify (#9) invest in the numerical infrastructure beneath all of this. Governance also appears in unexpected places — motion unlearning for humanoids (#8), privacy-by-sensing (#11) — signalling that deployment concerns are moving from footnotes to first-class research problems. Note: Semantic Scholar h-index data was unavailable for this batch, so papers appear in arXiv listing order.

📂 Papers by Research Area

VLA Post-Training, Memory & Data

How to adapt, remember with, and cheaply feed data to pretrained vision-language-action policies.

#3 Watch, Recall, Act: Always-On Robots in Concurrent E...
#12 Beyond Future Prediction: Denoising as Generative Ad...
#15 TANDEM: Task and Motion Planning with As-Needed Demo...
#21 MemBodied: Recurrent Associative Memory for Vision-L...
#28 Dissecting Advantage-Guided Post-Training for Vision...
#30 Distillation for Efficient Multitask Manipulation Po...

World Models & Generative Dynamics

Predictive models of how scenes and objects move — and making them fast and faithful enough to plan or act with.

#2 LiMA: Bridging Long-term Imagination to Real-time De...
#5 Frozen Flows Forget: Diagnosing and Restoring Lost M...
#7 PointCast: One World Model for Rigid, Articulated, a...
#20 Generalizable Robotic Insertion with World Models

Contact-Rich & Precision Manipulation

Grasping, insertion, assembly and contact-mode discovery where millimetres and contact choices matter.

#16 VGM-VS: Rethinking Visual Geometry Model for High-Pr...
#17 Contact-Implicit Stein Projected ADMM for Discovery ...
#19 BrickCraft-Duo: Efficient Dual-Arm Skill Learning an...
#29 DEAL-Grasp: Decoupled Alignment Representation for G...
#24 VLMs Can Describe, But Not Measure: Object-Centric S...

Safe Control, Formal Methods & Optimization Tooling

Certificates, temporal-logic rewards, sampling-based MPC and the numerical libraries underneath them.

#10 LEAP-CBF: A Safety Filter for Uncertain Systems with...
#6 Tractable Reinforcement Learning for Full Class of S...
#13 Motoneuron-Inspired Sampling for Model Predictive Pa...
#9 Amplify: A Lightweight Library for Reproducible Nonl...
#25 GLASS: Architecture-Tuned, Composable, Device-Side L...

Humanoid Skills & Motion Policies

Whole-body humanoid policies — what they learn from vision, and what they can be made to forget.

#8 ForgetMimic: Motion Unlearning for Reinforcement Lea...
#26 DAVIS: A Depth-Only End-to-End Active-Vision Framewo...

Navigation, Localization & Perception

Getting robots and drones to know where they are and what they see, robustly and privately.

#18 Talk2Escape: Conversational Grounding for Vision-and...
#23 Large-Scale Geometric Map-Based Localization of UAVs...
#11 Privacy-Preserving Semantic Segmentation from High-R...

Social & Human-Centered Robotics

Robots that read groups, conversations and individual preferences to act appropriately around people.

#1 Where Should I Join? Robot Group Joining via Languag...
#14 Multimodal Voice Activity Projection for Social Robo...
#4 Context-Continuous Preference Learning for Exoskelet...

Multi-Robot Systems & Swarms

Coordinating many robots — through language feedback or decentralized sensing protocols.

#22 Controlling Collectives of AI Agents in Reasoning Sp...
#27 Dynamic, Decentralized Spatial Code Reuse for OCDMA ...

VLA Post-Training, Memory & Data

#3 h=—

Watch, Recall, Act: Always-On Robots in Concurrent Embodied Streams

2026-09-23cs.RO👤 Ding Yi (h n/a)

Ding Yi, Peiwen Sun, Chenchu Rong, Jianan Wang, Xili Dai

Core Contributions

Targets 'always-on' robots whose stream never resets: instructions arrive mid-task, lapse, and depend on the robot's own earlier actions — a regime today's fixed-instruction VLAs simply don't model.
Deliberately minimal: keeps a single pretrained π0.5 backbone and adds three light context modules (live perception, embodied state, agent-causal self-history of which arm did what, when).
Modules update asynchronously so watching and recalling never block acting, enabling true dual-arm concurrency rather than serialized per-arm decisions.
The ARMS Dataset's staged construction script auto-labels every module from teleoperation, avoiding extra annotation; ARMS hits 45% on the combined task vs. 28% for the best baseline, and ablations show memory, state head and concurrency are each necessary.

Show abstract

An always-on robot faces an endless stream that never resets: instructions arrive and lapse, the scene changes, and its own past actions reshape what it must reason about. Today's action models are built for the opposite: a fixed instruction, no mid-task intervention, single-step reasoning. In an open-ended world a robot must watch a live stream for far-future cues, recall its own far-past actions, and act on them under dual-arm concurrency. We present ARMS (Always-on Robot in Multi-modal Streams), a deliberately simple streaming policy: a single pretrained π0.5 backbone augmented by three lightweight modules that turn live perception, embodied states, and the robot's own past actions into context the backbone reads before it acts. The modules update this context asynchronously, so watching and recalling never block acting and the two arms act at once. Rather than inventing new mechanisms, ARMS integrates these learned context providers with an agent-causal self-history that logs which arm did what, and when. To supervise them without extra annotation, we build ARMS Dataset, whose staged construction script itself labels every module from real dual-arm teleoperation. Trained on it, ARMS reaches 45% on the combined task against 28% for the strongest of our four main baselines, and ablations confirm the memory module, the embodied-state head, and asynchronous concurrency are each necessary.

#12 h=—

Beyond Future Prediction: Denoising as Generative Adaptation for Robot Control

2026-09-23cs.RO👤 Zanyi Wang (h n/a)

Zanyi Wang, Yuheng Lei, Dengyang Jiang, Ping Luo, Mengdi Wang

Core Contributions

Challenges the assumption that video/image DiT priors help control because they predict the future: NowWAM denoises only the current frame alongside action prediction.
Controlled comparisons show past and future visual targets perform about the same, while training only on the clean endpoint hurts robustness — the denoising trajectory, not foresight, is the useful interface.
On LIBERO-Plus reaches 87.7% with FLUX2-Klein (+6.1 over future-target co-training) while halving visual tokens (784→392) and cutting step time 2.85 s→1.63 s (1.8×).
Works equally with a pure text-to-image backbone (Z-Image, 87.8%), suggesting expensive video-generation backbones are not required for generative policy transfer.

Show abstract

Pretrained generative Diffusion Transformers (DiTs) capture rich pixel-level visual and language-conditioned structure through large-scale image and video generation training. A growing line of robot policies builds on this generative prior, but how it should be transferred to control remains unclear, and existing approaches commonly instantiate this transfer through future visual prediction. We ask a more basic question: what a pretrained generative DiT actually contributes to action learning, and how this prior should be adapted for control. We introduce NowWAM, a future-target-free co-training formulation that denoises the current observation and predicts robot actions from the same visual stream, directly coupling the native generative objective to the action-facing representation across the denoising trajectory. Under matched controlled settings, past and future visual targets perform comparably, while restricting training to the clean endpoint substantially reduces robustness, suggesting that a separate future target is not essential for generative adaptation, while the denoising trajectory remains an effective interface for control. On LIBERO-Plus, NowWAM reaches 87.7% with FLUX2-Klein, improving over the future-target co-training baseline by 6.1 points while halving training visual tokens (784 to 392) and reducing step time from 2.85 s to 1.63 s, a 1.8x speedup. With the pure text-to-image Z-Image backbone, NowWAM further reaches 87.8%, showing that strong control adaptation is not tied to video generation or image-editing backbones.

#15 h=—

TANDEM: Task and Motion Planning with As-Needed Demonstrations for Efficient Vision-Language-Action Model Fine-tuning

2026-09-23cs.RO👤 Samrat Sahoo (h n/a)

Samrat Sahoo, Liang Ji, Tom Silver, Yixuan Huang

Core Contributions

Observes that teleoperators waste time demonstrating steps a planner could do; TANDEM treats human help as an on-demand planning operator ('magic operator') inserted only where TAMP falls short.
VLMs extend the planning domain with missing predicates and human operators from the instruction and image, so no task-specific intervention points need to be hand-coded.
Re-perceives and verifies effects after each human stage, and aligns planner motions to the target VLA's pretraining distribution so generated data is actually useful for fine-tuning.
Collects 2.9× more demonstrations per unit of human time than full teleoperation; 20 TANDEM demos per task lift π0.5-DROID from 0% to 60% average success on five long-horizon tasks.

Show abstract

Human teleoperators spend substantial time demonstrating behaviors that robots can already perform autonomously, limiting the scalability of data collection for robot foundation models. Task and motion planning (TAMP) can automate many of these behaviors, but a fixed planning domain may not support every stage of a long-horizon manipulation task. We present TANDEM (Tamp with As-Needed Demonstrations for Efficient Model fine-tuning), a system that combines TAMP with selective human teleoperation to collect demonstrations for tasks beyond the planner's capabilities. Our key idea is to represent human assistance as an on-demand planning capability. Given a language instruction and visual observation, TANDEM uses pretrained vision-language models to extend the planning domain with missing predicates and human-executed magic operators. This allows the planner to interleave autonomous and human-executed stages without task-specific intervention points. After each human stage, TANDEM re-perceives the scene and checks whether the intended effects hold before resuming autonomous planning. To support fine-tuning vision-language-action (VLA) models, TANDEM also uses example pretraining trajectories to align planner-generated motions with the target model's pretraining distribution. We evaluate TANDEM on five long-horizon manipulation tasks beyond the TAMP domain's capabilities. On a representative long-horizon task, TANDEM collects 2.9x as many demonstrations as full-task teleoperation at the same human intervention time. Fine-tuning a pretrained π_{0.5}-DROID model on 20 TANDEM demonstrations per task increases average task success from 0% to 60% across the five tasks.

#21 h=—

MemBodied: Recurrent Associative Memory for Vision-Language-Action Models

2026-09-23cs.ROcs.AIcs.CV👤 Tej Deep Pala (h n/a)

Tej Deep Pala, Navonil Majumder, Bryce Goh, Raphael Yee, Jianfei Yang

Core Contributions

Most VLAs are stateless; stuffing past frames into context bloats latency. MemBodied adds a fixed-size memory: a recurrent associative state across calls plus an 'episode anchor' of the initial scene.
The anchor preserves what the scene looked like before the robot changed it — exactly the information history-dependent tasks need and vanilla recurrence tends to wash out.
On five memory-requiring RMBench tasks: 7.81× the success of a stateless policy, 2.98× vanilla recurrent memory, and 1.3× the best memory baseline with 10× fewer added parameters.
Does not hurt fully observable tasks — 90.6% on LIBERO-Long, +5.4 over stateless π0 — making it a drop-in rather than a specialist add-on.

Show abstract

Vision-Language-Action models provide a strong foundation for general-purpose robot control, yet a vast majority of policies do not preserve and leverage episode-level information beyond the current observation. This limitation is consequential in history-dependent manipulation tasks that depend on information available only in past observations. Retaining past observations in context can aid in recovering this information, but at the significant cost of ever-growing, bloated context and inference latency. We thus introduce MemBodied, a fixed-size episodic memory with two complementary components: an associative state that records interactions across policy calls and an episode anchor that preserves a compact representation of the initial scene as a reference. At each policy call, the model conditions action generation on the current input and the memory components, rather than directly using past observations. Across five evaluated RMBench tasks requiring memory, MemBodied achieves 7.81\times the mean success rate of a stateless policy and 2.98\times of vanilla recurrent memory, while outperforming the strongest memory-augmented baseline by 1.3\times with 10\times fewer added parameters. On the fully observable LIBERO-Long suite, it reached 90.6%, a 5.4% improvement over the stateless π_0 policy. These findings support MemBodied as a practical alternative to expanding the policy context for history-dependent manipulation.

#28 h=—

Dissecting Advantage-Guided Post-Training for Vision-Language-Action Policies

2026-09-23cs.RO👤 Jiahang Cao (h n/a)

Jiahang Cao, Hanye Zhao, Hang Lai, Shenyu Zhang, Xiaoshen Han

Core Contributions

Disentangles advantage-guided RL post-training for VLAs into its coupled choices — advantage construction, calibration, and how advantages weight policy updates — which prior recipes bundled together.
Develops stage-specific offline evaluations to screen each choice without real-robot rollouts for every combination, which is the main cost barrier to such ablations.
Arrives at a modular recipe: TD advantages, group-wise calibration and continuous advantage weighting, raising task progress by 0.42 and success by 0.63 over SFT on four real bimanual tasks.
Shows the offline diagnostics broadly track real-world outcomes, giving practitioners a cheaper way to choose post-training designs.

Show abstract

Advantage-guided reinforcement learning provides a practical way to post-train vision-language-action (VLA) policies using limited robot data. However, its performance depends on several coupled choices, including how critic-derived advantages are constructed, calibrated, and used for policy training. Existing recipes often combine these choices into a single end-to-end procedure, making their individual effects difficult to identify. In this work, we dissect advantage-guided VLA post-training through a controlled empirical study that separates these design choices while accounting for their distinct estimands. We develop stage-specific offline evaluation methods to screen alternative choices efficiently, without requiring extensive real-robot policy evaluations for every possible combination. The staged evaluation identifies a modular recipe that combines temporal-difference advantage construction, group-wise calibration, and continuous advantage weighting. Across four real-world bimanual tasks, the resulting recipe improves mean task progress and success over the SFT initialization by 0.42 and 0.63, respectively. Moreover, the proposed evaluation diagnostics show an overall alignment with downstream real-world performance, supporting their use for interpreting empirical outcomes and selecting advantage-guided post-training designs in practice.

#30 h=—

Distillation for Efficient Multitask Manipulation Policies via Conditional Flow Matching

2026-09-23cs.ROcs.AIcs.LG👤 Shreya Deshmukh (h n/a)

Shreya Deshmukh, Imen Mahdi, Nick Heppert, Abhinav Valada

Core Contributions

Addresses a practical cost: training one flow-matching policy per task is expensive, but naively pooling tasks degrades performance unless model size grows.
Distills single-task CFM experts into one multi-task student by matching their learned velocity fields, not just their sampled actions — transferring the full denoising dynamics.
Combines the distillation signal with the original demonstration CFM loss so the student stays faithful to data rather than inheriting expert errors.
On RLBench, improves multi-task success over naive joint training at a fixed model size.

Show abstract

Advances in generative modeling have recently been extensively employed in robotics for policy learning. In particular, Conditional Flow Matching (CFM) trained with expert demonstrations has been shown to outperform existing methods on robot manipulation benchmarks. While prior work has mainly focused on single-task settings, we study the problem from a multi-task perspective, as training independent models for each task is computationally expensive. Multi-Task policy learning comes with its own set of challenges, as naively training on a concatenated dataset of demonstrations would either require increased model capacity to accommodate the added complexity or result in drops in performance. We propose to distill knowledge from single-task CFM experts into a shared multi-task policy by transferring their learned velocity fields. We combine this distillation signal with the original CFM objective to retain fidelity to the demonstrations. Experiments on RLBench show that our approach improves multi-task policy performance over naive training while maintaining a fixed model size.

World Models & Generative Dynamics

#2 h=—

LiMA: Bridging Long-term Imagination to Real-time Dexterous Manipulation via Asynchronous Diffusion

2026-09-23cs.RO👤 Ning Chen (h n/a)

Ning Chen, Yankai Fu, Junkai Zhao, Qianpu Sun, Guocai Yao

Core Contributions

Diagnoses a temporal-misalignment problem: VLAs reason well but lack physical foresight, while world-action models foresee but are too slow — so intent is stale by the time contact changes.
Splits computation into an asynchronous slow 'imagination' system producing sparse long-horizon intent and a fast system densifying it into high-rate actions, letting each run at its natural frequency.
A Latent Schrödinger Bridge couples sparse intent to dense trajectories as entropy-regularized transport — a principled alternative to naive interpolation or re-planning when the two systems disagree.
Cuts inference latency 45.8% versus Cosmos-Policy and reaches 70.8% overall / 78.9% subtask success on six bimanual dexterous tasks, while holding up in unseen scenes.

Show abstract

Dexterous manipulation demands long-term foresight and rapid reactive control. Vision-Language-Action (VLA) models, while proficient in high-level reasoning, often lack a fine-grained understanding of physical dynamics and spatial perception. Conversely, World-Action Models (WAMs) typically suffer from high inference latency due to iterative generation. These deficiencies result in a critical temporal misalignment where the model's intent fails to adapt to rapid physical contact changes. To overcome this fundamental bottleneck, we propose LiMA, an asynchronous dual-system generative framework that systematically decouples intent planning from reactive execution. LiMA organizes computation into a multi-scale hierarchy: a slow system handles sparse long-horizon spatiotemporal intent generation, while a fast system focuses on dense high-frequency motion refinement. To align sparse intent predictions with dense action trajectories, we introduce a Latent Schrödinger Bridge Coupling mechanism that formulates refinement as an entropy-regularized probabilistic transport process. LiMA reduces inference latency by 45.8% compared with Cosmos-Policy via asynchronous decoupling. Evaluated across six bimanual dexterous manipulation tasks spanning multiple horizons, LiMA achieves an overall success rate of 70.8% and an average subtask success rate of 78.9%, while maintaining performance in unseen scenarios. The project website is available at https://ccdcs.github.io/LiMA_repo/

#5 h=—

Frozen Flows Forget: Diagnosing and Restoring Lost Motion in a Latent-flow World Model

2026-09-23cs.CVcs.AIcs.RO👤 Xiwen Chen (h n/a)

Xiwen Chen, Rigaudiere Z. Li, Zhiruo Zhou, Xiaojun Zhu, Houde Liu

Core Contributions

Identifies a silent failure in latent-flow world models built on frozen self-supervised features: the pretrained flow never moves the manipulated object, and latent-only retraining swaps stillness for teleport-like jumps.
Traces the root cause to supervision, not representation — anchor-sparse latent losses never specify when along the horizon change should occur — which reframes where fixes should go.
DART adds decode-path supervision during rollout training while keeping the encoder frozen, restoring temporal motion structure and closing nearly half the gap to an oracle interpolation reference at scale.
Cautionary evaluation finding: pixel error alone rewards frozen (non-moving) predictions, so world-model leaderboards using only pixel metrics may be ranking the wrong models.

Show abstract

Latent world models that integrate a flow in a frozen self supervised latent space train stably and cheaply, yet silently lose the property manipulation depends on most: motion. The pretrained flow never moves the manipulated object; retraining it with latent-only losses only trades stillness for teleport-like motion. We trace the failure to the training signal, not the representation: anchor-sparse, latent-only supervision never says where along the horizon change belongs. Decode-augmented rollout training (DART) repairs this while keeping the representation frozen, retraining only the flow with decode-path supervision. DART outperforms its latent only parent on the full protocol, restores the temporal structure of motion, and re-couples predicted motion to the scene; at larger scale it further improves prediction quality, closing nearly half the remaining gap to an oracle-informed interpolation reference. Finally, we report an unexpected finding about evaluation: pixel error alone rewards frozen predictions.

#7 h=—

PointCast: One World Model for Rigid, Articulated, and Deformable Object Manipulation

2026-09-23cs.RO👤 Hantao Ye (h n/a)

Hantao Ye, Ross Worobel, Zhuoli Xie, Mingen Li, Houjian Yu

Core Contributions

One point-set world model spans rigid objects, articulated cabinets, cloth and rope — the state is just tracked 3D points on object and gripper, so no mesh, topology or per-class state design is needed.
Each point keeps identity and is supervised on its own trajectory, teaching where every point goes rather than just the resulting shape — crucial for deformables where shape-only losses are ambiguous.
A compact 19.8M-parameter diffusion transformer with alternating local/global attention and end-effector cross-attention; best on 3 of 4 sim regimes and lowest error in 4 of 6 real teleop categories, beating the dataset's own model in all six.
Fast enough to sit frozen inside sampling-based MPC at one network evaluation per window, competitive or better than every baseline across four planning tasks — evidence that unified world models are usable for control, not just prediction.

Show abstract

World models are useful for robotic manipulation because robots can predict how actions change the states of objects before executing them. We present PointCast, a point-set world model that spans rigid, articulated, and deformable object manipulation. Its state is a set of 3D points on the object and the end-effector, mesh-free and topology-agnostic. Each point keeps its identity and is supervised on its own trajectory, which teaches the model where every point goes rather than only the shape the points form. Its backbone is a diffusion transformer that denoises a short window of future point positions, conditioned on the points' recent history and the commanded end-effector motion. The backbone's attention alternates between local and global, and cross-attention to the end-effector carries the coupling. This one architecture at 19.8M parameters and one training recipe cover four regimes, rigid objects, cloth, rope, and multi-joint cabinets, with a separate checkpoint trained for each. Trained on randomized simulation and scored against four baselines on the same metric, it is best on three of four regimes and second on rigid. Trained on a real-world robot teleoperation dataset, it has the lowest mean error in four of its six categories, is second in the other two, and improves on the dataset's own model in all six; zero-shot, its simulation checkpoints are best on two of four captures. Frozen inside sampling-based model-predictive control at one network evaluation per window, it plans four simulated tasks over 64 episodes, competitive with or outperforming every baseline on each. Project website at https://pointcast-wm.github.io.

#20 h=—

Generalizable Robotic Insertion with World Models

2026-09-23cs.ROcs.CVcs.LG👤 Nicklas Hansen (h n/a)

Nicklas Hansen, Iretiayo Akinola, Yijie Guo, Jie Xu, Bingjie Tang

Core Contributions

Trains one world model on up to 90 geometrically diverse insertion tasks from wrist-camera images plus proprioception, instead of a specialist policy per part.
Model-based beats model-free by a wide margin on generalization: 56% zero-shot success on unseen parts of unknown geometry vs. 7% for a model-free baseline.
Performance scales with the number of training objects, and fine-tuning the generalist on held-out parts is more data-efficient — sometimes with better asymptotic success — than training from scratch.
Claimed as the first fully data-driven system to assemble unseen objects, a key step for high-mix manufacturing where per-part engineering is the main cost.

Show abstract

Robotic assembly in high-mixture settings requires adaptable systems that can handle diverse parts, yet current approaches typically rely on policies specialized to each insertion task. Although this can reach high success rates, it makes the process of deploying systems for new problems tedious and time consuming. We present a framework for generalizable insertion using world models that combine robot proprioceptive information with raw visual observations captured by a wrist-mounted camera. Our model-based approach trains a single world model on up to 90 insertion tasks with geometrically diverse parts, achieving 56% zero-shot success on unseen objects with unknown geometry compared to just 7% with a model-free baseline. Importantly, performance improves as more objects are included in the training dataset, demonstrating strong scalability. Lastly, finetuning the generalist model on held-out objects significantly enhances data-efficiency compared to training from scratch and, in some cases, achieves better asymptotic performance. To our knowledge, this is the first system capable of assembling unseen objects in an entirely data-driven manner, and thus represents a significant step toward scalable, generalizable robotic assembly systems.

Contact-Rich & Precision Manipulation

#16 h=—

VGM-VS: Rethinking Visual Geometry Model for High-Precision Visual Servoing

2026-09-23cs.ROcs.CV👤 Yimin Pan (h n/a)

Yimin Pan, Sen Wang, You Zhou, Jianfeng Gao, Pengbo Sun

Core Contributions

Repurposes a pretrained feed-forward visual geometry model as the pose estimator inside classic PBVS, inheriting robustness to occlusion, low texture and small targets that break feature-based servoing.
Resolves the scale ambiguity of such models with a self-supervised, scene-specific metric adaptation that also learns the hand-eye transform — eliminating a separate calibration step.
Runs at 30 Hz and converges to sub-millimetre accuracy on USB-C cable picking and insertion; 90–100% success when the target moves during servoing.
Converges in all trials from 30 cm initial offsets and with 50% target occlusion, outperforming compared visual-servoing baselines on RAM and cable insertion.

Show abstract

We present VGM-VS, a visual servoing method built on a pretrained feed-forward visual geometry model. Given the current view and a reference image captured at the target configuration, we estimate the relative camera pose with a visual geometry model and apply it iteratively as the pose increment of a closed-loop pose-based visual servoing (PBVS) scheme. The geometry-aware representation acquired from large-scale pretraining keeps this estimate reliable when the target is occluded, weakly textured, or covers only a small part of the image. However, the scale ambiguity inherent to these models leaves the predicted translation defined up to an unknown scale, while the pose increment must be metric for robot control. We close this gap with a scene-specific metric adaptation: the robot autonomously records image--pose pairs along a predefined motion starting from the target pose, and we fine-tune the camera head on these data, jointly learning the hand--eye transform and thus removing the need for a dedicated calibration process. We evaluate our method on three real-world assembly tasks with demanding tolerances: USB-C cable picking, cable insertion, and RAM insertion. Running in real time at 30Hz, VGM-VS converges to submillimeter terminal accuracy on the cable tasks, and reaches success rates of 90--100% when the target is moved during servoing. It converges in all trials under initial displacements of up to 30cm from the reference pose and with 50% of the target object occluded, outperforming the compared visual servoing baselines.

#17 h=—

Contact-Implicit Stein Projected ADMM for Discovery of Diverse Contact-Rich Manipulation Strategies

2026-09-23cs.RO👤 Hrishikesh Sathyanarayan (h n/a)

Hrishikesh Sathyanarayan, Christian Hughes, Ian Abraham

Core Contributions

Contact-implicit trajectory optimization normally collapses to one contact mode; this solver returns a diverse set of valid strategies (different grasps, push directions, handovers) in one run.
Combines consensus ADMM with Stein variational inference; the key insight is applying the Stein repulsion to ADMM's split variable instead of the primal, which spreads solutions without stalling convergence.
Diversity yields robustness: a manipulation system can switch to an alternative contact mode when the first becomes infeasible, instead of re-solving from a fragile initialization.
Simpler in form than existing diverse-solution solvers while discovering unique contact modes across pushing, grasping and multi-robot handover.

Show abstract

Contact-implicit trajectory optimization formulates contact-rich manipulation as a single constrained program; however, that single program run collapses onto one local optimum out of many equally valid contact modes, grasps, or push directions. As a consequence, the resulting manipulation strategy is reluctant to change and sensitive to initialization. In order to promote robust manipulation, this paper investigates how contact-implicit solvers can discover diverse contact-rich strategies. Our approach derives a variation of Consensus Alternating Direction Method of Multipliers (ADMM) combined with Stein variational inference methods to output a set of distinct contact-rich solutions. We find that applying the Stein repulsive force to ADMM's split variable (rather than its primal form) allows for effective coverage over the set of feasible contact strategies without prematurely stalling the solver. We demonstrate the effectiveness of our approach on a variety of contact-rich manipulation tasks, including pushing, grasping, and multi-robot handover. Last, we find the proposed solver is simpler in form and capable of discovering unique contact modes when compared with existing solvers. Videos and code with examples are found in https://anon-website-submission.github.io/stein-admm-website/.

#19 h=—

BrickCraft-Duo: Efficient Dual-Arm Skill Learning and Refinement for Compositional Long-Horizon Assembly

2026-09-23cs.RO👤 Jichuan Yu (h n/a)

Jichuan Yu, Zhenyu Xiao, Ze Wang, Ruixuan Liu, Changliu Liu

Core Contributions

Uses interlocking brick assembly as a demanding real-world testbed: tight insertion tolerances, inter-step dependencies and partially supported structures.
Learns reusable single- and dual-arm skills with bilateral symmetry alignment, so a skill learned on one arm transfers to the mirrored arm and to swapped assemble/support roles — cutting demo requirements.
Stability-aware assembly reasoning sequences heterogeneous skills, and human-in-the-loop corrections refine only failing skills rather than retraining everything.
Achieves ≥60% full long-horizon success and ≥95% step completion across five real tasks up to nine steps long.

Show abstract

Interlocking brick assembly provides a representative testbed for evaluating real-world robotic manipulation capabilities, where diverse structural designs, complex inter-step dependencies, intricate mechanical interactions and tight insertion tolerances pose substantial challenges. We present BrickCraft-Duo, a modular framework for long-horizon dual-arm collaborative assembly of interlocking bricks through data-efficient skill learning and composition. BrickCraft-Duo learns reusable single- and dual-arm assembly skills from diverse demonstrations, with bilateral symmetry alignment facilitating skill sharing across symmetric arms and assembly--support role assignments. Guided by stability-aware assembly reasoning, BrickCraft-Duo composes heterogeneous skills to achieve autonomous long-horizon execution, and further integrates human-in-the-loop correction for targeted skill refinement. The resulting system achieves long-horizon success rates of at least 60% and step-level completion rates of at least 95% across five real-world assembly tasks involving partially supported configurations, with horizons of up to nine steps. Project website: https://jichuan-yu.github.io/BrickCraft-Duo.

#29 h=—

DEAL-Grasp: Decoupled Alignment Representation for Geometry-Aware Dexterous Grasp Generation

2026-09-23cs.ROcs.GR👤 Fuqiang Zhao (h n/a)

Fuqiang Zhao, Qian Liu

Core Contributions

Standard dexterous grasp generators denoise a joint space that entangles wrist pose and finger articulation, producing unstable samples; DEAL separates them.
Generates task-space geometric anchors plus articulation, then recovers the rigid transform in closed form via Procrustes alignment — the global pose is derived, not generated.
Heterogeneous-state flow matching with per-component vector fields and time-adaptive physics regularization during training removes the need for test-time optimization.
High force-perturbation success with low penetration and high diversity on MultiDex and zero-shot RealDex, at much lower latency than optimization-heavy baselines.

Show abstract

Synthesizing realistic articulated hand-object interactions is a fundamental problem in virtual reality, embodied intelligence, and digital human applications. Existing methods for dexterous grasp synthesis typically regress or denoise poses in a joint space that couples global rigid motion with local articulation, which often yields unstable samples and physically implausible contacts. We introduce DEAL-Grasp, built upon the Decoupled Alignment (DEAL) representation, which reformulates grasp synthesis as alignment-space generation: the interaction state comprises task-space geometric anchors and articulation parameters, from which the rigid transform is recovered via closed-form Procrustes alignment while preserving local articulation. On this mixed state, we model grasp generation using heterogeneous-state flow matching with component-wise vector fields, incorporating time-adaptive physical regularization during training. At inference, grasps are synthesized solely by integrating the learned vector field, without test-time optimization or auxiliary physical guidance. Across MultiDex and zero-shot RealDex benchmarks, DEAL-Grasp attains high force-perturbation success rates alongside minimal penetration and high diversity of generated grasps, while substantially reducing native inference latency compared to optimization-heavy baselines. The project page is available at https://wmtlab.github.io/DEAL-Grasp/.

#24 h=—

VLMs Can Describe, But Not Measure: Object-Centric Scene Understanding for Robotic Manipulation

2026-09-23cs.ROcs.CV👤 Enrico Saccon (h n/a)

Enrico Saccon, Tommaso Faraci, Iñigo De La Ossa Zarzuelo, Luigi Palopoli, Marco Roveri

Core Contributions

Quantifies a widely suspected gap: VLMs label objects well but their direct metric estimates (location, depth) are unreliable for manipulation.
Proposes a modular pipeline — segment, let the VLM annotate, ground in depth — so the VLM does only what it's good at and geometry comes from the sensor.
Across 151 tabletop scenes, keeps semantic accuracy while substantially improving localization and depth versus asking the VLM directly.
Produces a task-independent object-centric representation that plugs into a task planner, a pragmatic counterpoint to end-to-end VLAs that must learn metric reasoning implicitly.

Show abstract

Robotic operation in previously unseen environments requires both semantic understanding and reliable metric information. While vision--language models (VLMs) provide strong semantic capabilities, their geometric estimates remain less reliable. In this paper, we propose a VLM-driven, modular perception framework for scene understanding using off-the-shelf approaches. Starting from a single RGB-D observation, the scene is segmented into object-level regions, annotated by a VLM, and grounded with depth information to construct a task-independent object-centric representation. Experiments on 151 tabletop scenes show that the proposed decomposition preserves strong semantic performance while substantially improving localization and depth estimation over direct VLM inference. The resulting representation is also integrated with a task-planning framework for robotic execution.

Safe Control, Formal Methods & Optimization Tooling

#10 h=—

LEAP-CBF: A Safety Filter for Uncertain Systems with Least-Effort Adversarial Potentials

2026-09-23cs.ROcs.LGmath.OC👤 Oswin So (h n/a)

Oswin So, Eric Yu, Chuchu Fan

Core Contributions

Replaces hard-to-construct, conservative robust CBFs with a new certificate — Least-Effort Adversarial Potential — measuring how much cumulative disturbance 'effort' is needed to push a state into failure.
Proven to be a CBF for the nominal system while also yielding a filter robust to any disturbance with bounded cumulative effort, a more natural budget than per-step worst-case bounds.
Constructed via on-policy deep RL, so it scales to high-dimensional, input-constrained multi-agent systems where hand-designed robust CBFs are impractical.
Hardware validation on a quadruped and quadrotors shows the certificate handles real modeling error, not just injected simulation noise.

Show abstract

Control barrier functions (CBF) are a popular safety filter to ensure safety for nonlinear dynamical systems. However, when the system is subject to uncertainties and disturbances, this requires the use of robust variants of CBFs, which can be difficult to construct and can be overly conservative, especially for high-dimensional systems under input constraints. In this work, we propose a new approach to solve these challenges by introducing Least-Effort Adversarial Potentials (LEAP), a certificate that quantifies the robustness of a given state against disturbances in terms of the effort required by the disturbance to cause failure. We show that LEAP is a CBF for the undisturbed system, but can also be used to construct a safety filter that is robust to disturbances whose cumulative effort is bounded. We propose a method for constructing LEAPs with on-policy deep reinforcement learning. Next, we demonstrate LEAPs in simulation on a variety of multi-agent systems with disturbances and uncertainties. Finally, hardware experiments on a quadruped and quadrotors validate that LEAPs are well suited to tackle the disturbances and uncertainties from real-world robotic systems.

#6 h=—

Tractable Reinforcement Learning for Full Class of Signal Temporal Logic Specifications Using Spatiotemporal Tube Reward

2026-09-23cs.ROeess.SY👤 Vaishnavi Jagabathula (h n/a)

Vaishnavi Jagabathula, P Sangeerth, Pushpak Jagtap

Core Contributions

Handles the full class of Signal Temporal Logic by compiling it into time-varying geometric 'spatiotemporal tubes' — the RL agent only has to stay inside a moving region, not evaluate logic online.
Unlike robustness-based STL RL, which collapses satisfaction into one scalar and often needs state-history augmentation, this uses time-augmented states and a geometry-aware dense reward, so the policy stays history-free.
Unlike analytic STT controllers, which can demand unbounded inputs, the learned SAC policy respects hard actuator limits — important for non-holonomic and underactuated platforms with unknown dynamics.
Offers a practical bridge between formal task specifications and deep RL that scales without memory-heavy automaton products.

Show abstract

This paper addresses the control problem for robotic systems, including non-holonomic and underactuated platforms operating under unknown dynamics and strict actuator limits to satisfy complex high-level specifications. We denote these high-level specifications using Signal Temporal Logic (STL) and propose a novel time-aware Reinforcement Learning (RL) framework that leverages the geometric properties of Spatiotemporal Tubes (STTs). While traditional analytical STT controllers often struggle to enforce input constraints, and existing RL approaches rely on memory-intensive state history, our method natively overcomes both limitations. By mapping the logical and temporal complexities of the full class of STL into time-varying geometric boundaries, we directly constrain the multidimensional system state without relying on scalar robustness metrics. Augmenting the state space with time, we train a time-aware Soft Actor-Critic (SAC) agent using a continuous, geometry-aware reward function that eliminates the need to explicitly evaluate complex logical semantics during execution. The proposed framework offers a history-free, computationally efficient approach to learn continuous control policies that ensure robust satisfaction of specifications while strictly adhering to system input constraints.

#13 h=—

Motoneuron-Inspired Sampling for Model Predictive Path Integral Control

2026-09-23cs.RO👤 Alexis Poignant (h n/a)

Alexis Poignant, Jan Babič

Core Contributions

Replaces MPPI's default white Gaussian perturbations with a proposal generated by a simplified motoneuron model, producing temporally structured, spike-driven control noise.
Structured sampling markedly improves executed-control smoothness on torque-driven and antagonistically actuated MuJoCo Ant, though task-performance gains depend on rollout budget and actuation.
Spectrum-matched Gaussian controls reproduce much — but not all — of the effect, isolating a contribution from higher-order statistics beyond the power spectrum.
Frames MPPI proposal design as two separable levers (second-order spectrum + higher-order organization), a useful lens for anyone tuning sampling-based MPC under tight budgets.

Show abstract

Model Predictive Path Integral (MPPI) control relies on stochastic trajectory sampling, and its performance under limited rollout budgets depends strongly on the structure of the proposal distribution. Standard implementations commonly perturb control sequences with Gaussian noise, despite growing evidence that temporally correlated and structured sampling can improve finite-budget control. We introduce Spike-MPPI, a motoneuron-inspired proposal that generates temporally structured perturbations through a simplified model of motoneuron dynamics. The proposal is evaluated within a common MPPI framework on torque-actuated and antagonistically actuated MuJoCo Ant models against standard Gaussian sampling and spectrum-matched Gaussian controls. Results show that structured sampling substantially improves executed-control smoothness, while its effect on task performance depends on rollout condition and robot actuation. Spectrum matching reproduces a substantial part of the observed behavior, while the full Spike proposal retains additional effects beyond second-order spectral structure. These results support treating proposal design as a combination of second-order spectral structure and higher-order statistical organization.

#9 h=—

Amplify: A Lightweight Library for Reproducible Nonlinear Programming Problems in Robotics

2026-09-23cs.RO👤 Nelson Rosa (h n/a)

Nelson Rosa

Core Contributions

Attacks reproducibility of robotics trajectory optimization with a 537-line library that needs only AMPL, a text editor and an internet connection.
Novel in encoding the algorithms themselves — dynamics recursions, trajectory and reference-motion generation — as declarative constraints inside the optimization model, rather than as imperative code calling a solver.
Makes the full problem (not just the solver call) portable and inspectable, so results can be re-run across solvers and labs without re-implementing dynamics.
Compared against three other libraries on benchmark problems including bipedal locomotion and grasp planning, and gives a recipe other transcription libraries can adopt.

Show abstract

Optimization problems (OPs) are key to solving many challenging research problems in robotics. However, reproducibility still remains a major issue. In this paper, we present Amplify, a lightweight nonlinear programming library aimed at reproducible results of robotic-related trajectory optimization problems. The minimalistic requirements for the 537-line library (80 characters per line) are an Internet connection, familiarity with the AMPL modeling language, and a text editor. Our primary contribution is the formulation of a library where trajectory optimization algorithms are represented directly within the optimization model. Specifically, we implement the algorithms used to compute the dynamics, trajectories, and reference motions as constraints of the OP in a declarative programming paradigm. We outline how our formulation of objectives, decisions variables, and constraints can be implemented in other transcription libraries that want to be lightweight and reproducible. We also compare the Amplify framework with 3 other libraries across examples of benchmark optimization problems across several fields, including bipedal locomotion and grasp planning.

#25 h=—

GLASS: Architecture-Tuned, Composable, Device-Side Linear Algebra for Edge Robotics and Beyond

2026-09-23cs.ROcs.DCcs.MS👤 Brian Plancher (h n/a)

Brian Plancher

Core Contributions

Fills a gap in GPU robotics: a header-only CUDA C++ library of robotics-scale linear algebra and geometry with thread-, warp-, block- and vendor-backed variants under one composable device API.
Treats implementation, execution scope and launch packing as architecture-specific decisions measured offline and resolved at compile time — best vs. worst placement differs by a median 4.9× (max 81×).
Placement recommendations differ markedly across devices (145/396 change between Jetson AGX Orin and RTX 5090), so hard-coded choices are a real performance loss on edge hardware.
Up to 73× faster than the best of PyTorch/JAX on Orin; integrating into published systems gave up to 1.5× embedded speedups and surfaced a pre-existing numerical bug.

Show abstract

GPU robotics lacks the reusable numerical infrastructure of mature CPU stacks, instead relying on compiler frameworks that introduce overhead or repeatedly reimplementing numerical libraries. To address this, we introduce GLASS (GPU Linear Algebra Simple Subroutines), a header-only CUDA C++ library that provides thread-, warp-, block-, and NVIDIA-backed implementations of robotics-scale linear algebra and geometric computations under one composable device API. GLASS treats implementation choice, execution scope, and launch packing as architecture-specific placement decisions determined by offline measurement and resolved statically at compile time. This is critical as the best and worst placements differ by a median of 4.9x (max 81x), with 145 of 396 recommended placements changing between a Jetson AGX Orin and an RTX 5090, and 162 of 396 versus an AGX Xavier. These stakes are highest at the edge as GLASS's advantage over the best of PyTorch and JAX is as much as 73x on the Orin versus 12x on the RTX 5090. GLASS is released open source with independent numerical oracles and source-bound local-GPU test attestation. Finally, integrating GLASS with published robotics systems both exposed a pre-existing numerical bug and improved embedded runtimes by up to 1.5x.

Humanoid Skills & Motion Policies

#8 h=—

ForgetMimic: Motion Unlearning for Reinforcement Learning Humanoid Control

2026-09-23cs.ROcs.CRcs.LG👤 Xukun Luan (h n/a)

Xukun Luan, Zhongxiang Lei, Chen Gong, Shaowei Li, Yuanguo Bi

Core Contributions

Introduces motion-level machine unlearning for humanoid RL — removing specific skills (e.g. Fight, Flip) from a policy trained on N motions while preserving the other N−K.
Motivated by concrete governance needs: purging poisoned or malicious demonstrations and honouring right-to-be-forgotten / copyright requests for human mocap data under GDPR.
Identifies two training mechanisms specific to robot control that make naive unlearning fail — a distinct problem from LLM or vision unlearning, where there is no closed-loop physics to recover the forgotten behavior.
Validated on real Unitree G1 and H2 humanoids across 12 motions, showing targeted forgetting without degrading retained skills.

Show abstract

Humanoid control, leveraging human demonstrations, has achieved diverse, agile, and natural locomotion behaviors through reinforcement learning (RL). While this paradigm has yielded remarkable performance in physical humanoid control, how to eliminate specific motions from learned policies remains insufficiently explored. Addressing this issue is motivated by pressing safety and privacy concerns: the removal of malicious, poisoned, or suboptimal motions, as well as copyright-protected motions subject to the right to be forgotten under regulations such as the GDPR, is of critical importance. To this end, we propose {ForgetMimic}, the first motion-level unlearning method designed specifically for physical-world humanoid control. The core idea of ForgetMimic is as follows: given a policy π_θ trained on N motions, our method degrades performance on a target subset of K motions while preserving the effectiveness of the remaining N-K motions. Furthermore, we identify and resolve two key training mechanisms in robot control that lead to unlearning failure. We conduct extensive experiments on the Unitree G1 and H2 humanoid robots across 12 motions, including Dance, Fight, Flip, and others. Experimental results demonstrate that ForgetMimic effectively eliminates memory of designated motions while maintaining the normal operation of all other motions.

#26 h=—

DAVIS: A Depth-Only End-to-End Active-Vision Framework for Humanoid Soccer Skills

2026-09-23cs.RO👤 Jiakang Jin (h n/a)

Jiakang Jin, Yixiao Huo, Pengyuan Wang, Yinan Han, Tingxuan Zhang

Core Contributions

Asks whether a humanoid can learn soccer contact skills from only a head-mounted depth image and proprioception, outputting 25-DoF joint targets with no separate perception or planning module.
Learns visibility-aware auxiliary geometry so the policy copes with the ball leaving view — a constant problem when the robot's own motion swings the camera.
GT-to-prediction annealing smoothly moves from privileged ball state to predicted estimates, combined with curricula and AMP motion priors for natural movement.
Instantiates goal-directed shooting and directional dribbling, validated on a real Noetix E1 humanoid.

Show abstract

Humanoid soccer contact skills require more than producing high-impact foot-ball contacts: the robot must close the loop over perception, approach, alignment, impact, and recovery while its own motion induces substantial viewpoint changes, frequent loss of the ball from view, and uncertain contact outcomes. In this work, we ask a compact yet stricter question: can a humanoid learn soccer contact skills using only a head-mounted depth image, proprioceptive history, and an optional low-dimensional task command, and directly output 25-DoF joint PD targets without extra runtime perception or planning modules? To this end, we propose DAVIS, a depth-only end-to-end framework for humanoid soccer skills that learns visibility-aware auxiliary geometry during training, and combines GT-to-prediction annealing, task curricula, and AMP-style motion priors to smoothly bridge privileged supervision and real deployment. Built on this framework, we instantiate representative soccer contact skills, including goal-directed shooting and directional dribbling, through task-specific definitions of objects, commands, rewards, and curricula, and validate them through simulation, Noetix E1 real-robot experiments, and ablations.

Navigation, Localization & Perception

#18 h=—

Talk2Escape: Conversational Grounding for Vision-and-Language Navigation

2026-09-23cs.ROcs.HC👤 Zerui Li (h n/a)

Zerui Li, Sihao Lin, Yanyan Shao, Jiwen Zhang, Xiangyu Shi

Core Contributions

Argues single-turn VLN is fundamentally open-loop — drift and aliasing compound with no recovery — and turns navigation into a closed-loop dialogue.
A lightweight VLM monitors kinematics for looping or divergence and only then asks a concise, grounded question, so help is proactive but sparse.
Model-agnostic: wraps diverse base agents and consistently improves them on R2R-CE, RxR-CE and VLNVerse, reaching 66.0% SR on R2R-CE, above both supervised and zero-shot SOTA.
Sim-to-real on a Unitree Go2 quadruped shows the intervention mechanism holds up with real sensor noise.

Show abstract

While Vision-and-Language Navigation (VLN) has demonstrated remarkable success, the prevailing single-turn paradigm exposes a fundamental vulnerability: agents operate in a strictly open-loop manner. In practice, factors such as perceptual aliasing, sensor noise, and odometry drift can cause minor deviations to accumulate over time, often leading to catastrophic mission failures with no built-in mechanism for error recovery. To address this, we introduce \textit{Talk2Escape}, a proactive and model-agnostic dialogue intervention framework that reframes navigation as a closed-loop interactive process. At its core, a lightweight vision-language module continuously monitors agent kinematics. Upon detecting localized looping or severe trajectory divergence, it translates raw egocentric observations into concise, grounded queries to solicit targeted corrective feedback from either an algorithmic oracle or a human-in-the-loop. Extensive evaluations in high-fidelity simulators, including R2R-CE, RxR-CE, and VLNVerse, demonstrate that \textit{Talk2Escape} exhibits consistent improvements across diverse base agents. Empirically, \textit{Talk2Escape} achieves a 66.0% Success Rate on R2R-CE, outperforming the current supervised and zero-shot state-of-the-art methods. We further validate its sim-to-real transfer on a Unitree Go2 quadruped, proving that proactive dialogue drastically improves navigation robustness in physical environments.

#23 h=—

Large-Scale Geometric Map-Based Localization of UAVs in GNSS-Denied Urban Environments

2026-09-23cs.RO👤 Garth Terlizzi (h n/a)

Garth Terlizzi, Kaveh Fathian

Core Contributions

Localizes UAVs without GNSS by matching building footprints seen from a downward camera against a public footprint database — using geometry, not appearance.
A new descriptor combines local triangle structure of nearby buildings with per-building shape features, robust to seasons, lighting and imagery age that break satellite-image retrieval.
100% Recall@1 over ~113 km² and ~254 km², 71.4% over ~452 km² (~277k buildings), while baselines fall to 0% at 254 km² and beyond.
Accumulating detections across frames into a local map makes the search discriminative at metropolitan scale — relevant for urban delivery and GNSS-jamming resilience.

Show abstract

Unmanned aerial vehicles (UAVs) operating in GNSS-denied urban environments require alternative methods for position estimation. Existing approaches based on satellite image retrieval or learned descriptors are sensitive to appearance variation and degrade rapidly as the search area grows. We present a vision-based localization system that matches building patterns observed from a downward-facing UAV camera against a reference building footprint database. Our approach detects buildings in aerial imagery, accumulates observations across frames into a unified map, and matches local building arrangements against reference footprints using a novel geometry-driven descriptor that augments local triangle structure with per-building shape features. By encoding spatial relationships between nearby buildings rather than visual appearance, the system is robust to appearance variations and remains discriminative over large search areas. Evaluations on seven flights across four municipalities in a large metropolitan area demonstrate 100% Recall@1 at search areas of approximately 113 km^2 and 254 km^2, and 71.4% Recall@1 when expanded to approximately 452 km^2, encompassing up to 277,000 buildings. In contrast, baseline methods degrade rapidly and achieve 0% Recall@1 at 254 km^2 and 452 km^2.

#11 h=—

Privacy-Preserving Semantic Segmentation from High-Resolution Depth and Ultra-Low-Resolution RGB

2026-09-23cs.CVcs.RO👤 Xuying Huang (h n/a)

Xuying Huang, Swithinraj Moses Daniel, Sicong Pan, Sebastian Houben, Maren Bennewitz

Core Contributions

Proposes an asymmetric privacy-by-design sensor setup: high-resolution depth plus ultra-low-resolution RGB, so faces and documents are never captured while geometry stays dense.
Uses HR geometry to guide semantics-oriented RGB reconstruction rather than trying to super-resolve appearance, then fuses into RGB-D segmentation — exploiting the information imbalance instead of fighting it.
An end-to-end 2D-to-3D pipeline consolidates frame predictions into consistent 3D semantics; best 2D and 3D results among privacy-preserving methods on ScanNet with the strongest zero-shot transfer to SUN RGB-D and SceneNN.
Privacy recoverability tests show sensitive content is harder to recover, and real-robot object-goal navigation demonstrates the semantics are usable for downstream tasks.

Show abstract

As mobile robots become increasingly integrated into everyday environments, privacy risks arising from onboard cameras have become a growing concern. Ultra-low-resolution (ULR) RGB can mitigate visual privacy exposure at the source, but ULR appearance alone substantially limits semantic and spatial understanding. We therefore introduce a privacy-preserving asymmetric sensing setting that combines high-resolution (HR) depth with ULR RGB, preserving dense geometry while restricting fine-grained visual information. To address the severe information imbalance between HR depth and ULR RGB, we propose a joint 2D framework using HR geometry to guide semantic-oriented RGB reconstruction and RGB-D segmentation. Despite reliable frame-level predictions, consistent scene-level understanding remains challenging under the asymmetric HR depth--ULR RGB setting. We therefore develop an end-to-end 2D-to-3D pipeline that consolidates 2D semantic features for 3D segmentation. Experiments on ScanNet show that our method achieves the best 2D and 3D segmentation performance among privacy-preserving approaches and delivers the strongest zero-shot transfer to SUN RGB-D and SceneNN. Privacy recoverability analysis shows that our proposed HR depth--ULR RGB input reduces the recoverability of sensitive data, and real-robot experiments demonstrate the utility of the resulting 3D semantics for object-goal navigation.

Social & Human-Centered Robotics

#1 h=—

Where Should I Join? Robot Group Joining via Language-Guided Goal Prediction

2026-09-23cs.ROcs.AI👤 Zilin Fang (h n/a)

Zilin Fang, Zishuo Wang, Gim Hee Lee, David Hsu

Core Contributions

Reframes social navigation from 'reach a given goal politely' to 'decide where the goal should be' — the robot must infer a joining pose from a group's formation and a language description, a problem prior crowd-navigation work sidesteps.
Recursive spectral partitioning proposes structured candidate subgroups before language ranking, turning an open-ended referring-expression problem into ranking a small set of plausible groups — key to its sub-second inference.
Outputs a multimodal energy–orientation map using human F-formation priors rather than a single pose, so the robot can pick among several socially valid slots (e.g. end of a queue vs. gap in a circle).
Directly relevant to guide-dog robots and autonomous mobility scooters, where the user says 'join my friends by the fountain' rather than giving coordinates; validated on real robots with groups that reconfigure mid-approach.

Show abstract

Social navigation typically assumes a specified goal and focuses on reaching it while respecting social conventions, whereas robot group joining requires predicting where to join based on the group's real-time activity and formation. This is a highly semantic task, yet an important capability for applications such as robotic guide dogs and autonomous mobility scooters. We formulate language-grounded robot group joining: given an observation and a natural-language description of a target group, the robot identifies the relevant group members and predicts socially compliant joining poses. For grounding, we generate structured candidate subsets through recursive spectral partitioning and rank them with a language-conditioned image--geometry model. Given the grounded group, a goal predictor leverages human-formation priors to produce a multimodal energy--orientation map over feasible robot poses. Experiments on conversations, queues, and audiences across varying group sizes, crowd densities, and visual ambiguities show that our method achieves competitive grounding accuracy with sub-second inference and outperforms all baselines in joining-pose prediction. Real-robot experiments further demonstrate group joining in both static and dynamically changing interactions.

#14 h=—

Multimodal Voice Activity Projection for Social Robot Mediation: Expected Behavior and Deployment Constraints

2026-09-23cs.RO👤 Antonio Cano (h n/a)

Antonio Cano, Guillermo Perez, Luis Merino, Randy Gomez

Core Contributions

Reframes turn-taking for a mediator robot: the right action is often not to speak but to orient, wait or prepare a balanced intervention — so perception must predict floor dynamics, not just detect speech.
Predicts future voice activity from synchronized audio-visual input and derives Hold, Shift, backchannel and overlap events zero-shot from those projections, rather than training a classifier per event.
Uses pretrained VA encoders with LoRA and inter-speaker attention; evaluated on NoXi, NoXi+J and Haru EDR, strongest for floor-management events that map to gaze and active listening.
Explicitly specifies the robot-facing output interface and deployment constraints (latency, sync, input-quality monitoring), bridging a research model to an actual social robot stack.

Show abstract

Turn-taking prediction is especially relevant for social robots that act as mediators in human-human interaction, where the expected action is often not to speak, but to orient, wait, avoid interruption, or prepare a balanced intervention. This paper presents Multimodal Voice Activity Projection (MM-VAP) as a human state-aware perception layer for future robot mediation behavior. The model estimates the future evolution of the conversational floor from synchronized audio-visual evidence and derives turn-taking events such as Hold, Shift, Shift prediction, Backchannel prediction, and overlap-related states. The approach uses VA-related pretrained audio-visual encoders, LoRA adaptation, inter-speaker attention, and zero-shot event inference from future voice activity projections. Experiments on NoXi, NoXi+J, and Haru EDR support the feasibility of this formulation, especially for floor management events that can be connected to gaze preparation, active listening, and conservative intervention. Finally, the paper defines the expected robot output interface and discusses the main deployment constraints, including real-time inference, preprocessing latency, multimodal synchronization, and input-quality monitoring.

#4 h=—

Context-Continuous Preference Learning for Exoskeleton Personalization

2026-09-23cs.LGcs.RO👤 Sunin Baek (h n/a)

Sunin Baek, Sungwoo Park, Daekyum Kim

Core Contributions

Asks an empirical question before building a method: do exoskeleton users' preference landscapes vary smoothly across walking speeds/loads? Reference data from nine adults say yes, for nearby conditions.
A Gaussian-process preference model shares comparisons across nearby contexts while keeping context-specific utilities — sitting between fully independent and fully pooled learning.
With five exposures per context, reconstruction correlation rises from 0.644→0.720 (ankle) and 0.476→0.526 (elbow), equivalent to ~37% and ~17% fewer queries than independent learning needs.
Honestly reports negative transfer when continuity is weak and that online human benefits remain unproven — a useful map of when context sharing is safe for personalization.

Show abstract

Personalizing exoskeleton assistance across operating conditions is constrained by the time and physical effort required to collect user feedback. We examined whether a user's preference landscape varies smoothly across operating conditions and when this continuity supports learning from limited feedback. We propose Context-Continuous Preference Learning (CCPL), a Gaussian-process preference model that shares observations across nearby contexts while retaining context-specific utility estimates. We evaluated CCPL through simulations and retrospective analyses of ankle and elbow exoskeleton preference data from nine healthy adults. In simulations, CCPL improved reconstruction and preference-based Bayesian optimization relative to independent learning when preferences varied smoothly, but showed negative transfer when continuity was weak. In both human studies, full-data reference landscapes estimated separately for each participant and context tended to be more similar between nearby operating conditions. With five exposures per context, CCPL increased mean reconstruction correlation with these references from 0.644 to 0.720 for ankle assistance and from 0.476 to 0.526 for elbow assistance relative to independent learning. The five-exposure budget was approximately 37% lower for ankle and 17% lower for elbow than the estimated independent-learning budgets needed to match these correlations. CCPL also improved held-out response prediction relative to independent learning, while benefits over pooled learning varied. These findings support context continuity as a basis for sharing preference observations under limited feedback, although benefits for online personalization in humans remain to be established.

Multi-Robot Systems & Swarms

#22 h=—

Controlling Collectives of AI Agents in Reasoning Space with Spatial Transformers

2026-09-23cs.ROcs.AI👤 Frederic Vatnsdal (h n/a)

Frederic Vatnsdal, Roshan Gopal, Romina Garcia Camargo, Vijay Kumar, Alejandro Ribeiro

Core Contributions

Shows LLM-driven multi-robot planning breaks down as teams grow, and proposes COMPASS: decentralized per-robot LLM agents closed-loop controlled in 'reasoning space'.
A spatial transformer aggregates multi-hop messages into a learned feedback token for each robot's LLM; hand-engineered feedback with raw state in the language channel destroys cohesion.
Finds structured diversity in input commands cancels individual LLM biases — a collective-intelligence effect that persists at scale.
Flies up to 1024 simulated robots under natural-language commands, 16× training scale, and generalizes zero-shot to ambiguous instructions; beats a centralized frontier-LLM policy.

Show abstract

Large Language Models (LLMs) introduce an exciting new paradigm for planning and navigation in robotics, but fail on even simple multi-robot tasks as team sizes grow. We propose COMPASS, a scalable, decentralized multi-robot architecture for controlling large collectives of agentic robots with reasoning space feedback control. Feedback is generated locally on each robot by a spatial transformer which aggregates multi-hop messages across the fleet into a learned feedback token. Our experiments find that collectives of language models demonstrate performance gains from structured diversity of the input command, which can cancel biases; an advantage that is held across scale. Compared against a centralized frontier LLM policy and a language-only communication ablation, we find that the coupled design of COMPASS decisively produces cohesive flocking formations that accurately fly the commanded intent. We show that reasoning feedback works best when composed with a compact learned token. Our ablations show that hand engineered feedback with raw state appearing in the language channel obliterates cohesion. COMPASS generalizes zero-shot to unseen instructions of ambiguous meaning while commanding flocks up to 16 times its training scale, flying up to 1024 robots under natural language commands.

#27 h=—

Dynamic, Decentralized Spatial Code Reuse for OCDMA LiDAR in Robot Swarms

2026-09-23cs.RO👤 Mohammad Hani Alomari (h n/a)

Mohammad Hani Alomari

Core Contributions

Tackles mutual interference among LiDAR-equipped robots using optical CDMA codes, where static assignment needs one code per robot (Θ(N)).
A decentralized protocol reassigns reusable codes from a beacon-maintained interference graph; proven to need only O(log N / log log N) codes under constant density.
Monte Carlo validation under mobility and imperfect detection shows the advantage over static assignment growing from ~2× at 15 robots to ~12× at 120.
Against a coordination-free baseline, achieves higher code reuse and 30–40% lower collision risk on the same code budget — arguing coordination, not reactivity, closes the scaling gap.

Show abstract

Robots in a LiDAR-equipped swarm mutually interfere when their optical ranging codes collide. Existing mitigations either assign codes statically -- requiring L=N distinguishable codes for N robots -- or react to detected interference without a scalable, coordinated assignment rule beneath them; prior work explicitly identifies the code-assignment scaling problem as unsolved. We propose a decentralized protocol in which robots dynamically reassign spatial reuse codes based on a live, beacon-maintained interference-neighborhood graph, and prove that the number of codes required grows as O(\log N/\log\log N) under constant robot density -- an unbounded improvement over the Θ(N) growth of static assignment. We validate this result under conditions substantially beyond the idealized proof -- robot mobility, imperfect beacon-based detection, and reactive reassignment -- via Monte Carlo simulation (30 seeds per condition, 95% confidence intervals): the advantage over static assignment widens from roughly 2\times at 15 robots to 12\times at 120. Against a structurally faithful, fairly constructed model of an existing coordination-free approach, our protocol achieves both substantially greater code-reuse efficiency and 30--40% lower collision risk under an identical, constrained code budget, demonstrating that coordination -- not merely reactivity -- is what closes the scaling gap.