🤖 Robotics arXiv Digest

August 24, 2026
30 papers 7 research areas Generated by Claude

Research Landscape

Today's batch is dominated by a single quiet argument playing out across a third of the papers: scaling data is not the only, or even the best, lever for generalization in robot learning. Physics Filtering Favors the Generalization of Robot Learning states this most directly, showing a lightweight physics-filtered feedback module lets quadrupeds, drones, and aerial manipulators generalize under dynamics uncertainty with limited training data. The seven VLA papers in the VLA & Foundation Models group make the same bet from different angles rather than pushing more demonstrations through bigger models: Act with Intent (INDI) distills behavior-level intent from a teacher VLM instead of only cloning motor commands (lifting GR00T-N1.7 from 64.3% to 84.7% on SimplerEnv-Bridge); UniMem unifies memory and control under one backbone rather than bolting on a second VLM; and Pointing-VLA and TOWN-VLA both attack brittle prompt/coordinate interfaces — the latter's discovery that meaningless text appends collapse success from 92.47% to 3.00% just as badly as meaningful ones is a striking demonstration that VLA failure often lives in interface design, not semantics.

A second thread is the move from raw pixel or text signals toward structured, geometry- or certification-aware intermediate representations. GeoWAM makes this case explicitly for driving world models, arguing point-cloud geometry is a more natural state space than pixels because it aligns directly with the space where actions execute; Pointing-VLA makes the analogous argument for manipulation, replacing text-serialized coordinates with typed geometric heads; and CSymPlan pushes structure furthest of all, offering formally certified reach-avoid policies for high-DOF manipulators with zero safety violations in both simulation and real Franka FR3 trials. This connects to a cluster of papers solving reliability under partial observability without dedicated sensing: RAFT adapts to thruster faults using a privileged critic that never needs a fault sensor at deployment, the Reward-Free Continual Adaptation paper adapts space robots to hardware degradation without ever observing a reward, and RACO treats UAV navigation's coarse goal as a correctable runtime hypothesis rather than ground truth — three different domains converging on the same design pattern of training-time privilege or structure that's discarded or hidden at deployment.

Elsewhere, the batch reveals a field increasingly willing to challenge its own assumptions: the Shaping the Evolutionary Dynamics of Robot Morphology paper argues the well-known morphological Baldwin effect is largely an artifact of premature fitness evaluation rather than genuine evolutionary tendency, and the Spinning Quadrotor paper inverts the standard assumption that yaw must always be suppressed in hover, turning it into a source of aerodynamic lift instead (22% thrust reduction in preliminary tests). Meanwhile RoboRacer Arena and MIVIFI both tackle the perennial data/asset bottleneck in simulation and generative modeling — one by generating racing environments directly from occupancy maps and natural language, the other by bridging fisheye and perspective camera domains to compensate for scarce fisheye training data — underscoring that infrastructure for scale, not just algorithms, remains a first-class research problem in robotics.

Papers by Research Area

VLA & Foundation Models for Manipulation

Fixing what scaling alone can't: memory, intent, prompt robustness, spatial grounding, and real-hardware deployment for VLA policies.

7 papers

  • #8 ROS2SmolVLA: Enabling Small Vision-Language-Action Models fo…
  • #17 Triplet2Track: A Hierarchical System with Object-Centric Rep…
  • #19 Act with Intent: Distilling Behavior Intent for Vision-Langu…
  • #22 InstructMove: A Text-Indispensable Benchmark for Instruction…
  • #24 UniMem: Unifying Multimodal Memory and Control for Vision-La…
  • #26 Think Only When Needed: Prompt-Authority Control for Selecti…
  • #27 Pointing-VLA: Typed Spatial Grounding Interfaces for Vision-…

Navigation & Spatial Perception

Self-supervised pathfinding, semantic SLAM, and hybrid VLM+geometric navigation for ground and aerial robots.

6 papers

  • #1 VikPath: A Vision Kansformer Framework for Effective Obstacl…
  • #5 Contextrast++: Robust Multi-Scale Contextual Contrastive Lea…
  • #10 OptiSight: Bridging Semantic Reasoning and Geometric Control…
  • #16 SuperMap: A Spatio-Temporal SLAM System for Visual-Language …
  • #20 Macro-Action Topological Navigation under Noisy Localization…
  • #30 RACO: Reliability-Aware Coarse-Goal Optimization for Inspect…

Autonomous Driving & Scene Understanding

World models, temporal memory, and multi-view generation for perception and planning in driving.

3 papers

  • #3 GeoWAM: Visual Geometry World Action Models for Autonomous D…
  • #9 MomADv2: Reliable Temporal Memory for End-to-End Autonomous …
  • #15 MIVIFI: Bridging Perspective and Fisheye Domains for Trainin…

Robot Learning, Control & Adaptation

Sample-efficient control, certified planning, and fault/degradation adaptation without full observability.

6 papers

  • #4 Guided Riemannian Optimization (GuRO): Bridging Model Predic…
  • #6 Reward-Free Continual Adaptation for Resilient Space Robots
  • #11 Physics Filtering Favors the Generalization of Robot Learnin…
  • #14 CSymPlan: Certified Symbolic Planning and Control for High-D…
  • #25 Privileged Critic Training Enables Sensor-Free Thruster Faul…
  • #28 Switched Turn-based Adaptive Source Seeking Strategy using E…

Hardware Design & Bio-inspired Systems

Novel sensing skins, aerodynamic hover strategies, and reproducible open-source manipulator builds.

3 papers

  • #7 Design of a Biomimetic Joint-Covering Skin with Tissue-Like …
  • #12 Spinning Quadrotor: Hover Thrust Augmentation with Passive L…
  • #29 Reproducible Vision-Guided 6-DoF Robotic Manipulator with a …

Simulation, Benchmarks & Evolutionary Design

Scalable environment generation, morphology-control co-design theory, and exact combinatorial packing analysis.

3 papers

  • #2 RoboRacer Arena: Scaling High-Fidelity Autonomous Racing in …
  • #13 Shaping the Evolutionary Dynamics of Robot Morphology via Ad…
  • #18 Exact Finite-Length Theory of Uniform Car Parking: Spatial L…

Social & Adaptive Human-Robot Interaction

Online motor plasticity and socially-aware nonverbal behavior generation for physical and virtual agents.

2 papers

  • #21 Free-Energy-Gated Plasticity for Real-Time Online Motor Lear…
  • #23 LLM-Based Selection of Incongruent Verbal and Nonverbal Beha…

Papers by Category

VLA & Foundation Models for Manipulation

7 papers
#8
h=12

ROS2SmolVLA: Enabling Small Vision-Language-Action Models for Integration into Industrial-Grade Lightweight Robots

2026-08-24cs.ROL. Mikelsons (h=12)
Nils Mandischer, Noah Böckmann, Ludwig Holl, Lars Mikelsons
Core Contributions
  • Targets the deployment gap left open by big VLA models: rather than scaling further, adapts Hugging Face's SmolVLA to run entirely on-premise on a Universal Robots UR10e, sidestepping the compliance/security concerns of cloud-hosted large VLAs in factory settings.
  • Ships as an open-source ROS 2 bridge (ROS2SmolVLA) rather than a new model architecture — the contribution is integration engineering that makes a small VLA usable on industrial-grade hardware instead of lab-only arms.
  • Validated on a real pick-and-place task on production-grade hardware, a step most VLA papers skip in favor of simulation or lab manipulators.
  • Positions small, locally-computable VLAs as the practical answer for short-batch, high-mix manufacturing lines where large cloud models are a non-starter.
Show abstract

Industrial demand changes the paradigms of production. Due to smaller batch sizes and more variations in products, companies face a growing challenge to adopt more adaptive production systems. In particular, robot-based automation is usually static and fails to respond to constantly changing processes. Vision-Language-Action (VLA) Models are a promising opportunity to mitigate this challenge by generating robot actions based on the observed system state. However, current research either focuses on large models that cannot be computed on premise, creating compliance and security challenges, or use lab-grade robot hardware that obscures exploitation in real industrial settings. In this work, we adapt Hugging Face's SmolVLA for Universal Robots lightweight robots. Further, we release the open-source repository ROS2SmolVLA that implements an interface for ROS 2 to SmolVLA, and makes it applicable for industrial-grade hardware. By this, we allow a lenient adoption into lab and industrial environments. We validate the functionality of SmolVLA for a Universal Robots UR10e using a pick-and-place task and give implementation guidelines. Our findings support that SmolVLA is a well-suited option for small-sized tasks that need to be computed on premise.

#17
h=5

Triplet2Track: A Hierarchical System with Object-Centric Representations for Reliable Long-Horizon Manipulation

2026-08-24cs.RO cs.AIChuan Wen (h=5)
Jianxiang Liu, Gaojing Zhang, Chuan Wen, Qipeng Liu, Yuxuan Zhao
Core Contributions
  • Splits the difference between end-to-end VLA and classical hierarchical planning: represents subgoals as instance-grounded triplets that get translated into continuous track priors, keeping the interpretability of a pipeline while staying closed-loop.
  • Learns from human videos rather than robot teleop data, cutting the dependency on expensive robot-collected demonstrations that plague end-to-end VLA training.
  • Adds online progress monitoring from observations to trigger replanning — directly targeting the open-loop hallucination failure mode that hierarchical pipelines are typically criticized for.
  • Reports 74.8% average success on diverse long-horizon real-world tasks with both object-level and compositional generalization, a regime where flat VLA policies typically degrade sharply.
Show abstract

Ensuring reliability in uncertain environments remains difficult for long-horizon robotic manipulation. End-to-end VLA models are data-heavy and opaque, making diagnosis and verification difficult. Hierarchical pipelines are more interpretable, but their plans are often weakly grounded in observations, weakly aligned with low-level actions, and computed without online feedback, leading to open-loop behavior and hallucinations. To address these issues, we introduce the Triplet-to-Track System (TTS), a closed-loop long-horizon imitation learning system that uses human videos to reduce reliance on robot-collected data. TTS represents high-level subgoals as instance-grounded triplets, translates them into continuous track priors for execution, and monitors task progress from observations for online replanning. Across diverse real-world long-horizon tasks, TTS achieves a 74.8% average success rate and supports object-level and compositional generalization.

#19
h=3

Act with Intent: Distilling Behavior Intent for Vision-Language-Action Models

2026-08-24cs.RO cs.AI cs.CVWook-Shin Han (h=3)
Sangoh Lee, Sangwoo Mo, Wook-Shin Han
Core Contributions
  • Reframes what action decoders should learn from: instead of only cloning the demonstrated motor command, distills the intent behind a behavior segment from a frozen teacher VLM watching the execution video — closer to 'why' than 'what.'
  • Differs from prior future-prediction supervision (frames, trajectories, motion) by targeting the shared semantic objective rather than one particular realization of what might happen next, making the signal less brittle to nuisance variation.
  • Lifts GR00T-N1.7 from 64.3% to 84.7% on SimplerEnv-Bridge and from 64.1% to 70.3% on RoboCasa Kitchen, with consistent gains transferring to π0.5 on both benchmarks — a large jump for what is essentially an auxiliary training signal.
  • On real hardware, improves average success from 62.0% to 68.7%, with the biggest gains (up to 12 points) concentrated on the longest-horizon tasks, suggesting intent modeling pays off most exactly where behavior cloning struggles most.
Show abstract

Vision-Language-Action (VLA) models can turn multimodal context into robot actions, but their action decoders are still trained largely by behavior cloning. This supervises which motor command was demonstrated while leaving implicit the local objective served by the behavior under the instruction. Future-based supervision enriches action learning with frames, latent observations, trajectories, or motion representations, but these signals capture particular realizations of what may happen rather than the shared semantic objective of the forthcoming behavior. We propose Intention Distillation (INDI), which distills behavior-level intent into the action decoder. During training, a frozen teacher VLM interprets a demonstrated segment from the current observation, instruction, coarse action summary, and corresponding execution video. From its standard inputs, the deployed VLA recovers the resulting multimodal intent representation at an intermediate decoder layer and uses it to organize action prediction together with representations of how the behavior unfolds and what it achieves. On SimplerEnv-Bridge, INDI improves GR00T-N1.7 from 64.3% to 84.7%, and on RoboCasa Kitchen it improves the controlled GR00T-N1.7 baseline from 64.1% to 70.3%, with consistent gains on pi-0.5 across both benchmarks. In real-world tasks, INDI improves average success from 62.0% to 68.7%, with gains of up to 12.0 pp on longer-horizon tasks. Further analyses show that the recovered latent is used by the decoder, captures behavior objective and execution progress, and organizes downstream predictions in an objective-dependent manner. These results show that action decoders benefit from explicitly modeling the semantic objective of the behavior they generate.

#22
h=3

InstructMove: A Text-Indispensable Benchmark for Instruction-Following Manipulation

2026-08-24cs.ROZhizhong Su (h=3)
Mengao Zhao, Ziang Li, Chaodong Huang, Mengchen Ma, Haoyi Jiang
Core Contributions
  • Diagnoses a hidden flaw in existing manipulation benchmarks: because the correct object/destination is often visually obvious or the only feasible choice, policies can score well while ignoring the language instruction entirely.
  • Introduces a 'text-indispensable' design principle — scenes are built with semantic distractors so multiple actions are physically plausible but only one matches the instruction, forcing genuine language grounding to be measured rather than assumed.
  • Decomposes instruction-following into category identification, attribute discrimination, spatial reasoning, and compositional pick-and-place, giving a diagnostic breakdown rather than a single pass/fail number.
  • Shows that training on InstructMove's simulation data transfers to improve real-world instruction-following, indicating the benchmark also functions as a data source, not just an eval.
Show abstract

Vision-language-action (VLA) models have made general-purpose robot manipulation increasingly plausible by conditioning robot actions on natural-language instructions. A key test of such generality is whether policies actually follow language instructions. Yet many manipulation benchmarks leave this ability underdetermined: the intended object or destination is often visually salient or uniquely feasible, allowing policies to succeed without grounding the instruction. We argue that instruction-following evaluation should be text-indispensable: multiple actions should be visually and physically plausible, while only one should be consistent with the language instruction. We introduce InstructMove, a text-indispensable benchmark for instruction-following manipulation. InstructMove instantiates this principle in pick-and-place scenes with semantic distractors, decomposing instruction following into category identification, attribute discrimination, spatial reasoning, and compositional pick-and-place. InstructMove supports a train-eval protocol with InstructMove training data and held-out evaluation tasks, with additional diagnostics for language dependence. Experiments with representative VLA policies show that InstructMove provides a controlled testbed for diagnosing visual shortcuts and that InstructMove simulation data can improve real-world instruction-following manipulation performance. Code: https://github.com/HorizonRobotics/RoboOrchardSim

#24
h=2

UniMem: Unifying Multimodal Memory and Control for Vision-Language-Action Models

2026-08-24cs.RO cs.CVLars W. Osterberg (h=2)
Lars Osterberg, Maggie Wang, Mac Schwager
Core Contributions
  • Attacks VLA memory with a single backbone instead of the common pattern of bolting a separate long-term-memory VLM onto the policy, removing the memory bottleneck and the fractured two-model training pipeline that approach creates.
  • Uses an event classifier to decide when to update memory and a keyframe encoder/caching scheme to keep dense spatial memory affordable at rollout time, rather than conditioning on frames sampled at arbitrary fixed intervals.
  • Outperforms fixed-interval frame sampling baselines by a wide margin (93.4% vs. 68.2% in simulation) and hierarchical memory baselines by an even larger one (80.0% vs. 43.5% on hardware), while running faster at inference.
  • Evaluated across nine tasks spanning both simulation and four hardware setups targeting sequential and spatial memory specifically, rather than folding memory evaluation into generic long-horizon benchmarks.
Show abstract

While Vision-Language-Action (VLA) models have leveraged internet-scale pretraining and task-focused finetuning to achieve strong performance on long-horizon tasks, they often struggle with non-Markovian tasks that require memory. Existing approaches to memory typically involve additional Vision-Language-Models (VLMs) for long-term memory management, introducing a memory bottleneck and a fractured training pipeline. Conditioning on multiple historical frames can provide the VLA with access to more descriptive features of past scenes, but can degrade performance if frames are chosen at arbitrary, fixed intervals. To address these limitations, we present UniMem, a framework that unifies high-level, multimodal memory and low-level control under one backbone. UniMem employs an event classifier for memory updates, a keyframe encoder for dense spatial memory, and a keyframe caching technique to minimize overhead during policy rollouts. We evaluate UniMem across five simulation and four hardware tasks targeting sequential and spatial memory, demonstrating that our unified, single-model system outperforms fixed-interval image sampling baselines (93.4% vs. 68.2%) in simulation and hierarchical baselines (80.0% vs. 43.5%) in hardware, while offering faster inference and a simple training pipeline for easy adoption. Project website: https://losterberg3.github.io/unimem-vla/

#26
h=0

Think Only When Needed: Prompt-Authority Control for Selective Slow-Path Intervention in Vision-Language-Action Manipulation

2026-08-24cs.RO cs.AI cs.CVZhiruo Zhou (h=0)
Zhiruo Zhou, Zelin Li, Xiwen Chen, Jiazhuo Li, Chenwei Wang
Core Contributions
  • Surfaces a sharp failure mode in retrieval-augmented VLA: appending retrieved text to the executed prompt collapses success from 92.47% to 3.00%, and critically, semantically meaningless appends of matched length fail just as badly — the form of the prompt, not its content, is what breaks the policy.
  • Names this 'prompt-form collapse' and responds with a prompt-authority interface that only allows a canonical, pre-authorized compact instruction to alter the policy input; anything else is rejected and the original base prompt is restored exactly.
  • Runs a large-scale audit (900 routes) to verify the contract holds: 525 routes correctly fall back to the base prompt with matching hashes, and all 375 authorized prompts preserve the task signature — a rare example of a VLA paper auditing its own safety mechanism at this granularity.
  • Improves LIBERO-Plus success from 69.5% to 73.1% and, on a physical PiPER arm with a frozen π0.5 checkpoint, raises success from 52.7% to 78.7% over 150 trials (p = 3.16×10⁻⁶) — without retraining the underlying policy at all.
Show abstract

Retrieval can efficiently and effectively augment a frozen vision--language--action (VLA) policy without retraining, yet retrieved text becomes a control intervention once it enters the executed prompt. In a matched audit, raw appended text reduces mean success from 92.47% to 3.00%, while meaningful and length-matched meaningless appends both fail on all 500 states. This result identifies prompt-form collapse: changing the instruction form, rather than adding useful semantics, can dominate execution. We introduce TOWN-VLA (Think Only When Needed), a prompt-authority interface that separates candidate generation from permission to alter the policy input. A fixed compatibility rule authorizes a canonical compact instruction; otherwise, the interface restores the original Base prompt exactly. Across 900 audited routes, every route follows this contract: 525 routes recover Base with matching hashes, and all 375 authorized prompts preserve the task signature. On a matched 4x7 LIBERO-Plus evaluation with 10,030 episodes per method, success rises from 69.5% to 73.1% (+362 episodes; 95% CI 1.89--5.45 points), improving on six perturbation axes and all four suites. On a physical PiPER arm with a frozen pi-0.5 checkpoint, success rises from 52.7% to 78.7% over 150 trials per method (p=3.16x10^-6). Prompt authority is enforceable for a frozen controller; oracle-free admission calibration is the next deployment target.

#27
h=0

Pointing-VLA: Typed Spatial Grounding Interfaces for Vision-Language-Action Manipulation

2026-08-24cs.RO cs.AI cs.CVXiwen Chen (h=0)
Xiwen Chen, Zelin Li, Zhiruo Zhou, Huiming Chen, Chenwei Wang
Core Contributions
  • Replaces the usual VLA spatial-grounding interfaces — autoregressive text coordinates or opaque action tokens — with typed hidden-state heads that predict normalized points, object-functional-grounding heatmaps, and trajectories directly, avoiding the brittleness of serializing geometry as text.
  • Assigns an explicit execution contract per manipulation stage (source-conditioned grounding for PICK, point prediction for PLACE) rather than using one generic spatial output for the whole task.
  • Achieves SOTA on Bridge/WidowX (72.9% average across four tasks) without dataset-specific finetuning, and the typed heads run 6.68–6.90× faster than Embodied-R1's text-decoding approach on a shared benchmark.
  • As a plug-in spatial guidance module for a π0.5 policy, raises real-robot autonomous success from 52.7% to 80.7% across three visual contexts, and separately cuts controller latency by over 20× when transferred to NORA-1.5.
Show abstract

Vision-language-action (VLA) models often expose spatial grounding through autoregressive text coordinates or opaque action tokens, creating brittle interfaces between multimodal reasoning and robot execution. We present Pointing-VLA, a typed hidden-state spatial readout built on Embodied-R1. Geometry-specific heads predict normalized points, object-functional grounding (OFG) heatmaps, and visual trajectories without serializing geometry as text. For the evaluated Bridge/WidowX and physical pick-place deployments, an explicit execution contract assigns PICK to source-conditioned OFG and PLACE to Pointing, providing direct stage-aligned spatial targets. Pointing-VLA achieves SOTA performance on Bridge/WidowX, averaging 72.9% across the evaluated four-task set without Bridge-specific finetuning under collision-enabled CuRobo execution. Pointing and OFG show complementary strengths across native and cross-dataset evaluations. The OFG/contact readout transfers to NORA-1.5, preserving or improving success while reducing recorded controller time by more than 20x; typed heads are also 6.68--6.90x faster than Embodied-R1 text decoding on a shared external suite. When integrated as spatial guidance for a pi-0.5 action policy, Pointing-VLA raises autonomous real-robot success from 52.7% to 80.7% across three visual contexts. These results establish typed spatial readouts as an efficient, inspectable interface between embodied reasoning and robot execution.

Navigation & Spatial Perception

6 papers
#1
h=40

VikPath: A Vision Kansformer Framework for Effective Obstacle Avoidance in Self-Supervised Pathfinding

2026-08-24cs.ROM. A. Faruque (h=40)
Junyao Wang, Yulin Xu, Mohammad Abdullah Al Faruque
Core Contributions
  • Moves pathfinding away from supervised imitation of conventional planners: its Vision Kansformer module learns obstacle-distribution representations self-supervised, so it isn't capped by the quality of hand-crafted heuristic labels the way prior learned planners are.
  • Explicitly optimizes for obstacle clearance and trajectory smoothness together, not just path length — addressing a gap where prior learned pathfinders produce technically short but hard-to-execute routes.
  • Adds a sharp-turn penalty specifically to discourage jagged, difficult-to-track paths, a concrete mechanism rather than a generic smoothness loss term.
  • Reports 3.28% greater obstacle clearance and, more strikingly, 87.07% lower inference latency than SOTA baselines — a latency reduction large enough to matter for real-time replanning.
Show abstract

Pathfinding is a fundamental problem in artificial intelligence and autonomous systems. Traditional heuristic-based algorithms, such as A*, rely on predefined heuristic functions to guide the search process. Although effective in structured environments, their search efficiency can degrade substantially in complex, obstacle-rich scenarios, where handcrafted heuristics may provide limited guidance. Recent studies have explored learning-based approaches to improve pathfinding efficiency; however, most existing methods rely on supervised learning and require labels generated by conventional planners or obtained through manual annotation. As a result, their performance is inherently influenced by the quality of the underlying supervision and may degrade when the labeling heuristics fail to capture complex environmental structures. Moreover, existing methods primarily optimize for path length while paying limited attention to obstacle clearance and trajectory smoothness, which can lead to paths that are difficult or unsafe to execute in real-world environments. To address these limitations, we propose $\Design$, a self-supervised pathfinding framework that jointly considers obstacle proximity and path smoothness. At its core, our novel Vision Kansformer module learns representations of obstacle distributions without relying on labeled trajectories, enabling the model to better adapt to complex environments. We further introduce a sharp-turn penalty to encourage smoother and more practically executable paths. Extensive experiments demonstrate that, compared with state-of-the-art (SOTA) approaches, achieves an average of 3.28% greater obstacle clearance and 87.07% lower inference latency while maintaining smooth path generation.

#5
h=20

Contextrast++: Robust Multi-Scale Contextual Contrastive Learning for Semantic Segmentation

2026-08-24cs.CV cs.ROHyungtae Lim (h=20)
Changki Sung, Hyungtae Lim, Wanhee Kim, Youngwoo Seo, Hyun Myung
Core Contributions
  • Extends contrastive learning for semantic segmentation to tackle two separate problems at once — multi-scale local/global context fusion and long-tailed class imbalance — rather than the single-axis focus of most contrastive segmentation work.
  • Adds an adaptive fusion module that dynamically balances local and global features per-pixel, feeding both a pixel-to-anchor loss and an anchor-to-anchor loss that draws on a class-balanced memory bank to specifically counter rare-class underrepresentation.
  • Introduces boundary-aware negative (BANE) sampling that mines hard negatives specifically from misclassified boundary regions, sharpening fine-grained segment edges rather than relying on random negative sampling.
  • Improves over prior contrastive-learning segmentation SOTA with zero additional inference-time compute, since all the added machinery (fusion, losses, sampling) is training-only.
Show abstract

Semantic segmentation has rapidly advanced with deep learning; however, challenges remain in effectively capturing local and global contexts as well as addressing the long-tailed distribution problem. To tackle these issues, we present Contextrast++, a robust contrastive learning method for semantic segmentation that improves multi-scale feature integration and mitigates class imbalance issues. Our method consists of two key components: 1) contextual contrastive learning (CCL) and 2) boundary-aware negative (BANE) sampling. CCL includes three subcomponents: adaptive fusion module, pixel-to-anchor (PA) loss, and anchor-to-anchor (AA) loss. The adaptive fusion module dynamically balances local and global feature integration, resulting in a more context-aware representation. While the PA loss leverages the fused multi-scale features to improve feature representation learning, the AA loss focuses on addressing the long-tailed distribution problem by utilizing a memory bank that stores a fixed number of class-balanced representative anchors. Meanwhile, BANE sampling enhances segmentation precision by selecting hard negatives from misclassified boundary regions, which refines fine-grained details during contrastive learning. As verified in extensive experiments using public datasets, we demonstrate that Contextrast++ substantially improves semantic segmentation performance over existing contrastive learning-based state-of-the-art approaches, while introducing no additional computational overhead during inference.

#10
h=10

OptiSight: Bridging Semantic Reasoning and Geometric Control for Embodied Navigation

2026-08-24cs.RO cs.CVJordi Sanchez-Riera (h=10)
Alperen Avan, Jordi Sanchez-Riera
Core Contributions
  • Splits the navigation problem cleanly along its two hard parts: uses a VLM only for high-level semantic reasoning at sparse decision points, and hands continuous control to deterministic visual servoing via camera-projection geometry — avoiding the cost of querying a VLM every step.
  • Uses Grounded-SAM for open-vocabulary target localization, meaning the system isn't restricted to a fixed object vocabulary the way earlier geometric navigation stacks were.
  • Skips dense mapping entirely by converting visual observations straight into navigation commands, cutting the memory and compute overhead of full SLAM-based approaches.
  • Runs zero-shot across diverse indoor scenarios, including obstacle avoidance and semantic ambiguity, within an 8 GB VRAM budget — a constraint that matters for onboard deployment rather than workstation-class GPUs.
Show abstract

Autonomous indoor navigation requires both semantic understanding and precise geometric control. We propose OptiSight, a hybrid framework that combines Vision-Language Model reasoning with deterministic visual servoing through a finite-state Chain-of-Thought architecture. Grounded-SAM localizes open-vocabulary targets, while camera projection geometry converts visual observations into navigation commands without requiring dense mapping. The VLM is queried only at key decision points, reducing computational overhead while geometric control handles continuous navigation. Experiments in AI Habitat demonstrate reliable zero-shot navigation across diverse indoor scenarios, including obstacle avoidance and semantic ambiguity, while operating within an 8 GB VRAM budget. The source code is available at https://github.com/avanalperen/OptiSight-Python-Multimodal-CoT-for-Visual-Reasoning.

#16
h=7

SuperMap: A Spatio-Temporal SLAM System for Visual-Language Navigation

2026-08-24cs.ROShibo Zhao (h=7)
Shibo Zhao, Guofei Chen, Honghao Zhu, Zhiheng Li, Changwei Yao
Core Contributions
  • Targets the specific failure of naively bolting foundation-model perception onto SLAM: open-vocabulary detections are intermittent and view-dependent, which causes identity drift and stale semantics if just dumped into a map over time.
  • Introduces a consistency-driven mapping engine with 3D-aware instance association/re-activation plus confidence-weighted existence-and-label updates, explicitly designed to prune outdated content as scenes change — a temporal-consistency problem largely unaddressed by prior open-vocabulary mapping work.
  • Produces a queryable 4D scene graph (not just a static point cloud with labels), enabling compositional VLM queries over objects and relations rather than simple nearest-neighbor semantic lookup.
  • Validated on both benchmarks and real robots in dynamic scenes with object appearance, disappearance, and relocation, and released fully open-source as a deployable baseline.
Show abstract

Robotic navigation in human environments requires a spatio-temporal semantic representation that can reconcile open-vocabulary perception with long-term environmental changes. While foundation models provide strong zero-shot recognition, their predictions are intermittent and view-dependent, and naively integrating them into mapping pipelines leads to identity drift and stale semantics over time. We present SuperMap, a 4D spatio-temporal mapping framework for language-guided navigation that integrates high-frequency geometric SLAM with asynchronous open-vocabulary perception. Our core contribution is a consistency-driven mapping engine that combines 3D-aware instance association/re-activation with a principled existence-and-label confidence update to maintain stable object identities and prune outdated map content under occlusions and scene changes. SuperMap produces a queryable 4D scene-graph representation that interfaces naturally with Vision-Language Models by supporting compositional queries over object semantics, relations. We demonstrate SuperMap on benchmarks and real robots, including dynamic scenes with appearance/disappearance and relocation, and provide ablations and runtime analysis. We release the full system as open-source to provide the community with a deployable baseline for open-vocabulary spatio-temporal mapping. Project website: superodometry.com/supermap.

#20
h=3

Macro-Action Topological Navigation under Noisy Localization using Reinforcement Learning

2026-08-24cs.LG cs.ROSimon Hakenes (h=3)
Simon Hakenes, Tobias Glasmachers
Core Contributions
  • Challenges the assumption that plain RL can't handle photorealistic pixel-based indoor navigation, by replacing the earlier version's reliance on ground-truth simulator pose with an onboard, object-centric pose estimate built from ORB features and a minimal EKF.
  • Deliberately trades full SLAM for local pose consistency — since the agent and nearby objects drift together, only a locally-consistent estimate is needed to follow a topological edge and visually home in on a target, a much lighter-weight approach argued to be closer to biological navigation.
  • Handles moving target positions across episodes, forcing genuine exploration rather than memorized routes.
  • Demonstrates the approach works purely from vision in the photorealistic Habitat simulator despite the injected pose drift and noisy executed motions.
Show abstract

Navigating large, photorealistic 3D apartments from raw pixels is widely considered infeasible for plain reinforcement learning. We build an agent that does it anyway, estimating its own pose from the camera alone. The agent has to reach several target objects in sequence, and their positions change between episodes, so it must explore to find them. It builds on our earlier object-centric topological controller, which still read the agent's true pose and its object detections from the simulator. Here we replace that true pose with an onboard, object-centric estimate. For each object we keep a bank of ORB features that, when the object is seen again, yield a rough pose measurement, which a minimal Extended Kalman Filter (EKF) fuses with a motion model. As on a real robot, the executed motions are noisy. The estimate drifts, but the agent and the nearby objects drift together, so a locally consistent pose is enough to follow each short edge and then home in visually on the target, which lets us replace full SLAM with a much smaller model, closer to how biological navigation appears to work. In the photorealistic Habitat simulator, the agent reaches its target objects from vision alone, with a pose that only needs to be locally consistent.

#30
h=0

RACO: Reliability-Aware Coarse-Goal Optimization for Inspection-Oriented UAV Vision-Language Navigation

2026-08-24cs.RO cs.AI cs.CVSen Wang (h=0)
Sen Wang, Yiming Sun, Jiaxuan He, Pengfei Zhu
Core Contributions
  • Identifies a specific weakness in coarse-to-fine UAV vision-language navigation: the coarse goal predicted before local refinement is usually trusted outright, so if it drifts toward a plausible-but-wrong region, the fine stage has no way to recover.
  • Introduces LG-UVI, a new inspection-oriented evaluation setting (built on CityNav/CityRefer) with hard distractors and type-aware inspection regions — designed specifically to expose false-positive confirmations, which standard goal-reaching UAV-VLN metrics don't penalize.
  • Treats the coarse goal as a runtime hypothesis to be checked and corrected using object-level candidate anchors, both before and at the coarse-to-fine handoff, rather than as a fixed waypoint.
  • Improves success rate over a reproduced HETT baseline by 9.53 and 7.98 percentage points on validation-unseen and test-unseen respectively, while also reducing false verification risk — the two metrics moving together suggests the reliability mechanism is addressing the right failure mode.
Show abstract

UAV vision-language navigation (UAV-VLN) is commonly evaluated as goal reaching, but inspection-oriented deployment requires the agent to stop within a valid inspection region and avoid falsely confirming visually or semantically similar distractors. This requirement exposes a key weakness in existing coarse-to-fine UAV-VLN policies: the coarse goal predicted before local refinement is often treated as reliable, although it may drift toward plausible but incorrect object regions and limit the ability of the local stage to recover. To systematically evaluate this problem, we introduce LG-UVI, an object-centric inspection evaluation setting derived from CityNav/CityRefer. LG-UVI extends standard UAV-VLN episodes with target objects, hard distractors, type-aware inspection regions, and diagnostics for inspection-region arrival and object-level confirmation. To address this inspection-oriented setting, we further propose RACO, a reliability-aware adaptive coarse-to-fine navigation framework. Instead of treating the predicted coarse goal as a fixed waypoint, RACO views it as a runtime hypothesis and uses object-level candidate anchors to check and correct coarse localization before Stage 1 and at the Stage 1-to-Stage 2 boundary. RACO also applies scale-adaptive terminal refinement to handle terminal near-miss cases using runtime-observable geometric and anchor-based evidence. Under a unified online evaluation protocol, RACO improves SR over the reproduced HETT baseline by 9.53 and 7.98 percentage points on validation-unseen and test-unseen, respectively. It also improves inspection-region arrival and reduces false verification risk, showing that coarse-goal reliability optimization is an effective complement to existing coarse-to-fine UAV-VLN policies.

Autonomous Driving & Scene Understanding

3 papers
#3
h=20

GeoWAM: Visual Geometry World Action Models for Autonomous Driving

2026-08-24cs.CV cs.ROBurhaneddin Yaman (h=20)
Yiren Lu, Xin Ye, Jiaming Liu, Jin Yao, Yi-chung Chen
Core Contributions
  • Makes the case that pixel-space world models for driving are solving a harder problem than necessary: pixels entangle geometry with appearance, texture, and lighting, forcing the model to infer 3D structure from 2D — GeoWAM instead pretrains directly on future point-cloud geometry.
  • Departs from the common WAM recipe of video-generation-backbone-plus-action-head by making future-geometry prediction the pretraining objective itself, arguing geometry aligns more directly with the space where driving actions actually happen.
  • Uses a geometry-conditioned action head to turn the learned geometric dynamics into ego-trajectory predictions, rather than bolting trajectory prediction onto pixel-space features after the fact.
  • Shows substantially stronger driving policies than image-based world models in both open-loop and closed-loop evaluation, positioning future-geometry prediction as a competing pretraining paradigm to video generation for driving.
Show abstract

World action models (WAMs) have recently gained increasing attention as a framework for jointly modeling scene evolution and ego actions in autonomous driving. Most existing WAMs learn scene dynamics in pixel space by combining a video-generation backbone for future-observation prediction with an action head for ego-trajectory prediction. Pixels, however, provide only an indirect representation of these dynamics: they entangle geometry and motion with appearance, texture, and illumination, forcing the model to infer three-dimensional transformations from two-dimensional observations. We argue that geometry, represented by point clouds, offers a more natural state space for driving because it explicitly captures spatial structure and the rigid and non-rigid transformations that govern scene evolution while directly aligning with the space in which driving actions are executed. Building on this insight, we introduce GeoWAM, a visual geometry world action model for autonomous driving. Rather than predicting future images, GeoWAM is pretrained to forecast future scene geometry, yielding representations that jointly encode spatial structure and temporal evolution. A geometry-conditioned action head then leverages these learned geometric dynamics to predict future ego trajectories. Extensive open-loop and closed-loop evaluations show that visual geometry world modeling yields substantially stronger driving policies than image-based alternatives, establishing future-geometry prediction as an effective pretraining objective for autonomous driving.

#9
h=10

MomADv2: Reliable Temporal Memory for End-to-End Autonomous Driving

2026-08-24cs.CV cs.ROCaiyan Jia (h=10)
Ziying Song, Shengkai Zhang, Lin Liu, Peiliang Wu, Lei Yang
Core Contributions
  • Targets a specific brittleness in temporal-memory-based driving planners: stored planning history can become actively misleading once the driving command changes, so the model needs to be selective about which memory to trust rather than just accumulating it.
  • Introduces a Selective State-Space Planning Memory Query Module that filters historical queries by both temporal continuity and command consistency — memory that no longer matches the current command gets suppressed rather than blended in.
  • Adds a Flow-Matching Trajectory Residual Refiner that learns a continuous correction field from the model's raw output toward the expert trajectory, reducing long-horizon drift while keeping the stability benefits of anchor-based planning.
  • Cuts average collision rate by 15.6% over the original MomAD under 6-second planning horizons, validated across NAVSIM, Bench2Drive (closed-loop), and nuScenes (open-loop) — a multi-benchmark validation that's relatively rare in this space.
Show abstract

Long-horizon planning is critical for safe autonomous driving in complex scenarios. Existing methods improve planning continuity with temporal memory, but such memory may become invalid and mislead decisions when the driving command changes. Thus, selectively leveraging useful history while suppressing command-inconsistent memory remains a key challenge. To address this issue, we propose MomADv2, a reliable state-space memory framework for long-horizon end-to-end autonomous driving. At its core, MomADv2 introduces a Selective State-Space Planning Memory Query Module, which filters historical planning queries based on temporal continuity and command consistency, selects planning modes relevant to the current command, and models the evolution of planning intentions through a selective state-space mechanism. To further alleviate local trajectory deviations and error accumulation in long-horizon planning, we design a Flow-Matching Trajectory Residual Refiner. It learns a continuous residual correction field from the refined planning output to the expert trajectory, enabling fine-grained trajectory refinement while preserving the stability of anchor-based planning. Extensive experiments on closed-loop NAVSIM and Bench2Drive, as well as open-loop nuScenes, demonstrate that MomADv2 improves long-horizon planning consistency and reduces the average collision rate by 15.6% over MomAD under 6-second planning.

#15
h=7

MIVIFI: Bridging Perspective and Fisheye Domains for Training Multi-View Fisheye Image Generation Models

2026-08-24cs.CV cs.RODaniel Cremers (h=7)
Matthias Neuwirth-Trapp, Begüm Altunbas, Jiayi Wang, Yan Xia, Maarten Bieshaar
Core Contributions
  • Opens up a largely unexplored generation problem: synthesizing multi-view fisheye imagery, which matters because fisheye rigs give full 360° autonomous-vehicle coverage with as few as two cameras but have almost no training data to draw on.
  • First tries adapting the existing SyntheOcc architecture directly to fisheye (SyntheOcc-FE), then shows its limits are fundamentally a data-scarcity problem, motivating a second, cross-domain approach.
  • MIVIFI proper bridges domains via Equirectangular Projections, combining KITTI-360 fisheye images with nuScenes multi-view standard images — using standard-camera data to compensate for the shortage of labeled fisheye data rather than requiring more fisheye captures.
  • Enables structural editing of scenes (adding/removing actors) and rendering novel weather/lighting conditions absent from existing fisheye datasets, useful specifically for generating corner cases without expensive 3D simulation.
Show abstract

Achieving 360 degree coverage is critical for the visual perception systems of autonomous vehicles. Fisheye cameras offer a cost-effective solution by enabling full surround coverage with as few as two sensors. However, existing multi-view fisheye datasets are limited, and synthesizing rare corner cases typically requires computationally expensive 3D simulations, hindering the training. While generative models have achieved significant success in standard perspective imagery, their application to wide-angle distortion remains unexplored. In this work, we formally introduce the novel problem of multi-view fisheye image generation conditioned on volumetric semantic representations and present two distinct methods. We first propose SyntheOcc-FE, which adapts the SyntheOcc architecture to fisheye data. While effective, this method is constrained by the scarcity of fisheye datasets, which limits its generalization. To overcome these limitations, we propose our second method, MIVIFI (multi-view fisheye), which leverages cross-domain learning with Equirectangular Projections. By bridging the gap between dataset domains using KITTI-360 fisheye images alongside nuScenes multi-view standard images, our approach enables high-fidelity manipulation of scene content. This framework enables the structural modification of semantic occupancy inputs to introduce or eliminate specific actors and facilitates the rendering of diverse meteorological conditions and illumination scenarios absent in the limited fisheye datasets. Quantitative and qualitative experiments demonstrate that our methods achieve robust photorealistic multi-view fisheye image generation and highlight the specific advantages of our cross-domain strategy for handling data scarcity.

Robot Learning, Control & Adaptation

6 papers
#4
h=20

Guided Riemannian Optimization (GuRO): Bridging Model Predictive Control and Decision Transformers

2026-08-24cs.ROSatya Dash (h=20)
Hossein Abdi, Satya Prakash Dash, Mingfei Sun
Core Contributions
  • Bridges MPC and decision-transformer-style sequence modeling by using MPC's locally-optimal trajectory predictions to guide the transformer's training, removing the need for the extensive offline pretraining decision transformers usually require.
  • Addresses the notoriously non-convex loss landscape of transformer-based decision-making by training the policy in a Riemannian parameter space with a curvature-aware optimizer, rather than relying on standard Euclidean optimizers that converge slowly on this class of problem.
  • Targets high-dimensional quadruped control specifically, a regime where both pure MPC (model-inaccuracy-limited) and pure RL (sample-inefficient) struggle individually.
  • Outperforms TRPO, SAC, and Online Decision Transformer on both final returns and convergence speed, suggesting the MPC-guidance-plus-Riemannian-optimization combination addresses both baselines' weaknesses at once.
Show abstract

Decision-making in high-dimensional, nonlinear systems remains a central challenge in robotics. While model-based methods like Model Predictive Control (MPC) offer sample efficiency and interpretability, their performance degrades when the dynamics model is inaccurate or long-horizon predictions are required. Conversely, model-free reinforcement learning (RL) learns policies directly from interaction but suffers from high sample complexity and unstable optimization. Recent advances in sequence modeling have inspired transformer-based decision-making frameworks that can unify MPC and RL, but their training typically faces significant optimization challenges due to highly non-convex loss landscapes. In this work, we propose a novel framework that integrates MPC with RL in a sequence decision-making framework and leverages a curvature-aware optimization to efficiently tackle non-convex loss landscapes. MPC provides predictions of locally optimal trajectories that guide the decision transformer, removing the need for extensive offline pretraining. To address the slow and unstable convergence of traditional optimizers, we train the policy in a Riemannian parameter space using an efficient Riemannian (curvature-aware) method, leading to faster and more robust optimization. We evaluate our framework on high-dimensional quadruped control tasks and demonstrate consistent improvements over strong baselines, including TRPO, SAC, and Online Decision Transformer, achieving higher returns and faster convergence.

#6
h=18

Reward-Free Continual Adaptation for Resilient Space Robots

2026-08-24cs.RO cs.AI cs.LGM. Olivares-Mendez (h=18)
Andrej Orsula, Miguel Olivares-Mendez, Carol Martinez
Core Contributions
  • Tackles a deployment reality that most continual-RL work sidesteps: space robots can't get a reward signal in the field at all, since there's no external tracking system to compute one, yet still need to adapt when hardware degrades.
  • Pretrains a latent-state world model across diverse simulations so the reward structure is learned in advance, then at deployment freezes the observation encoder and reward predictor and updates only the transition dynamics — adapting to new dynamics without ever seeing a new reward.
  • Trains the policy entirely on imagined rollouts from the updated world model rather than requiring real interaction data post-degradation, important when real trials on damaged space hardware are risky or unrepeatable.
  • Validated across three distinct domains — planetary traversal, orbital navigation, and precision assembly — each subjected to severe morphological failure, showing the reward-free adaptation mechanism isn't tuned to one task type.
Show abstract

Space robots operate in extreme environments where hardware degradation can critically compromise traditional control strategies. While continual reinforcement learning offers a promising mechanism for online adaptation, it inherently requires access to a reward signal during deployment. However, precise reward computation in space is often infeasible due to the lack of external tracking systems and the overall complexity of the environment. To address the challenge of unobservable rewards, we introduce a reward-free continual learning framework that leverages latent-state world models. By pre-training a model-based agent across diverse simulations, the world model learns a robust predictor of the reward structure within its latent space. Upon deployment to an environment with severe hardware degradation, we freeze the observation encoder and reward predictor to update only the transition dynamics of the world model through unsupervised rollouts. By training the policy entirely on imagined trajectories generated by this updated world model, the agent adapts to altered dynamics without receiving new rewards. We demonstrate our approach across simulated planetary traversal, orbital navigation, and precision assembly tasks subjected to severe morphological failures.

#11
h=9

Physics Filtering Favors the Generalization of Robot Learning

2026-08-24cs.ROJindou Jia (h=9)
Jindou Jia, Shixuan Han, Meng Wang, Gen Li, Zihan Yang
Core Contributions
  • Pushes back on the scaling-data-to-generalize trend in robot learning by showing that a physics-informed feedback correction can substitute for the massive datasets that are impractical to collect on real robots.
  • PhyFilter is lightweight and model-agnostic, correcting policy outputs using physics-filtered learning residuals whose parameters are tuned by an auto-learning algorithm — no manual tuning per robot, unlike many physics-in-the-loop control schemes.
  • Validated across four genuinely different embodiments (quadrupeds, drones, aerial manipulators, acceleration differentiators), each facing a distinct uncertainty type — terrain, wind, mass, distribution shift — rather than one platform under one disturbance.
  • Reports centimeter-level in-air capture for aerial manipulators despite wind and mass uncertainty, a precision benchmark that's demanding relative to typical drone-disturbance-robustness papers.
Show abstract

Living organisms exhibit extraordinary adaptability to unseen environments through their intrinsic physical structures and lifelong feedback-driven learning. Endowing robots with comparable generalization is critical for reliable operation in the real world. While recent approaches attempt to improve generalization by scaling training data, such strategies remain impractical for robotics, where collecting real-world demonstrations at the scale of large language models is prohibitively costly and slow. Contrary to this reliance on massive datasets, we show that robots can generalize effectively under dynamics uncertainties even with limited training data by leveraging a feedback mechanism, namely PhyFilter, that corrects learning outputs with physics-filtered learning residuals. PhyFilter operates as a lightweight, model-agnostic module whose parameters can be automatically optimized through an auto-learning algorithm, eliminating manual tuning and enabling seamless integration with diverse robot policies. We validate PhyFilter across four representative robotic systems, demonstrating that it enables quadruped robots to generalize to unseen terrains, payload variations, and speed ranges; drones to flight under unseen wind disturbances; aerial manipulators to achieve centimeter-level in-air capture despite wind and mass uncertainties; and acceleration differentiators to remain robust with distribution shift. These results show that physics-filtered feedback can serve as a powerful alternative to massive data scaling.

#14
h=8

CSymPlan: Certified Symbolic Planning and Control for High-DOF Manipulators

2026-08-24cs.RO eess.SYMahmoud Khaled (h=8)
Aditya Narendra, Ashok Kumar Saini, Mahathi Anand, Mahmoud Khaled, Fares J. Abu-Dakka
Core Contributions
  • Addresses a specific failure mode of the standard plan-then-track manipulator stack: references from a collision-free planner can be nearly impossible to track exactly under actuator limits and model mismatch, especially near tight obstacle clearances.
  • Offers two complementary implementations — an offline one that precomputes certified reach-avoid policies for known workspaces, and an online one that synthesizes/updates symbolic policies at runtime as perception changes — covering both static and dynamic deployment needs with the same underlying abstraction.
  • Reduces manipulator dynamics to a sampled perturbed double-integrator via feedback linearization and treats torque, model, and measurement errors as bounded disturbances, giving formal certification rather than empirical robustness claims.
  • Reports zero safety violations across randomized simulated benchmarks and real Franka FR3 experiments, with an explicit fallback: when no certified action exists, the robot holds or replans instead of executing an uncertified command — a safety property most learned planners can't offer.
Show abstract

Robot manipulators are commonly engineered around a decoupled motion-generation stack: a planner computes a collision-free path and a lower-level controller tracks the resulting reference. This separation is computationally convenient, but it can produce references that are difficult to execute under actuator limits, tracking error, model mismatch, and small obstacle clearances. We present CSymPlan, a certified symbolic planning and control framework for high-DOF manipulators with two complementary implementations: an offline implementation that precomputes certified reach-avoid feedback policies for known workspaces; and an online implementation that synthesizes or updates symbolic policies at runtime from changing task and perception information using parallelization. The offline implementation reduces the manipulator dynamics to a sampled perturbed double-integrator model in operational space through feedback linearization, treats torque-realization errors, modeling inaccuracies, and measurement uncertainty as bounded disturbances, and refines the synthesized symbolic policy to the Franka FR3 through a quantization--lookup--torque realization pipeline. The online implementation uses the same abstraction and refinement interface, but replaces the precomputed policy table with a runtime pFaces request--synthesis--execution loop. In randomized simulated benchmarks and perception-driven Franka FR3 experiments, both implementations complete reach-avoid tasks with zero safety violations; whenever no certified action exists, the robot holds, replans, or stops safely instead of executing an uncertified command.

#25
h=1

Privileged Critic Training Enables Sensor-Free Thruster Fault Adaptation in End-to-End RL

2026-08-24cs.RORicard M. Castan (h=1)
Ricard Marsal I Castan, Miguel A. Olivares-Méndez
Core Contributions
  • Solves sensor-free fault adaptation using an asymmetric information trick: the PPO critic sees the true thruster degradation state during training while the actor only ever sees standard task observations, so the deployed policy compensates for failures it can't directly detect.
  • Targets a realistic fault model — thrusters that degrade continuously, fail dead, or jam stuck-open — rather than the simpler binary fail/no-fail assumption common in fault-tolerant control literature.
  • Adds recurrent memory so the policy can infer degradation from how the system responds over time, rather than requiring an instantaneous fault signal.
  • On an 8-thruster floating platform under up to four simultaneous failures, RAFT achieves 70.2% success at four concurrent failures — closing 84% of the gap between a failure-naive baseline (4.8%) and an oracle with full degradation-state access (82.4%) — a concrete measure of how much sensor-free adaptation recovers.
Show abstract

Fault-tolerant navigation for thruster-actuated robots requires online adaptation to failures that are neither binary nor fully observable: thrusters may degrade continuously, fail dead, or jam stuck-open. Classical fault detection pipelines require dedicated sensors unavailable at deployment; oracle controllers that observe the true failure state are equally impractical. We show that privileged critic training is sufficient for sensor-free fault adaptation: giving the PPO value function access to the true degradation state dgt during training, while the actor receives only standard task observations, shapes a policy that compensates for failures at deployment without any dedicated fault sensing. We propose RAFT (Recurrent Asymmetric Fault Tolerant), a policy with recurrent memory trained with a privileged asymmetric critic. Evaluated on a floating-platform robot (8 thrusters, 1 reaction wheel) under up to four simultaneous thruster failures, RAFT achieves 70.2% success at four concurrent failures, closing 84% of the gap from a failure-naive baseline (4.8%) to an oracle policy that sees the full degradation state at deployment (82.4%). All code, checkpoints, and data are open-source.

#28
h=0

Switched Turn-based Adaptive Source Seeking Strategy using Estimation and Information-driven Direction of Improvement

2026-08-24cs.RO eess.SYShubhra Banerjee (h=0)
Shubhra Banerjee, Satadal Ghosh
Core Contributions
  • Couples estimation and motion explicitly for source seeking, arguing prior loop-based geometric strategies leave estimation uncertainty out of the direction-selection decision even though the two are fundamentally linked — better trajectories yield better measurements and vice versa.
  • Combines an Extended Kalman Filter for the source estimate with Fisher Information Matrix-based direction selection, so heading changes at loop boundaries are driven by both current uncertainty and predicted information gain rather than by geometry alone.
  • Uses a measurement-based stopping condition to detect convergence without needing prior knowledge of the source location, useful for genuinely unknown-field scenarios like gas leaks or radiation sources.
  • Shows improved tracking and reduced estimation error over both purely information-driven and purely estimate-driven baselines, under both stationary and moving source scenarios.
Show abstract

Source seeking arises in applications such as gas leak localization, radiation monitoring, and environmental surveillance, where the origin of an unknown signal field must be estimated from spatial measurements. In practice, the source location is not directly observable and must be inferred from noisy scalar measurements collected during motion. In robotic source seeking, estimation and motion are closely linked: measurements improve the source estimate, while the chosen trajectory affects the quality of future measurements. Existing loop-based geometric strategies generate feasible motion but do not explicitly use estimation uncertainty to regulate direction updates. This paper presents a loop-based source-seeking framework that combines Extended Kalman Filter (EKF) estimation with Fisher Information Matrix (FIM)-based direction selection. The source estimate is updated during motion, and the heading is changed at loop boundaries using both estimation uncertainty and predicted information gain. A measurement-based stopping condition is used to detect convergence without requiring prior knowledge of the source location. Simulation results under stationary and moving source scenarios demonstrate improved tracking performance and reduced estimation error compared to purely information-driven or estimate-driven strategies.

Hardware Design & Bio-inspired Systems

3 papers
#7
h=17

Design of a Biomimetic Joint-Covering Skin with Tissue-Like Structure to Enhance Proprioception in a Musculoskeletal Humanoid

2026-08-24cs.ROKento Kawaharazuka (h=17)
Akihiro Miki, Shun Hasegawa, Yoshimoto Ribayashi, Kento Kawaharazuka, Kei Okada
Core Contributions
  • Targets an underexplored proprioceptive channel in musculoskeletal humanoids: most systems infer joint state from muscle sensing alone, leaving out the cutaneous deformation signal that biological mechanoreceptors provide around joints.
  • Designs a tissue-like layered skin combining pressure- and stretch-sensitive elements specifically shaped to the joint-covering geometry, rather than a generic flat tactile patch.
  • Demonstrated on the Musashi-W musculoskeletal humanoid, the skin alone estimates joint angle with about 3 degrees average error, and fusing it with existing muscle sensing further improves accuracy — showing the two modalities are complementary, not redundant.
  • Argues the joint-covering structure may also mechanically shield muscles from external disturbances, suggesting a dual mechanical-plus-sensing benefit beyond pure proprioception.
Show abstract

Proprioception in musculoskeletal humanoids is typically estimated primarily from muscle sensing, while the role of cutaneous deformation around joints remains insufficiently explored. In biological systems, mechanoreceptors distributed within soft tissue complement muscle feedback and support reliable joint state estimation. This study presents the design of a biomimetic joint-covering skin with a tissue-like layered structure that integrates pressure- and stretch-sensitive elements within the joint-covering tissue. The proposed skin is implemented on the musculoskeletal humanoid Musashi-W, and its independent proprioceptive capability as well as its integration with muscle sensing are evaluated. Experimental results show that the proposed skin alone achieves joint angle estimation with an average error of approximately 3 degrees. Furthermore, integration with muscle sensing improves estimation accuracy. Owing to its joint-covering structure, the skin may mechanically mitigate the influence of external disturbances on the muscles, and the integration of multiple modalities suggests the possibility of contributing to the identification of external stimuli that are difficult to interpret using muscle sensing alone. This work presents a design methodology for biomimetic joint-covering skin and demonstrates that such tissue-structured skin can serve as an effective approach for extending proprioceptive systems in musculoskeletal humanoids.

#12
h=8

Spinning Quadrotor: Hover Thrust Augmentation with Passive Lifting Surfaces

2026-08-24cs.ROHarikumar Kandath (h=8)
Aniketh Parkala, Harikumar Kandath
Core Contributions
  • Inverts a standard design assumption in multirotor flight: instead of actively suppressing yaw during hover (which costs power for no force-balance benefit), it intentionally sustains a yaw rate and converts that rotation into useful aerodynamic lift via passive lifting surfaces.
  • Requires new airfoil selection for the low-Reynolds-number regime the spinning blades operate in, since standard hover aerodynamics don't directly transfer.
  • Preliminary hardware tests show a 22% reduction in required thrust — a substantial efficiency gain if it holds up at larger scale, since yaw suppression is normally treated as a fixed hover cost.
  • Reframes yaw rotation from a disturbance to be cancelled into a design variable to be exploited, which could inform future efficient-hover multirotor designs beyond this specific implementation.
Show abstract

Conventional multirotor aerial vehicles actively suppress yaw rotation during hover, expending power to maintain a fixed heading despite the fact that yaw regulation is not required for force balance or altitude control. This paper challenges that paradigm by proposing a spinning quadrotor architecture that intentionally operates at a sustained yaw rate, converting power traditionally spent on yaw regulation into useful aerodynamic effects. A dynamic model of the spinning quadrotor is developed, analysis for low Re range is conducted to choose an airfoil for lifting surfaces. Preliminary hardware tests show a 22% reduction in thrust required. These findings suggest that intentional yaw rotation, rather than being suppressed, can be exploited as a design mechanism for efficient and robust multirotor flight.

#29
h=0

Reproducible Vision-Guided 6-DoF Robotic Manipulator with a Mixed Stepper-Driver Architecture and Browser-Native Control

2026-08-24cs.RO cs.CV eess.SYLasan Perera (h=0)
Lasan Perera, Deneth Priyadarshana, Dulana Pitiwaduge, Isitha Dinujaya, Mokshan Colambage
Core Contributions
  • Documents a full reproducible build of an open, low-cost 6-DOF manipulator by an undergraduate team, explicitly including the debugging experience needed to reproduce it — a rare level of build-process transparency for a hardware paper.
  • Mixes two stepper-driver strategies on a single STM32H743 controller (push-pull for onboard TMC2209 wrist drivers, open-drain for external CL57T/DM542 proximal-joint drivers), a hybrid electronics approach chosen to balance cost and torque needs across joint types.
  • Pairs a MATLAB/Simscape inverse-kinematics pipeline with a browser-native control interface via the Web Serial API, avoiding the need for a dedicated desktop control application.
  • Actuates all six axes under coordinated control at 2 kHz with a lightweight vision pipeline for autonomous pick-and-place, and releases all design files and firmware openly for reproduction.
Show abstract

We present the NeuralNexus Arm, an open, low-cost 6-DOF robotic manipulator built by an undergraduate engineering team, together with the design decisions and debugging experience needed to reproduce it. The arm is driven by a single STM32H743 microcontroller on a custom printed circuit board (PCB) and combines two stepper-driver strategies on one controller: push-pull 3.3V step/direction outputs for onboard TMC2209 drivers on the three wrist joints, and open-drain outputs for external CL57T and DM542 drivers on the three high-torque proximal joints. We describe the mechanical design, mixed-driver electronics, interrupt-driven firmware, a MATLAB/Simscape-based inverse-kinematics pipeline, a browser-native control interface using the Web Serial API, and a lightweight vision pipeline for object localisation and autonomous pick-and-place tasks. We also document non-obvious hardware and firmware failure modes encountered during the transition from a development board to the custom PCB as reproducibility guidance. All design files and firmware are released openly. The platform actuates all six axes under coordinated control at a 2 kHz update rate and executes both manual and pre-recorded motions from the browser interface.

Simulation, Benchmarks & Evolutionary Design

3 papers
#2
h=39

RoboRacer Arena: Scaling High-Fidelity Autonomous Racing in Isaac Sim

2026-08-24cs.ROE. Bartocci (h=39)
Mihaela-Larisa Clement, Agnes Poks, Ezio Bartocci
Core Contributions
  • Solves a scalability bottleneck in racing-policy research: occupancy-grid simulators are easy to extend with new maps but lack physical contact, while full 3D simulators need each track hand-built as a separate asset — RoboRacer Arena generates 3D Isaac Sim environments directly from occupancy maps.
  • Extracts drivable corridors and track boundaries via flood fill, then builds barriers from a computed distance field and assembles the whole environment as a USD stage automatically, rather than requiring manual asset creation per track.
  • Supports natural-language track generation using Gemma to produce a track specification with no explicit coordinates, then validates it through geometric screening and raster-level checks — letting users describe a track in words rather than draw it.
  • Currently ships 130 tracks and hits 8,707 vehicle-steps/second with 256 parallel rigid-body vehicles, with environment initialization scaling linearly with raster size (1.18–2.48s) — throughput numbers aimed squarely at large-scale RL training use.
Show abstract

RoboRacer offers a standardized platform for research using 1:10-scale autonomous vehicles, but the variety of available tracks hinders the process of acquiring policies. Although existing occupancy-grid simulators allow for the quick addition of new maps, they fail to include physical contact, while 3D simulators require each circuit to be implemented as a separate asset, thus limiting their scalability. In order to overcome this issue, we have developed RoboRacer Arena, a system that creates 3D racing environments directly from occupancy maps. Our method starts by using a flood fill algorithm to extract the drivable corridors and to identify the track boundaries, which are then used to establish the barriers. A distance field is calculated to define the collision boundaries. The track surfaces, collision properties, and materials are assembled into a USD stage, which allows for the automated and reproducible generation of the environment in Isaac Sim. The input maps can be obtained from SLAM sessions, from rescaled Formula 1 circuits, or from natural-language descriptions. When the input is based on natural language, we use Gemma 4 31B to generate a track specification without specifying any coordinates or geometry. To guarantee consistency and reproducibility, we apply geometric screening, procedural generation, and raster-level validation. The simulation environments are initialized in a time range of 1.18 to 2.48 seconds, with the initialization time increasing linearly as the raster size increases. In 30 matched trials involving 10 tracks and 3 seeds, 21 maps were generated and all passed validation. RoboRacer Arena currently contains 130 tracks and supports the generation of tracks from natural language. In benchmark tests, the system attains 8,707 vehicle-steps per second when using 256 parallel rigid-body vehicles, excluding the time taken for rendering and policy execution.

#13
h=8

Shaping the Evolutionary Dynamics of Robot Morphology via Adaptive Control Learning

2026-08-24cs.RO cs.AIYing Wen (h=8)
Junru Song, Yang Yang, Yaqing Xu, Ying Wen, Wei Peng
Core Contributions
  • Separates two things that co-design research usually conflates: how fast a morphology's controller converges (morphological intelligence) versus that morphology's actual performance ceiling (true potential) — and shows these are orthogonal, not the same axis.
  • Argues a well-known result in the field — the morphological Baldwin effect — is actually an artifact of premature fitness evaluation biasing selection toward fast learners, rather than a genuine evolutionary tendency, a fairly direct challenge to prior interpretation.
  • AdaControl monitors this selection bias during evolution and allocates just enough control-learning budget per morphology for an unbiased fitness estimate, instead of using a fixed learning budget for every candidate.
  • Matches state-of-the-art generative-model-based co-design methods on design diversity and performance while cutting computation by up to 80% versus exhaustive control learning, on simulated voxel-based soft robots.
Show abstract

Robot co-design via bi-level optimization couples within-lifetime controller learning for fitness evaluation with cross-generational morphological evolution. Prior work has established that well-adapted morphology facilitates faster control learning, a property termed morphological intelligence. Yet how control learning reciprocally shapes morphological evolution remains unexplored. This paper examines both directions for a holistic account of brain-body interplay. We first show that morphological contributions to control learning decouple into two orthogonal dimensions. We formalize the convergence speed as morphological intelligence and identify the performance ceiling as a complementary quantity termed true potential. A concise functional relation is then established to jointly characterize both quantities from individual learning curves, which, when aggregated at the population level, capture evolutionary profiles. Through extensive experiments on simulated voxel-based soft robots, we reveal that premature fitness evaluation systematically underestimates true potential and biases selection towards fast learners. This restricts design space exploration, compromising both optimization efficiency and morphological diversity. Notably, the widely recognized morphological Baldwin effect emerges as an artifact of this bias rather than a general evolutionary tendency. We therefore propose AdaControl, which monitors disproportionate selection for morphological intelligence during evolution and allocates minimally sufficient control learning for unbiased fitness evaluation. With AdaControl, a simple genetic algorithm rivals state-of-the-art generative-model-based co-design methods in discovering diverse high-performing designs while cutting computation by up to 80% versus exhaustive control.

#18
h=5

Exact Finite-Length Theory of Uniform Car Parking: Spatial Laws, Absorption, and Aggregation

2026-08-24cs.RO cs.DS cs.SC math.PR math.RAGanesh P. Kumar (h=5)
Ganesh P Kumar
Core Contributions
  • Develops exact (not asymptotic) finite-length theory for the classic 1D random sequential adsorption 'car parking' process, where prior work has largely focused on infinite-length limiting behavior.
  • Resolves the joint density of parked-car positions into 'jamming cells' that are each rational functions, computed via a subset recursion in O(2ⁿn) — an explicit, exact algorithm rather than a Monte Carlo estimate.
  • Expresses marginal and gap order statistics as hyperlogarithms whose weight is determined by how many coordinates get integrated out, connecting a robotics-adjacent packing problem to techniques from exact integration theory.
  • Provides closed-form treatment of absorption counts and aggregate quantities via the classical Rényi integral equation, giving a fully analytic finite-size theory useful anywhere sequential non-overlapping placement matters (e.g., parking, sensor placement, resource allocation).
Show abstract

The uniform car-parking process is the one-dimensional random sequential adsorption of unit cars on a segment of finite length s: cars arrive at uniformly random positions and park wherever they fit, until no gap admits another. This paper develops the exact finite-s theory. The joint density of the parked positions is resolved into jamming cells, on each of which it is a rational function, and evaluated by a subset recursion in O(2^n n) operations; the marginal and gap order statistics are obtained as hyperlogarithms whose weight is fixed by the number of coordinates integrated out; and the absorption count and the aggregate quantities are treated through the integral equation descending from Rényi.

Social & Adaptive Human-Robot Interaction

2 papers
#21
h=3

Free-Energy-Gated Plasticity for Real-Time Online Motor Learning in Physical Human--Robot Interaction

2026-08-24cs.RO cs.HCJun Tani (h=3)
Hiroki Sawada, Jun Tani
Core Contributions
  • Extends a Predictive-Coding-inspired recurrent network (PV-RNN) to keep learning online during physical interaction, rather than the usual train-then-deploy split, tackling the core continual-learning tension of acquiring new behavior without erasing old behavior.
  • Gates the effective learning rate by variational free energy itself, so plasticity is allocated according to model-environment mismatch rather than a fixed or manually scheduled learning rate.
  • A randomly initialized network learned three cyclic motor patterns purely from live human-robot interaction with no offline pretraining, replay buffer, or explicit task-boundary signal — a demanding test of true online learning.
  • Controlled experiments show neither a constant learning rate matched to FEGP's time-averaged rate, nor replay with disrupted temporal structure, reproduce its retention benefits — indicating it's specifically the timing of plasticity relative to mismatch, not just its average magnitude, that matters.
Show abstract

Fully online embodied learning requires synaptic adaptation to acquire new behaviors while preserving previously learned dynamics during ongoing interaction. We extend the Predictive-Coding-inspired Variational Recurrent Neural Network (PV-RNN) to continuously adapt its synaptic weights and propose Free-Energy-Gated Plasticity (FEGP), which regulates the effective learning rate according to variational free energy. In real-time physical human--robot interaction, a randomly initialized network acquired three cyclic motor patterns without offline pretraining, replay, or task-boundary signals, with all three patterns emerging in autonomous rollouts. Controlled experiments over ten randomized teaching streams and five network initializations per stream showed that FEGP substantially improved repertoire coverage and retention of previously acquired patterns after they left the recent observation window. Neither a constant learning rate matched to the gate's time-averaged effective rate nor replay of the same gain values with disrupted temporal organization reproduced these improvements. These results indicate that the temporal allocation of plasticity relative to model--environment mismatch, rather than simply its average magnitude or distribution, is critical for maintaining previously acquired behaviors during continued online learning.

#23
h=3

LLM-Based Selection of Incongruent Verbal and Nonverbal Behavior for Virtual Humans

2026-08-24cs.AI cs.HC cs.ROP. Torshizi (h=3)
Parisa Ghanad Torshizi, Stacy Marsella
Core Contributions
  • Pushes virtual-agent nonverbal behavior generation beyond the common assumption that gestures should simply emphasize or illustrate speech content, modeling instead how nonverbal channels can reinforce, weaken, qualify, or contradict what's being said.
  • Builds a taxonomy of mismatch categories grounded in Ekman's verbal-nonverbal framework, giving a structured vocabulary for incongruent behavior rather than treating mismatches as noise.
  • Uses LLMs to select contextually appropriate mismatched verbal-nonverbal pairs from dialogue and social context — a generation target most nonverbal-behavior systems don't attempt, since congruence is usually treated as the goal.
  • Validates in a human-subject study specifically aimed at training contexts requiring nuanced social read, such as counseling simulations with virtual patients, where incongruent cues (e.g., hidden distress) are the actual signal to detect.
Show abstract

Nonverbal behavior generation systems for virtual agents often take an utterance as input and generate nonverbal behaviors that emphasize or illustrate the content of the verbal channel. However, human nonverbal behavior is shaped by more than the content of the speech. It is also influenced by speaker roles, interpersonal relationships, social context, and the cognitive and emotional states of the interactants. As a result, the nonverbal channel may reinforce, weaken, qualify, or even contradict the verbal channel. It may also reveal internal states that are hidden or only indirectly implied in speech, including emotional leakage that may be incidental to the immediate interaction. Modeling this richer relationship between verbal and nonverbal behavior is important for designing virtual agents that exhibit realistic, human-like behavior. It is especially critical in training contexts that require nuanced social interpretation, such as counseling simulations involving virtual patients. Drawing on Ekman's framework of verbal nonverbal relationships, we propose a taxonomy of categories in which mismatches between verbal and nonverbal behavior can occur. We then examine alternative approaches for realizing these behaviors using large language models, focusing on whether LLMs can select contextually appropriate mismatched verbal and nonverbal behaviors from a given dialogue and social interaction context. Finally, we evaluate the resulting behaviors in a human-subject study, assessing whether context-driven nonverbal behavior, when embodied in a virtual human, produces the intended effects on observers.