Robotics | Erwin POUSSI

GAIT-Manip: Whole-Body Robot Control for Loco-Manipulation

Tue, 01 Sep 2026 00:00:00 +0000

Demo

Whole-body robot control (G1 and Spot) for loco-manipulation, using GAIT scene reasoning.

Status: More to come soon!

GAIT: Graph-Memory Agent with Interrupt-Triggered Reasoning

Sat, 01 Aug 2026 00:00:00 +0000

Demo

GAIT (Graph-memory Agent with Interrupt-Triggered reasoning) builds a persistent, structured memory for vision-language-action (VLA) models by constructing semantic scene graphs online during navigation. The agent incrementally builds a graph-based spatial memory from egocentric observations, encoding object identities, spatial relations, and semantic attributes as it explores. An interrupt-triggered reasoning mechanism allows the agent to re-plan on the fly when novel or task-relevant cues are detected — bridging reactive control with deliberative graph-based reasoning.

This enables open-vocabulary navigation: given a free-form language goal, the agent retrieves and reasons over its scene graph memory to localize targets it has previously observed, or to plan exploration strategies for unseen goals — without requiring a fixed object vocabulary or pre-built map.

Status: More to come soon!

Language-Steered Drones

Thu, 01 Jan 2026 00:00:00 +0000

Drone navigating to a leaf blower

Developed a vision-language navigation (VLN) policy for autonomous drone flight in photorealistic 3D Gaussian Splatting environments. Given a natural language instruction like “go to the green leafblower,” the drone autonomously identifies and navigates to the target — collision-free.

The video shows the drone’s onboard view: RGB (left) and semantic similarity field (right) for the query “green and pink leafblower.” In the right view, red indicates high similarity with the query and blue indicates low similarity — the drone navigates towards the high-similarity region while avoiding obstacles. The system first encodes the language instruction via CLIP embeddings, localizes the target using CLIPSeg semantic segmentation, and generates real-time control commands to navigate through a cluttered indoor environment.

The control policy is a lightweight neural network (SqueezeNet Commander MLP) trained via Behavioral Cloning from an ACADOS-based MPC expert. A key contribution is the design and implementation of a full DAgger (Dataset Aggregation) pipeline — including mixed-policy rollouts, expert annotation filtering, iterative retraining with best-model checkpointing, and automated benchmarking — to systematically correct for compounding errors under distribution shift. A second key contribution is the introduction of explicit geometric features — bearing and elevation — extracted from the CLIPSeg heatmap centroid, providing the policy with a direct spatial signal for goal-directed control. This replaces the previous approach where target localization had to be implicitly learned from visual embeddings alone.

Results: 90% navigation success rate (up from 52%), collision rate reduced from 20% to 8%, with generalization to unseen RRT-planned trajectories.

This research is conducted under Prof. Mac Schwager (Multi-Robot Systems Lab, Stanford) in collaboration with PhD student Maximilian Adang (MSL Stanford).

GitHub Repository

Tech Stack

PyTorch · 3D Gaussian Splatting (gsplat) · CLIPSeg · ACADOS optimal control · CUDA

Autonomous Spacecraft Docking

Thu, 01 Jun 2023 00:00:00 +0000

Docking demonstration

This project was conducted as an undergraduate research project in collaboration with Nyx Exploration Company and Antoine Pallois, focusing on autonomous spacecraft docking.

The work investigated guidance, navigation, and control (GNC) strategies for close-proximity operations, with a particular emphasis on vision-based relative navigation and closed-loop docking maneuvers. Experimental validation was performed using a quadcopter drone as a hardware test platform.

Technical report

📄 Final technical report (PDF)