Deep Learning | Erwin POUSSI

GAIT-Manip: Whole-Body Robot Control for Loco-Manipulation

Tue, 01 Sep 2026 00:00:00 +0000

Demo

Whole-body robot control (G1 and Spot) for loco-manipulation, using GAIT scene reasoning.

Status: More to come soon!

GAIT: Graph-Memory Agent with Interrupt-Triggered Reasoning

Sat, 01 Aug 2026 00:00:00 +0000

Demo

GAIT (Graph-memory Agent with Interrupt-Triggered reasoning) builds a persistent, structured memory for vision-language-action (VLA) models by constructing semantic scene graphs online during navigation. The agent incrementally builds a graph-based spatial memory from egocentric observations, encoding object identities, spatial relations, and semantic attributes as it explores. An interrupt-triggered reasoning mechanism allows the agent to re-plan on the fly when novel or task-relevant cues are detected — bridging reactive control with deliberative graph-based reasoning.

This enables open-vocabulary navigation: given a free-form language goal, the agent retrieves and reasons over its scene graph memory to localize targets it has previously observed, or to plan exploration strategies for unseen goals — without requiring a fixed object vocabulary or pre-built map.

Status: More to come soon!

Growing a Virtual Embryo with 3D Convolutional Models

Sat, 01 Aug 2026 00:00:00 +0000

Embryo Growth Simulation

This project trains foundational 3D convolutional models to reproduce embryogenesis in silico, capturing both cell-level gene expression prediction and 3D morphogenetic dynamics. The goal is to learn a generative model of embryonic development that jointly predicts how cells divide, differentiate, and spatially organize — bridging gene regulatory networks with tissue-scale morphology.

The approach leverages volumetric deep learning architectures to model the spatiotemporal evolution of developing embryos, learning from multi-modal developmental biology datasets that combine single-cell transcriptomics with 3D imaging.

I am one of the builders of the Virtual Embryo Challenge — a community benchmark for computational models of embryonic development.

Virtual Embryo Challenge — check it out and don’t hesitate to participate!

Status: More to come soon!

Language-Steered Drones

Thu, 01 Jan 2026 00:00:00 +0000

Drone navigating to a leaf blower

Developed a vision-language navigation (VLN) policy for autonomous drone flight in photorealistic 3D Gaussian Splatting environments. Given a natural language instruction like “go to the green leafblower,” the drone autonomously identifies and navigates to the target — collision-free.

The video shows the drone’s onboard view: RGB (left) and semantic similarity field (right) for the query “green and pink leafblower.” In the right view, red indicates high similarity with the query and blue indicates low similarity — the drone navigates towards the high-similarity region while avoiding obstacles. The system first encodes the language instruction via CLIP embeddings, localizes the target using CLIPSeg semantic segmentation, and generates real-time control commands to navigate through a cluttered indoor environment.

The control policy is a lightweight neural network (SqueezeNet Commander MLP) trained via Behavioral Cloning from an ACADOS-based MPC expert. A key contribution is the design and implementation of a full DAgger (Dataset Aggregation) pipeline — including mixed-policy rollouts, expert annotation filtering, iterative retraining with best-model checkpointing, and automated benchmarking — to systematically correct for compounding errors under distribution shift. A second key contribution is the introduction of explicit geometric features — bearing and elevation — extracted from the CLIPSeg heatmap centroid, providing the policy with a direct spatial signal for goal-directed control. This replaces the previous approach where target localization had to be implicitly learned from visual embeddings alone.

Results: 90% navigation success rate (up from 52%), collision rate reduced from 20% to 8%, with generalization to unseen RRT-planned trajectories.

This research is conducted under Prof. Mac Schwager (Multi-Robot Systems Lab, Stanford) in collaboration with PhD student Maximilian Adang (MSL Stanford).

GitHub Repository

Tech Stack

PyTorch · 3D Gaussian Splatting (gsplat) · CLIPSeg · ACADOS optimal control · CUDA

Pantheon: LLM-Based Bio Assistant

Wed, 01 Oct 2025 00:00:00 +0000

Demo

System overview

This project focuses on the development of Pantheon, an open-source multi-agent LLM platform for biological scientific workflows
(Pantheon Project).

My work specifically targets Gene Panel Selection (GPS), a core capability of Pantheon that automates the design of gene panels for spatial transcriptomics experiments.

The GPS system coordinates five specialized LLM agents, built on open-source models and extended with custom toolchains and domain-specific reasoning. A Leader Agent interprets ambiguous biological intent, decomposes high-level scientific questions into structured computational steps, and orchestrates agents such as the Selection Expert, with iterative validation and error handling.

I am currently fine-tuning and continually pretraining Qwen3 LLMs to power Pantheon’s self-evolving agent backbone, leveraging reinforcement learning from verifiable rewards (RLVR) and tool-augmented inference to build domain-adaptive, evolvable agents for autonomous genomic reasoning. I also collaborate with Vizgen to integrate a reinforcement learning module for autonomous gene panel design by Pantheon.

This research is conducted in collaboration with Prof. Xiaojie Qiu (Qiu Lab, Stanford) and Dr. Weize Xu (Stanford Profile).

Deployed in collaboration with Vizgen, Pantheon processes large-scale multi-omics datasets and produces curated gene panels that previously required extensive manual expertise.

Publication: Xu, Poussi et al., PantheonOS: An Evolvable Multi-Agent Framework for Automatic Genomics Discovery, bioRxiv 2026. Read preprint

GitHub Repository

SelectSmart: Autonomous Gene Panel Design

Wed, 01 Oct 2025 00:00:00 +0000

Training Overview

SelectSmart is a reinforcement-learning framework for gene panel selection in single-cell transcriptomics.
This project was developed as part of coursework with Prof. Mykel Kochenderfer
(AA228 / CS238 – Decision Making under Uncertainty) and extended through research assistantship work with Prof. Xiaojie Qiu
(Qiu Lab, Stanford).

The method combines meta-voted candidate genes, an actor–critic architecture, and a reward balancing clustering fidelity (ARI) with panel-size regularization.

Trained on a 30k-cell kidney dataset and evaluated on an independent CZ Kidney dataset, SelectSmart produces a 500-gene panel that preserves transcriptomic geometry and outperforms classical gene panel selection methods.

Status: 🧪 Active development

Report

📄 Download the technical report (PDF)

Physics-Informed Random Forest Surrogate Modeling for Parachute Fabric Permeability

Sun, 01 Jun 2025 00:00:00 +0000

Parachute Inflation

This project presents a physics-informed surrogate modeling framework based on Random Forests for optimizing fabric permeability in supersonic Mars landing parachutes, conducted as part of a research project under (Prof. Jason Rabinovitch) at Stevens Institute of Technology.

The method combines a Darcy–Forchheimer analytical trend model with a Random Forest residual learner, enabling accurate prediction of permeability coefficients across a high-dimensional geometric design space. The surrogate is embedded in a multi-objective optimization pipeline to identify pore geometries consistent with experimental permeability measurements.

The framework integrates:

Pore-scale CFD simulations (NGA2)
Physics-informed Random Forest surrogate modeling
Global multi-objective optimization (NSGA-II, CTAEA)
Local high-fidelity refinement

Optimization results

Trained on tens of thousands of CFD simulations, the surrogate achieves <10% worst-case relative error, while optimized geometries reproduce experimental permeability with <1% mean error. The resulting models provide efficient and physically consistent closures for large-scale FSI simulations of parachute inflation.

Status: 📝 Paper in preparation