Reinforcement Learning-Guided MPC for Whole-Body Loco-Manipulation

A scalable and data-efficient framework for safety- and precision-critical loco-manipulation.

Jan 1, 0001

Go to Project Site

Framework

We introduce the first framework that combines RL and whole-body MPC, where the MPC is embedded in training. The high-level planner is trained using RL to provide task-space references to a low-level MPC. We are thus able to leverage the benefits of both learning- and optimization-based methods including safety guarantees, great generalization, minimum sim2real gap and data-efficient training.

During a pre-training stage, we use a simplified kinematic model for fast simulation. When training with MPC, we use the MPC solution as our simulator to avoid the additional cost of physics simulation. During deployment, the RL is deployed in closed-loop with the MPC.

Deploying MPC with the planner trained using RL (Right) alleviates the issue of getting stuck in a local minimum when deploying MPC alone (Left) due to its shortsightedness.

Results

We are currently working to apply our framework in whole-body navigation to target end-effector positions in cluttered scenes in the real-world.

Stefanos Charalambous

Robotics Intern

My research interests include motion planning, optimal control and learning.