Visual Inertial SLAM Introduction

Posted on Edited on In ,

This post talks about the theory for Visual Inertial SLAM. Notes of book SLAM in Autonomous Driving.

IMU

In Visual SLAM, only observation equation exists due to the lack of input signal. Though constant velocity or constant accelerations could be used for initial guess of the pose at new frame time, it cannot provide the uncertainty/covariance propagation, so the Visual SLAM usually degrades to Bundle Adjustment, or MLE estimation.

Visual Inertial SLAM could use the IMU measurements as the input signal, which could provide the control equation to propagate the system state and covariance. This could turn the SLAM as MAP estimation and could provide higher accuracy.

Since IMU measurements also contains noise like accel (accelerometer) bias, gyro (gyroscope) bias, and random noise, the system state needs to be augmented to not only include 6DoF pose, but also the IMU parameters. This section talks about the IMU Kinematics model, and how to construct the system state in Visual Inertial SLAM system.

IMU Kinematics

Visual Inertial System Equations

IMU Pre-integration

Loosely Coupled

Loosely coupled method are usually recursive estimation like EKF. Where the IMU measurements are used to predict the system state with the system equations, and camera measurements are used to perform Visual SLAM, then the pose is used to update the system state.

Tightly Coupled

Tightly coupled method could be recursive estimation like IEKF, batch estimation. To achieve the realtime, sliding window batch estimation with IMU pre-integration are usually used.

Sliding Window Management

  • Which frames and mappoints to remove
  • How to marginalize

Backend Optimization

The mappoint position could be optimized in frontend, or it could be optimized in backend to leave frontend only optimize poses.

Frontend processing needs to estimate each frame pose, while the backend needs to do more processing on the keyframes, to reconstruct map and optimize more to get accurate pose.

Map Management

  • How to construct backend map
  • How to optimize the mappoint positions