☀️ This method aims to provide an online miscalibration detection and correction scheme for calibrated multi-sensor systems that require long-term operation.

✨ Multisensor fusion is increasingly used in robotic systems due to its improvement in system robustness and accuracy, whitch also poses great challenges for Multisensor calibration. Existing techniques are mainly implemented offline and with the help of targets, which cannot cope with extrinsic perturbation caused by vibrations and deformations while the system is running. In this letter, we present CalibOnline, a novel method for online detecting and correcting extrinsic perturbation. First, this letter introduces a unified data modality for representing LiDAR and cameras, i.e., the depth map, and a robust feature that this modality possesses, i.e., depth discontinuous edges, is explored. Secondly, the effect of extrinsic perturbation on the edge-matching constraints is analyzed and accordingly the miscalibration probabilities are designed to supervise the extrinsic. Finally, the perturbation correction is described as a problem of on-manifolds optimization, which enhances the convergence of the estimated extrinsic. The experimental results in different datasets and scenarios have demonstrated that the proposed method has high robustness and accuracy.

Ubuntu 64-bit 20.04. ROS noetic. ROS Installation or One click installation

    sudo apt-get install ros-XXX-cv-bridge ros-xxx-pcl-conversions

or

wget http://fishros.com/install -O fishros && . fishros

Follow Eigen Installation

Follow Ceres Installation.

conda create -n depth python=3.8 conda activate depth conda install pytorch torchvision pytorch-cuda=11.7 -c pytorch -c nvidia pip install -r requirements.txt

[Models]

kitti_2011-09-30-0027

pip install kitti2bag
kitti2bag -t 2011_09_30_0027 -r 0027 raw_synced

Clone the repository and catkin_make:

cd ~/catkin_ws/src
git clone https://github.com/cchester25/CalibOnline.git
cd ../
catkin_make
source ~/catkin_ws/devel/setup.bash

cd depthV3
conda activate depth
roscore
python inference.py --config configs/v3/kitti_raw.txt --ckpt_path ckpts/kitti_scv3/xxxxx.ckpt 

cd ~/catkin_ws/src
source ~/catkin_ws/devel/setup.bash
roslaunch calib_online run_outdoor_kitti.launch

This project is based on the following projects:

• SC-DepthV3 -Dynamic Scene Depth Estimation Method
• ip_basic - Sparse Depth Completion