weixin_39759107
weixin_39759107
2020-12-08 18:06

Problem with single imu - single cam extrinsic calibration

I am trying to run Kalibr Toolbox to calibrate the relative rotation and translation of the camera and imu in Google Pixel. I have recorded a calibration sequence and prepared all the necessary files - it can be viewed here: https://drive.google.com/drive/folders/1vgQEZtfHXTaIpiLiXLMKb_CfezB5XG6e?usp=sharing

I run the kalibr_calibrate_imu_camera cde code on virtualbox ubuntu with the following command:

../kalibr-cde/kalibr_calibrate_imu_camera --target checker.yaml --cam camchain.yaml --imu imu.yaml --bag ch_gentle.bag

And the output is:

importing libraries Initializing IMUs: Update rate: 400.0 Accelerometer: Noise density: 0.003 Noise density (discrete): 0.06 Random walk: 0.0003 Gyroscope: Noise density: 0.006 Noise density (discrete): 0.12 Random walk: 0.0006 Initializing imu rosbag dataset reader: Dataset: /home/viki/workspace/chessboard_imu4/ch_gentle.bag Topic: /imu0 Number of messages: 20239 Reading IMU data (/imu0) Read 20239 imu readings over 51.5 seconds
Initializing calibration target: Type: checkerboard Rows Count: 6 Distance: 0.05 [m] Cols Count: 9 Distance: 0.05 [m] Initializing camera chain: Camera chain - cam0: Camera model: pinhole Focal length: [642.406, 645.745] Principal point: [315.463, 238.763] Distortion model: radtan Distortion coefficients: [0.27883956940720067, -2.2111427587870867, -0.0025311661150160614, -0.0010815551002964759] baseline: no data available Initializing camera rosbag dataset reader: Dataset: /home/viki/workspace/chessboard_imu4/ch_gentle.bag Topic: /cam0/image_raw Number of images: 3057 Extracting calibration target corners Extracted corners for 3057 images (of 3057 images)

Building the problem Spline order: 6 Pose knots per second: 70 Do pose motion regularization: False xddot translation variance: 1000000.000000 xddot rotation variance: 100000.000000 Bias knots per second: 50 Do bias motion regularization: True Blake-Zisserman on reprojection errors -1 Acceleration Huber width (m/s^2): -1.000000 Gyroscope Huber width (rad/s): -1.000000 Do time calibration: False Max iterations: 30 Time offset padding: 0.020000

Estimating imu-camera rotation prior

Initializing a pose spline with 5145 knots (100.000000 knots per second over 51.452952 seconds) Orientation prior camera-imu found as: (T_i_c) [[-0.00191114 -0.99996987 0.00752429] [-0.99994817 0.00198624 0.00998568] [-0.01000033 -0.00750482 -0.99992183]] Gyro bias prior found as: (b_gyro) [ 0.0001671 0.00031136 -0.00018996]

Initializing a pose spline with 5153 knots (100.000000 knots per second over 51.532952 seconds)

Initializing the bias splines with 2577 knots

Adding camera error terms (/cam0/image_raw) Added 3057 camera error terms

Adding accelerometer error terms (/imu0) Added 20236 of 20239 accelerometer error terms (skipped 3 out-of-bounds measurements)

Adding gyroscope error terms (/imu0) Added 20236 of 20239 gyroscope error terms (skipped 3 out-of-bounds measurements)

Before Optimization

Reprojection error squarred (cam0): mean 0.467676793977, median 0.207836145574, std: 0.797330713119 Gyro error squarred (imu0): mean 0.189767406562, median 0.0968219978587, std: 0.308269812229 Accelerometer error squarred (imu0): mean 54354.8262336, median 54130.3940099, std: 2950.85395155

Transformation T_cam0_imu0 (imu0 to cam0, T_ci): [m] [[-0.00191114 -0.99996987 0.00752429 0. ] [-0.99994817 0.00198624 0.00998568 0. ] [-0.01000033 -0.00750482 -0.99992183 0. ] [ 0. 0. 0. 1. ]]

Optimizing... No linear system solver set in the options. Defaulting to the sparse_cholesky solver Using the sparse_cholesky linear system solver Using the levenberg_marquardt trust region policy No linear system solver set in the options. Defaulting to the sparse_cholesky solver Using the sparse_cholesky linear system solver Using the levenberg_marquardt trust region policy Initializing Optimization problem initialized with 10325 design variables and 207385 error terms The Jacobian matrix is 537706 x 46448 [0.0]: J: 1.1e+09 [pid 8460] +++ killed by SIGKILL +++PANIC: handle_group_exit: 8460 leader 8455 [1]: J: 1.02855e+08, dJ: 9.97149e+08, deltaX: 1.74884, LM - lambda:100 mu:2 [2]: J: 1.20735e+07, dJ: 9.07815e+07, deltaX: 1.28185, LM - lambda:33.3333 mu:2 [3]: J: 4.07142e+06, dJ: 8.00203e+06, deltaX: 0.889082, LM - lambda:16.7371 mu:2 [4]: J: 375400, dJ: 3.69602e+06, deltaX: 0.627697, LM - lambda:15.5062 mu:2 Last step was a regression. Reverting [5]: J: 3.11681e+06, dJ: -2.74141e+06, deltaX: 0.719216, LM - lambda:5.16873 mu:2 Last step was a regression. Reverting [6]: J: 3.12445e+06, dJ: -2.74905e+06, deltaX: 0.128326, LM - lambda:20.6749 mu:4 Last step was a regression. Reverting [7]: J: 3.1491e+06, dJ: -2.7737e+06, deltaX: 0.0148867, LM - lambda:165.399 mu:8 Last step was a regression. Reverting [8]: J: 3.14763e+06, dJ: -2.77223e+06, deltaX: 0.00119014, LM - lambda:2646.39 mu:16 Last step was a regression. Reverting [9]: J: 3.15405e+06, dJ: -2.77865e+06, deltaX: 0.000214775, LM - lambda:84684.5 mu:32 Last step was a regression. Reverting [10]: J: 3.16637e+06, dJ: -2.79097e+06, deltaX: 8.99753e-07, LM - lambda:5.41981e+06 mu:64

After Optimization (Results)

Reprojection error squarred (cam0): mean 0.551160888183, median 0.251199784795, std: 0.855047993769 Gyro error squarred (imu0): mean 0.260294320249, median 0.142024223516, std: 0.355983314094 Accelerometer error squarred (imu0): mean 11.6355964031, median 3.8276315513, std: 20.4208009338

Transformation T_cam0_imu0 (imu0 to cam0, T_ci): [m] [[-0.09970554 -0.98924099 -0.10705641 -0.23471977] [-0.99416647 0.09459437 0.05181634 0.04647423] [-0.04113191 0.11159827 -0.9929018 -0.00756453] [ 0. 0. 0. 1. ]]

Saving calibration to file: camchain-imucam-homevikiworkspacechessboard_imu4ch_gentle.yaml Detailed results written to file: results-imucam-homevikiworkspacechessboard_imu4ch_gentle.txt Generating result report...

(python:8446): GLib-WARNING **: getpwuid_r(): failed due to unknown user id (1000) Gtk-Message: Failed to load module "unity-gtk-module" libproxy.so.1: cannot open shared object file: No such file or directory Failed to load module: /usr/lib/x86_64-linux-gnu/gio/modules/libgiolibproxy.so

(python:8446): GVFS-RemoteVolumeMonitor-WARNING **: cannot open directory /usr/share/gvfs/remote-volume-monitors: Error opening directory '/usr/share/gvfs/remote-volume-monitors': No such file or directory Report written to report-imucam-homevikiworkspacechessboard_imu4ch_gentle.pdf

I repeated this process several times with different test sequences. Each time I get very good rotation matrix, but completely wrong translation values (23cm is more than the dimensions of the phone!). Also, the translation results are completely different for similar sequences.

Does anyone have an idea what I am doing wrong / what can be done to get correct calibration results? I have imu sampling rate 400Hz and camera 60Hz. I have tried to excite all gyro and acc axes in the test sequence. I get the same kind of results when using Aprilgrid.

Thank you for any help!

该提问来源于开源项目:ethz-asl/kalibr

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

4条回答

  • weixin_39759107 weixin_39759107 5月前

    Dear Zachary, thank you for your fast reply! Running with --time-calibration yields virtually identical results. They are surely wrong, as the reprojection error increases compared to the initial transformation input into the program.

    However, increasing noise parameters does change a lot, and results seem much more sensible:

    viki:~/workspace/chessboard_imu4$ ../kalibr-cde/kalibr_calibrate_imu_camera --target checker.yaml --cam camchain.yaml --imu imu.yaml --bag ch_gentle.bag importing libraries Initializing IMUs: Update rate: 400.0 Accelerometer: Noise density: 0.05 Noise density (discrete): 1.0 Random walk: 0.001 Gyroscope: Noise density: 0.02 Noise density (discrete): 0.4 Random walk: 4e-05 Initializing imu rosbag dataset reader: Dataset: /home/viki/workspace/chessboard_imu4/ch_gentle.bag Topic: /imu0 Number of messages: 20239 Reading IMU data (/imu0) Read 20239 imu readings over 51.5 seconds
    Initializing calibration target: Type: checkerboard Rows Count: 6 Distance: 0.05 [m] Cols Count: 9 Distance: 0.05 [m] Initializing camera chain: Camera chain - cam0: Camera model: pinhole Focal length: [642.406, 645.745] Principal point: [315.463, 238.763] Distortion model: radtan Distortion coefficients: [0.27883956940720067, -2.2111427587870867, -0.0025311661150160614, -0.0010815551002964759] baseline: no data available Initializing camera rosbag dataset reader: Dataset: /home/viki/workspace/chessboard_imu4/ch_gentle.bag Topic: /cam0/image_raw Number of images: 3057 Extracting calibration target corners Extracted corners for 3057 images (of 3057 images)

    Building the problem Spline order: 6 Pose knots per second: 70 Do pose motion regularization: False xddot translation variance: 1000000.000000 xddot rotation variance: 100000.000000 Bias knots per second: 50 Do bias motion regularization: True Blake-Zisserman on reprojection errors -1 Acceleration Huber width (m/s^2): -1.000000 Gyroscope Huber width (rad/s): -1.000000 Do time calibration: False Max iterations: 30 Time offset padding: 0.020000

    Estimating imu-camera rotation prior

    Initializing a pose spline with 5145 knots (100.000000 knots per second over 51.452952 seconds) Orientation prior camera-imu found as: (T_i_c) [[-0.00191114 -0.99996987 0.00752429] [-0.99994817 0.00198624 0.00998568] [-0.01000033 -0.00750482 -0.99992183]] Gyro bias prior found as: (b_gyro) [ 0.0001671 0.00031136 -0.00018996]

    Initializing a pose spline with 5153 knots (100.000000 knots per second over 51.532952 seconds)

    Initializing the bias splines with 2577 knots

    Adding camera error terms (/cam0/image_raw) Added 3057 camera error terms

    Adding accelerometer error terms (/imu0) Added 20236 of 20239 accelerometer error terms (skipped 3 out-of-bounds measurements)

    Adding gyroscope error terms (/imu0) Added 20236 of 20239 gyroscope error terms (skipped 3 out-of-bounds measurements)

    Before Optimization

    Reprojection error squarred (cam0): mean 0.467676793978, median 0.207836145574, std: 0.797330713119 Gyro error squarred (imu0): mean 0.0170790665906, median 0.00871397980728, std: 0.0277442831007 Accelerometer error squarred (imu0): mean 195.677374441, median 194.869418436, std: 10.6230742256

    Transformation T_cam0_imu0 (imu0 to cam0, T_ci): [m] [[-0.00191114 -0.99996987 0.00752429 0. ] [-0.99994817 0.00198624 0.00998568 0. ] [-0.01000033 -0.00750482 -0.99992183 0. ] [ 0. 0. 0. 1. ]]

    Optimizing... No linear system solver set in the options. Defaulting to the sparse_cholesky solver Using the sparse_cholesky linear system solver Using the levenberg_marquardt trust region policy No linear system solver set in the options. Defaulting to the sparse_cholesky solver Using the sparse_cholesky linear system solver Using the levenberg_marquardt trust region policy Initializing Optimization problem initialized with 10325 design variables and 207385 error terms The Jacobian matrix is 537706 x 46448 [0.0]: J: 4.03572e+06 [1]: J: 604408, dJ: 3.43132e+06, deltaX: 1.01076, LM - lambda:100 mu:2 [2]: J: 7699.49, dJ: 596709, deltaX: 0.537287, LM - lambda:33.3333 mu:2 [3]: J: 6463.22, dJ: 1236.27, deltaX: 0.0242896, LM - lambda:11.1111 mu:2 [4]: J: 5851.1, dJ: 612.112, deltaX: 0.0570309, LM - lambda:3.7037 mu:2 [5]: J: 5519.1, dJ: 332.006, deltaX: 0.103663, LM - lambda:1.23457 mu:2 [6]: J: 5516.45, dJ: 2.65076, deltaX: 0.083261, LM - lambda:0.670455 mu:2 [7]: J: 5449.42, dJ: 67.0286, deltaX: 0.027262, LM - lambda:1.22799 mu:2 Last step was a regression. Reverting [8]: J: 5470.93, dJ: -21.5124, deltaX: 0.0379517, LM - lambda:0.792043 mu:2 Last step was a regression. Reverting [9]: J: 5476.01, dJ: -26.5953, deltaX: 0.00637397, LM - lambda:3.16817 mu:4 Last step was a regression. Reverting [10]: J: 5474.69, dJ: -25.2703, deltaX: 0.0020852, LM - lambda:25.3454 mu:8 Last step was a regression. Reverting [11]: J: 5474.79, dJ: -25.3691, deltaX: 4.92446e-05, LM - lambda:405.526 mu:16

    After Optimization (Results)

    Reprojection error squarred (cam0): mean 0.0245299856825, median 0.00839543575126, std: 0.128245414253 Gyro error squarred (imu0): mean 0.016187034165, median 0.00845718763541, std: 0.0269381155127 Accelerometer error squarred (imu0): mean 0.0562023224036, median 0.0276993077609, std: 0.0807926874988

    Transformation T_cam0_imu0 (imu0 to cam0, T_ci): [m] [[-0.00346152 -0.99998936 -0.00304782 0.02950604] [-0.9996563 0.00338113 0.02599696 0.02170828] [-0.02598638 0.00313676 -0.99965738 0.06945846] [ 0. 0. 0. 1. ]]

    Saving calibration to file: camchain-imucam-homevikiworkspacechessboard_imu4ch_gentle.yaml Detailed results written to file: results-imucam-homevikiworkspacechessboard_imu4ch_gentle.txt Generating result report...

    I have uploaded the new result report to GDrive as an additional file with subscript _largenoise.

    The big question is - how do we know if they are correct? I have several recorded sequences and running the calibration on each with larger noise still produces different translations by several cm...

    点赞 评论 复制链接分享
  • weixin_39628070 weixin_39628070 5月前

    Hi! any improvements on this? I can not find any doc in which the results can be evaluated.. any hint?

    点赞 评论 复制链接分享
  • weixin_39942351 weixin_39942351 5月前

    Having a quick look through your results the thing that is really jumping out at me is the plot of your estimated imu biases. Over the space of a minute they change by over 0.5 m/s-2. Over this short of a time period I would expect the bias to be near constant.

    Your re-projection errors while on the high side, but not completely terrible which means things are at least somewhat working though.

    I am wondering if the issue could be time offsets, try running with the --time-calibration flag and see if this helps (if things are really out you may need to increase the range of times it allows, it will throw an error if this is the case).

    点赞 评论 复制链接分享
  • weixin_39942351 weixin_39942351 5月前

    Also looking at your imu yaml file it could be that you have the noise in your sensors set far too low. For cheap consumer imus I usually use the following parameters

    accelerometer_noise_density: 0.05 #continous accelerometer_random_walk: 0.001 gyroscope_noise_density: 0.02 #continous gyroscope_random_walk: 4.0e-05

    点赞 评论 复制链接分享

相关推荐