Skip to content

Quick start

Get a camera–LiDAR match in three steps.

1. Install

pip install XCalib

Requires Python 3.10+. For ONNX export add XCalib[onnx]; for HDF5 training add XCalib[train]; for the xcalib-review window add XCalib[gui].

2. Load a pretrained matcher

from xcalib import Matcher

# Downloads and caches the released checkpoint + config on first use.
matcher = Matcher.from_pretrained("crlite", site="a9_dataset_r02_s01")

3. Match one frame

Pass an RGB image, a LiDAR point cloud, and your detector's 2D/3D boxes:

import numpy as np

result = matcher.match(
    image=image,             # H,W,3 uint8 RGB
    point_cloud=points,      # N,3 float32 (x, y, z) in meters
    bboxes_2d=bboxes_2d,     # K,4 (x1, y1, x2, y2) in pixels
    bboxes_3d=bboxes_3d,     # M,6 (xmin, ymin, zmin, xmax, ymax, zmax) in meters
)

print(result.similarity.shape)  # (K, M) similarity matrix
print(result.matches)           # [(img_idx, lidar_idx, score), ...]

That's it — result.matches gives the camera box ↔ LiDAR box pairs.

Optional: one pair per box

result.matches holds the best-scoring LiDAR box for each camera box, so two camera boxes can end up with the same LiDAR box. Ask for the optimal one-to-one assignment instead:

result = matcher.match(
    image=image,
    point_cloud=points,
    bboxes_2d=bboxes_2d,
    bboxes_3d=bboxes_3d,
    one_to_one=True,
)

print(result.assignment)  # [(img_idx, lidar_idx, score), ...], each box used at most once

Once you have a successful calibration (for example from matcher.calibrate(..., intrinsics=K)), pass it as calibration=calib as well. Pairs whose 3D center projects behind the camera or far from the 2D box are then ruled out before the assignment, unless that would leave a camera box with no candidate at all. See Pair selection.

Next steps