Quick start¶
Get a camera–LiDAR match in three steps.
1. Install¶
pip install XCalib
Requires Python 3.10+. For ONNX export add XCalib[onnx]; for HDF5 training add
XCalib[train]; for the xcalib-review window add XCalib[gui].
2. Load a pretrained matcher¶
from xcalib import Matcher
# Downloads and caches the released checkpoint + config on first use.
matcher = Matcher.from_pretrained("crlite", site="a9_dataset_r02_s01")
3. Match one frame¶
Pass an RGB image, a LiDAR point cloud, and your detector's 2D/3D boxes:
import numpy as np
result = matcher.match(
image=image, # H,W,3 uint8 RGB
point_cloud=points, # N,3 float32 (x, y, z) in meters
bboxes_2d=bboxes_2d, # K,4 (x1, y1, x2, y2) in pixels
bboxes_3d=bboxes_3d, # M,6 (xmin, ymin, zmin, xmax, ymax, zmax) in meters
)
print(result.similarity.shape) # (K, M) similarity matrix
print(result.matches) # [(img_idx, lidar_idx, score), ...]
That's it — result.matches gives the camera box ↔ LiDAR box pairs.
Optional: one pair per box¶
result.matches holds the best-scoring LiDAR box for each camera box, so two camera boxes can
end up with the same LiDAR box. Ask for the optimal one-to-one assignment instead:
result = matcher.match(
image=image,
point_cloud=points,
bboxes_2d=bboxes_2d,
bboxes_3d=bboxes_3d,
one_to_one=True,
)
print(result.assignment) # [(img_idx, lidar_idx, score), ...], each box used at most once
Once you have a successful calibration (for example from matcher.calibrate(..., intrinsics=K)),
pass it as calibration=calib as well. Pairs whose 3D center projects behind the camera or far
from the 2D box are then ruled out before the assignment, unless that would leave a camera box
with no candidate at all. See Pair selection.
Next steps¶
- Input protocol — exact shape/dtype contract for real frames.
- Pair selection — one-to-one assignment and the geometric gate.
- Deployed site — capture frames the matcher is lost on, refine an existing calibration.
- Review GUI — fix wrong pairs in
xcalib-review. - Continual refine — learn from verified pairs behind a validation gate.
- Models and datasets — available checkpoints and sites.
- Custom models — load your own weights or add an architecture.