API reference¶

The curated public API — everything here is importable from the top-level xcalib package and stable across minor versions.

Matcher¶

`xcalib.engine.matcher.Matcher` ¶

High-level wrapper around any of the xcalib models.

`from_pretrained(model, weights=None, config=None, device=None, *, site='a9_dataset_r02_s01', repo_id=None, revision=None, token=None)` `classmethod` ¶

Construct a matcher from local files or the HuggingFace Hub.

Resolution rules for weights:

None — download checkpoints/{model}_{site}_best.pth (+ the matching YAML when config is None) from the configured Hub repo. Use repo_id / revision / token only when you need to override the released defaults.
hf://org/repo[@rev]/path — explicit Hub file.
directory — output of :meth:save_pretrained; loads <dir>/{model}.pth (+ <dir>/{model}.yaml if config is None).
anything else — local checkpoint path (original behaviour).

config may likewise be an hf:// URI, a local path outside the package, an EdgeConfig, or a plain dict. If omitted for local weights, the packaged xcalib/cfg/{model}_{site}.yaml is used when present.

`match(image, point_cloud, bboxes_2d, bboxes_3d, top_k=1, match_threshold=0.0, return_latency=False, validate='warn')` ¶

Run the full pipeline on a single frame.

Parameters:

Name	Type	Description	Default
`image`	`ndarray`	[H, W, 3] uint8 RGB.	required
`point_cloud`	`ndarray`	[P, 3+] float32 (X, Y, Z[, intensity ...]).	required
`bboxes_2d`	`ndarray`	[K, 4] image bboxes (x1, y1, x2, y2).	required
`bboxes_3d`	`ndarray`	[M, 6] LiDAR bboxes either (xmin,ymin,zmin,xmax,ymax,zmax) or (cx, cy, cz, dx, dy, dz).	required
`top_k`	`int`	How many top matches to keep in `matches` per image.	`1`
`match_threshold`	`float`	Minimum similarity to include in `matches`.	`0.0`
`return_latency`	`bool`	Force the forward-pass timing into the result.	`False`
`validate`	`str`	Input-protocol policy — "strict" raises on any violation, "warn" (default) raises on hard errors and logs soft warnings once, "off" skips checks. See docs/protocol.md.	`'warn'`

`pair(*args, **kwargs)` ¶

Alias of :meth:match — "pairing" in the partner's vocabulary.

`match_frame(frame)` ¶

Skip cropping, score a pre-prepared FrameData.

Used by scripts/paper/validate_paper.py where the UTC HDF5 loader has already produced FrameData. Returns (scores [N,M], elapsed_ms).

`calibrate(image, point_cloud, bboxes_2d, bboxes_3d, intrinsics, *, match_threshold=0.5, ransac_reproj_px=8.0, validate='warn')` ¶

Estimate the camera-LiDAR projection matrix from one frame.

Runs :meth:match, keeps each image detection's best LiDAR match above match_threshold, and solves PnP/RANSAC over the (2D bbox center, 3D bbox center) correspondences. For better conditioning across multiple frames use :meth:oneshot / CalibrationSession instead.

`oneshot(intrinsics, **kwargs)` ¶

Open a projection-matrix-supervised one-shot learning session.

The session observes frames, refines the camera-LiDAR projection matrix, harvests geometry-confirmed pseudo-pairs into a feature bank, and applies confidence-gated adapter updates. See xcalib.oneshot.session.OneShotSession.

`train(train_data, val_data, *, output_dir='runs/finetune', **kwargs)` ¶

Fine-tune the currently loaded weights on HDF5 caches.

Thin wrapper over :func:xcalib.engine.trainer.train that trains this matcher's model in place, then reloads the best (val-MRR) checkpoint so the matcher immediately serves the improved weights. train_data / val_data accept local .h5 paths or Hub site names. Extra keyword arguments (epochs, lr, scheduler, ...) are forwarded to the trainer.

Returns:

Type	Description
`Path`	Path to `best.pth` inside output_dir.

`build(target='onnx', output_dir=None, *, precision='fp16', device=None, onnx_dir=None, trtexec=None, extra_trt_args=(), verify=True)` ¶

Export the currently loaded weights to a deployment artifact.

Works for any weights — pretrained, fine-tuned, or one-shot adapted — because it traces self.model as it currently is.

Parameters:

Name	Type	Description	Default
`target`	`str`	"onnx" (any host) or "trt" (needs trtexec, i.e. Jetson/JetPack or a TensorRT install).	`'onnx'`
`output_dir`	`Union[str, Path, None]`	Where artifacts land. Defaults to `./onnx/<model>/` or `./engines/<model>/`.	`None`
`precision`	`str`	TensorRT precision ("fp32" \| "fp16" \| "best").	`'fp16'`
`device`	`Union[str, device, None]`	Device used for ONNX tracing (default cpu).	`None`
`onnx_dir`	`Union[str, Path, None]`	For target="trt": where to look for / export the intermediate ONNX graphs (default ./onnx//).	`None`
`trtexec`	`Optional[str]`	Explicit trtexec path override.	`None`
`extra_trt_args`	`Tuple[str, ...]`	Extra flags forwarded verbatim to trtexec.	`()`
`verify`	`bool`	Run the onnxruntime parity check after export.	`True`

Returns:

Type	Description
`'BuildResult'`	`xcalib.engine.exporter.BuildResult` with artifact paths and
`'BuildResult'`	(for ONNX) the max \|torch - onnx\| parity per output.

`save_pretrained(save_dir)` ¶

Write <dir>/{model}.pth + <dir>/{model}.yaml.

The checkpoint round-trips through :meth:from_pretrained (pass the directory as weights) and can be exported via :meth:build. One-shot adapter weights, when present, are stored under adapters_state_dict and re-attached automatically on load.

Results¶

`xcalib.engine.matcher.MatchResult` `dataclass` ¶

Container for matcher.match() output.

`xcalib.engine.exporter.BuildResult` `dataclass` ¶

What Matcher.build / export_onnx hand back to the caller.

`xcalib.oneshot.calibration.CalibrationResult` `dataclass` ¶

Output of a PnP/RANSAC solve.

`extrinsics` `property` ¶

[3,4] = [R|t] (without intrinsics).

`project(points_3d)` ¶

Project LiDAR-frame points to pixels.

Returns (uv [N,2], depth [N]); depth is the camera-frame Z, points behind the camera have depth <= 0.

Training¶

`xcalib.engine.trainer.train(model, train_data, val_data, *, model_name=None, config=None, site=None, epochs=10, lr=0.0001, weight_decay=0.0001, optimizer='adamw', warmup_epochs=0, scheduler='none', min_lr=1e-05, temperature=0.07, label_smoothing=0.05, stage1_weight=1.0, stage2_weight=1.0, weights=None, output_dir='runs/train', device='auto', seed=42, limit_train_frames=None, limit_val_frames=None, pairwise_batch_size=64)` ¶

Train (from scratch) or fine-tune a matcher; returns the best checkpoint.

Parameters:

Name	Type	Description	Default
`model`	`Union[str, Module]`	Registry name (`"crlite"`, ...) or an already-built model module (used by :meth:`Matcher.train`).	required
`train_data`	`Union[str, Path]`	Local `.h5` cache path or Hub site name (resolved through `xcalib.hub.datasets`).	required
`val_data`	`Union[str, Path]`	Same as train_data, for the validation split.	required
`model_name`	`Optional[str]`	Required when model is a module.	`None`
`config`	`Union[EdgeConfig, str, Path, None]`	Reference YAML / :class:`EdgeConfig`. Defaults to the packaged config for (model, site).	`None`
`site`	`Optional[str]`	Site used to pick the default config (inferred from train_data when it is a site name; falls back to `utc4`).	`None`
`weights`	`Union[str, Path, None]`	Optional checkpoint to warm-start from (registry-name mode).	`None`
`output_dir`	`Union[str, Path]`	Where per-epoch checkpoints, `best.pth` (selected by validation MRR), and `history.json` land.	`'runs/train'`

Returns:

Type	Description
`Path`	Path to `best.pth` inside output_dir.

Datasets¶

`xcalib.hub.datasets.load_dataset(site, split='test', *, repo_id=None, revision=None, token=None, local=None)` ¶

Return an HDF5 frame loader for one split of a site's cache.

Resolution order: explicit local path → checkout-local default → Hub download (cached by huggingface_hub). Requires h5py (the [train] extra).

Visualization¶

`xcalib.visualization.draw_matching_overlay(image, point_cloud, bboxes_2d, bboxes_3d, matches, *, output_height=420, match_threshold=None)` ¶

Draw camera detections, LiDAR BEV detections, and matched links.

Parameters:

Name	Type	Description	Default
`image`	`ndarray`	RGB image, `[H, W, 3]` `uint8`.	required
`point_cloud`	`ndarray`	LiDAR points with XYZ in the first three columns.	required
`bboxes_2d`	`ndarray`	Camera boxes as `(x1, y1, x2, y2)`.	required
`bboxes_3d`	`ndarray`	LiDAR boxes as extents or center+dimensions.	required
`matches`	`Iterable[Sequence[float]]`	Iterable of `(camera_index, lidar_index, score)` triples, such as `MatchResult.matches`.	required
`output_height`	`int`	Height of each visual pane.	`420`
`match_threshold`	`float \| None`	Optional score floor for display; lower-scoring matches are omitted from the drawn links.	`None`

Returns:

Type	Description
`ndarray`	RGB `uint8` side-by-side overlay.

`xcalib.visualization.draw_calibration_overlay(image, point_cloud, *, projection=None, intrinsics=None, rotation=None, translation=None, bboxes_3d=None, max_points=6000)` ¶

Project LiDAR points and optional 3D-box centers onto the camera image.

Parameters:

Name	Type	Description	Default
`image`	`ndarray`	RGB image, `[H, W, 3]` `uint8`.	required
`point_cloud`	`ndarray`	LiDAR points with XYZ in columns 0-2.	required
`projection`	`ndarray \| None`	Optional `3x4` camera projection matrix `K[R\|t]`.	`None`
`intrinsics`	`CameraIntrinsics \| ndarray \| None`	Camera intrinsics used with `rotation` and `translation` when `projection` is not supplied.	`None`
`rotation`	`ndarray \| None`	`3x3` extrinsic rotation.	`None`
`translation`	`ndarray \| None`	`3` or `3x1` extrinsic translation.	`None`
`bboxes_3d`	`ndarray \| None`	Optional LiDAR detections; their centers are projected and marked with labels.	`None`
`max_points`	`int`	Upper bound on projected point count for readable overlays.	`6000`

Returns:

Type	Description
`ndarray`	RGB `uint8` camera overlay.

Input protocol¶

`xcalib.protocol.validate_frame_inputs(image, point_cloud, bboxes_2d, bboxes_3d)` ¶

Check one frame's raw inputs against the input protocol.

Returns the (possibly empty) list of violations; never raises. Use enforce (or Matcher.match(validate=...)) to act on them.

`xcalib.protocol.CameraIntrinsics` `dataclass` ¶

Pinhole intrinsics, pixels. Distortion follows OpenCV ordering.

`K` `property` ¶

3x3 camera matrix.

`dist_coeffs` `property` ¶

Distortion coefficients as a float64 vector (zeros when None).

`xcalib.protocol.ProtocolError` ¶

Bases: ValueError

Raised when frame inputs violate the protocol (see docs/protocol.md).

`xcalib.protocol.ProtocolViolation` `dataclass` ¶

One contract violation found by validate_frame_inputs.

API reference¶

Matcher¶

xcalib.engine.matcher.Matcher ¶

from_pretrained(model, weights=None, config=None, device=None, *, site='a9_dataset_r02_s01', repo_id=None, revision=None, token=None) classmethod ¶

match(image, point_cloud, bboxes_2d, bboxes_3d, top_k=1, match_threshold=0.0, return_latency=False, validate='warn') ¶

pair(*args, **kwargs) ¶

match_frame(frame) ¶

calibrate(image, point_cloud, bboxes_2d, bboxes_3d, intrinsics, *, match_threshold=0.5, ransac_reproj_px=8.0, validate='warn') ¶

oneshot(intrinsics, **kwargs) ¶

train(train_data, val_data, *, output_dir='runs/finetune', **kwargs) ¶

build(target='onnx', output_dir=None, *, precision='fp16', device=None, onnx_dir=None, trtexec=None, extra_trt_args=(), verify=True) ¶

save_pretrained(save_dir) ¶

Results¶

xcalib.engine.matcher.MatchResult dataclass ¶

xcalib.engine.exporter.BuildResult dataclass ¶

xcalib.oneshot.calibration.CalibrationResult dataclass ¶

extrinsics property ¶

project(points_3d) ¶

Training¶

Datasets¶

xcalib.hub.datasets.load_dataset(site, split='test', *, repo_id=None, revision=None, token=None, local=None) ¶

Visualization¶

xcalib.visualization.draw_matching_overlay(image, point_cloud, bboxes_2d, bboxes_3d, matches, *, output_height=420, match_threshold=None) ¶

xcalib.visualization.draw_calibration_overlay(image, point_cloud, *, projection=None, intrinsics=None, rotation=None, translation=None, bboxes_3d=None, max_points=6000) ¶

Input protocol¶

xcalib.protocol.validate_frame_inputs(image, point_cloud, bboxes_2d, bboxes_3d) ¶

xcalib.protocol.CameraIntrinsics dataclass ¶

K property ¶

dist_coeffs property ¶

xcalib.protocol.ProtocolError ¶

xcalib.protocol.ProtocolViolation dataclass ¶

`xcalib.engine.matcher.Matcher` ¶

`from_pretrained(model, weights=None, config=None, device=None, *, site='a9_dataset_r02_s01', repo_id=None, revision=None, token=None)` `classmethod` ¶

`match(image, point_cloud, bboxes_2d, bboxes_3d, top_k=1, match_threshold=0.0, return_latency=False, validate='warn')` ¶

`pair(*args, **kwargs)` ¶

`match_frame(frame)` ¶

`calibrate(image, point_cloud, bboxes_2d, bboxes_3d, intrinsics, *, match_threshold=0.5, ransac_reproj_px=8.0, validate='warn')` ¶

`oneshot(intrinsics, **kwargs)` ¶

`train(train_data, val_data, *, output_dir='runs/finetune', **kwargs)` ¶

`build(target='onnx', output_dir=None, *, precision='fp16', device=None, onnx_dir=None, trtexec=None, extra_trt_args=(), verify=True)` ¶

`save_pretrained(save_dir)` ¶

`xcalib.engine.matcher.MatchResult` `dataclass` ¶

`xcalib.engine.exporter.BuildResult` `dataclass` ¶

`xcalib.oneshot.calibration.CalibrationResult` `dataclass` ¶

`extrinsics` `property` ¶

`project(points_3d)` ¶

`xcalib.hub.datasets.load_dataset(site, split='test', *, repo_id=None, revision=None, token=None, local=None)` ¶

`xcalib.visualization.draw_matching_overlay(image, point_cloud, bboxes_2d, bboxes_3d, matches, *, output_height=420, match_threshold=None)` ¶

`xcalib.visualization.draw_calibration_overlay(image, point_cloud, *, projection=None, intrinsics=None, rotation=None, translation=None, bboxes_3d=None, max_points=6000)` ¶

`xcalib.protocol.validate_frame_inputs(image, point_cloud, bboxes_2d, bboxes_3d)` ¶

`xcalib.protocol.CameraIntrinsics` `dataclass` ¶

`K` `property` ¶

`dist_coeffs` `property` ¶

`xcalib.protocol.ProtocolError` ¶

`xcalib.protocol.ProtocolViolation` `dataclass` ¶