MapillaryFeatureExtractor

MapillaryFeatureExtractor#

class rapidtools.processing.MapillaryFeatureExtractor(classes: Sequence[str], access_token: str | None = None, region: BoundingBox | PolygonRegion | None = None, start_date: str = '', end_date: str = '', filter_rapid_only: bool = True, detection_source: str = 'auto', label_mapper: Any = None, camera_height_m: float = 2.4, min_range_m: float = 2.0, max_range_m: float = 60.0, max_single_view_range_m: float = 30.0, witness_radius_m: float = 12.0, min_object_width_m: float = 1.0, object_height_m: float = 1.5, min_area_fraction: float | None = None, cluster_radius_m: float = 4.0, localization_method: str = 'tracks', bearing_sigma_deg: float = 0.75, max_merge_m: float | None = None, track_gap_frames: int = 2, min_path_distance_m: float = 1.5, object_size_m: float = 4.5, merge_pieces_of_neighbours: bool = True, close_range_m: float = 20.0, min_sighting_deg: float = 6.0, report_single_views: bool = False, abeam_window_deg: float = 0.0, reid: bool = False, reid_model: str = 'openai/clip-vit-base-patch16', reid_min_similarity: float | None = None, reid_max_distance_m: float = 6.0, reid_min_width_px: float = 100, reid_image_size: str = '2048', reid_embedder: Any = None, min_observations: int = 1, frame_spacing_m: float = 0.0, ego_filter: bool = True, ego_min_recurrence: float = 0.5, asset_type: str | None = None, id_prefix: str = 'street', save_directory: str | Path = 'street_detections', sam3_model: Any = None, sam3_model_id: str = 'facebook/sam3', sam3_threshold: float = 0.5, sam3_mask_threshold: float = 0.5, sam3_image_size: str = '1024', sam3_batch_size: int = 4, load_in_4bit: bool = True, device: str = 'auto', max_workers: int = 10, frame_batch_size: int = 200, simplify_tolerance: float = 0.002, cancel_event: Event | None = None, client: MapillaryClient | None = None)[source]#

Bases: object

Discover objects along a Mapillary survey and place them on the ground.

The extractor lists every image in a region (optionally only RAPID Facility uploads within a date window), reads Mapillary’s segmentation detections for the requested classes as metadata, estimates where each detected object stands from the camera pose and the detection’s position in the frame, and merges repeated sightings into one asset per object. Objects seen from two or more camera positions are triangulated; single sightings fall back to the one-view estimate and are labelled as such.

Detections of the collection vehicle itself recur at the same place in every frame of a sequence and are removed before localisation.

Classes outside the Mapillary vocabulary can be detected with SAM 3 on downsampled thumbnails when detection_source allows it; the default only does so for classes Mapillary cannot supply.

Parameters:

  • classes (Sequence[str]) – Objects to find, in plain English ('cars', 'utility poles') or as exact Mapillary labels ('object--vehicle--truck').

  • access_token (str | None) – Mapillary API token. Not needed when client is given.

  • region (BoundingBox | PolygonRegion | None) – Area to survey. May instead be passed when the extractor is called.

  • start_date (str) – Inclusive lower bound on capture date (YYYY-MM-DD).

  • end_date (str) – Inclusive upper bound on capture date.

  • filter_rapid_only (bool) – Only use imagery uploaded by the RAPID Facility. Defaults to True.

  • detection_source (str) – 'auto' (Mapillary map features for static point classes such as utility poles and hydrants, Mapillary detections for the rest, SAM 3 for unresolved classes), 'mapillary' (detections only; skip unresolved classes with a warning), 'sam3' (SAM 3 for every class) or 'map_features' (only Mapillary’s pre-triangulated map features; other classes are skipped).

  • label_mapper (Any | None) – Optional MapillaryLabelMapper to translate class names the built-in aliases do not cover.

  • camera_height_m (float) – Camera height above the road, used for single-view ranges. Defaults to 2.4 (roof-mounted rig).

  • min_range_m (float) – Closest plausible object. Defaults to 2.0.

  • max_range_m (float) – Farthest triangulated position reported, and the farthest single-view range kept as a prior. Defaults to 60.0.

  • max_single_view_range_m (float) – A single-view object whose nearest sighting is farther than this is not reported; beyond it the ground-contact range is a guess and the track tends to hop between neighbouring objects. 0 keeps them. Defaults to 30.0.

  • witness_radius_m (float) – Negative-evidence check: a weak object (a single view, or a triangulation never seen closer than 25 m) that same-day frames within this distance had in view, without any detection of the class along its bearing, is dropped (at least two such frames, outnumbering frames that did see it, tallied per survey day). 0 disables the check. Defaults to 12.0.

  • min_object_width_m (float) – Objects narrower than this, judged from the outline’s angular width at the estimated distance, are fragments or false detections and are dropped. Resolution-independent and tolerant of partial occlusion. Defaults to 1.0.

  • object_height_m (float) – Typical height of the class. When a track’s ground-contact ranges disagree with its triangulation, the track is called moving only if enough of an object this tall is visible for occlusion to be ruled out. Defaults to 1.5 (vehicles).

  • min_area_fraction (float | None) – Deprecated. Pixel-area floor below which a detection is ignored; None (the default) keeps only a tiny floor for decoding noise. Size is judged with min_object_width_m instead.

  • cluster_radius_m (float) – Sightings closer than this are the same object in the legacy 'cluster' method; with 'tracks' it only sets the default max_merge_m (1.5 times this value). Defaults to 4.0.

  • localization_method (str) – How sightings become objects. 'tracks' (default) links detections frame to frame by bearing, triangulates each track robustly, drops moving vehicles and merges estimates with a covariance-aware gate (see street_tracking). 'voting' accumulates rays on a ground grid and takes the peaks, a baseline to compare against. 'cluster' is the previous behaviour: single-view positions clustered by radius.

  • bearing_sigma_deg (float) – One-sigma bearing error of a sighting. Defaults to 0.75.

  • max_merge_m (float | None) – Farthest two estimates can be and still be merged into one object. None uses 1.5 * cluster_radius_m.

  • track_gap_frames (int) – Frames a track may miss before it is closed. Defaults to 2.

  • min_path_distance_m (float) – Objects closer than this to the line the camera drove are dropped; the survey vehicle passed through that spot. Defaults to 1.5.

  • object_size_m (float) – Object footprint used by the voting method. Defaults to 4.5.

  • merge_pieces_of_neighbours (bool) – After localisation, fold an object whose closest sightings show a better-located neighbour into that neighbour: the second track left by a split outline (a car cut in two by a pole, or into a front and a rear) or by the tracker sliding between vehicles lined up along the line of sight. Such pairs share frames, which the position merge takes as proof of two objects, so they are judged on what their sightings show (see merge_pieces()). Defaults to True.

  • close_range_m (float) – Range within which an object must have been seen to be reported. Ground-contact ranges read from farther away are off by a third and the tracker hops between vehicles lined up down the road, so an object never seen this close whose closest rays point at a vehicle within 15 m already placed from the close pass is a far sighting of that vehicle, and is dropped unless it is well triangulated regardless (parallax of 45 degrees or more and uncertainty of half a metre or less). One whose rays point elsewhere, a car set back in a driveway or down a side street, stays. The piece merge judges sightings by the same range. 0 keeps every object. Defaults to 20.

  • min_sighting_deg (float) – An object must have been seen at least once with an outline this wide or tall, in degrees of the panorama (6 degrees is 34 pixels on a 2048-pixel thumbnail, 225 on the original). What was never more than a speck cannot be identified by anyone and is not reported; a drive-by survey sees anything worth counting properly in some frame. 0 keeps every object. Defaults to 6.

  • report_single_views (bool) – Keep objects placed from ground-contact ranges alone, without parallax (localization='single_view'). Their positions are guesses; defaults to False.

  • abeam_window_deg (float) – Objects are built only from the sightings made while passing them, within this many degrees of abeam on either side of the direction of travel. A drive-by survey passes everything worth counting, and the passing frames carry the widest parallax and the largest outlines; the frames looking ahead or behind down the road cannot place anything and are where tracks slide between vehicles that line up. Those sightings are attached afterwards to the objects their rays point at, so records and crops stay complete, but never create an object. Off (0) by default: the track classification is tuned on tracks that include the approach and a pass alone trips it, losing real vehicles on the surveys tried. Defaults to 0.

  • reid (bool) – After localisation, compare the appearance of objects from different sequences that lie within reid_max_distance_m and merge look-alikes, removing double counts from repeated passes. Downloads one thumbnail per compared object and loads reid_model. Appearance only ever adds merges: an object without a sighting at least reid_min_width_px wide is left as geometry placed it. Defaults to False.

  • reid_model (str) – Hugging Face vision backbone for the embeddings. Defaults to 'openai/clip-vit-base-patch16'; DINOv2 checkpoints work too.

  • reid_min_similarity (float | None) – Cosine similarity at which two crops count as the same object. None (the default) takes the value recommended for reid_model: 0.89 for CLIP, 0.83 for DINOv2.

  • reid_max_distance_m (float) – Only objects this close are compared. Defaults to 6.0.

  • reid_min_width_px (float) – Narrowest detection whose crop is embedded, in pixels of a 2048-pixel-wide thumbnail (scaled to reid_image_size). On the Spokane survey, crops below this width scored as alike whether or not they showed the same vehicle. 0 embeds every object. Defaults to 100.

  • reid_image_size (str) – Thumbnail size the re-identification crops are cut from: '2048' (default, the size the thresholds were tuned on) or '1024'.

  • reid_embedder (Any | None) – A ready AppearanceEmbedder (or anything with embed(images)) to use instead of loading reid_model.

  • min_observations (int) – Distinct frames an object must have been seen in; 3 is a sound choice for a survey sampled every few metres. Defaults to 1.

  • frame_spacing_m (float) – Keep one frame per this many metres along each sequence before detecting; 0 keeps every frame. Defaults to 0.

  • ego_filter (bool) – Remove detections of the collection vehicle. Defaults to True.

  • ego_min_recurrence (float) – Fraction of a sequence’s frames a box must recur in to count as the collection vehicle. Defaults to 0.5.

  • asset_type (str | None) – asset_type attribute written to every asset. Defaults to the class name each object was found under.

  • id_prefix (str) – Prefix of the generated asset IDs. Defaults to 'street'.

  • save_directory (str | Path) – Where SAM 3 thumbnails are cached when that source is used.

  • sam3_model (Any | None) – Pre-loaded SAM 3 wrapper. Loaded on demand otherwise.

  • sam3_model_id (str) – Hugging Face repository of the SAM 3 weights. Defaults to 'facebook/sam3'.

  • sam3_threshold (float) – SAM 3 detection confidence threshold. Defaults to 0.5.

  • sam3_mask_threshold (float) – SAM 3 mask binarisation threshold. Defaults to 0.5.

  • sam3_image_size (str) – Thumbnail size SAM 3 runs on: '1024' (default) or '2048'.

  • sam3_batch_size (int) – Thumbnails per SAM 3 inference call. Defaults to 4.

  • load_in_4bit (bool) – Load SAM 3 with 4-bit quantisation. Defaults to True.

  • device (str) – Compute device for SAM 3 ('auto', 'cuda', 'cpu').

  • max_workers (int) – Threads for metadata and thumbnail downloads. Defaults to 10.

  • frame_batch_size (int) – Frames whose metadata is fetched and converted at a time. Each image’s detection payload (every Mapillary label, as encoded geometry) is dropped as soon as the requested classes have been read from it, so memory use is bounded by one batch rather than by the size of the survey. Defaults to 200.

  • simplify_tolerance (float) – Douglas-Peucker tolerance, in normalised image units, applied to every detection outline before it is stored (0.002 is about four pixels of a 2048-wide image), capped at 1 % of the outline’s longest side so small, distant objects keep their detail. Mapillary outlines trace every pixel step of a mask, so this cuts the memory of a city-wide run several-fold without changing which objects are found. 0 keeps the raw outlines. Defaults to 0.002.

  • cancel_event (Event | None) – Cooperative cancellation flag checked between images and batches.

  • client (MapillaryClient | None) – Pre-configured client; one is created from access_token otherwise.

Example

>>> from rapidtools.core import BoundingBox
>>> extractor = MapillaryFeatureExtractor(
...     classes=['vehicles'],
...     access_token='MLY|...',
...     start_date='2025-08-01',
...     min_observations=2,
... )
>>> region = BoundingBox(-117.45, 47.69, -117.44, 47.70)
>>> vehicles = extractor(region)
>>> vehicles[0].attributes['localization']
'triangulated'