SAM3ImageSegmenter

SAM3ImageSegmenter#

class rapidtools.processing.SAM3ImageSegmenter(prompt: str | list[str] = '', model_id: str = 'facebook/sam3', device: str = 'auto', load_in_4bit: bool = True, batch_size: int = 4, threshold: float = 0.5, mask_threshold: float = 0.5, image_filter: Callable[[ImageAsset], bool] | None = None, cancel_event: Event | None = None)[source]#

Bases: object

Pipeline component that uses local SAM 3 to segment images attached to assets.

This segmenter gathers filtered images across all PhysicalAsset objects, batches them together (to maximize GPU utilization without causing OOM errors), and stores the resulting segmentation masks directly in the corresponding asset’s attributes under the following keys:

  • 'sam3_masks': {image_id: masks} where masks is the per-image mask stack returned by the model (typically an (N, H, W) boolean array).

  • 'sam3_bounding_boxes': {image_id: boxes} (only when the model returns boxes).

  • 'ai_model_used': The Hugging Face model ID that was used.

Parameters:

  • prompt (str | list[str], optional) – The text prompt (or list of prompts) to guide the segmentation (e.g., 'building' or ['building', 'tree']). Lists are joined into a single period-separated prompt. Defaults to ''.

  • model_id (str, optional) – The Hugging Face repository ID for the SAM 3 model. Defaults to 'facebook/sam3'.

  • device (str, optional) – The compute device to use ('cuda', 'cpu', 'auto'). Defaults to 'auto'.

  • load_in_4bit (bool, optional) – Whether to load the model using 4-bit quantization. Defaults to True.

  • batch_size (int, optional) – Number of images to process simultaneously across assets. Defaults to 4.

  • threshold (float, optional) – Confidence threshold for predictions. Defaults to 0.5.

  • mask_threshold (float, optional) – Threshold for binarizing the masks. Defaults to 0.5.

  • image_filter (Callable[[ImageAsset], bool] | None, optional) – A function that takes an ImageAsset and returns True if the image should be segmented. If None, all downloaded images attached to the asset are processed. Defaults to None.

  • cancel_event (Event | None, optional) – Cooperative cancellation flag. When set, the segmenter raises OperationCancelled before starting the next batch. Defaults to None.

Example

Segment only the aerial crops of each asset:

>>> from rapidtools.processing import SAM3ImageSegmenter
>>> segmenter = SAM3ImageSegmenter(
...     prompt='building',
...     batch_size=8,
...     image_filter=lambda img: img.id.endswith('_aerial'),
... )
>>> collection = segmenter(my_collection)
>>> masks = collection['bldg_01'].attributes['sam3_masks']
>>> masks['bldg_01_aerial'].shape  # (num_instances, height, width)
(2, 512, 512)