HFVisionInference

class rapidtools.models.HFVisionInference(model_id: str | None = None, device: str = 'auto', load_in_4bit: bool = False, temperature: float = 0.4, max_tokens: int = 512, trust_remote_code: bool = False)

Bases: BaseLocalInferenceModel

Run any Hugging Face vision-language checkpoint that ships a chat template.

Family-specific subclasses override the class attributes below.

Variables:

• MODEL_CATALOG (list[dict[str, str]]) – Curated checkpoint records for list_available_models().

• DEFAULT_MODEL (str) – Checkpoint used when model_id is omitted.

• MIN_TRANSFORMERS_VERSION (str | None) – Minimum Transformers release required by the family; a clear ImportError is raised when it is not met.

• FAMILY_NAME (str) – Human-readable family label used in logs.

Example

>>> from rapidtools.models import HFVisionInference
>>> model = HFVisionInference('Qwen/Qwen2.5-VL-3B-Instruct', load_in_4bit=True)
>>> out = model.run_inference('roof.jpg', 'Describe the roof damage.')
>>> print(out.text)
>>> batch = model.run_inference_batch(
...     [['a.jpg'], ['b.jpg']], ['Rate damage 0-5.', 'Rate damage 0-5.']
... )
>>> [o.text for o in batch]
['2', '4']

Methods

list_available_models	Return the curated catalogue of checkpoints for this family.
run_batch	Process assets sequentially to avoid GPU out-of-memory crashes.
run_inference	Run the model on one prompt and its image(s).
run_inference_batch	Run the model on several prompt/image sets at once (left-padded).
search_hub_models	List the most downloaded image-text-to-text checkpoints on the Hub.

classmethod list_available_models(auth_key: str | None = None) → list[str]

Return the curated catalogue of checkpoints for this family.

Parameters:

auth_key – Unused; present for signature parity with API wrappers.

Returns:

list[str] – Hugging Face repository IDs, newest first.

Example

>>> from rapidtools.models import HFVisionInference
>>> 'google/gemma-3-4b-it' in HFVisionInference.list_available_models()
True

classmethod search_hub_models(limit: int = 40) → list[str]

List the most downloaded image-text-to-text checkpoints on the Hub.

The curated catalogue is returned first; Hub results that are not already listed are appended. Falls back to the catalogue alone when the Hub cannot be reached.

Parameters:

limit – Maximum number of Hub results to fetch.

Returns:

list[str] – Repository IDs.

Example

>>> from rapidtools.models import HFVisionInference
>>> ids = HFVisionInference.search_hub_models(limit=10)
>>> ids[0]
'Qwen/Qwen3.8-27B'

run_inference(image_inputs: str | Path | list[str | Path], prompt: str | Path, **kwargs: Any) → ModelOutput | None

Run the model on one prompt and its image(s).

Parameters:

• image_inputs – A path/URL, a list of them, or None/[] for a text-only prompt.

• prompt – Prompt text or path to a prompt file.

• **kwargs – temperature, max_tokens, json_mode overrides or a config=GenerationConfig(...).

Returns:

ModelOutput | None – Generated text, or None when every image failed to load or generation raised.

Example

>>> from rapidtools.models import HFVisionInference
>>> model = HFVisionInference('Qwen/Qwen2.5-VL-3B-Instruct')
>>> model.run_inference([], 'Reply with the word ready.').text
'ready'

run_inference_batch(batch_image_inputs: list[list[str | Path]], batch_prompts: list[str], **kwargs: Any) → list[ModelOutput | None]

Run the model on several prompt/image sets at once (left-padded).

Entries whose images all failed to load yield None at their index. If the batched call raises (typically out-of-memory), the batch is retried one asset at a time.

Parameters:

• batch_image_inputs – One list of image paths per asset.

• batch_prompts – One prompt per asset (same length).

• **kwargs – temperature and max_tokens overrides.

Returns:

list[ModelOutput | None] – Results aligned with the inputs.

Example

>>> from rapidtools.models import HFVisionInference
>>> model = HFVisionInference('Qwen/Qwen2.5-VL-3B-Instruct')
>>> outs = model.run_inference_batch(
...     [['a.jpg'], ['missing.jpg']], ['Describe.', 'Describe.']
... )
>>> outs[1] is None
True