QwenVisionInference

class rapidtools.models.QwenVisionInference(model_id: str | None = None, device: str = 'auto', load_in_4bit: bool = False, temperature: float = 0.4, max_tokens: int = 512, trust_remote_code: bool = False)

Bases: HFVisionInference

Local inference with open-weight Qwen vision-language checkpoints.

A specialisation of HFVisionInference whose catalogue lists the Qwen3.8 / Qwen3.6 / Qwen3.5 natively multimodal models and the Qwen3-VL / Qwen2.5-VL families.

Example

>>> from rapidtools.models import QwenVisionInference
>>> QwenVisionInference.list_available_models()[0]
'Qwen/Qwen3.8-27B'
>>> model = QwenVisionInference('Qwen/Qwen3.5-9B', load_in_4bit=True)
>>> model.run_inference('roof.jpg', 'Describe the roof.').text

Methods

list_available_models	Return the curated catalogue of checkpoints for this family.
run_batch	Process assets sequentially to avoid GPU out-of-memory crashes.
run_inference	Run the model on one prompt and its image(s).
run_inference_batch	Run the model on several prompt/image sets at once (left-padded).
search_hub_models	List the most downloaded image-text-to-text checkpoints on the Hub.