Gemma4Inference

class rapidtools.models.Gemma4Inference(model_id: str = 'google/gemma-4-E2B-it', device: str = 'auto', temperature: float = 0.4, max_tokens: int = 2048)

Bases: BaseLocalInferenceModel

Local inference model for Google’s Gemma 4 vision-language models.

Supports every instruction-tuned Gemma 4 size (31B, 26B-A4B, 12B, E4B, E2B). Images are loaded with the robust PIL loader of BaseLocalInferenceModel and generation runs sequentially (or in explicit batches through run_inference_batch()) to protect GPU memory.

Example

>>> from rapidtools.models import Gemma4Inference
>>> Gemma4Inference.list_available_models()[-1]
'google/gemma-4-E2B-it'
>>> model = Gemma4Inference('google/gemma-4-E4B-it', device='cuda')
>>> out = model.run_inference(
...     'house.jpg', 'Rate the damage from 0 (none) to 5 (destroyed).'
... )
>>> out.text
'4'

Methods

list_available_models	Return the curated instruction-tuned Gemma 4 checkpoints.
run_batch	Process assets sequentially to avoid GPU out-of-memory crashes.
run_inference	Run multimodal inference on a single prompt and image(s).
run_inference_batch	Run batched multimodal inference on several prompt/image sets.

classmethod list_available_models(auth_key: str | None = None) → list[str]

Return the curated instruction-tuned Gemma 4 checkpoints.

Parameters:

auth_key – Unused; present for signature parity with API wrappers.

Returns:

list[str] – Repository IDs, largest first.

Example

>>> from rapidtools.models import Gemma4Inference
>>> Gemma4Inference.list_available_models()[0]
'google/gemma-4-31B-it'

run_inference(image_inputs: str | Path | list[str | Path] | None, prompt: str | Path, **kwargs: Any) → ModelOutput | None

Run multimodal inference on a single prompt and image(s).

Parameters:

• image_inputs – A path/URL, a list of them, or None/[] for a text-only prompt.

• prompt – Prompt text or path to a prompt file.

• **kwargs – temperature, max_tokens, json_mode overrides or a config=GenerationConfig(...).

Returns:

ModelOutput | None – Generated text and raw response, or None when every requested image failed to load or generation raised.

Example

>>> from rapidtools.models import Gemma4Inference
>>> model = Gemma4Inference()
>>> model.run_inference([], 'Reply with OK.').text
'OK'

run_inference_batch(batch_image_inputs: list[list[str | Path]], batch_prompts: list[str], **kwargs: Any) → list[ModelOutput | None]

Run batched multimodal inference on several prompt/image sets.

Assets whose images all fail to load receive None at their index so the output aligns with the input. If the batched call itself raises, every index is None.

Parameters:

• batch_image_inputs – One list of image paths/URLs per asset.

• batch_prompts – One prompt per asset (same length).

• **kwargs – temperature and max_tokens overrides.

Returns:

list[ModelOutput | None] – Results aligned with the inputs.

Example

>>> from rapidtools.models import Gemma4Inference
>>> model = Gemma4Inference()
>>> outs = model.run_inference_batch(
...     [['a.jpg'], ['b.jpg']], ['Rate damage 0-5.'] * 2
... )
>>> [o.text for o in outs]
['1', '4']