GenerationConfig

GenerationConfig#

class rapidtools.models.GenerationConfig(temperature: float | None = None, max_tokens: int | None = None, json_mode: bool = False, system_instruction: str | None = None)[source]#

Bases: object

Text-generation options shared by every vision-language wrapper.

Variables:

  • temperature (float | None) – Sampling temperature; None keeps the model default.

  • max_tokens (int | None) – Maximum number of generated tokens; None keeps the model default.

  • json_mode (bool) – Ask for a JSON-only answer (native flag on APIs that support it, prompt instruction otherwise).

  • system_instruction (str | None) – System prompt override for this call.

Example

>>> from rapidtools.models import GenerationConfig
>>> GenerationConfig(temperature=0.0, json_mode=True).json_mode
True

Methods

merged

Return a copy with the non-None overrides applied.
merged(**overrides: Any) → GenerationConfig[source]#

Return a copy with the non-None overrides applied.

Example

>>> GenerationConfig(temperature=0.4).merged(max_tokens=5).max_tokens
5