pixano_inference.models.vlm

VLM (Vision-Language Model) base class.

The I/O types live in :mod:pixano_inference_client.vlm and are re-exported here so from pixano_inference.models.vlm import VLMInput keeps working.

VLMModel(config)

Bases: InferenceModel

Base class for vision-language models.

Example

@register_model("my-vlm")
class MyVLM(VLMModel):
    def load_model(self):
        self.model = load_weights(self.config.model_params["path"])

    def predict(self, input: VLMInput) -> VLMOutput:
        text = self.model.generate(input.prompt, input.images)
        return VLMOutput(generated_text=text, usage=..., generation_config=...)

Source code in pixano_inference/models/base.py

def __init__(self, config: ModelDeploymentConfig) -> None:
    """Initialize the model with deployment config.

    Args:
        config: Model deployment configuration.
    """
    self._config = config

predict(input) abstractmethod

Run vision-language generation.

Parameters:

Name	Type	Description	Default
input	VLMInput	VLM input with prompt, images, and generation parameters.	required

Returns:

Type	Description
VLMOutput	VLM output with generated text, usage info, and generation config.

Source code in pixano_inference/models/vlm.py

@abstractmethod
def predict(self, input: VLMInput) -> VLMOutput:
    """Run vision-language generation.

    Args:
        input: VLM input with prompt, images, and generation parameters.

    Returns:
        VLM output with generated text, usage info, and generation config.
    """