Skip to content

pixano_inference.models.base

Shared base class for deployed inference models.

InferenceModel(config)

Bases: ABC, Generic[InputT, OutputT]

Abstract base class for all inference models deployed on Ray Serve.

The class is generic in the type predict receives and the type it returns. Each capability base class (SegmentationModel, DetectionModel, ...) fixes both, so a model subclasses the capability base and implements predict with that capability's Input and Output.

Example 
from pixano_inference.models import InferenceModel, register_model

@register_model("my_model")
class MyModel(InferenceModel[MyInput, MyOutput]):
    def load_model(self) -> None:
        self._model = ...  # Load your model

    def predict(self, input: MyInput) -> MyOutput:
        return MyOutput(result=self._model(input))

Parameters:

Name Type Description Default
config ModelDeploymentConfig

Model deployment configuration.

 required
Source code in pixano_inference/models/base.py 
53
54
55
56
57
58
59
def __init__(self, config: ModelDeploymentConfig) -> None:
    """Initialize the model with deployment config.

    Args:
        config: Model deployment configuration.
    """
    self._config = config

capability property

Capability handled by this model.

config property

Model deployment configuration.

interface property

How the model is called, for clients: the descriptor of its capability.

A capability's Input is the union of what every model of that capability might accept; the descriptor says what this model reads and returns (its prompts, its class set, its outputs...), so a client knows how to call it and what to show for it. The server fetches it once per deployment, after :meth:load_model, and publishes it on GET /v1/models; values that depend on the loaded weights (a class set, an embedding size) can therefore be read from the model. Each capability base class names the descriptor type it expects.

The default, None, means the model does not describe itself.

metadata property

Model metadata. Override for custom metadata.

model_name property

Unique model name.

load_model() abstractmethod

Load model artifacts.

Called once in the Ray actor __init__. Implement this to load weights, initialize processors, etc.

Source code in pixano_inference/models/base.py 
100
101
102
103
104
105
106
@abstractmethod
def load_model(self) -> None:
    """Load model artifacts.

    Called once in the Ray actor ``__init__``. Implement this to load
    weights, initialize processors, etc.
    """

predict(input) abstractmethod

Run inference.

Parameters:

Name Type Description Default
input InputT

The capability's Input object.

 required

Returns:

Type Description
OutputT

The capability's Output object.
Source code in pixano_inference/models/base.py 
108
109
110
111
112
113
114
115
116
117
@abstractmethod
def predict(self, input: InputT) -> OutputT:
    """Run inference.

    Args:
        input: The capability's Input object.

    Returns:
        The capability's Output object.
    """

predict_batch(inputs)

Run inference on a batch of inputs.

Only used when the deployment sets max_batch_size > 1. The default runs :meth:predict sequentially; override it to exploit true batched execution (e.g. a single padded forward pass).

Parameters:

Name Type Description Default
inputs list[InputT]

A list of task-specific Input objects.

 required

Returns:

Type Description
list[OutputT]

A list of task-specific Output objects, one per input, in order.
Source code in pixano_inference/models/base.py 
119
120
121
122
123
124
125
126
127
128
129
130
131
132
def predict_batch(self, inputs: list[InputT]) -> list[OutputT]:
    """Run inference on a batch of inputs.

    Only used when the deployment sets ``max_batch_size > 1``. The default runs
    :meth:`predict` sequentially; override it to exploit true batched execution
    (e.g. a single padded forward pass).

    Args:
        inputs: A list of task-specific Input objects.

    Returns:
        A list of task-specific Output objects, one per input, in order.
    """
    return [self.predict(inp) for inp in inputs]

unload()

Free resources. Override for custom cleanup.

Source code in pixano_inference/models/base.py 
134
135
def unload(self) -> None:
    """Free resources. Override for custom cleanup."""