Skip to content

pixano_inference.models.base

Shared base class for deployed inference models.

InferenceModel(config)

Bases: ABC

Abstract base class for all inference models deployed on Ray Serve.

Subclass this to implement custom inference models that can be deployed on Ray Serve.

Example 
from pixano_inference.models import InferenceModel, register_model

@register_model("my_model")
class MyModel(InferenceModel):
    def load_model(self) -> None:
        self._model = ...  # Load your model

    def predict(self, input: MyInput) -> MyOutput:
        return MyOutput(result=self._model(input))

Parameters:

Name Type Description Default
config ModelDeploymentConfig

Model deployment configuration.

 required
Source code in pixano_inference/models/base.py 
47
48
49
50
51
52
53
def __init__(self, config: ModelDeploymentConfig) -> None:
    """Initialize the model with deployment config.

    Args:
        config: Model deployment configuration.
    """
    self._config = config

capability property

Capability handled by this model.

config property

Model deployment configuration.

metadata property

Model metadata. Override for custom metadata.

model_name property

Unique model name.

load_model() abstractmethod

Load model artifacts.

Called once in the Ray actor __init__. Implement this to load weights, initialize processors, etc.

Source code in pixano_inference/models/base.py 
79
80
81
82
83
84
85
@abstractmethod
def load_model(self) -> None:
    """Load model artifacts.

    Called once in the Ray actor ``__init__``. Implement this to load
    weights, initialize processors, etc.
    """

predict(input) abstractmethod

Run inference.

Parameters:

Name Type Description Default
input BaseModel

Task-specific Input object (subclasses narrow this type).

 required

Returns:

Type Description
BaseModel

Task-specific Output object (subclasses narrow this type).
Source code in pixano_inference/models/base.py 
87
88
89
90
91
92
93
94
95
96
@abstractmethod
def predict(self, input: BaseModel) -> BaseModel:
    """Run inference.

    Args:
        input: Task-specific Input object (subclasses narrow this type).

    Returns:
        Task-specific Output object (subclasses narrow this type).
    """

predict_batch(inputs)

Run inference on a batch of inputs.

Only used when the deployment sets max_batch_size > 1. The default runs :meth:predict sequentially; override it to exploit true batched execution (e.g. a single padded forward pass).

Parameters:

Name Type Description Default
inputs list[BaseModel]

A list of task-specific Input objects.

 required

Returns:

Type Description
list[BaseModel]

A list of task-specific Output objects, one per input, in order.
Source code in pixano_inference/models/base.py 
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
def predict_batch(self, inputs: list[BaseModel]) -> list[BaseModel]:
    """Run inference on a batch of inputs.

    Only used when the deployment sets ``max_batch_size > 1``. The default runs
    :meth:`predict` sequentially; override it to exploit true batched execution
    (e.g. a single padded forward pass).

    Args:
        inputs: A list of task-specific Input objects.

    Returns:
        A list of task-specific Output objects, one per input, in order.
    """
    return [self.predict(inp) for inp in inputs]

unload()

Free resources. Override for custom cleanup.

Source code in pixano_inference/models/base.py 
113
114
def unload(self) -> None:
    """Free resources. Override for custom cleanup."""