pixano_inference.models.base

Shared base class for deployed inference models.

InferenceModel(config)

Bases: ABC

Abstract base class for all inference models deployed on Ray Serve.

Subclass this to implement custom inference models that can be deployed on Ray Serve.

Example

from pixano_inference.models import InferenceModel, register_model

@register_model("my_model")
class MyModel(InferenceModel):
    def load_model(self) -> None:
        self._model = ...  # Load your model

    def predict(self, input: MyInput) -> MyOutput:
        return MyOutput(result=self._model(input))

Parameters:

Name	Type	Description	Default
config	ModelDeploymentConfig	Model deployment configuration.	required

Source code in pixano_inference/models/base.py

def __init__(self, config: ModelDeploymentConfig) -> None:
    """Initialize the model with deployment config.

    Args:
        config: Model deployment configuration.
    """
    self._config = config

capability property

Capability handled by this model.

config property

Model deployment configuration.

metadata property

Model metadata. Override for custom metadata.

model_name property

Unique model name.

load_model() abstractmethod

Load model artifacts.

Called once in the Ray actor __init__. Implement this to load weights, initialize processors, etc.

Source code in pixano_inference/models/base.py

@abstractmethod
def load_model(self) -> None:
    """Load model artifacts.

    Called once in the Ray actor ``__init__``. Implement this to load
    weights, initialize processors, etc.
    """

predict(input) abstractmethod

Run inference.

Parameters:

Name	Type	Description	Default
input	BaseModel	Task-specific Input object (subclasses narrow this type).	required

Returns:

Type	Description
BaseModel	Task-specific Output object (subclasses narrow this type).

Source code in pixano_inference/models/base.py

@abstractmethod
def predict(self, input: BaseModel) -> BaseModel:
    """Run inference.

    Args:
        input: Task-specific Input object (subclasses narrow this type).

    Returns:
        Task-specific Output object (subclasses narrow this type).
    """

unload()

Free resources. Override for custom cleanup.

Source code in pixano_inference/models/base.py

def unload(self) -> None:
    """Free resources. Override for custom cleanup."""