pixano_inference.ray.config

Ray Serve configuration models.

ResourceConfig, AutoscalingConfig and ModelDeploymentConfig live in :mod:pixano_inference.configs.deployment; they are re-exported here so from pixano_inference.ray.config import ModelDeploymentConfig keeps working.

RayServeConfig(**data)

Bases: BaseModel

Top-level Ray Serve configuration.

Attributes:

Name	Type	Description
host	str	Host to bind to. Defaults to loopback; bind 0.0.0.0 explicitly to expose the server (do so only with API-key auth enabled — see pixano_inference.security).
port	int	Port to serve on.
num_cpus	int | None	Total number of CPUs available to Ray. None means auto-detect.
num_gpus	int | None	Total number of GPUs available to Ray. None means auto-detect.
pip_packages	list[str] | None	List of pip packages to install in Ray workers runtime environment.
working_dir	str | None	Working directory for Ray workers.
models	list[ModelDeploymentConfig]	List of models to deploy at startup.
default_resources	ResourceConfig	Default resource configuration for deployments.
default_autoscaling	AutoscalingConfig	Default autoscaling configuration for deployments.
serve_start_timeout_s	float	How long to wait for Ray Serve's controller to come up before aborting startup. Ray's own serve.start() waits forever, so this is the only bound on a cluster that can never schedule the controller actor.
deploy_timeout_s	float	How long to wait for a model's Serve application to reach RUNNING.

Source code in pydantic/main.py

def __init__(self, /, **data: Any) -> None:
    """Create a new model by parsing and validating input data from keyword arguments.

    Raises [`ValidationError`][pydantic_core.ValidationError] if the input data cannot be
    validated to form a valid model.

    `self` is explicitly positional-only to allow `self` as a field name.
    """
    # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    __tracebackhide__ = True
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    if self is not validated_self:
        warnings.warn(
            'A custom validator is returning a value other than `self`.\n'
            "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
            f'See the `model_validator` docs (https://pydantic.dev/docs/validation/{version_short()}/concepts/validators/#model-validators) for more details.',
            stacklevel=2,
        )