pixano_inference.schemas.tracking

Tracking I/O types.

A tracking model follows objects through the frames of a video. It is either prompted (the request names the objects with points, boxes or masks, as SAM2 expects) or prompt-free (tracking by detection, as in ByteTrack: the model detects the objects and creates the tracks itself). Both return the same :class:TrackingOutput: for each frame, the objects tracked in it.

TrackedFrame(**data)

Bases: _BaseModel

The objects tracked in one frame.

Attributes:

Name	Type	Description
frame_index	int	Index of the frame, 0-based and relative to the submitted video frames.
objects	list[TrackedObject]	Objects tracked in the frame, each track at most once.

Source code in pydantic/main.py

def __init__(self, /, **data: Any) -> None:
    """Create a new model by parsing and validating input data from keyword arguments.

    Raises [`ValidationError`][pydantic_core.ValidationError] if the input data cannot be
    validated to form a valid model.

    `self` is explicitly positional-only to allow `self` as a field name.
    """
    # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    __tracebackhide__ = True
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    if self is not validated_self:
        warnings.warn(
            'A custom validator is returning a value other than `self`.\n'
            "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
            f'See the `model_validator` docs (https://pydantic.dev/docs/validation/{version_short()}/concepts/validators/#model-validators) for more details.',
            stacklevel=2,
        )

TrackedObject(**data)

Bases: _BaseModel

One tracked object in one frame.

A mask-based tracker (SAM2) fills mask; a detection-based tracker (ByteTrack) fills box and score. At least one of box and mask is present.

Attributes:

Name	Type	Description
track_id	int	Identity of the object across frames. A prompted model returns the object ID of the request; a prompt-free model assigns it.
box	list[float] | None	Bounding box [x1, y1, x2, y2] in pixels of the frame (top-left and bottom-right corners), or None.
score	float | None	Confidence of the object in this frame, or None.
class_name	str | None	Class name of the object, or None. Serialized as class.
mask	CompressedRLE | None	Mask of the object in compressed-RLE format, or None.

Source code in pydantic/main.py

def __init__(self, /, **data: Any) -> None:
    """Create a new model by parsing and validating input data from keyword arguments.

    Raises [`ValidationError`][pydantic_core.ValidationError] if the input data cannot be
    validated to form a valid model.

    `self` is explicitly positional-only to allow `self` as a field name.
    """
    # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    __tracebackhide__ = True
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    if self is not validated_self:
        warnings.warn(
            'A custom validator is returning a value other than `self`.\n'
            "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
            f'See the `model_validator` docs (https://pydantic.dev/docs/validation/{version_short()}/concepts/validators/#model-validators) for more details.',
            stacklevel=2,
        )

TrackingBoxPrompt(**data)

Bases: _BaseModel

Box prompt for a tracking keyframe.

Source code in pydantic/main.py

def __init__(self, /, **data: Any) -> None:
    """Create a new model by parsing and validating input data from keyword arguments.

    Raises [`ValidationError`][pydantic_core.ValidationError] if the input data cannot be
    validated to form a valid model.

    `self` is explicitly positional-only to allow `self` as a field name.
    """
    # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    __tracebackhide__ = True
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    if self is not validated_self:
        warnings.warn(
            'A custom validator is returning a value other than `self`.\n'
            "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
            f'See the `model_validator` docs (https://pydantic.dev/docs/validation/{version_short()}/concepts/validators/#model-validators) for more details.',
            stacklevel=2,
        )

TrackingInput(**data)

Bases: _BaseModel

Input for video tracking.

A prompted request names the objects to track: objects_ids with one keyframe (or one legacy point/box prompt) per object. A prompt-free request gives only the video, and optionally classes and box_threshold: the model detects the objects and assigns the track ids.

Attributes:

Name	Type	Description
video	list[str | Path | bytes] | str | Path | bytes	Path to the video, list of frame paths, or base64 encoded video/frames.
points	list[list[list[int]]] | None	Legacy point prompts [num_objects, num_points, 2].
labels	list[list[int]] | None	Legacy labels for points [num_objects, num_points].
boxes	list[list[int]] | None	Legacy box prompts [num_objects, 4].
propagate	bool | None	Whether to propagate masks beyond the prompted frames.
interval	TrackingInterval | None	Optional propagation interval relative to the provided frame window.
keyframes	list[TrackingKeyframe] | None	Optional structured prompt payloads for each object.
objects_ids	list[int]	IDs of the prompted objects. Empty for a prompt-free request.
frame_indexes	list[int]	Indexes of the prompted frames. Empty for a prompt-free request.
classes	list[str] | str | None	Class names to detect and track (prompt-free tracking). None means the model's own class set.
box_threshold	float | None	Minimum detection confidence for an object to be tracked (prompt-free tracking). None means the model's default.

Source code in pydantic/main.py

def __init__(self, /, **data: Any) -> None:
    """Create a new model by parsing and validating input data from keyword arguments.

    Raises [`ValidationError`][pydantic_core.ValidationError] if the input data cannot be
    validated to form a valid model.

    `self` is explicitly positional-only to allow `self` as a field name.
    """
    # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    __tracebackhide__ = True
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    if self is not validated_self:
        warnings.warn(
            'A custom validator is returning a value other than `self`.\n'
            "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
            f'See the `model_validator` docs (https://pydantic.dev/docs/validation/{version_short()}/concepts/validators/#model-validators) for more details.',
            stacklevel=2,
        )

TrackingInterval(**data)

Bases: _BaseModel

Optional propagation window relative to the provided video frames.

Source code in pydantic/main.py

def __init__(self, /, **data: Any) -> None:
    """Create a new model by parsing and validating input data from keyword arguments.

    Raises [`ValidationError`][pydantic_core.ValidationError] if the input data cannot be
    validated to form a valid model.

    `self` is explicitly positional-only to allow `self` as a field name.
    """
    # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    __tracebackhide__ = True
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    if self is not validated_self:
        warnings.warn(
            'A custom validator is returning a value other than `self`.\n'
            "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
            f'See the `model_validator` docs (https://pydantic.dev/docs/validation/{version_short()}/concepts/validators/#model-validators) for more details.',
            stacklevel=2,
        )

TrackingKeyframe(**data)

Bases: _BaseModel

Prompt payload for a single tracking keyframe.

Source code in pydantic/main.py

def __init__(self, /, **data: Any) -> None:
    """Create a new model by parsing and validating input data from keyword arguments.

    Raises [`ValidationError`][pydantic_core.ValidationError] if the input data cannot be
    validated to form a valid model.

    `self` is explicitly positional-only to allow `self` as a field name.
    """
    # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    __tracebackhide__ = True
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    if self is not validated_self:
        warnings.warn(
            'A custom validator is returning a value other than `self`.\n'
            "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
            f'See the `model_validator` docs (https://pydantic.dev/docs/validation/{version_short()}/concepts/validators/#model-validators) for more details.',
            stacklevel=2,
        )

TrackingOutput(**data)

Bases: _BaseModel

Output for video tracking: for each frame, the objects tracked in it.

This is the shape a tracker produces, one frame at a time: append one :class:TrackedFrame per frame, holding one :class:TrackedObject per object alive in that frame. Use :meth:tracks to read the same result as trajectories.

Attributes:

Name	Type	Description
frames	list[TrackedFrame]	Tracked frames, in the order the model processed them. A frame appears at most once; a frame where nothing is tracked may be omitted or have no objects.

Source code in pydantic/main.py

def __init__(self, /, **data: Any) -> None:
    """Create a new model by parsing and validating input data from keyword arguments.

    Raises [`ValidationError`][pydantic_core.ValidationError] if the input data cannot be
    validated to form a valid model.

    `self` is explicitly positional-only to allow `self` as a field name.
    """
    # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    __tracebackhide__ = True
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    if self is not validated_self:
        warnings.warn(
            'A custom validator is returning a value other than `self`.\n'
            "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
            f'See the `model_validator` docs (https://pydantic.dev/docs/validation/{version_short()}/concepts/validators/#model-validators) for more details.',
            stacklevel=2,
        )

tracks()

Group the result by track.

Returns:

Type	Description
dict[int, list[tuple[int, TrackedObject]]]	For each track ID, its (frame_index, object) pairs sorted by frame index.

Source code in pixano_inference/schemas/tracking.py

def tracks(self) -> dict[int, list[tuple[int, TrackedObject]]]:
    """Group the result by track.

    Returns:
        For each track ID, its ``(frame_index, object)`` pairs sorted by frame index.
    """
    tracks: dict[int, list[tuple[int, TrackedObject]]] = {}
    for frame in sorted(self.frames, key=lambda frame: frame.frame_index):
        for tracked in frame.objects:
            tracks.setdefault(tracked.track_id, []).append((frame.frame_index, tracked))
    return tracks

TrackingPointPrompt(**data)

Bases: _BaseModel

Point prompt for a tracking keyframe.

Source code in pydantic/main.py

def __init__(self, /, **data: Any) -> None:
    """Create a new model by parsing and validating input data from keyword arguments.

    Raises [`ValidationError`][pydantic_core.ValidationError] if the input data cannot be
    validated to form a valid model.

    `self` is explicitly positional-only to allow `self` as a field name.
    """
    # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    __tracebackhide__ = True
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    if self is not validated_self:
        warnings.warn(
            'A custom validator is returning a value other than `self`.\n'
            "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
            f'See the `model_validator` docs (https://pydantic.dev/docs/validation/{version_short()}/concepts/validators/#model-validators) for more details.',
            stacklevel=2,
        )