Options
All
  • Public
  • Public/Protected
  • All
Menu

Module @pixano/ai

@pixano/ai

Pixano toolbox of AI algorithms for smart annotation in the browser.

Import

import { PixelToBoundingBox } from "@pixano/ai/lib/pixel-to-bounding-box";

Example: Bounding box detection guided by a user click

Example usage:

const image = new Image();
image.onload = () => {
    const detection = await this.boundingBoxCreator.predict(
    {x: 200, y: 500},
    image
);
}
image.src = '/path/to/your/image.jpg';

Index

Functions

change

  • change(r: tf.Tensor2D): Tensor<Rank>
  • Parameters

    • r: tf.Tensor2D

    Returns Tensor<Rank>

cropImage

  • cropImage(image: HTMLImageElement | HTMLCanvasElement, roi: Rectangle): HTMLCanvasElement
  • Crop an image.

    Parameters

    • image: HTMLImageElement | HTMLCanvasElement
    • roi: Rectangle

    Returns HTMLCanvasElement

cropImage

  • cropImage(image: HTMLImageElement | HTMLCanvasElement, roi: Rectangle): HTMLCanvasElement
  • Crop an image.

    Parameters

    • image: HTMLImageElement | HTMLCanvasElement
    • roi: Rectangle

    Returns HTMLCanvasElement

cxy_wh_2_rect

  • cxy_wh_2_rect(pos: number[], sz: number[]): number[]
  • Convert center + size to top left + size

    Parameters

    • pos: number[]
    • sz: number[]

    Returns number[]

generatePoints

  • Generate extra points from the original point p

    Parameters

    • p: Point
    • Default value num: number = 5

    Returns Point[]

get_axis_aligned_bbox

  • get_axis_aligned_bbox(box: { h: number; w: number; x: number; y: number }): { cx: number; cy: number; h: number; w: number }
  • Convert top left + size into center + size

    Parameters

    • box: { h: number; w: number; x: number; y: number }
      • h: number
      • w: number
      • x: number
      • y: number

    Returns { cx: number; cy: number; h: number; w: number }

    • cx: number
    • cy: number
    • h: number
    • w: number

get_subwindow_tracking

  • get_subwindow_tracking(imTensor: tf.Tensor3D, pos: number[], modelSz: number, originalSz: number, avgChans: Tensor): tf.Tensor3D
  • Parameters

    • imTensor: tf.Tensor3D
    • pos: number[]
    • modelSz: number
    • originalSz: number
    • avgChans: Tensor

    Returns tf.Tensor3D

isInside

  • Test if a point is inside the given rectangle.

    Parameters

    Returns boolean

isInside

  • Test if a point is inside the given rectangle.

    Parameters

    Returns boolean

Const isModelCached

  • isModelCached(url: string): Promise<unknown>
  • Parameters

    • url: string

    Returns Promise<unknown>

Const loadGraphModel

  • loadGraphModel(url: string): Promise<GraphModel | null>
  • Parameters

    • url: string

    Returns Promise<GraphModel | null>

loadImage

  • loadImage(url: string): Promise<HTMLImageElement>
  • Parameters

    • url: string

    Returns Promise<HTMLImageElement>

Const logModelLoad

  • logModelLoad(p: number): void
  • Parameters

    • p: number

    Returns void

python2round

  • python2round(f: number): number
  • Parameters

    • f: number

    Returns number

rectifyFisheyeFromPolyline

  • rectifyFisheyeFromPolyline(mask: ImageData, polylineVerticesFisheye: [number, number][]): { correctedMask: ImageData; countModifNeg: number; countModifPos: number; countOtherClass: number }
  • This function rectifies a border defined by a polyline. It finds a pair of dominate semantic labels in the mask, one on each side. Then it corrects "wrong" labels within an adaptive band witdh. The assigned instance id is the one belonging to the dominate instance of corresponding class.

    Parameters

    • mask: ImageData

      mask of semantic and instance segmentation. Supposed to be RGBA where 2 first channels used for instance id, the 3d one is class id.

    • polylineVerticesFisheye: [number, number][]

      Array of points representing the border polyline

    Returns { correctedMask: ImageData; countModifNeg: number; countModifPos: number; countOtherClass: number }

    "correctedMask": ImageData; "countModifPos": number; "countModifNeg": number; "countOtherClass": number;} Return value description

    • correctedMask: ImageData
    • countModifNeg: number
    • countModifPos: number
    • countOtherClass: number

sz_wh

  • sz_wh(wh: number[]): number
  • Parameters

    • wh: number[]

    Returns number

toSize

  • toSize(w: tf.Tensor2D, h: tf.Tensor2D): Tensor<Rank>
  • Parameters

    • w: tf.Tensor2D
    • h: tf.Tensor2D

    Returns Tensor<Rank>

update_tracks

  • update_tracks(model: GraphModel, template: tf.Tensor3D, search: tf.Tensor3D, targetPos: number[], targetSz: number[], window: tf.Tensor2D, scaleZ: number, p: { context_amount: number; lr: number; penalty_k: number; ratio: number; score_size: number; search_size: number; template_size: number; total_stride: number; window_influence: number; windowing: string }): [number, number][]
  • Update track. Return pred_targetPos and pred_targetSz.

    Parameters

    • model: GraphModel
    • template: tf.Tensor3D
    • search: tf.Tensor3D
    • targetPos: number[]
    • targetSz: number[]
    • window: tf.Tensor2D
    • scaleZ: number
    • p: { context_amount: number; lr: number; penalty_k: number; ratio: number; score_size: number; search_size: number; template_size: number; total_stride: number; window_influence: number; windowing: string }
      • context_amount: number
      • lr: number
      • penalty_k: number
      • ratio: number
      • score_size: number
      • search_size: number
      • template_size: number
      • total_stride: number
      • window_influence: number
      • windowing: string

    Returns [number, number][]

Object literals

Const COCO_CLASSES

COCO_CLASSES: object
copyright

CEA-LIST/DIASI/SIALV/LVA (2019)

author

CEA-LIST/DIASI/SIALV/LVA pixano@cea.fr

license

CECILL-C

airplane

airplane: number = 5

apple

apple: number = 53

backpack

backpack: number = 27

banana

banana: number = 52

baseball bat

baseball bat: number = 39

baseball glove

baseball glove: number = 40

bear

bear: number = 23

bed

bed: number = 65

bench

bench: number = 15

bicycle

bicycle: number = 2

bird

bird: number = 16

boat

boat: number = 9

book

book: number = 84

bottle

bottle: number = 44

bowl

bowl: number = 51

broccoli

broccoli: number = 56

bus

bus: number = 6

cake

cake: number = 61

car

car: number = 3

carrot

carrot: number = 57

cat

cat: number = 17

cell phone

cell phone: number = 77

chair

chair: number = 62

clock

clock: number = 85

couch

couch: number = 63

cow

cow: number = 21

cup

cup: number = 47

dining table

dining table: number = 67

dog

dog: number = 18

donut

donut: number = 60

elephant

elephant: number = 22

fire hydrant

fire hydrant: number = 11

fork

fork: number = 48

frisbee

frisbee: number = 34

giraffe

giraffe: number = 25

hair drier

hair drier: number = 89

handbag

handbag: number = 31

horse

horse: number = 19

hot dog

hot dog: number = 58

keyboard

keyboard: number = 76

kite

kite: number = 38

knife

knife: number = 49

laptop

laptop: number = 73

microwave

microwave: number = 78

motorcycle

motorcycle: number = 4

mouse

mouse: number = 74

orange

orange: number = 55

oven

oven: number = 79

parking meter

parking meter: number = 14

person

person: number = 1

pizza

pizza: number = 59

potted plant

potted plant: number = 64

refrigerator

refrigerator: number = 82

remote

remote: number = 75

sandwich

sandwich: number = 54

scissors

scissors: number = 87

sheep

sheep: number = 20

sink

sink: number = 81

skateboard

skateboard: number = 41

skis

skis: number = 35

snowboard

snowboard: number = 36

spoon

spoon: number = 50

sports ball

sports ball: number = 37

stop sign

stop sign: number = 13

suitcase

suitcase: number = 33

surfboard

surfboard: number = 42

teddy bear

teddy bear: number = 88

tennis racket

tennis racket: number = 43

tie

tie: number = 32

toaster

toaster: number = 80

toilet

toilet: number = 70

toothbrush

toothbrush: number = 90

traffic light

traffic light: number = 10

train

train: number = 7

truck

truck: number = 8

tv

tv: number = 72

umbrella

umbrella: number = 28

vase

vase: number = 86

wine glass

wine glass: number = 46

zebra

zebra: number = 24