Spatial Attributes
How a prediction says where something is — boxes, points, outlines and masks
Every position in a prediction is measured the same way, so the code that draws a person's box also draws the face found inside it, the key points on that face and the outline of a segment.
The source frame
A prediction's source_width and source_height define its coordinate system: the full frame of the media, in pixels.
The origin is the top-left corner.
xgrows to the right andygrows downward.Positions are in the source's own pixels, not in the resolution a model ran at. A model that saw a 640 × 640 copy of a 1280 × 720 image still reports 1280 × 720 coordinates.
Values are decimals. Coordinates and angles are cut to 3 decimal places, and confidences to 4 — truncated, not rounded.
(0,0) ───────────────── x ───────────────▶ (1280,0)
│
│ (503,251) ┌──────┐
│ │ │ height 464
y │person│
│ │ │
│ └──────┘
▼ width 166
(0,720)Boxes
An object's position is an axis-aligned box: x and y are its top-left corner, width and height its extent.
The box covers x to x + width and y to y + height. Its center is at x + width / 2, y + height / 2.
Nested results use the same frame
A Pop that crops a person and looks for a face inside the crop still reports the face in the full frame, not relative to the crop:
The face's x of 554 is 51 pixels to the right of the person's, not 554 pixels into the crop. You never add a parent's position to a child's; draw every result directly on the original image.
The same holds for a region of interest: the Pop sees only that region, but the positions it reports are in the full frame.
Orientation and outline
Some objects have a direction of their own: a face points from chin to forehead, whichever way the head is tilted. An ability that knows this reports it as orientation, and adds an outline, the corners of the box turned to match:
orientation is an angle in radians, measured from the x axis. Because y grows downward, negative angles point up: a perfectly upright face reports −π/2, about -1.571. This face, at -1.509, leans about 3.5° toward the right of the image.
x, y, width and height stay axis-aligned, so a box can always be drawn without trigonometry; the outline is the turned shape. An ability that reports no direction gives an orientation of 0 and no outline.
Points
A point is an x and a y in the same frame. Points appear wherever a result is a location rather than an area:
Key points — the landmarks of a body, a hand or a face, each with a
classLabelsuch asnoseorleft shoulder. See Key Points and Meshes.Outlines and contours — the corners of a shape, in order. See Masks, Contours and Depth.
A point can also carry a z. That is the model's own depth estimate relative to the object, in the model's own units, not a distance in meters.
Masks
A segmentation mask marks exactly which pixels belong to an object. It is a bitmap the size of the object's box, laid over that box. See Masks, Contours and Depth for how to decode it.
Positions in meters
Everything above is in pixels: where in the image. A Pop with a depth map can also say where in the room, adding worldX, worldY and worldZ in meters to points, contours and masks. That is covered in Depth and World Coordinates.
Next steps
Temporal Attributes — when in the media
Objects — everything an object carries besides its box
Last updated
