Depth and World Coordinates

Turn a Pop's predictions into positions measured in meters

A normal prediction tells you where something is in the frame — pixels across, pixels down. Depth and world coordinates tell you where it is in the room, in meters.

That turns questions a 2D box cannot answer into ordinary arithmetic: how far apart two people are standing, whether someone crossed a line on the floor, how tall a stack is, which object is nearest the camera.

Depth from a single image

EyePop provides state-of-the-art monocular depth estimation models: they predict a depth map from one ordinary image, with no stereo rig, no depth sensor, and no second camera. Any source a Pop already accepts works unchanged — an uploaded photo, a video frame, an RTSP camera — and the model infers the geometry of the scene from the picture alone.

The result is a depth map: one distance value per pixel, at the source frame's aspect ratio, telling you how far that part of the scene sits from the camera. The depth abilities EyePop ships for this are metric — see Models — so those values carry real scale rather than a relative ordering, which is what makes them measurable rather than merely comparable.

A depth map is useful on its own — for masking by distance, sorting objects front to back, or rendering the scene's structure. Combined with a camera calibration, it becomes the basis for world coordinates: the platform back-projects each predicted point through the map and returns its position in meters.

Depth Estimation Features

Three pieces, each covered by its own page:

1. A depth map — a metric depth ability estimates how far away every pixel is. The Pop names it once, in depthMap.

2. A calibration — the lens and, optionally, where the camera stands. Without one the platform falls back to a guess, and distances along the optical axis are only as good as that guess.

3. An opt-in — components set toWorld, and their point-shaped predictions come back carrying worldX, worldY and worldZ.

Requires Python SDK and Node SDK 3.20.0 or newer.

A first example

Detect people, and get their key points in meters:

Python

from eyepop import EyePopSdk
from eyepop.worker.camera import Camera
from eyepop.worker.worker_types import (
    CropForward, InferenceComponent, Pop, PopDepthMap, SourceDefaults,
)

pop = Pop(
    components=[InferenceComponent(
        ability="eyepop.person:latest",
        toWorld=True,
        forward=CropForward(targets=[InferenceComponent(
            ability="eyepop.person.2d-body-points:latest",
            toWorld=True,
        )]),
    )],
    depthMap=PopDepthMap(ability="eyepop.depth.metric.large:latest"),
    defaults=SourceDefaults(camera=Camera(hfovDegrees=60.0)),
)

with EyePopSdk.sync_worker(pop=pop) as endpoint:
    result = endpoint.upload("street.jpg").predict()

for obj in result.get("objects", []):
    for keypoints in obj.get("keyPoints", []):
        for point in keypoints.get("points", []):
            if point.get("worldZ") is not None:
                print(point["classLabel"], round(point["worldZ"], 2), "m away")

Node

import { EyePop, PopComponentType, ForwardOperatorType } from '@eyepop.ai/eyepop'

const endpoint = await EyePop.workerEndpoint({
    pop: {
        components: [{
            type: PopComponentType.INFERENCE,
            ability: 'eyepop.person:latest',
            toWorld: true,
            forward: {
                operator: { type: ForwardOperatorType.CROP },
                targets: [{
                    type: PopComponentType.INFERENCE,
                    ability: 'eyepop.person.2d-body-points:latest',
                    toWorld: true,
                }],
            },
        }],
        depthMap: { ability: 'eyepop.depth.metric.large:latest' },
        defaults: { camera: { hfovDegrees: 60 } },
    },
}).connect()

const results = await endpoint.process({ source: { path: 'street.jpg' } })
for await (const result of results) {
    for (const obj of result.objects ?? []) {
        for (const keypoints of obj.keyPoints ?? []) {
            for (const point of keypoints.points ?? []) {
                if (point.worldZ !== undefined) {
                    console.log(point.classLabel, point.worldZ.toFixed(2), 'm away')
                }
            }
        }
    }
}

Key points like these, plotted in 3D beside the frame they came from. The four nearest people stand between 2.1 and 5.6 meters from the camera. Source photo: Sint Antoniesbreestraat, Amsterdam, by Fons Heijnsbroek (CC0).

Two things worth knowing up front

Use a metric depth ability. A relative one is accepted and silently produces no world coordinates at all — no error, just absence. See Models for the four to choose from.

Calibrate before you measure. Without a camera, the platform assumes a 60° horizontal field of view. Lateral measurements survive that guess; distances along the optical axis do not.

Next steps

• Depth Maps — asking for a map, and what comes back

• Camera Calibration — the lens, the pose, and the frames they define

• World Coordinates — reading the meters, and point clouds

• Visualizing in 3D — plotting the result