> For the complete documentation index, see [llms.txt](https://docs.eyepop.ai/developer-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.eyepop.ai/developer-documentation/platform/depth-and-world-coordinates/depth-maps.md). # Depth Maps The frame-level map of distance that world coordinates are back-projected from A **depth estimation** ability produces one value per pixel: how far that part of the scene is from the camera. That map is what everything else in this section is built on — [world coordinates](/developer-documentation/platform/depth-and-world-coordinates/world-coordinates.md) are the map read through a [calibration](/developer-documentation/platform/depth-and-world-coordinates/camera-calibration.md). Only **metric** depth can be back-projected. See [Models](/developer-documentation/platform/models.md#depth-estimation) for the four abilities to choose from. ### Asking for one A depth map is not an inference component. A Pop names it once, in [`depthMap`](/developer-documentation/platform/pop/pop-object.md), because the platform back-projects every prediction through a single map — a second depth source would have nowhere to go. {% tabs %} {% tab title="Python" %} ```python from eyepop.worker.worker_types import InferenceComponent, Pop, PopDepthMap pop = Pop( components=[InferenceComponent(ability="eyepop.person:latest")], depthMap=PopDepthMap(ability="eyepop.depth.metric.large:latest", toWorld=True), ) ``` {% endtab %} {% tab title="Node" %} ```typescript const pop = { components: [{ type: PopComponentType.INFERENCE, ability: 'eyepop.person:latest' }], depthMap: { ability: 'eyepop.depth.metric.large:latest', toWorld: true }, } ``` {% endtab %} {% endtabs %} Give exactly one of `ability` and `abilityUuid`. Both SDKs reject naming neither or both before the request leaves your process. {% hint style="warning" %} **`toWorld` on the depth map is what reveals the map itself.** Name a `depthMap` without it and the platform still builds the depth stage and still enriches your predictions — but keeps the map out of the response. That is the useful default, not an oversight. A depth map is roughly a megabyte of base64 per frame; on a live stream, the difference between a few enriched key points and a megabyte a frame is the difference between a working pipeline and a saturated one. Ask for the map only when you want the map. {% endhint %} A `depthMap` with `toWorld` is also a **complete Pop on its own**. The map is the only consumer, so nothing needs to be detected for it to mean something: {% tabs %} {% tab title="Python" %} ```python pop = Pop( components=[], depthMap=PopDepthMap(ability="eyepop.depth.metric.large:latest", toWorld=True), ) ``` {% endtab %} {% tab title="Node" %} ```typescript const pop = { components: [], depthMap: { ability: 'eyepop.depth.metric.large:latest', toWorld: true }, } ``` {% endtab %} {% endtabs %} ### What comes back The map arrives as a frame-level `depth` member on the prediction. | Field | Type | Description | | ---------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `width` | integer | Map width in map pixels. | | `height` | integer | Map height in map pixels. | | `values` | string | Base64 of `width * height` little-endian float32 values, row-major. | | `semantic` | string | What the values mean. See below. | | `world` | string | The scene point cloud, present when `depthMap.toWorld` was asked for. See [World Coordinates](/developer-documentation/platform/depth-and-world-coordinates/world-coordinates.md). | The map has the **aspect ratio of the source frame** but not its resolution, so a source coordinate maps onto it proportionally: `(x * width / source_width, y * height / source_height)`. Both SDKs do that for you when you pass the source dimensions. Sky pixels carry **`+Infinity`**. #### What the values mean `semantic` is always present in prediction v2, `"unknown"` included, so an absent one means a worker predating the field rather than a map declining to say. | `semantic` | Meaning | Back-projectable | | ------------------ | ------------------------------------------------------------------------ | ---------------- | | `metric` | The value is already meters. | yes | | `canonical_metric` | Meters up to a factor fixed by the lens, which the calibration supplies. | yes | | `relative` | Scale- **and** shift-invariant: ordering is meaningful, distance is not. | no | | `unknown` | The ability declared nothing. | no | {% hint style="info" %} `canonical_metric` values are **not meters as they stand** — converting them needs the focal length in pixels, rescaled to the map's own resolution. Rather than doing that arithmetic yourself, ask for [world coordinates](/developer-documentation/platform/depth-and-world-coordinates/world-coordinates.md): the platform applies your calibration and returns meters directly. {% endhint %} A `relative` or `unknown` map is accepted and simply produces no world coordinates. There is no error — which is why the ability you name matters. ### Reading it {% tabs %} {% tab title="Python" %} ```python from eyepop import DepthMap depth_map = DepthMap.from_prediction(prediction) if depth_map is not None: print(depth_map.array.shape) # (height, width), float32, +inf for sky print(depth_map.sky_mask) # boolean array, True where sky print(depth_map.finite_min, depth_map.finite_max) # sample by source frame coordinate value = depth_map.at(x, y, prediction["source_width"], prediction["source_height"]) if not depth_map.is_sky(x, y, prediction["source_width"], prediction["source_height"]): print(value) ``` Decoding is lazy — the base64 is not touched until `.array` is first read. {% endtab %} {% tab title="Node" %} ```typescript import { decodeDepthMap } from '@eyepop.ai/eyepop' if (prediction.depth) { const depthMap = decodeDepthMap(prediction.depth) console.log(depthMap.width, depthMap.height) console.log(depthMap.finiteMin, depthMap.finiteMax) // undefined if all sky // sample by source frame coordinate const value = depthMap.at(x, y, prediction.source_width, prediction.source_height) if (!depthMap.isSky(x, y, prediction.source_width, prediction.source_height)) { console.log(value) } } ``` {% endtab %} {% endtabs %} Called without the source dimensions, `at()` and `isSky()` index the map's own grid instead. Coordinates are clamped to the map rather than rejected. ### Seeing it Both SDKs render a depth map as a **turbo heatmap** over the frame: near is warm, far is cool, and sky is left transparent. {% tabs %} {% tab title="Python" %} ```python import matplotlib.pyplot as plt from eyepop.visualize import EyePopPlot plot = EyePopPlot(plt.gca()) plot.depth(prediction, opacity=0.5) plt.show() ``` {% endtab %} {% tab title="Node" %} ```typescript import { Render2d } from '@eyepop.ai/eyepop-render-2d' const renderer = Render2d.renderer(context, [ Render2d.renderDepth({ opacity: 0.5, renderSky: false }), ]) renderer.draw(prediction) ``` {% endtab %} {% endtabs %}
A busy street beside the same frame under a turbo depth heatmap

A source frame and its depth map from eyepop.depth.metric.large. The near pavement and foreground figures are warm, the street recedes through green, the far end is cool, and sky pixels are left untouched. Source photo: Sint Antoniesbreestraat, Amsterdam, by Fons Heijnsbroek (CC0).

### Next steps * [Camera Calibration](/developer-documentation/platform/depth-and-world-coordinates/camera-calibration.md) — the lens and pose that turn this map into meters * [World Coordinates](/developer-documentation/platform/depth-and-world-coordinates/world-coordinates.md) — reading positions off the map * [Models](/developer-documentation/platform/models.md#depth-estimation) — the four metric depth abilities