> For the complete documentation index, see [llms.txt](https://docs.eyepop.ai/developer-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.eyepop.ai/developer-documentation/platform/depth-and-world-coordinates/camera-calibration.md).

# Camera Calibration

Describing the lens and the pose that turn a depth map into measurable meters

A depth map on its own is a picture of distance. Turning it into positions you can measure takes a **calibration**: what the lens does to the scene, and where the camera stands in it.

Without one the worker assumes a **60° horizontal field of view**. That is a development scaffold, not something to ship — and it fails in a way worth understanding, because it is quiet. For canonical metric depth the assumed focal length cancels out of X and Y and survives only in Z, so lateral measurements stay exact while every distance along the optical axis is wrong by however wrong the guess was. Two people standing side by side measure the right distance apart; how far away they are is fiction.

### The lens

Exactly one of `intrinsics` and `hfovDegrees` describes the lens. Supplying both is rejected rather than resolved by precedence — two descriptions of one lens that disagree have no right answer — and supplying neither is rejected too, since defaulting a focal length would be inventing a lens.

| Field           | Type  | Description                                                                         |
| --------------- | ----- | ----------------------------------------------------------------------------------- |
| `intrinsics.fx` | float | Focal length as a fraction of the frame's **width**. Positive and finite.           |
| `intrinsics.fy` | float | Focal length as a fraction of the frame's **height**. Positive and finite.          |
| `intrinsics.cx` | float | Principal point across the frame, in `[0, 1]`. A centred lens is `0.5`.             |
| `intrinsics.cy` | float | Principal point down the frame, in `[0, 1]`. A centred lens is `0.5`.               |
| `hfovDegrees`   | float | Horizontal field of view, in `(0, 180)`. The shorthand alternative to `intrinsics`. |

{% hint style="info" %}
**Intrinsics are normalised to the frame, not given in pixels.** A camera calibrated at 1920x1080 and later streamed at 1280x720 keeps working. Pixel values carrying no calibration resolution are wrong by exactly the resolution ratio, and nothing downstream can detect it.

To convert an OpenCV matrix, divide by the resolution it was calibrated at: `fx_normalised = fx_pixels / width`, `cy_normalised = cy_pixels / height`.
{% endhint %}

`hfovDegrees` is for an operator who knows their lens's field of view but not its calibration matrix. It assumes square pixels and a centred principal point, so it approximates a real calibration rather than replacing one — but it describes the actual lens rather than the worker's fallback guess, and it composes with distortion and pose exactly as `intrinsics` do.

### Distortion

`distortion` carries the OpenCV Brown-Conrady coefficients — `k1`, `k2`, `p1`, `p2`, `k3`, each defaulting to `0.0`. All zero is a rectilinear lens, which is also what you get by omitting the field. Supply them for a wide-angle or fisheye lens, where straight lines in the scene do not arrive straight.

### The pose

`extrinsics` places the camera in the world. Omit it and the meters come back in the camera's own frame.

| Field                    | Type   | Description                                                                                |
| ------------------------ | ------ | ------------------------------------------------------------------------------------------ |
| `extrinsics.rotation`    | object | A unit quaternion, **`w` first**: `{ w, x, y, z }`, defaulting to the identity `{ w: 1 }`. |
| `extrinsics.translation` | object | Where the camera sits, in meters: `{ x, y, z }`, defaulting to the origin.                 |

The pose is **camera to world**:

```
P_world = R(rotation) · P_camera + translation
```

so the rotation carries the camera's axes onto the world's, and the translation is the camera's own position.

{% hint style="warning" %}
**This is the inverse of what `cv2.solvePnP` returns.** Holding its `rvec` / `tvec`, invert both halves — `R = R_cv.T` and `t = -R_cv.T @ t_cv` — then convert `R` to a quaternion. `tvec` is **not** the camera position, and passing it as `translation` puts the camera somewhere it never was.
{% endhint %}

Either half means something on its own: a rotation with no translation is a camera at the world origin, a translation with no rotation is one looking along the world axes.

### Which frame the meters are in

This is the single thing to get straight, because a prediction does not say which frame it holds — and the two are oriented differently.

|                          | Frame                               | Axes                            |
| ------------------------ | ----------------------------------- | ------------------------------- |
| **With** `extrinsics`    | The world frame the pose defines    | Z **up**, ground plane at Z = 0 |
| **Without** `extrinsics` | The camera frame, OpenCV convention | X right, Y **down**, Z forward  |

A source that supplies no pose — or an identity one — gets the camera frame, so Y growing *downwards* is the common case rather than the exotic one. Anything that draws these coordinates has to know which it is looking at, or it renders the scene upside down.

### Supplying it

A calibration belongs to a source. Give it per source, or once on the Pop's [`defaults`](/developer-documentation/platform/sources-and-options/pop-defaults.md) for every source that Pop processes.

{% tabs %}
{% tab title="Python" %}

```python
from eyepop.worker.camera import Camera, CameraExtrinsics, CameraIntrinsics, Quaternion, Vector3d

camera = Camera(
    intrinsics=CameraIntrinsics(fx=0.9, fy=1.6, cx=0.5, cy=0.5),
    extrinsics=CameraExtrinsics(
        rotation=Quaternion(w=0.5736, x=-0.8192, y=0.0, z=0.0),
        translation=Vector3d(x=0.0, y=0.0, z=3.0),
    ),
)

job = endpoint.load_from("rtsp://camera.example.com/stream1", camera=camera)
```

The lens shorthand is `Camera(hfovDegrees=72.0)`.
{% endtab %}

{% tab title="Node" %}

```typescript
const results = await endpoint.process({
    source: { url: 'rtsp://camera.example.com/stream1' },
    camera: {
        intrinsics: { fx: 0.9, fy: 1.6, cx: 0.5, cy: 0.5 },
        extrinsics: {
            rotation: { w: 0.5736, x: -0.8192, y: 0, z: 0 },
            translation: { x: 0, y: 0, z: 3 },
        },
    },
})
```

The lens shorthand is `camera: { hfovDegrees: 72 }`.
{% endtab %}
{% endtabs %}

On the Pop, the same object goes under `defaults`:

{% tabs %}
{% tab title="Python" %}

```python
from eyepop.worker.worker_types import Pop, SourceDefaults

pop = Pop(
    components=[...],
    defaults=SourceDefaults(camera=Camera(hfovDegrees=72.0)),
)
```

{% endtab %}

{% tab title="Node" %}

```typescript
const pop = {
    components: [...],
    defaults: { camera: { hfovDegrees: 72 } },
}
```

{% endtab %}
{% endtabs %}

{% hint style="info" %}
Defaults merge field by field, so a source giving its own `roi` but no `camera` keeps its roi and takes the Pop's camera. The camera itself merges as **one whole field**: a source declaring its own lens replaces the defaulted calibration outright rather than mixing the two. There is no way to inherit a default pose and override only the lens.
{% endhint %}

### With a region of interest

A [`roi`](/developer-documentation/platform/sources-and-options/options/region-of-interest.md) crops the frame before inference, but a calibration always describes the source **as captured** — the original media resolution, not the crop. Intrinsics are normalised against the full frame, and the platform accounts for the crop itself. Do not pre-adjust a calibration to match a region of interest.

{% hint style="warning" %}
This is also why you should not recompute world coordinates yourself from a prediction's `x` / `y` and your own calibration. Under an `roi` the emitted coordinates are original-frame pixels while the projection happened against the cropped frame, so a hand-rolled back-projection is wrong in exactly the case where calibration matters most. Read the `worldX` / `worldY` / `worldZ` the platform returns.
{% endhint %}

### Failing early

Both SDKs validate a calibration before the request leaves your process, so a malformed one raises locally — a `ValidationError` in Python, a thrown error in Node — rather than coming back as a `400` halfway through an upload. The worker validates authoritatively; the two are meant to agree.

What is checked: exactly one lens description; `hfovDegrees` inside `(0, 180)`; `fx` and `fy` positive and finite; `cx` and `cy` within `[0, 1]`; every distortion coefficient finite; every translation component finite; and the rotation a unit quaternion, within `1e-3` of norm 1.0 so it survives a float round trip through JSON without letting an un-normalised one through.

### Next steps

* [World Coordinates](/developer-documentation/platform/depth-and-world-coordinates/world-coordinates.md) — reading the meters a calibration produces
* [Depth Maps](/developer-documentation/platform/depth-and-world-coordinates/depth-maps.md) — the map the coordinates are back-projected from
* [Pop Defaults](/developer-documentation/platform/sources-and-options/pop-defaults.md) — setting one calibration for every source