> For the complete documentation index, see [llms.txt](https://docs.eyepop.ai/developer-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.eyepop.ai/developer-documentation/platform/response-format/predictions/key-points-and-meshes.md).

# Key Points and Meshes

Named landmarks and dense 3D surfaces on an object

Some abilities do not answer with a box but with a set of **points**: the joints of a body, the eyes and ears of a face, the hundreds of vertices of a face's surface. Named landmarks come as **key points**, dense surfaces as **meshes**.

### Key points

A body-points ability run on a person returns the person's joints, each a named point in the frame:

<figure><img src="https://1956543008-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2tRktyRCGgE1tPlAfT4h%2Fuploads%2Fgit-blob-4b59f349736476ba3baa5afd33a11b2ce528464e%2Fresponse-format-body-points.jpg?alt=media" alt="The person in the lobby with 17 dots on her nose, eyes, ears, shoulders, elbows, wrists, hips, knees and ankles, joined into a stick figure"><figcaption><p>The 17 key points of <code>eyepop.person.2d-body-points:latest</code>. The person is generated.</p></figcaption></figure>

```json
{
  "classLabel": "person",
  "confidence": 0.9775,
  "x": 503.281, "y": 251.718, "width": 166.562, "height": 464.765,
  "keyPoints": [
    {
      "confidence": 0.603,
      "points": [
        { "classLabel": "nose",           "confidence": 0.603,  "x": 578.792, "y": 293.622 },
        { "classLabel": "left eye",       "confidence": 0.9102, "x": 590.817, "y": 285.438 },
        { "classLabel": "right eye",      "confidence": 0.8992, "x": 569.875, "y": 284.466 },
        { "classLabel": "left shoulder",  "confidence": 0.8839, "x": 629.105, "y": 357.93 },
        { "classLabel": "right shoulder", "confidence": 0.9546, "x": 532.056, "y": 357.289 }
      ]
    }
  ]
}
```

(The response lists all 17 points; five are shown.)

`keyPoints` is a list of **groups**. Each group is one set of landmarks that belong together, such as one body's skeleton or one face's features, with its points in `points`. The order of the points within a group is fixed by the ability, so the same index always names the same landmark.

| Key          | Type   | Meaning                                                 |
| ------------ | ------ | ------------------------------------------------------- |
| `confidence` | number | How certain the model is of the group as a whole        |
| `category`   | string | The group's category, when the Pop or ability names one |
| `points`     | array  | The landmarks                                           |

Each point:

| Key          | Type   | Meaning                                                                             |
| ------------ | ------ | ----------------------------------------------------------------------------------- |
| `classLabel` | string | Which landmark: `nose`, `left shoulder`, `right ankle`                              |
| `confidence` | number | How certain the model is of this landmark                                           |
| `x`, `y`     | number | Its position, in source pixels like every other position                            |
| `z`          | number | Its depth relative to the other points of the group, when the ability estimates one |

**Left and right are the subject's own.** The person's `left shoulder` above is at `x` 629, on the right side of the image, because she faces the camera.

#### Depth of a key point

A 3D body-points ability adds a `z` to each point:

```json
{ "classLabel": "nose",     "confidence": 0.9998, "x": 584.552, "y": 294.117, "z": -283.182 }
{ "classLabel": "left hip", "confidence": 0.9999, "x": 604.449, "y": 506.436, "z": 3.157 }
```

`z` places the points of one group relative to each other, in the model's own scale: here the hips are near 0, and the nose, which is closer to the camera, is at -283. It is not a distance from the camera and not in meters; compare `z` values only within one group. For positions in meters, see [Depth and World Coordinates](https://docs.eyepop.ai/developer-documentation/platform/depth-and-world-coordinates.md).

### Meshes

A mesh is a dense set of 3D points covering a surface. A face mesh ability run on a face returns 478 of them:

```json
{
  "classLabel": "face",
  "x": 554.438, "y": 263.858, "width": 65.296, "height": 65.296,
  "meshes": [
    {
      "category": "3d-face-mesh",
      "confidence": 5.4395,
      "points": [
        { "x": 580.481, "y": 304.095, "z": -6.246 },
        { "x": 580.819, "y": 296.528, "z": -10.141 }
      ]
    }
  ]
}
```

(Two of the 478 points are shown.)

| Key          | Type   | Meaning                                                                                |
| ------------ | ------ | -------------------------------------------------------------------------------------- |
| `confidence` | number | The model's score for the mesh. Its scale is the model's own and is not limited to 0–1 |
| `category`   | string | The mesh's category, when the Pop or ability names one                                 |
| `points`     | array  | The vertices, each with `x`, `y` and `z`                                               |

Mesh points have no names. Like key points, they come in a fixed order, so the same index is the same spot on the face in every frame. `x` and `y` are source pixels and `z` is relative depth, as for key points.

### Example: shoulder width in pixels

Because points are named, measuring between landmarks is a lookup and a subtraction:

{% tabs %}
{% tab title="Python" %}

```python
import math

def shoulder_width(person):
    for group in person.get("keyPoints", []):
        points = {p["classLabel"]: p for p in group["points"]}
        left, right = points.get("left shoulder"), points.get("right shoulder")
        if left and right:
            return math.hypot(left["x"] - right["x"], left["y"] - right["y"])
    return None

for person in prediction.get("objects", []):
    print(f"shoulders {shoulder_width(person):.0f} px apart")
```

{% endtab %}

{% tab title="Node" %}

```typescript
import { Prediction, PredictedObject } from '@eyepop.ai/eyepop'

function shoulderWidth(person: PredictedObject): number | undefined {
    for (const group of person.keyPoints ?? []) {
        const left = group.points.find(p => p.classLabel === 'left shoulder')
        const right = group.points.find(p => p.classLabel === 'right shoulder')
        if (left && right) {
            return Math.hypot(left.x - right.x, left.y - right.y)
        }
    }
    return undefined
}

function printShoulders(prediction: Prediction) {
    for (const person of prediction.objects ?? []) {
        console.log(`shoulders ${shoulderWidth(person)?.toFixed(0)} px apart`)
    }
}
```

{% endtab %}
{% endtabs %}

```
shoulders 97 px apart
```

The distance is in pixels of the image. A person farther from the camera has narrower shoulders in pixels; for a measurement in meters, use [world coordinates](https://docs.eyepop.ai/developer-documentation/platform/depth-and-world-coordinates/world-coordinates.md).

### Next steps

* [Masks, Contours and Depth](https://docs.eyepop.ai/developer-documentation/platform/response-format/predictions/masks-contours-and-depth.md) — the exact shape of an object, and depth for the whole frame
* [Objects](https://docs.eyepop.ai/developer-documentation/platform/response-format/predictions/objects.md) — the objects key points and meshes are attached to
