> For the complete documentation index, see [llms.txt](https://docs.eyepop.ai/developer-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.eyepop.ai/developer-documentation/platform/response-format/predictions/raw-and-details.md).

# Raw Output and Details

Tensor output such as embeddings, and structured JSON answers from abilities with a response schema

Most results fit one of the shapes on the previous pages: a box, a label, a text, a set of points. Two kinds do not, and both are open by design:

* **Raw output** is a model's answer as plain numbers, in tensors. An embedding model's vector is the typical case.
* **Details** are structured answers in JSON, shaped by the ability that produced them. A vision-language ability with a response schema answers this way.

### Raw output

A face-ID ability turns a face into a vector of 512 numbers. Faces of the same person give vectors that point in nearly the same direction, so comparing vectors recognizes people. The vector comes back as raw output on the face:

```json
{
  "classLabel": "face",
  "confidence": 0.9716,
  "x": 137.293, "y": 69.366, "width": 168.836, "height": 168.835,
  "raw": [
    {
      "confidence": 1,
      "tensors": [
        {
          "name": "embedding",
          "type": "float32",
          "dimensions": [512, 1],
          "data": [[-0.0153, -0.011, -0.0668, …]]
        }
      ]
    }
  ]
}
```

(The response holds all 512 values.)

`raw` is a list of outputs. Each output holds one or more **tensors**, a tensor being an array of numbers with a shape:

| Key          | Type              | Meaning                                                |
| ------------ | ----------------- | ------------------------------------------------------ |
| `name`       | string            | What the tensor is, as the model names it: `embedding` |
| `type`       | string            | The number type the model produced, such as `float32`  |
| `dimensions` | array of integers | The tensor's shape, **innermost dimension first**      |
| `data`       | nested arrays     | The numbers, nested to match `dimensions`              |

The order of `dimensions` is the one detail to get right. It lists the sizes from the innermost array outward, the reverse of the order of a NumPy shape. A tensor with `dimensions` of `[512, 1]` is **one** array of **512** numbers: `data` holds one list, and the vector is `data[0]`.

The same format serves any model, which is the point of it: whatever a model computes, `raw` carries it without EyePop having to know what it means. The meaning is in `name`, and in the documentation of the ability.

#### Example: read an embedding

{% tabs %}
{% tab title="Python" %}

```python
import numpy as np

def tensor(obj, name):
    for output in obj.get("raw", []):
        for t in output["tensors"]:
            if t["name"] == name:
                return np.array(t["data"], dtype=np.float32).reshape(t["dimensions"][::-1])
    return None

vector = tensor(face, "embedding")[0]
print(vector.shape, np.linalg.norm(vector))
```

{% endtab %}

{% tab title="Node" %}

```typescript
import { PredictedObject } from '@eyepop.ai/eyepop'

type Tensor = { name: string, type: string, dimensions: number[], data: unknown }
type RawOutput = { confidence?: number, tensors: Tensor[] }

function tensor(obj: PredictedObject, name: string): Tensor | undefined {
    const raw = (obj as PredictedObject & { raw?: RawOutput[] }).raw ?? []
    return raw.flatMap(output => output.tensors).find(t => t.name === name)
}

function printEmbedding(face: PredictedObject) {
    const vector = (tensor(face, 'embedding')?.data as number[][])[0]
    console.log(vector.length, Math.hypot(...vector))
}
```

{% endtab %}
{% endtabs %}

Reversing `dimensions` gives the shape NumPy expects. An embedding's length is not normalized; normalize it before comparing vectors by cosine similarity. [Example: Best Face-ID Vector from Walk-Bys](https://docs.eyepop.ai/developer-documentation/platform/temporal-context/example-face-id-from-walk-bys.md) collects and compares face-ID vectors end to end.

### Details

`details` holds answers whose shape the **ability** defines. Any valid JSON can appear there; what it looks like is up to the ability.

The common case is a vision-language ability with a **response schema**. The schema is a JSON Schema the model's answer must follow, and the answer comes back parsed, as an entry of `details`. An ability that names the make, model and color of a car, with a schema of three required strings, answers on each car:

```json
{
  "classLabel": "car",
  "x": 412.5, "y": 120.25, "width": 310.75, "height": 240.5,
  "details": [
    { "make": "Toyota", "model": "Camry", "color": "silver" }
  ]
}
```

Its schema:

```json
{
  "type": "object",
  "properties": {
    "make": { "type": "string" },
    "model": { "type": "string" },
    "color": { "type": "string" }
  },
  "required": ["make", "model", "color"]
}
```

* `details` is a **list**; the answer is an entry of it, usually the only one.
* Each entry follows the ability's response schema. Read the schema to know which keys to expect; the format adds nothing of its own.
* An ability without a schema can still return details. Their shape is then whatever that ability documents.
* Like other results, `details` sits on the object the ability ran on, or on the prediction when it ran on the whole frame.

[Example: Make and Model Insights from Car Drive-Bys](https://docs.eyepop.ai/developer-documentation/platform/temporal-context/example-car-make-and-model.md) creates such an ability with its schema and reads the answers.

### Next steps

* [Tracks and Selected Predictions](https://docs.eyepop.ai/developer-documentation/platform/response-format/predictions/tracks-and-selected-predictions.md) — results that span a video
* [User defined VLM tasks](https://docs.eyepop.ai/developer-documentation/platform/abilities/user-defined-vlm-tasks.md) — the fields of an ability you define, including its response schema
