> For the complete documentation index, see [llms.txt](https://docs.eyepop.ai/developer-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.eyepop.ai/developer-documentation/platform/temporal-context/example-face-id-from-walk-bys.md).

# Example: Best Face-ID Vector from Walk-Bys

Keep one face-ID vector per person walking past a camera, from the frame that shows their face best

A camera at an office entrance sees people walk in and out all day, and many of them more than once. In this example you build a Pop that detects and tracks every person on every frame, picks the frame where each person's face is seen best, and computes a face-ID vector from that one frame. Comparing the vectors then recognizes who came back, even when they changed their clothes.

The example video shows ten walk-bys of eight people, the way an entrance sees them: people come in through the center door or the doors on the left and walk on past the camera without stopping, most of them alone, some in a pair, and one group of three, a few steps apart. Two of them come back later in different clothes: one in a navy blazer instead of a yellow rain jacket and backpack, the other in a trench coat instead of a green cardigan.

The complete, runnable example is in the Python SDK repository, under [`examples/temporal_context`](https://github.com/eyepop-ai/eyepop-sdk-python/tree/main/examples/temporal_context), together with the video. The steps below build it piece by piece.

{% hint style="warning" %}
A face-ID vector identifies a person. Before you collect them from real cameras, make sure you have the consent and the legal basis your jurisdiction requires, and store them as the personal data they are.
{% endhint %}

<figure><img src="https://1956543008-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2tRktyRCGgE1tPlAfT4h%2Fuploads%2Fgit-blob-093afff724319ec1f131e7072986f89f421045d1%2Ftemporal-context-walk-by.webp?alt=media" alt="The example video, played at three times its speed: people walk in through the doors of an office lobby and on past the camera, alone, in pairs and in a group of three"><figcaption><p>The example video at 3× speed. Everyone in it is generated.</p></figcaption></figure>

### Before you start

* Python SDK **3.24.0** or newer, and an [API key](https://docs.eyepop.ai/developer-documentation/api-keys.md) in `EYEPOP_API_KEY`.
* The example code and video:

```bash
git clone https://github.com/eyepop-ai/eyepop-sdk-python.git
cd eyepop-sdk-python/examples/temporal_context
pip install "eyepop>=3.24.0"
```

### Step 1: Detect and track people

The efficient side of the Pop runs on every frame. `eyepop.person.medium:latest` finds people, and a `tracking` component with `eyepop.person.reid:latest` follows each one by appearance as well as motion, so a person keeps their `trackId` when they cross paths with someone else:

```python
from eyepop.worker.worker_types import (
    CropForward, InferenceComponent, Pop, SelectForward, TrackingComponent,
)

InferenceComponent(
    ability="eyepop.person.medium:latest",
    forward=CropForward(targets=[TrackingComponent(
        reidModel="eyepop.person.reid:latest",
        forward=...,  # the selection, next
    )]),
)
```

### Step 2: Select the frame where the face is seen best

A person's box alone says little about their face: the largest view of someone walking away shows the back of their head. A **relevancy model** fixes that. With a face detector as the relevancy model, a frame is only as relevant as the face found in it, and a person whose face is never found is never selected:

```python
FACE_DETECTOR = "eyepop.person.face.short-range:latest"

TrackingComponent(
    reidModel="eyepop.person.reid:latest",
    forward=SelectForward(
        relevancyModel=FACE_DETECTOR,
        minTrackLengthSeconds=2.0,
        targets=[...],  # the face-ID stages, next
    ),
)
```

`minTrackLengthSeconds=2.0` ignores tracks shorter than two seconds. A walk-by through the lobby takes longer than that, but when a group comes in through the doors together, the tracker can briefly start a second track on one of them before it settles; that fragment is never selected.

The relevancy model runs on every tracked person of every frame, so it should be small and fast. The short-range face detector is both, and it is made for faces close to the camera: the views a face-ID model compares best. Without `intervalSeconds`, each person is selected once, when they leave the frame — the right choice for walk-bys, which last seconds.

See [Select Operators](https://docs.eyepop.ai/developer-documentation/platform/temporal-context/select-operators.md) for exactly how relevance and timing work.

<figure><img src="https://1956543008-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2tRktyRCGgE1tPlAfT4h%2Fuploads%2Fgit-blob-5bbed9355bc7ee4678f67ced5947be50f52231a7%2Ftemporal-context-walk-by-selections.jpg?alt=media" alt="Ten walk-bys from the example video, each cropped from the frame the select operator chose: close to the camera and facing it"><figcaption><p>The ten selections from the example video: for each walk-by, the frame where the face detector found the face best. #3 and #5 are the same person, and so are #4 and #9. Everyone in the video is generated.</p></figcaption></figure>

### Step 3: Detect, align and embed the face

The selection forwards a crop of the person from the selected frame. Three stages turn it into a vector:

```python
targets=[InferenceComponent(
    ability=FACE_DETECTOR,
    forward=CropForward(
        boxPadding=1.25,
        orientationTargetAngle=-90,
        targets=[InferenceComponent(ability="eyepop.face-id.large:latest")],
    ),
)]
```

1. The face detector finds the face inside the person crop.
2. `CropForward` cuts it out with some margin, and `orientationTargetAngle=-90` rotates it upright, using the eye line the detector reports. Face-ID models compare best on aligned faces.
3. `eyepop.face-id.large:latest` embeds the aligned face as a 512-dimensional vector.

These stages run once per person instead of on every frame, which is what makes the large face-ID model affordable on a busy entrance.

### Step 4: Read the vectors

Selected predictions carry the result, one per person. The vector is attached to the face object nested under the selected person, as a tensor named `embedding`:

```python
def find_face_id(obj):
    for raw in obj.get("raw") or []:
        for tensor in raw.get("tensors") or []:
            if tensor.get("name") == "embedding":
                return tensor["data"][0]
    for child in obj.get("objects") or []:
        if found := find_face_id(child):
            return found
    return None

faces = []
with EyePopSdk.sync_worker(pop=pop) as endpoint:
    job = endpoint.upload("media/office_walk_by.mp4")
    while result := job.predict():
        if not result.get("selected"):
            continue  # a live prediction, one per frame
        person = result["objects"][0]
        if (face_id := find_face_id(person)) is not None:
            faces.append({"trackId": person["trackId"], "seconds": result["seconds"], "faceId": face_id})
```

The vectors are not normalized. Normalize them before storing them in a vector database that expects unit vectors.

### Step 5: Compare walk-bys

Two vectors of the same face point in nearly the same direction, so their cosine similarity is high. Grouping walk-bys by similarity counts the people behind them, and shows who walked by more than once. A walk-by joins a group only when it is alike to every walk-by already in it, so one borderline match cannot chain two different people together:

```python
import numpy as np

def group_by_face(faces, threshold):
    vectors = np.array([face["faceId"] for face in faces])
    vectors /= np.linalg.norm(vectors, axis=1, keepdims=True)
    similarity = vectors @ vectors.T  # cosine similarity of every pair of walk-bys

    groups = []
    for i in range(len(faces)):
        # the groups this walk-by is alike to all of, by its least-alike member
        fits = [(similarity[i, group].min(), group) for group in groups]
        fits = [(sim, group) for sim, group in fits if sim > threshold]
        if fits:
            max(fits, key=lambda fit: fit[0])[1].append(i)
        else:
            groups.append([i])
    return [[faces[i] for i in group] for group in groups]

people = group_by_face(faces, threshold=0.25)
```

The threshold trades missed matches against false ones. Tune it on your own cameras, with the script's `--threshold`: lighting, distance and angle all move it.

Face-ID compares faces, not outfits. In the example video, the two people who come back are matched to their first walk-by with a cosine similarity of 0.42 and 0.29, although both changed their clothes. No two different people score higher than 0.16, so a threshold of 0.25 sits between the two.

### Run the example

```bash
python walk_by_face_id.py --output faces.jsonl
```

It prints one line per person as their selection arrives, writes every vector to `faces.jsonl`, and ends with how many different people walked by, each repeat visit with its lowest similarity to the rest of its group:

```
   8.71s  person #1   face-ID with 512 dimensions
  20.38s  person #2   face-ID with 512 dimensions
  24.04s  person #3   face-ID with 512 dimensions
  37.29s  person #4   face-ID with 512 dimensions
  48.83s  person #7   face-ID with 512 dimensions
  53.00s  person #6   face-ID with 512 dimensions
  57.83s  person #5   face-ID with 512 dimensions
  70.58s  person #8   face-ID with 512 dimensions
  81.62s  person #10  face-ID with 512 dimensions
  86.83s  person #9   face-ID with 512 dimensions

10 walk-bys, 8 different people
  walked by 2 times: #3 at 24.0s (0.42), #5 at 57.8s (0.42)
  walked by 2 times: #4 at 37.3s (0.29), #9 at 86.8s (0.29)
```

<figure><img src="https://1956543008-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2tRktyRCGgE1tPlAfT4h%2Fuploads%2Fgit-blob-84f9a9aa700c7dcbaf0026b30cb9460b0cd95ba1%2Ftemporal-context-walk-by-returns.jpg?alt=media" alt="The face crops of the two people who walked by twice, side by side: #3 and #5 with a cosine similarity of 0.42, and #4 and #9 with 0.29"><figcaption><p>The face crops face-ID embedded for the two people who came back, in different clothes. Everyone in the video is generated.</p></figcaption></figure>

Every walk-by is selected exactly once, while the live predictions cover every frame of the video, and the two people who came back are recognized by their faces alone.

Point `--video` at an `rtsp://` camera instead of the file to run the same Pop on a live entrance.

### Next steps

* [Example: Make and Model Insights from Car Drive-Bys](https://docs.eyepop.ai/developer-documentation/platform/temporal-context/example-car-make-and-model.md) — the same pattern with a custom VLM ability
* [Select Operators](https://docs.eyepop.ai/developer-documentation/platform/temporal-context/select-operators.md) — relevance, timing, and selected predictions in full
* [Components](https://docs.eyepop.ai/developer-documentation/platform/pop/pop-components.md#tracking) — tuning the tracker and its re-identification
* [Pretrained, fixed classes](https://docs.eyepop.ai/developer-documentation/platform/abilities/pretrained-fixed-classes.md) — the person and face abilities used here
