> For the complete documentation index, see [llms.txt](https://docs.eyepop.ai/developer-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.eyepop.ai/developer-documentation/cli/on-premise.md).

# On-Premise Instances

Install and operate the EyePop runtime on your own hardware

Run inference on your own hardware. An **instance** is the EyePop runtime installed on one machine, serving one pop. While a machine has an instance the CLI is **on-premise**: a `--pop` run, a `--model` run naming a published model, or one with no target at all goes to the instance, and nothing stands up EyePop compute.

### What you need

* Docker with Compose v2
* An API key, created in the EyePop dashboard — pass it as `EYEPOP_API_KEY` or `--api-key`, or let `init` prompt for it on a terminal

### Set up an instance

```bash
eyepop instance init --pop eyepop.person:latest
```

One command does the whole thing, and shows its work: detects the hardware profile, checks prerequisites, installs a registry credential if docker does not already hold one, registers the instance with your account, pulls the runtime image — several gigabytes, and quiet while it runs unless you set `EYEPOP_LOG_LEVEL=debug` to see docker's own progress — and brings the container up. When it finishes, the machine is on-premise and ready to run.

Everything the instance needs lives under one directory, the **instance root** (`~/.eyepop` unless you name another with `--config-dir`, or point every command including `run` at one with `EYEPOP_INSTANCE_DIR`). Every file is written `0600` in a `0700` directory:

| File                  | What it is                                                     |
| --------------------- | -------------------------------------------------------------- |
| `docker-compose.yml`  | The compose project the instance runs as                       |
| `eyepop-instance.yml` | The instance's configuration — holds your API key in plaintext |
| `pop.json`            | The pop the instance serves                                    |
| `license.pem`         | The runtime licence, when the platform issued one              |

`init` takes these flags:

| Flag          | What it does                                                                                                                                                                     |
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--pop`       | The pop the instance serves: a pop name, an ability, or the pop as JSON/YAML (inline or a file)                                                                                  |
| `--name`      | Instance name; defaults to this machine's hostname                                                                                                                               |
| `--tag`       | Runtime image tag; defaults to the tag the platform recommends, or `latest` when it cannot answer                                                                                |
| `--profile`   | Hardware family selecting the runtime image (`cpu`, `cuda`, `cuda-jetpack6`, `openvino`, `qnn`); detected when omitted. Only `cpu`, `cuda-jetpack6` and `qnn` can be provisioned |
| `--bind`      | Host interface to publish on                                                                                                                                                     |
| `--wait`      | Seconds to wait for the container to report healthy                                                                                                                              |
| `--log-level` | The **runtime container's** log level, not the CLI's — set `EYEPOP_LOG_LEVEL` for that                                                                                           |
| `--dump`      | Render the files into `./eyepop-instance/`, or that name under `--config-dir`, and stop — nothing registered, pulled, or started, and the real instance root is untouched        |
| `--yes`       | Accept the defaults without prompting; the pop defaults to `eyepop.person:latest`                                                                                                |

### Change the pop it serves

The pop is not fixed for the life of the instance. Point it at a different one:

```bash
eyepop instance set pop eyepop.person:latest
```

The argument is a pop name or an ability alias, and the CLI resolves it against your account before it touches anything — an unknown pop fails there, leaving the running instance untouched. On success it rewrites `pop.json` and force-recreates the container; `docker-compose.yml` and `eyepop-instance.yml` are left alone, the runtime image is not pulled again, and the model cache volume survives. The instance is down for the recreate, and the command waits for the container to report healthy — up to `--wait` seconds, 300 by default.

Nothing changes on the EyePop side: the account record, the instance identity, and the billing history keyed to it all stay as they are. The next `eyepop run` targets the new pop with no further step.

### Hardware profiles

`init` detects the profile, and `eyepop system` reports what it found. Three profiles are provisionable:

| Profile         | Hardware                                                      |
| --------------- | ------------------------------------------------------------- |
| `cpu`           | Any machine, no accelerator                                   |
| `cuda-jetpack6` | NVIDIA Jetson on JetPack 6                                    |
| `qnn`           | Qualcomm Dragonwing, with the QAIRT SDK installed on the host |

`cuda` and `openvino` are not provisionable yet: their compose projects need device access the CLI does not generate. That applies to the detected profile as well as an explicit `--profile` — on a machine with a discrete NVIDIA GPU or an Intel GPU, `init` refuses before it registers anything. Pass `--profile cpu` to run those machines without the accelerator, or install them with the on-premise Compose package instead — see [On-Premise](https://docs.eyepop.ai/developer-documentation/deploying/on-premise).

#### QNN prerequisites

The `qnn` runtime image carries no QAIRT SDK — it expects one installed on the host and bind-mounts it read-only at `/opt/qairt`. Install the SDK before `init`, then confirm with `eyepop system --json`: the report names the QAIRT root, the Hexagon variant, and the ARM library directory it will use.

The CLI reads the target identity from `/proc/device-tree/compatible` and looks for the SDK in `/opt/qcom/aistack/qairt`, `/opt/qairt`, `/usr/local/qairt`, and `$XDG_DATA_HOME/eyepop/qairt`. It requires exactly one match — no installation and several installations are both errors, so the message names the paths it searched. Set `QAIRT_SDK_ROOT` to an absolute path to override the search for a nonstandard installation.

The host also needs the `fastrpc` group and its `/dev/fastrpc-*` nodes — without them the machine is not detected as `qnn` at all — plus the `dmaheap` group the Qualcomm BSP creates; without `dmaheap`, `eyepop system` reports `QNN.ready` as false.

### Run against it

Name the target with `--pop`, which routes to the instance before any catalog lookup:

```bash
eyepop run --pop eyepop.person:latest --media-path image.jpg
```

`--model` is the flag for an ability everywhere else, and on-premise it splits. An alias that resolves as a published model composes into a single-component pop and runs on the instance, subject to the same ability-mismatch refusal as `--pop`. An alias that only names a VLM ability is not on-premise aware — it falls through to cloud inference and bills. `--pop` has one behaviour here, so prefer it.

Or name no target at all, and the run uses whichever pop the instance already serves:

```bash
eyepop run --media-path image.jpg
```

On an on-premise machine this goes to the instance. No compute is stood up and no session is created — `eyepop get sessions` will show nothing new. When the target names an ability — directly, or as a pop file or inline pop with one inference component — and the instance serves a single ability, a mismatch is refused rather than answered with the wrong model. A pop-name target, a multi-component pop, or an instance serving a composed pop is not checked — the run goes to the instance and it serves the pop it was configured with. Either way — a `--pop` target or none — nothing is sent to the cloud. A `--model` target naming a published model also runs on the instance and bills no compute, but it resolves the alias against the platform first, so that one is not offline. To serve a different pop, see [Change the pop it serves](#change-the-pop-it-serves).

Commands that would start EyePop compute are disabled while on-premise — `create deployment`, for one, answers that it is disabled when running on-premise. Reads (`get models`, `get sessions`, `get instances`, …) work as always.

### Operate it

| Command                          | What it does                                       |
| -------------------------------- | -------------------------------------------------- |
| `eyepop instance stop`           | Stop the container; state and registration survive |
| `eyepop instance start`          | Start it again, recreating anything missing        |
| `eyepop instance restart`        | Stop and start                                     |
| `eyepop instance set pop <pop>`  | Serve a different pop; recreates the container     |
| `eyepop instance logs --tail 50` | Runtime logs; `--follow` streams                   |
| `eyepop system`                  | This machine's hardware and the profile it maps to |
| `eyepop get instances`           | Your account's instances                           |

A stopped instance does not fall back to the cloud: `run` says the instance is not responding and names `eyepop instance start` as the fix.

### Repair

Re-running `eyepop instance init` repairs. A deleted container, a wiped volume, a half-written instance root — all heal with the same command, and the instance keeps its identity: the account record, and the billing history keyed to it, stay the same.

If the instance root is gone entirely but your account still holds a record with this machine's name, `init` adopts that record instead of minting a second one. One guard: a record that registered within the last 24 hours may be another machine that is still in use, so `init` refuses to adopt it silently — pass `--adopt <uuid>` if this machine really is its replacement.

### Remove it

```bash
eyepop instance delete --yes
```

Containers, volumes (the model cache included), network, the pulled runtime image, the instance root, the account record — all gone. The registry credential goes too: the CLI deletes the robot account it minted and runs `docker logout`. If a docker credential helper holds the credential the CLI cannot name the robot, so it warns that the robot was left in the registry and still logs out.

To delete the account record of a machine you no longer have, pass `--uuid <instance-uuid>`. That deletes the record alone — nothing on the machine you are standing at is touched, your registry credential included.

### Use it from the SDK

The SDKs talk to an on-premise instance in local mode, which points them at `http://127.0.0.1:8080` instead of the cloud. Pass the instance's pop when connecting:

{% tabs %}
{% tab title="Python" %}

```python
from eyepop import EyePopSdk
from eyepop.worker.worker_types import Pop, InferenceComponent

pop = Pop(components=[InferenceComponent(ability="eyepop.person:latest")])

async with EyePopSdk.async_worker(is_local_mode=True, pop=pop) as endpoint:
    job = await endpoint.upload("image.jpg")
    result = await job.predict()
```

{% endtab %}

{% tab title="Node" %}

```typescript
import { EyePop, Pop, PopComponentType } from '@eyepop.ai/eyepop'

const pop: Pop = {
    components: [{ type: PopComponentType.INFERENCE, ability: 'eyepop.person:latest' }],
}

const endpoint = await EyePop.workerEndpoint({ isLocalMode: true, pop }).connect()
try {
    const results = await endpoint.process({ source: { path: 'image.jpg' } })
    for await (const result of results) {
        console.log(result)
    }
} finally {
    await endpoint.disconnect()
}
```

`EYEPOP_LOCAL_MODE=true` in the environment sets `isLocalMode` for you.
{% endtab %}
{% endtabs %}

### Next steps

* [On-Premise](https://docs.eyepop.ai/developer-documentation/deploying/on-premise) — the same instance, covered from the deployment side
* [Command Reference](/developer-documentation/cli/reference.md) — every `instance` command and its flags