# EyePop On-Premise AI Runtime

Welcome to the EyePop On-Premise AI Runtime—your gateway to powerful, private, and production-ready AI computer vision. Whether you choose to run your workloads in the cloud at [eyepop.ai](https://www.eyepop.ai) or deploy them directly inside your own infrastructure, EyePop ensures performance, control, and seamless integration.

Each runtime instance is securely linked to your EyePop account for data isolation, model access, and full usage and health monitoring.

***

### Key Features

* **Runs Anywhere:** Cloud or local on Docker-compatible systems (Linux, Windows WSL2, Mac CPU Only)
* **Fully API-Driven:** REST API and official SDKs for Python and Node (Typescript)
* **Modular Inference Pipelines:** Supports all EyePop Pop definitions and components
* **GPU-Ready:** CUDA 12.9 support with certified NVIDIA drivers and runtime
* **Private by Design:** All data stays within your control on-premise

***

### What’s Included

The EyePop AI Runtime is provided as a **Docker container image** and exposes its functionality over a local HTTP API (default: port 8080). The runtime supports:

* Media uploads (images/videos)
* Stream setup
* Pipeline execution
* Inference result collection
* Telemetry reporting

You can interact via:

* [**Python SDK**](/developer-documentation/sdks/python-sdk.md)
* [**Node SDK**](/developer-documentation/sdks/react-node-sdk.md)

A valid **EyePop API Key** and **Account UUID** are required to start.

***

### System Requirements

**Host OS:**

* Ubuntu 24.04 (Intel/AMD 64-bit) 
* Docker ≥ 28.5.1

**Optional for GPU acceleration:**

* NVIDIA Driver 580 or 580-server
* NVIDIA Container Toolkit 1.17.8
* CUDA 12.9-compatible GPU

**SDKs:**

* Python SDK: Python ≥ 3.12
* Node SDK: Node.js ≥ 22.16

***

### Quick Start Guide

#### 1. Validate Docker Installation

```shell
docker --version
docker run --rm hello-world
```

#### 2. Get Your Deployment Credentials

* Go to [My Servers](https://dashboard.eyepop.ai/servers) in your EyePop.ai Dashboard
* Click Add Server and follow the instructions
* You will receive a YML file with your:
  * EyePop Account UUID
  * EyePop API Key
  * Docker Registry Username & Password

#### 3. Authenticate with EyePop Docker Registry

```shell
docker login registry.eyepop.ai -u <username> -p <password>
```

#### 4. Pull the Runtime Image

```shell
docker pull registry.eyepop.ai/ai/runtime-cpu:latest
```

#### 5. Create Your Provisioning File

You only need to create this file if you did not download it in Step 2

**filename:** *eyepop-instance.yml*

**file contents:**

```yaml
eyepop-account-uuid: YOUR_ACCOUNT_UUID
eyepop-api-key: YOUR_API_KEY
instance-name: OptionalName
```

#### 6. Start the AI Runtime

```shell
 docker run -d \
    --name eyepop-runtime-1 \
    --restart unless-stopped \
    -p 8080:8080 \
    -v "$HOME/eyepop-instance.yml:/etc/eyepop-instance.yml" \
    registry.eyepop.ai/ai/runtime-cpu:latest && \
  docker logs -f eyepop-runtime-1
```

Check status:

```shell
curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/version
```

***

### Using the Runtime Locally (Python Example)

```python
import asyncio, sys, json
from eyepop import EyePopSdk, Job
from eyepop.worker.worker_types import Pop, InferenceComponent

async def on_ready(job: Job, url: str):
    while result := await job.predict():
        print(url, json.dumps(result, indent=2))

async def main(pop: Pop, url: str):
    async with EyePopSdk.async_worker(is_local_mode=True) as endpoint:        
        await endpoint.set_pop(pop)
        job = await endpoint.load_from(url)
        await on_ready(job, url)

POP = Pop(components=[InferenceComponent(ability='eyepop.common-objects:latest')])

asyncio.run(main(
    POP,
    sys.argv[1]
))
```

Then run:

```shell
python3 -m venv .venv && \
. .venv/bin/activate && \
pip install eyepop && \
python demo.py "https://example.com/image.jpg"
```

***

### Hardware Acceleration (CUDA)

#### Confirm Setup

```shell
docker run --rm --gpus all nvidia/cuda:12.9.1-cudnn-runtime-ubuntu24.04 nvidia-smi
```

#### Run GPU Runtime

```shell
docker create \
  --name eyepop-runtime-1 \
  --restart unless-stopped \
  --publish 127.0.0.1:8080:8080 \
  --gpus=all \
  --volume $HOME/eyepop-instance.yml:/etc/eyepop-instance.yml \
  registry.eyepop.ai/ai/runtime-cuda:latest
```

***

### Configuration Options (`eyepop-instance.yml`)

| Key                   | Required        | Description                                                          |
| --------------------- | --------------- | -------------------------------------------------------------------- |
| `eyepop-api-key`      | Yes             | Your API key for authentication                                      |
| `eyepop-account-uuid` | Yes             | Used for tracking usage and linking to your account                  |
| `instance-name`       | No              | Human-friendly name shown in the EyePop dashboard                    |
| `log-level`           | No              | Logging level: "error", "warn", "info", or "debug" (default: "info") |
| `eyepop-url`          | Only in staging | Override for non-production environments                             |

***

### Uninstall

```shell
docker stop eyepop-runtime-1 && docker rm eyepop-runtime-1
```

***

### Support

Questions? Reach out to us anytime at **<support@eyepop.ai>**\
Or visit [www.eyepop.ai](https://www.eyepop.ai)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.eyepop.ai/developer-documentation/deployment/eyepop-on-premise-ai-runtime.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.