EyePop On-Premise AI Runtime

Deployable, Flexible, API-First Inference Engine

Welcome to the EyePop On-Premise AI Runtime—your gateway to powerful, private, and production-ready AI computer vision. Whether you choose to run your workloads in the cloud at eyepop.ai or deploy them directly inside your own infrastructure, EyePop ensures performance, control, and seamless integration.

Each runtime instance is securely linked to your EyePop account for data isolation, model access, and full usage and health monitoring.

---

Key Features

• Runs Anywhere: Cloud or local on Docker-compatible systems (Linux, Windows WSL2, Mac CPU Only)

• Fully API-Driven: REST API and official SDKs for Python and Node (Typescript)

• Modular Inference Pipelines: Supports all EyePop Pop definitions and components

• GPU-Ready: CUDA 12.9 support with certified NVIDIA drivers and runtime

• Private by Design: All data stays within your control on-premise

---

What’s Included

The EyePop AI Runtime is provided as a Docker container image and exposes its functionality over a local HTTP API (default: port 8080). The runtime supports:

• Media uploads (images/videos)

• Stream setup

• Pipeline execution

• Inference result collection

• Telemetry reporting

You can interact via:

• Python SDK

• Node SDK

A valid EyePop API Key and Account UUID are required to start.

---

System Requirements

Host OS:

• Ubuntu 24.04 (Intel/AMD 64-bit)

• Docker ≥ 28.5.1

Optional for GPU acceleration:

• NVIDIA Driver 580 or 580-server

• NVIDIA Container Toolkit 1.17.8

• CUDA 12.9-compatible GPU

SDKs:

• Python SDK: Python ≥ 3.12

• Node SDK: Node.js ≥ 22.16

---

Quick Start Guide

1. Validate Docker Installation

docker --version
docker run --rm hello-world

2. Get Your Deployment Credentials

From your EyePop Dashboard (“Your Instances” tab), collect:

• EyePop Account UUID

• EyePop API Key

• Docker Registry Username & Password

3. Authenticate with EyePop Docker Registry

docker login registry.eyepop.ai -u <username> -p <password>

4. Pull the Runtime Image

docker pull registry.eyepop.ai/ai/runtime-cpu:latest

5. Create Your Provisioning File

eyepop-account-uuid: YOUR_ACCOUNT_UUID
eyepop-api-key: YOUR_API_KEY
instance-name: OptionalName

6. Start the AI Runtime

docker create \
  --name eyepop-runtime-1 \
  --restart unless-stopped \
  --publish 127.0.0.1:8080:8080 \
  --volume $HOME/eyepop-instance.yml:/etc/eyepop-instance.yml \
  registry.eyepop.ai/ai/runtime-cpu:latest && \
docker start eyepop-runtime-1 && \
docker logs -f eyepop-runtime-1

Check status:

curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/version

---

Using the Runtime Locally (Python Example)

import asyncio, sys, json
from eyepop import EyePopSdk, Job
from eyepop.worker.worker_types import Pop, InferenceComponent

async def on_ready(job: Job, url: str):
    while result := await job.predict():
        print(url, json.dumps(result, indent=2))

async def main(pop: Pop, url: str):
    async with EyePopSdk.workerEndpoint(is_local_mode=True, is_async=True) as endpoint:
        await endpoint.set_pop(pop)
        job = await endpoint.load_from(url)
        await on_ready(job, url)

asyncio.run(main(
    Pop(components=[InferenceComponent(model='eyepop.common-objects:latest')]),
    sys.argv[1]
))

Then run:

python3 -m venv .venv && \
. .venv/bin/activate && \
pip install eyepop && \
python demo.py "https://example.com/image.jpg"

---

Hardware Acceleration (CUDA)

Confirm Setup

docker run --rm --gpus all nvidia/cuda:12.9.1-cudnn-runtime-ubuntu24.04 nvidia-smi

Run GPU Runtime

docker create \
  --name eyepop-runtime-1 \
  --restart unless-stopped \
  --publish 127.0.0.1:8080:8080 \
  --gpus=all \
  --volume $HOME/eyepop-instance.yml:/etc/eyepop-instance.yml \
  registry.eyepop.ai/ai/runtime-cuda-12.9-nvidia-580-server:latest

---

Configuration Options (eyepop-instance.yml)

Key	Required	Description
eyepop-api-key	Yes	Your API key for authentication
eyepop-account-uuid	Yes	Used for tracking usage and linking to your account
instance-name	No	Human-friendly name shown in the EyePop dashboard
log-level	No	Logging level: "error", "warn", "info", or "debug" (default: "info")
eyepop-url	Only in staging	Override for non-production environments

---

Uninstall

docker stop eyepop-runtime-1 && docker rm eyepop-runtime-1

---

Support

Questions? Reach out to us anytime at [email protected] Or visit www.eyepop.ai