# JavaScript SDK

EyePop.ai has introduced a dedicated npm package for JavaScript developers, streamlining the process of integrating your Pop in the web. This package ensures an efficient and user-friendly setup experience. 

<https://www.npmjs.com/package/@eyepop.ai/javascript-sdk>

**npm:**

**`npm install`**<mark style="color:yellow;">**`@eyepop.ai/javascript-sdk`**</mark>

**cdn:**

**`<script src="`**<mark style="color:yellow;">**`https://cdn.jsdelivr.net/npm/@eyepop.ai/javascript-sdk`**</mark>**`"></script>`**

***

### Quick Start

Refer to the following page to quickly get the EyePopSDK JavaScript demo repository up and running on your local machine.

{% content-ref url="javascript-sdk/javascript-sdk-demos" %}
[javascript-sdk-demos](https://docs.eyepop.ai/developer-documentation/sdks/javascript-sdk/javascript-sdk-demos)
{% endcontent-ref %}

## SDK Usage

Here's a barebones example of the SDK for uploading and displaying an image with object identification overlays.

<pre class="language-html" data-overflow="wrap"><code class="lang-html">
&#x3C;form>
    <a data-footnote-ref href="#user-content-fn-1">&#x3C;input type="file" id="file_upload"></a>
&#x3C;/form>

<a data-footnote-ref href="#user-content-fn-2">&#x3C;video id="myLocalVideo" playsinline controls muted style="width:1px;height:1px;">&#x3C;/video></a>

&#x3C;div style="height: 600px; width:600px;">
<a data-footnote-ref href="#user-content-fn-3">    &#x3C;canvas id="mobilecanvas" style="background-color: transparent; opacity: 1; margin:auto auto;z-index:1000;" width="100%" height="100%">&#x3C;/canvas></a>
&#x3C;/div>

&#x3C;script src="https://cdn.jsdelivr.net/npm/@eyepop.ai/javascript-sdk">&#x3C;/script>

&#x3C;script>
    var config = {};
    
    // replace with your endpoint UUID
    const pop_uuid = '&#x3C;POP_UUID>'; 
    
    // leave this empty to launch a 'login' popup, or enter a temporary token 
    const token = ''; 

    // first, fetch the Pop info    
    EyePopSDK.EyePopAPI.FetchPopConfig(pop_endpoint, token)
        .then((response) => {
            config = response;
            
            config.input = {
                "name": "file_upload"
            };
             
            console.log("EyePopSDK config: ", config);
            
            // then start the Pop
            EyePopSDK.EyePopSDK.init(config);
        });
    
&#x3C;/script>
</code></pre>

***

## API

#### Table of Contents

* <mark style="color:red;">EyePopSDK</mark>&#x20;
  * [<mark style="color:green;">EyePopSDK</mark>](#eyepopsdk)
    * [<mark style="color:yellow;">instance</mark>](#instance)
    * [<mark style="color:yellow;">init(config)</mark>](#init-config)
  * [<mark style="color:green;">EyePopAPI</mark> ](#eyepopapi)
    * [<mark style="color:yellow;">lastmsg</mark>](#lastmsg)
    * [<mark style="color:yellow;">OnDrawFrame</mark>](#ondrawframe)<mark style="color:yellow;">()</mark>
    * [<mark style="color:yellow;">OnDrawFrameEnd(frameData)</mark>](#ondrawframeend-jsondata)
    * [<mark style="color:yellow;">OnPrediction(jsonData)</mark>](#onprediction-jsondata)
    * [<mark style="color:yellow;">OnPredictionTarget()</mark>](#onpredictiontarget)
    * [<mark style="color:yellow;">OnPredictionEnd()</mark>](#onpredictionend)
    * [<mark style="color:yellow;">onPredictionEndBase()</mark>](#onpredictionendbase)
  * [<mark style="color:green;">Rules</mark>](#rules)
    * [<mark style="color:yellow;">FindObjects(label, objects)</mark> ](#findobject-label-object)
    * [<mark style="color:yellow;">Biggest(label, objects)</mark> ](#biggest-label-object)
    * [<mark style="color:yellow;">Area(object, source\_width, source\_height)</mark> ](#area-object-source_width-source_height)
    * [<mark style="color:yellow;">Between(x, min, max)</mark> ](#between-x-min-max)
    * [<mark style="color:yellow;">Amount(label, objects)</mark> ](#amount-label-objects)
    * [<mark style="color:yellow;">PosePoint(label, personObject)</mark> ](#posepoint-label-personobject)
    * [<mark style="color:yellow;">Emotion(emotionLabel, personObject)</mark> ](#emotion-emotionlabel-personobject)
    * [<mark style="color:yellow;">Gender(genderLabel, personObject)</mark> ](#gender-genderlabel-personobject)
    * [<mark style="color:yellow;">Position(object1, direction, object2)</mark> ](#position-object1-direction-object2)
    * [<mark style="color:yellow;">Check(resultSet, rules, rulesState)</mark>](#check-resultset-rules-rulesstate)

***

## **EyePopSDK**

#### <mark style="color:yellow;">`instance`</mark>

The static instance of theSDK.

#### <mark style="color:yellow;">`init(config)`</mark>

The initialization function required for the SDK. This starts all media streaming and uploading, as well as drawing on top of the provided canvas. This object is created with a call to EyePopAPI.FetPopConfig(pop\_endpoint, token) and appended to with the following options:

* <mark style="color:green;">Parameters:</mark>

<details>

<summary><code>config.input</code></summary>

This is where to set a Pop's media input type and source is set.

<pre class="language-java" data-full-width="true"><code class="lang-java">
// An object with the following properties:

config.input = {} 


    // A string name of the input media, options are: "webcam_on_page", "screen", "webcam_off_site", "url", "file_upload"

    <a data-footnote-ref href="#user-content-fn-4">config.input.name</a> = "" 
    

    // A string url of the input media, ex: "https://www.example.com/video.mp4"

    <a data-footnote-ref href="#user-content-fn-5">config.input.url</a> = ""
    
 
</code></pre>

</details>

<details>

<summary><code>config.draw</code></summary>

The SDK supports an array of drawing methods, organized alphabetically by type. To use, you must change the Draw parameter on the config object used to initiate the EyePopSDK.

Overwrite the config.Draw object to enable or disable drawing passes. Pass an empty array to disable all drawings.&#x20;

User configurable elements of the config are as follows:

```java

// An array of objects, each specifying a different drawing pass. You are free to enable as many drawing passes as required.
config.draw = [ {} ]


    // A string, options include: "box", "pose", "hand", "face", "posefollow", "clip", "custom"

    config.draw[ i ].type = ""  


    // An array of strings, options include: "*", "people",

    config.draw[ i ].targets = [ "" ] 


    // An array of strings, options include all pose labels, such as "right eye", "left eye", etc

    config.draw[ i ].anchors = [ "" ] 


    // A string path to an image to be anchored to our Anchor points, ex: "./fun/sunglasses3.png?raw=true", 

    config.draw[ i ].image = "" 


    // A number to scale the anchored image by, ex: 2.6

    config.draw[ i ].scale = 2.6 

```

### <mark style="color:purple;">Draw Types</mark>

#### <mark style="color:purple;">box:</mark>

* Targets: Specific objects or parts of objects you wish to encircle within a box. E.g., "person.face". A value of "\*" means all labels.

#### <mark style="color:purple;">box with Tracking:</mark>

* Targets: A list of objects to enclose within a box. E.g., "apple", "backpack", and more. A value of "\*" means all labels.
* Tracking:&#x20;
  * Tracking allows you to receive a unique ID for each person in a scene. This ID is held stable while a person remains on screen. Tracking has to be turned on for your account so please ask an EyePop.ai Team member to turn this on for you.
* Labels: An array of assigned labels for the tracking augmentation.&#x20;

#### <mark style="color:purple;">pose:</mark>

* Targets: Currently, in the present version, the pose type can only target "person", highlighting body key points.

#### <mark style="color:purple;">posefollow:</mark>

* Targets: Objects like "person" that you aim to track. A value of "\*" means all labels.
* Anchors: Specific points within the target object where the augmentation is anchored, such as "right wrist".
* Image: The image placed at the anchor point, like "./images/sunglasses3.png".
* Scale: Augmentation image's scaling factor, e.g., 1.

<mark style="color:purple;">posefollow with multiple Anchors:</mark>

* Targets: Objects to track, such as "person". A value of "\*" means all labels.
* Anchors: Multiple anchor points within the target object, like "right eye", "left eye".
* Image: Image to be superimposed on the anchor points, such as "./images/scream.png".
* Scale: The image's scaling factor, e.g., 3.5.

</details>

* <mark style="color:green;">Example:</mark>

```java
var config = {};
EyePopSDK.EyePopAPI.FetchPopConfig(pop_endpoint, token)
    .then((response) =>
    {
    
        config = response;


        // First we set our input type        
        config.input = {
            "name": "url", // "webcam_on_page", "screen", "webcam_off_site", "url", "file_upload"
            "url": url
        };


        // Then we enable the following visualization
        config.draw = [
            { "type": "box",  "targets": [ "*" ] },
            { "type": "pose", "targets": [ "*" ] },
            { "type": "hand", "targets": [ "*" ] },
            { "type": "face", "targets": [ "*" ] },
        ]


        EyePopSDK.EyePopSDK.init(config);

    }
);
```

***

## **EyePopAPI**

#### <mark style="color:yellow;">`lastmsg`</mark>

The last message recieved from the Pop WebSocket. Useful for synchronizing video and the drawing loop.

* <mark style="color:green;">Example:</mark>

```java
EyePopSDK.EyePopAPI.instance.OnDrawFrame = function ()
{   
    var closestIndex = findClosestIndex(cached_data, video.currentTime);
    EyePopSDK.EyePopAPI.instance.lastmsg = cached_data[ closestIndex ];
}
```

#### <mark style="color:yellow;">`OnDrawFrame()`</mark>

The callback method fired at the beginning of the draw loop.

* <mark style="color:green;">Example:</mark>

```java
EyePopSDK.EyePopAPI.instance.OnDrawFrame = function ()
{
    console.log("Drawing frame");      
}
```

#### <mark style="color:yellow;">`OnDrawFrameEnd(jsonData)`</mark>

The callback method fired at the end of the draw loop.

* <mark style="color:green;">Example:</mark>

```java
EyePopSDK.EyePopAPI.instance.OnDrawFrameEnd = function (jsonData)
{
    console.log("Finished drawing frame: ", jsonData);      
}
```

#### <mark style="color:yellow;">`OnPrediction(jsonData)`</mark>

The callback method fired when a new prediction message is received.&#x20;

* <mark style="color:green;">Example:</mark>

```java
EyePopSDK.EyePopAPI.instance.OnPrediction = function ()
{
    console.log("Finished drawing frame");      
}
```

#### <mark style="color:yellow;">`OnPredictionTarget()`</mark>

The callback method fired when a target is found in the prediction data.&#x20;

* <mark style="color:green;">Example:</mark>

```java
EyePopSDK.EyePopAPI.instance.OnPredictionTarget = function ()
{
    console.log("Target found!");      
}
```

#### <mark style="color:yellow;">`OnPredictionEnd()`</mark>

The callback method fired when the analysis is completed.&#x20;

* <mark style="color:green;">Example:</mark>

```java
EyePopSDK.EyePopAPI.instance.OnPredictionEnd = function ()
{
    console.log("Analyzed 100%");      
}
```

#### <mark style="color:yellow;">`onPredictionEndBase()`</mark>

The callback method fired when the Pop has closed for any reason.

* <mark style="color:green;">Example:</mark>

```java
EyePopSDK.EyePopAPI.instance.OnPredictionEndBase = function ()
{
    console.error("Pop socket closed!");      
}
```

***

## Rules

The Rules class has been specifically crafted for processing the outputs of EyePop.ai's computer vision system. It provides functionalities to construct semantic rules, helping in the identification and extraction of specific features and attributes from photos and videos.

#### <mark style="color:yellow;">`FindObject(label, object)`</mark>

Purpose: Filters the provided list of objects based on the specified class label.

* <mark style="color:green;">Parameters:</mark>
  * ‘<mark style="color:yellow;">**label**</mark>’: String representing the class label of the desired object.
  * ‘<mark style="color:yellow;">**objects**</mark>’: Array of objects.
* <mark style="color:red;">Returns:</mark>
  * Array of objects that match the specified class label.
* <mark style="color:green;">Example</mark>:

```javascript
var rulesState = [];
var rules = [{
  description: "Person detected!",
  condition: (resultSet) => {
    const resultArray = Rules.FindObject("person", resultSet.objects);
    return resultArray.length > 1;
  }
}];
var predictionData = GetPrediction(image);
var rulesResult = Rules.Check(predictionData, rules, rulesState);
console.log(rulesResult);
```

#### <mark style="color:yellow;">`Biggest(label, object)`</mark>

Purpose: Identifies the object with the largest bounding box area for a specific class label.

* <mark style="color:green;">Parameters:</mark>
  * ‘<mark style="color:yellow;">**label**</mark>’: String representing the class label of the object to compare.
  * ‘<mark style="color:yellow;">**objects**</mark>’: Array of objects.
* <mark style="color:red;">Returns:</mark>
  * Single object with the largest area.
* <mark style="color:green;">Example</mark>:

```javascript
var rulesState = [];
var rules = [{
  description: "Largest person found!",
  condition: (resultSet) => {
    return Rules.Biggest("person", resultSet.objects);
  }
}];
var predictionData = GetPrediction(image);
var rulesResult = Rules.Check(predictionData, rules, rulesState);
console.log(rulesResult);
```

#### <mark style="color:yellow;">`Area(object, source_width, source_height)`</mark>

Purpose: Computes the relative area of an object to the source's dimensions.

* <mark style="color:green;">Parameters:</mark>
  * ‘<mark style="color:yellow;">**object**</mark>’: Object whose area needs to be determined.
  * ‘<mark style="color:yellow;">**source\_width**</mark>’: Width of the source.
  * ‘<mark style="color:yellow;">**source\_height**</mark>’: Height of the source.
* <mark style="color:red;">Returns:</mark>
  * Relative area (fraction) of the object with respect to the source dimensions
* <mark style="color:green;">Example</mark>:

```javascript
var rulesState = [];
var rules = [{
  description: "Person found in normalized coordinates.",
  condition: (resultSet) => {
    return Rules.Area("person", source_width, source_height);
  }
}];
var predictionData = GetPrediction(image);
var rulesResult = Rules.Check(predictionData, rules, rulesState);
console.log(rulesResult);
```

#### <mark style="color:yellow;">`Between(x, min, max)`</mark>

Purpose: Checks if a given value lies between a specified range.

* <mark style="color:green;">Parameters:</mark>
  * ‘<mark style="color:yellow;">**x**</mark>’: The value to be checked.
  * ‘<mark style="color:yellow;">**min**</mark>’: Minimum value of the range.
  * ‘<mark style="color:yellow;">**max**</mark>’: Maximum value of the range.
* <mark style="color:red;">Returns:</mark>
  * Boolean value indicating whether x lies between min and max.
* <mark style="color:green;">Example</mark>:

```javascript
var rulesState = [];
var rules = [{
  description: "Is x between min_x and mix_x?",
  condition: (resultSet) => {
    return Rules.Between(x, min_x, max_x);
  }
}];
var predictionData = GetPrediction(image);
var rulesResult = Rules.Check(predictionData, rules, rulesState);
console.log(rulesResult);
```

#### <mark style="color:yellow;">`Amount(label, objects)`</mark>

Purpose: Counts the number of objects that match a specific class label.

* <mark style="color:green;">Parameters:</mark>
  * ‘<mark style="color:yellow;">**label**</mark>’: String representing the class label.
  * ‘<mark style="color:yellow;">**objects**</mark>’: Array of objects.
* <mark style="color:red;">Returns:</mark>
  * Boolean indicating presence of the pose point label.
* <mark style="color:green;">Example</mark>:

```javascript
var rulesState = [];
var rules = [{
  description: "Person detected!",
  condition: (resultSet) => {
    return Rules.Amount("person", resultSet.objects);
  }
}];
var predictionData = GetPrediction(image);
var rulesResult = Rules.Check(predictionData, rules, rulesState);
console.log(rulesResult);
```

#### <mark style="color:yellow;">`PosePoint(label, personObject)`</mark>

Purpose: Determines if a person object contains a specific pose point label.

* <mark style="color:green;">Parameters:</mark>
  * ‘<mark style="color:yellow;">**label**</mark>’: Pose point label.
  * ‘<mark style="color:yellow;">**personObject**</mark>’: Object containing pose information.
* <mark style="color:red;">Returns:</mark>
  * Boolean indicating presence of the pose point label.
* <mark style="color:green;">Example</mark>:

```javascript
var rulesState = [];
var rules = [{
  description: "Person detected!",
  condition: (resultSet) => {
    return Rules.PosePoint("person", resultSet.objects);
  }
}];
var predictionData = GetPrediction(image);
var rulesResult = Rules.Check(predictionData, rules, rulesState);
console.log(rulesResult);
```

#### <mark style="color:yellow;">`Emotion(emotionLabel, personObject)`</mark>

Purpose: Checks the inferred emotion on a person's face.

* <mark style="color:green;">Parameters:</mark>
  * ‘<mark style="color:yellow;">**emotionLabel**</mark>’: The desired emotion label.
  * ‘<mark style="color:yellow;">**personObject**</mark>’: Object containing facial information.
* <mark style="color:red;">Returns:</mark>
  * Boolean indicating the presence of the specified emotion.
* <mark style="color:green;">Example</mark>:

```javascript
var rulesState = [];
var rules = [{
  description: "Person detected!",
  condition: (person) => {
    return Rules.Emotion("happy", person);
  }
}];
var predictionData = GetPrediction(image);
var rulesResult = Rules.Check(predictionData, rules, rulesState);
console.log(rulesResult);
```

#### <mark style="color:yellow;">`Gender(genderLabel, personObject)`</mark>

Purpose: Checks the inferred gender label of a person based on the identified facial features.

* <mark style="color:green;">Parameters:</mark>
  * ‘<mark style="color:yellow;">**genderLabel**</mark>’: Gender label to check.
  * ‘<mark style="color:yellow;">**personObject**</mark>’: Person Object containing facial information.
* <mark style="color:red;">Returns:</mark>
  * Boolean indicating if the identified gender label matches the specified personObject.
* <mark style="color:green;">Example</mark>:

```javascript
var rulesState = [];
var rules = [{
  description: "Gender detected!",
  condition: (person) => {
    return Rules.Gender("female", person);
  }
}];
var predictionData = GetPrediction(image);
var rulesResult = Rules.Check(predictionData, rules, rulesState);
console.log(rulesResult);
```

#### <mark style="color:yellow;">`Position(object1, direction, object2)`</mark>

Compares the relative positions of two objects based on the specified direction.

* <mark style="color:green;">Parameters:</mark>
  * ‘<mark style="color:yellow;">**object1**</mark>’: First object.
  * ‘<mark style="color:yellow;">**direction**</mark>’: String representing the desired direction (above, below, left, right).
  * ‘<mark style="color:yellow;">**object2**</mark>’: Second object.
* <mark style="color:red;">Returns:</mark>
  * Boolean indicating the relative position of object1 with respect to object2 based on the given direction.
* <mark style="color:green;">Example</mark>:

```javascript
var rulesState = [];
var person1 = null, person2 = null;

var rules = [{
  description: "Person detected!",
  condition: (resultSet) => {
    return Rules.Position(person1, "left", person2);
  }
}];

var predictionData= null;
{prediction, person1, person2} = GetPrediction(image);
var rulesResult = Rules.Check(predictionData, rules, rulesState);
console.log(rulesResult);
```

#### <mark style="color:yellow;">`Check(resultSet, rules, rulesState)`</mark>

Purpose: Evaluates a set of conditions on the provided resultSet and tracks the state of rule evaluations.

* <mark style="color:green;">Parameters:</mark>
  * ‘<mark style="color:yellow;">**resultSet**</mark>’: Data to be evaluated.
  * ‘<mark style="color:yellow;">**rules**</mark>’: Array containing conditions to evaluate on the resultSet.
  * ‘<mark style="color:yellow;">**rulesState**</mark>’: Object to track the state of rule evaluations.
* <mark style="color:red;">Returns:</mark>
  * Array of results for each rule evaluation.
* <mark style="color:green;">Example</mark>:

```javascript
var rulesState = [];
var rules = [ ... ];
var predictionData = GetPrediction(image);
var rulesResult = Rules.Check(predictionData, rules, rulesState);
console.log(rulesResult);
```

[^1]: A form for selecting an image

[^2]: a hidden video component for background operation

[^3]: A canvas for visualization

[^4]: A string, such as: "webcam\_on\_page", "screen", "webcam\_off\_site", "url", or "file\_upload"

[^5]: A URL, such as: "<https://www.example.com/video.mp4>"