Classes and Texts
Labels and recognized text, for a whole frame or for one object
Not every answer has a place. A classifier says what kind — a scene, an expression, an activity — and a text reader says what is written. These answers come as classes and texts, attached to the whole frame or to one object.
Classes
A class is a label with a confidence:
{ "classLabel": "neutral", "confidence": 0.9966 }classLabel
string
The label
confidence
number
How certain the model is, from 0 to 1
category
string
The group the label belongs to, when the Pop or the ability names one
For the whole frame
A classifier that runs on the full image puts its classes on the prediction. Asked what the people in the lobby image are doing, eyepop.image-contents:latest answers:
{
"source_width": 1280,
"source_height": 720,
"timestamp": 0,
"classes": [
{ "classLabel": "standing", "confidence": 0.99, "category": "human activity" }
]
}For one object
A classifier that runs on a crop puts its classes on the object it was cropped from. An expression classifier run on each face:
The object's own classLabel says what it is, a face; its classes say what was learned about it, a neutral expression.
More than one class
classes is a list. An ability can return several labels for one thing, each with its own confidence. The confidences are independent: they do not have to add up to 1, and several can be high at once. Asked what the person in the lobby is wearing, eyepop.image-contents:latest names two pieces of clothing, both with high confidence:
Texts
A text is a string that was read, with a confidence:
text
string
What was read
confidence
number
How certain the reader is, from 0 to 1
category
string
The group, when the Pop or the ability names one
Reading text usually takes two steps: a detector finds where the text is, and a reader reads each place. So texts typically arrive on the object the reader ran on. Run on a sign that says "New York Pies", the text detector finds three words, and the reader adds what each says:

Two confidences are at play: the object's says how sure the detector is that there is text at this place, and the text's says how sure the reader is of what it says. A clearly visible but badly lettered word has a high first and a low second.
The objects are not in reading order. To put words in reading order, sort them by their position.
Example: the best label of each face
On the prediction from the person, face and expression Pop in Predictions:
Next steps
Key Points and Meshes — landmarks and surfaces on an object
Raw Output and Details — answers that are neither a label nor text
Last updated
