Multimodal Human Behavior Processing Unit
Facial expressions
Gestures
Multimodal Human
Behavior Processing Unit
Body pose
Physiology
Voice
Core technology
1
Multimodal
Recognition
2
More than 20 emotions
and behavioral patterns
3
Natural Data
Processing
4
Biggest
affective dataset
Multimodality
Emotions are multimodal. We do not use only words or facial expressions to express our emotions - we naturally engage several modalities. Neurodata Lab’s core technology analyses each of the modalities below: some manifestations are explicit and can be detected via facial expressions , body language, change in the tone of voice; some are implicit, like psychophysiological manifestations, and require subtler recognition methods.
Facial
Expressions
Vocal
Affects
Body
poses
Interpersonal
distance
Gestures
Heart
rate
Respiration
rate
  • Happiness
  • Sadness
  • Anger
  • Disgust
  • Surprise
  • Engagement
  • Disengagement
  • Hostility
  • Friendliness
  • Contempt
  • Admiration
  • Self-presentation
  • Mental Effort
  • Shame
  • Pride
  • Anxiety
  • Pleasure
  • Self-confidence
  • Self-disclosure
  • Neutral
Emotions
Our product can recognize more
than 20 scales of emotions, cognitive states, social behavior patterns.
Natural Data Processing
One of the main advantages of multimodality is in universal application in the changing environment. In the conditions of partial loss of information from one of the channels: temporary disappearance of the person from the frame; a noisy audio picture; fuzzy, unclear, overlapping image of the body — all these flaws will not affect continuity and accuracy of emotion recognition. This is emotion recognition ‘in the wild.’
Any head
position
Multi-person
analysis
Complex
approach
Accuracy
90–92%
Biggest affective dataset
We train our emotion and human behavior recognition algorithms on hundreds of hours of audiovisual multimodal affective data, manually annotated and labeled on our own global online platform Emotion Miner.
160 hours of pubic
audiovisual content
in English
More than 110 000 annotated
video fragments, 10+ annotators
for each fragment
50,000 registered users
from 35+ countries and
growing
INPUT MODALITIES
The product allows to work with streaming video and audio in real time as processing speed significantly exceeds 25 fps, as well as with pre-recorded material of all common video and audio file formats.
VIDEO
Live Stream
Audio
No specific client hardware is
needed to obtain affective data:
CLOUD SOLUTIONS
  • Emotion as a Service (EaaS)
  • Emotion API
  • Emotion SDK
  • Emotion AI application
  • Fully customized solution for specific goals
Demo
See how it works: