Context-Aware Attention Network for Human Emotion Recognition in Video

Abstract EN

Recognition of human emotion from facial expression is affected by distortions of pictorial quality and facial pose, which is often ignored by traditional video emotion recognition methods.

On the other hand, context information can also provide different degrees of extra clues, which can further improve the recognition accuracy.

In this paper, we first build a video dataset with seven categories of human emotion, named human emotion in the video (HEIV).

With the HEIV dataset, we trained a context-aware attention network (CAAN) to recognize human emotion.

The network consists of two subnetworks to process both face and context information.

Features from facial expression and context clues are fused to represent the emotion of video frames, which will be then passed through an attention network and generate emotion scores.

Then, the emotion features of all frames will be aggregated according to their emotional score.

Experimental results show that our proposed method is effective on HEIV dataset.