Feature Fusion Based Audio-Visual Speaker Identification Using Hidden Markov Model under Different Lighting Variations

Abstract EN

The aim of the paper is to propose a feature fusion based Audio-Visual Speaker Identification (AVSI) system with varied conditions of illumination environments.

Among the different fusion strategies, feature level fusion has been used for the proposed AVSI system where Hidden Markov Model (HMM) is used for learning and classification.

Since the feature set contains richer information about the raw biometric data than any other levels, integration at feature level is expected to provide better authentication results.

In this paper, both Mel Frequency Cepstral Coefficients (MFCCs) and Linear Prediction Cepstral Coefficients (LPCCs) are combined to get the audio feature vectors and Active Shape Model (ASM) based appearance and shape facial features are concatenated to take the visual feature vectors.

These combined audio and visual features are used for the feature-fusion.

To reduce the dimension of the audio and visual feature vectors, Principal Component Analysis (PCA) method is used.

The VALID audio-visual database is used to measure the performance of the proposed system where four different illumination levels of lighting conditions are considered.

Experimental results focus on the significance of the proposed audio-visual speaker identification system with various combinations of audio and visual features.