Multimodal affect recognition in learning environments
Top Cited Papers
- 6 November 2005
- proceedings article
- Published by Association for Computing Machinery (ACM)
- p. 677-682
- https://doi.org/10.1145/1101149.1101300
Abstract
We propose a multi-sensor affect recognition system and evaluate it on the challenging task of classifying interest (or disinterest) in children trying to solve an educational puzzle on the computer. The multimodal sensory information from facial expressions and postural shifts of the learner is combined with information about the learner's activity on the computer. We propose a unified approach, based on a mixture of Gaussian Processes, for achieving sensor fusion under the problematic conditions of missing channels and noisy labels. This approach generates separate class labels corresponding to each individual modality. The final classification is based upon a hidden random variable, which probabilistically combines the sensors. The multimodal Gaussian Process approach achieves accuracy of over 86%, significantly outperforming classification using the individual modalities, and several other combination schemes.Keywords
This publication has 8 references indexed in Scilit:
- Mixture of Gaussian Processes for Combining Multiple ModalitiesPublished by Springer Nature ,2005
- Probabilistic combination of multiple modalities to detect interestPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2004
- Toward an affect-sensitive multimodal human-computer interactionProceedings of the IEEE, 2003
- Automated Posture Analysis for Detecting Learner's Interest LevelPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- Toward machine emotional intelligence: analysis of affective physiological statePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2001
- Critic-driven ensemble classificationIEEE Transactions on Signal Processing, 1999
- On combining classifiersPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1998
- Adaptive Mixtures of Local ExpertsNeural Computation, 1991