VR & EEG
Mixed Computer Science Methods in the Evaluation of Facial Emotion Expression Recognition
DOI:
https://doi.org/10.54337/aau.icdvrat26.26Abstract
Facial Emotion Expression Recognition is widely agreed to be one aspect of social function that is impaired within the usual characterisation of Autism Spectrum Disorder. The ”Reading the Mind in the Eyes” test is used to this day in the diagnostic process. However, it disregards temporal data, has potential test format effects, and only observes performance rather than underlying behavioral triggers. With the increasing prevalence of VR and EEG tools, including this temporal data and investigating brain activity during testing may result in novel methods to investigate impairments in social function. This paper outlines a study protocol to be used to investigate the application of said tools to this purpose.
References
American Psychiatric Association (2022), Diagnostic and Statistical Manual of Mental Disorders, American Psychiatric Association Publishing.
Baron-Cohen, S., Wheelwright, S., Skinner, R., Martin, J. & Clubley, E. (2001), ‘The autism-spectrum quotient (aq): Evidence from asperger syndrome/high-functioning autism, males and females, scientists and mathematicians’, Journal of Autism and Developmental Disorders 31, 5–17.
Bender, A., Voytek, B. & Schaworonkow, N. (2025), ‘Resting-state alpha and mu rhythms change shape across development but lack diagnostic sensitivity for attention-deficit/hyperactivity disorder and autism’, Journal of Cognitive Neuroscience 37, 1581–1615.
Bernier, R., Dawson, G.,Webb, S. & Murias, M. (2007), ‘Eeg mu rhythm and imitation impairments in individuals with autism spectrum disorder’, Brain and Cognition 64, 228–237.
Darvishi-Bayazi, M.-J., Ghaemi, M. S., Rish, I. & Faubert, J. (2025), ‘Critical role of eeg signals in assessment of sex-specific insights in neurological diagnostics via machine learning approach’, Scientific Reports 16, 1060.
Ekman, P. (1992), ‘Facial expressions of emotion: New findings, new questions’, Psychological Science 3, 34–38.
Fang, Z., Cai, L. &Wang, G. (2021), Metahuman creator the starting point of the metaverse, in ‘2021 International Symposium on Computer Technology and Information Science (ISCTIS)’, IEEE, pp. 154–157.
Fox, N. A., Bakermans-Kranenburg, M. J., Yoo, K. H., Bowman, L. C., Cannon, E. N., Vanderwert, R. E., Ferrari, P. F. & van IJzendoorn, M. H. (2016), ‘Assessing human mirror activity with eeg mu rhythm: A meta-analysis.’, Psychological bulletin 142, 291–313.
Games, E. (2023), ‘Scan yourself for mesh to metahuman’. URL: https://dev.epicgames.com/community/learning/tutorials/Ovd2/capturing-reality-scan-yourself-formesh-to-metahuman
Higgins, W. C., Savalei, V., Polito, V. & Ross, R. M. (2026), ‘Reading the mind in the eyes test scores demonstrate poor structural properties in nine large non-clinical samples’, Assessment 33, 204–220.
Hobson, H. M. & Bishop, D. V. M. (2017), ‘The interpretation of mu suppression as an index of mirror neuron activity: past, present and future’, Royal Society Open Science 4, 160662.
Huang, Y., Yang, J., Liao, P. & Pan, J. (2017), ‘Fusion of facial expressions and eeg for multimodal emotion recognition’, Computational Intelligence and Neuroscience 2017, 1–8.
LeBeau, R. T., Mesri, B. & Craske, M. G. (2016), ‘The dsm-5 social anxiety disorder severity scale: Evidence of validity and reliability in a clinical sample’, Psychiatry Research 244, 94–96.
OpenBCI (2026), ‘Galea’.
Ramos, Y. E., Santos, M. T., Yamamoto, I. N. R., da Costa Accioly, C. B., Daneault, J.-F., de Almeida Filho, D. G. & Miranda, J. G. V. (2025), ‘Handedness and brain lateralization: A nonlinear motor approach combined with eeg’, Human Movement Science 104, 103425.
Vellante, M., Baron-Cohen, S., Melis, M., Marrone, M., Petretto, D. R., Masala, C. & Preti, A. (2013), ‘The “reading the mind in the eyes” test: Systematic review of psychometric properties and a validation study in italy’, Cognitive Neuropsychiatry 18, 326–354.
Wagner, J. B., Hirsch, S. B., Vogel-Farley, V. K., Redcay, E. & Nelson, C. A. (2013), ‘Eye-tracking, autonomic, and electrophysiological correlates of emotional face processing in adolescents with autism spectrum disorder’, Journal of Autism and Developmental Disorders 43, 188–199.
Wall, N. G., Smith, O., Campbell, L., Loughland, C. & Schall, U. (2026), ‘Using eeg and eye tracking to evaluate an emotion recognition ipad app for autistic children’, Clinical EEG and Neuroscience 57, 130–140.
