Discriminative Few Shot Learning of Facial Dynamics in Interview Videos for Autism Trait Classification

Na Zhang, Mindi Ruan, Shuo Wang, Lynn Paul, Xin Li

Research output: Contribution to journalArticlepeer-review

9 Scopus citations


Autism is a prevalent neurodevelopmental disorder characterized by impairments in social and communicative behaviors. Possible connections between autism and facial expression recognition have recently been studied in the literature. However, most works are based on facial images or short videos. Few works aim at Autism Diagnostic Observation Schedule (ADOS) videos due to their complexity (e.g., interaction between interviewer and interviewee) and length (e.g., usually last for hours). In this paper, we attempt to fill this gap by developing a novel discriminative few shot learning method to analyze hour-long video data and exploring the fusion of facial dynamics for the trait classification of ASD. Leveraging well-established computer vision tools from spatio-temporal feature extraction and marginal fisher analysis to few-shot learning and scene-level fusion, we have constructed a three-category system to classify an individual into Autism, Autism Spectrum, and Non-Spectrum. For the first time, we have shown that certain interview scenes carry more discriminative information for ASD trait classification than others. Experimental results are reported to demonstrate the potential of the proposed automatic ASD trait classification system (achieving 91.72% accuracy on the Caltech ADOS video dataset) and the benefits of few-shot learning and scene-level fusion strategy by extensive ablation studies.

Original languageEnglish
Pages (from-to)1110-1124
Number of pages15
JournalIEEE Transactions on Affective Computing
Issue number2
StatePublished - Apr 1 2023


  • Autism Spectrum Disorder (ASD)
  • autism trait classification
  • facial dynamic features
  • few-shot learning (FSL)
  • marginal fisher analysis (MFA)
  • scene-level fusion


Dive into the research topics of 'Discriminative Few Shot Learning of Facial Dynamics in Interview Videos for Autism Trait Classification'. Together they form a unique fingerprint.

Cite this