Multimodal investigations of emotional face processing and social trait judgment of faces

Hongbo Yu, Chujun Lin, Sai Sun, Runnan Cao, Kohitij Kar, Shuo Wang

Research output: Contribution to journalReview articlepeer-review

2 Scopus citations


Faces are among the most important visual stimuli that humans perceive in everyday life. While extensive literature has examined emotional processing and social evaluations of faces, most studies have examined either topic using unimodal approaches. In this review, we promote the use of multimodal cognitive neuroscience approaches to study these processes, using two lines of research as examples: ambiguity in facial expressions of emotion and social trait judgment of faces. In the first set of studies, we identified an event-related potential that signals emotion ambiguity using electroencephalography and we found convergent neural responses to emotion ambiguity using functional neuroimaging and single-neuron recordings. In the second set of studies, we discuss how different neuroimaging and personality-dimensional approaches together provide new insights into social trait judgments of faces. In both sets of studies, we provide an in-depth comparison between neurotypicals and people with autism spectrum disorder. We offer a computational account for the behavioral and neural markers of the different facial processing between the two groups. Finally, we suggest new practices for studying the emotional processing and social evaluations of faces. All data discussed in the case studies of this review are publicly available.

Original languageEnglish
Pages (from-to)29-48
Number of pages20
JournalAnnals of the New York Academy of Sciences
Issue number1
StatePublished - Jan 2024


  • amygdala
  • autism spectrum disorder
  • emotion
  • face processing
  • multimodal approaches
  • social trait judgment


Dive into the research topics of 'Multimodal investigations of emotional face processing and social trait judgment of faces'. Together they form a unique fingerprint.

Cite this