Transformer-based classifier with feature aggregation for cancer subtype classification on histopathological images

Chaojie Zhang, Zong Fan, Zhimin Wang, Lulu Sun, Yao Hao, Zhongwei Zhang, Wade Thorstad, Hiram Gay, Xiaowei Wang, Mark A. Anastasio, Hua Li

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Histopathology whole-slide image (WSI) captures detailed structural and morphological features of tumor tissue, offering rich histological and molecular information to support clinical practice. With the development of artificial intelligence, deep learning (DL) methods have emerged to assist in automatically analyzing histopathology WSIs. It alleviates the need for tedious, time-consuming, and error-prone inspections by clinicians. Up to now, employing DL models for histopathology WSI analysis is still challenging due to the intrinsic complexity of histology characteristics of tumor tissue, high image resolution, and large image size. In this study, we proposed a transformer-based classifier with feature aggregation for cancer subtype classification using histopathology WSIs while addressing these challenges. Our method shows three advantages to improve classification performance. First, an aggregate transformer decoder is employed to learn both global and local features from WSIs. Second, the transformer architecture facilitates the decoder to learn spatial correlations among different regions in a WSI. Third, the self-attention mechanism of the transformer facilitates the generation of saliency maps to highlight regions of interest in WSIs. We evaluated our model on three cancer subtype classification tasks and demonstrated its effectiveness and performance.

Original languageEnglish
Title of host publicationMedical Imaging 2024
Subtitle of host publicationImage Processing
EditorsOlivier Colliot, Jhimli Mitra
ISBN (Electronic)9781510671560
StatePublished - 2024
EventMedical Imaging 2024: Image Processing - San Diego, United States
Duration: Feb 19 2024Feb 22 2024

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
ISSN (Print)1605-7422


ConferenceMedical Imaging 2024: Image Processing
Country/TerritoryUnited States
CitySan Diego


Dive into the research topics of 'Transformer-based classifier with feature aggregation for cancer subtype classification on histopathological images'. Together they form a unique fingerprint.

Cite this