Computational Integration of CODEX and Brightfield Histology for Cell Annotation using Deep Learning

Nicholas Lucarelli, Zoltan Laszik, Seth Winfree, Tarek M. El-Achkar, Michael Eadon, Sanjay Jain, Pinaki Sarder

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Cell types present in a biopsy provide information on disease processes and organ health, and are useful in a research setting. Multiplex imaging technologies like CODEX can provide spatial context for protein expression and detect cell types on a whole slide basis. The CODEX workflow also allows for hematoxylin and eosin (H&E) staining on the same sections used in molecular imaging. Deep learning can automate the process of histological analysis, reducing time and effort required. We seek to automatically segment and classify cells from histologically stained renal tissue sections using deep learning, with CODEX generated cell labels as a ground truth. Image data consisted of brightfield H&E whole slide images (WSIs) from a single institution, collected from human reference kidneys. Nuclei were segmented using deep learning, and CODEX markers were measured for each nucleus. Cells and their markers were clustered in an unsupervised manner, and assigned labels according to upregulated markers and spatial biological priors. Classified cell types included: proximal tubules, distal tubules, vessels, interstitial cells, and general glomerular cells. Cell maps were used to train a Deeplab V3+ semantic segmentation network. Cell maps were successfully created in all sections, with ~65% used for training and ~35% used for testing. The trained network achieved a balanced accuracy of 0.75 across all cell types. We were able to automatically segment and classify nuclei from various cell types directly from H&E stained WSIs. In future work, we intend to expand the dataset to include more CODEX markers (and therefore more granular cell types), and more samples with more variability, to test the robustness of the model to new data.

Original languageEnglish
Title of host publicationMedical Imaging 2024
Subtitle of host publicationDigital and Computational Pathology
EditorsJohn E. Tomaszewski, Aaron D. Ward
PublisherSPIE
ISBN (Electronic)9781510671706
DOIs
StatePublished - 2024
EventMedical Imaging 2024: Digital and Computational Pathology - San Diego, United States
Duration: Feb 19 2024Feb 21 2024

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume12933
ISSN (Print)1605-7422

Conference

ConferenceMedical Imaging 2024: Digital and Computational Pathology
Country/TerritoryUnited States
CitySan Diego
Period02/19/2402/21/24

Keywords

  • CODEX
  • Deep learning
  • pathology
  • semantic segmentation

Fingerprint

Dive into the research topics of 'Computational Integration of CODEX and Brightfield Histology for Cell Annotation using Deep Learning'. Together they form a unique fingerprint.

Cite this