TY - GEN
T1 - Computational Integration of CODEX and Brightfield Histology for Cell Annotation using Deep Learning
AU - Lucarelli, Nicholas
AU - Laszik, Zoltan
AU - Winfree, Seth
AU - El-Achkar, Tarek M.
AU - Eadon, Michael
AU - Jain, Sanjay
AU - Sarder, Pinaki
N1 - Publisher Copyright:
© 2024 SPIE.
PY - 2024
Y1 - 2024
N2 - Cell types present in a biopsy provide information on disease processes and organ health, and are useful in a research setting. Multiplex imaging technologies like CODEX can provide spatial context for protein expression and detect cell types on a whole slide basis. The CODEX workflow also allows for hematoxylin and eosin (H&E) staining on the same sections used in molecular imaging. Deep learning can automate the process of histological analysis, reducing time and effort required. We seek to automatically segment and classify cells from histologically stained renal tissue sections using deep learning, with CODEX generated cell labels as a ground truth. Image data consisted of brightfield H&E whole slide images (WSIs) from a single institution, collected from human reference kidneys. Nuclei were segmented using deep learning, and CODEX markers were measured for each nucleus. Cells and their markers were clustered in an unsupervised manner, and assigned labels according to upregulated markers and spatial biological priors. Classified cell types included: proximal tubules, distal tubules, vessels, interstitial cells, and general glomerular cells. Cell maps were used to train a Deeplab V3+ semantic segmentation network. Cell maps were successfully created in all sections, with ~65% used for training and ~35% used for testing. The trained network achieved a balanced accuracy of 0.75 across all cell types. We were able to automatically segment and classify nuclei from various cell types directly from H&E stained WSIs. In future work, we intend to expand the dataset to include more CODEX markers (and therefore more granular cell types), and more samples with more variability, to test the robustness of the model to new data.
AB - Cell types present in a biopsy provide information on disease processes and organ health, and are useful in a research setting. Multiplex imaging technologies like CODEX can provide spatial context for protein expression and detect cell types on a whole slide basis. The CODEX workflow also allows for hematoxylin and eosin (H&E) staining on the same sections used in molecular imaging. Deep learning can automate the process of histological analysis, reducing time and effort required. We seek to automatically segment and classify cells from histologically stained renal tissue sections using deep learning, with CODEX generated cell labels as a ground truth. Image data consisted of brightfield H&E whole slide images (WSIs) from a single institution, collected from human reference kidneys. Nuclei were segmented using deep learning, and CODEX markers were measured for each nucleus. Cells and their markers were clustered in an unsupervised manner, and assigned labels according to upregulated markers and spatial biological priors. Classified cell types included: proximal tubules, distal tubules, vessels, interstitial cells, and general glomerular cells. Cell maps were used to train a Deeplab V3+ semantic segmentation network. Cell maps were successfully created in all sections, with ~65% used for training and ~35% used for testing. The trained network achieved a balanced accuracy of 0.75 across all cell types. We were able to automatically segment and classify nuclei from various cell types directly from H&E stained WSIs. In future work, we intend to expand the dataset to include more CODEX markers (and therefore more granular cell types), and more samples with more variability, to test the robustness of the model to new data.
KW - CODEX
KW - Deep learning
KW - pathology
KW - semantic segmentation
UR - http://www.scopus.com/inward/record.url?scp=85191352857&partnerID=8YFLogxK
U2 - 10.1117/12.3008457
DO - 10.1117/12.3008457
M3 - Conference contribution
AN - SCOPUS:85191352857
T3 - Progress in Biomedical Optics and Imaging - Proceedings of SPIE
BT - Medical Imaging 2024
A2 - Tomaszewski, John E.
A2 - Ward, Aaron D.
PB - SPIE
T2 - Medical Imaging 2024: Digital and Computational Pathology
Y2 - 19 February 2024 through 21 February 2024
ER -