No-gold-standard evaluation of quantitative imaging methods in the presence of correlated noise

Ziping Liu, Zekun Li, Joyce C. Mhlanga, Barry A. Siegel, Abhinav K. Jha

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations


Objective evaluation of quantitative imaging (QI) methods with patient data is highly desirable, but is hindered by the lack or unreliability of an available gold standard. To address this issue, techniques that can evaluate QI methods without access to a gold standard are being actively developed. These techniques assume that the true and measured values are linearly related by a slope, bias, and Gaussian-distributed noise term, where the noise between measurements made by different methods is independent of each other. However, this noise arises in the process of measuring the same quantitative value, and thus can be correlated. To address this limitation, we propose a no-gold-standard evaluation (NGSE) technique that models this correlated noise by a multi-variate Gaussian distribution parameterized by a covariance matrix. We derive a maximum-likelihood-based approach to estimate the parameters that describe the relationship between the true and measured values, without any knowledge of the true values. We then use the estimated slopes and diagonal elements of the covariance matrix to compute the noise-To-slope ratio (NSR) to rank the QI methods on the basis of precision. The proposed NGSE technique was evaluated with multiple numerical experiments. Our results showed that the technique reliably estimated the NSR values and yielded accurate rankings of the considered methods for 83% of 160 trials. In particular, the technique correctly identified the most precise method for 97% of the trials. Overall, this study demonstrates the efficacy of the NGSE technique to accurately rank different QI methods when the correlated noise is present, and without access to any knowledge of the ground truth. The results motivate further validation of this technique with realistic simulation studies and patient data.

Original languageEnglish
Title of host publicationMedical Imaging 2022
Subtitle of host publicationImage Perception, Observer Performance, and Technology Assessment
EditorsClaudia R. Mello-Thoms, Claudia R. Mello-Thoms, Sian Taylor-Phillips
ISBN (Electronic)9781510649453
StatePublished - 2022
EventMedical Imaging 2022: Image Perception, Observer Performance, and Technology Assessment - Virtual, Online
Duration: Mar 21 2022Mar 27 2022

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
ISSN (Print)1605-7422


ConferenceMedical Imaging 2022: Image Perception, Observer Performance, and Technology Assessment
CityVirtual, Online


  • medical imaging
  • no-gold-standard
  • objective evaluation
  • quantitative imaging


Dive into the research topics of 'No-gold-standard evaluation of quantitative imaging methods in the presence of correlated noise'. Together they form a unique fingerprint.

Cite this