TY - JOUR
T1 - Reproducibility of cognitive endpoints in clinical trials
T2 - lessons from neurofibromatosis type 1
AU - the NF Clinical Trials Consortium
AU - Payne, Jonathan M.
AU - Hearps, Stephen J.C.
AU - Walsh, Karin S.
AU - Paltin, Iris
AU - Barton, Belinda
AU - Ullrich, Nicole J.
AU - Haebich, Kristina M.
AU - Coghill, David
AU - Gioia, Gerard A.
AU - Cantor, Alan
AU - Cutter, Gary
AU - Tonsgard, James H.
AU - Viskochil, David
AU - Rey-Casserly, Celiane
AU - Schorry, Elizabeth K.
AU - Ackerson, Joseph D.
AU - Klesse, Laura
AU - Fisher, Michael J.
AU - Gutmann, David H.
AU - Rosser, Tena
AU - Packer, Roger J.
AU - Korf, Bruce
AU - Acosta, Maria T.
AU - North, Kathryn N.
N1 - Publisher Copyright:
© 2019 The Authors. Annals of Clinical and Translational Neurology published by Wiley Periodicals, Inc on behalf of American Neurological Association.
PY - 2019/12/1
Y1 - 2019/12/1
N2 - Objective: Rapid developments in understanding the molecular mechanisms underlying cognitive deficits in neurodevelopmental disorders have increased expectations for targeted, mechanism-based treatments. However, translation from preclinical models to human clinical trials has proven challenging. Poor reproducibility of cognitive endpoints may provide one explanation for this finding. We examined the suitability of cognitive outcomes for clinical trials in children with neurofibromatosis type 1 (NF1) by examining test-retest reliability of the measures and the application of data reduction techniques to improve reproducibility. Methods: Data were analyzed from the STARS clinical trial (n = 146), a multi-center double-blind placebo-controlled phase II trial of lovastatin, conducted by the NF Clinical Trials Consortium. Intra-class correlation coefficients were generated between pre- and post-performances (16-week interval) on neuropsychological endpoints in the placebo group to determine test-retest reliabilities. Confirmatory factor analysis was used to reduce data into cognitive domains and account for measurement error. Results: Test-retest reliabilities were highly variable, with most endpoints demonstrating unacceptably low reproducibility. Data reduction confirmed four distinct neuropsychological domains: executive functioning/attention, visuospatial ability, memory, and behavior. Test-retest reliabilities of latent factors improved to acceptable levels for clinical trials. Applicability and utility of our model was demonstrated by homogeneous effect sizes in the reanalyzed efficacy data. Interpretation: These data demonstrate that single observed endpoints are not appropriate to determine efficacy, partly accounting for the poor test-retest reliability of cognitive outcomes in clinical trials in neurodevelopmental disorders. Recommendations to improve reproducibility are outlined to guide future trial design.
AB - Objective: Rapid developments in understanding the molecular mechanisms underlying cognitive deficits in neurodevelopmental disorders have increased expectations for targeted, mechanism-based treatments. However, translation from preclinical models to human clinical trials has proven challenging. Poor reproducibility of cognitive endpoints may provide one explanation for this finding. We examined the suitability of cognitive outcomes for clinical trials in children with neurofibromatosis type 1 (NF1) by examining test-retest reliability of the measures and the application of data reduction techniques to improve reproducibility. Methods: Data were analyzed from the STARS clinical trial (n = 146), a multi-center double-blind placebo-controlled phase II trial of lovastatin, conducted by the NF Clinical Trials Consortium. Intra-class correlation coefficients were generated between pre- and post-performances (16-week interval) on neuropsychological endpoints in the placebo group to determine test-retest reliabilities. Confirmatory factor analysis was used to reduce data into cognitive domains and account for measurement error. Results: Test-retest reliabilities were highly variable, with most endpoints demonstrating unacceptably low reproducibility. Data reduction confirmed four distinct neuropsychological domains: executive functioning/attention, visuospatial ability, memory, and behavior. Test-retest reliabilities of latent factors improved to acceptable levels for clinical trials. Applicability and utility of our model was demonstrated by homogeneous effect sizes in the reanalyzed efficacy data. Interpretation: These data demonstrate that single observed endpoints are not appropriate to determine efficacy, partly accounting for the poor test-retest reliability of cognitive outcomes in clinical trials in neurodevelopmental disorders. Recommendations to improve reproducibility are outlined to guide future trial design.
UR - http://www.scopus.com/inward/record.url?scp=85076084771&partnerID=8YFLogxK
U2 - 10.1002/acn3.50952
DO - 10.1002/acn3.50952
M3 - Article
C2 - 31797581
AN - SCOPUS:85076084771
SN - 2328-9503
VL - 6
SP - 2555
EP - 2565
JO - Annals of Clinical and Translational Neurology
JF - Annals of Clinical and Translational Neurology
IS - 12
ER -