Assessing statistical applications in publications on Alzheimer's disease

Background/Aims: To evaluate statistical applications in publications on Alzheimer's disease (AD). Methods: Three instruments/checklists were developed: Assessment of Statistical Reporting (ASR; 44 items), Survey of Statistical Designs (SSD; 10 items), and Survey of Statistical Methods (SSM; 7 items). After a pilot testing on 5 AD publications, the instruments/checklists were revised and tested for reliability with a sample of 30 AD articles and for validity with another sample of 10 AD articles from MEDLINE. Results: Item-specific test-retest and interrater reliability for ASR ranged from 0.29 to 1.0 with the associated standard errors (SEs) ranging from 0.01 to 0.31. The test-retest reliability (intraclass correlation coefficient = 0.94, 95% CI: 0.88-0.97) and the interrater reliability (intraclass correlation coefficient = 0.84, 95% CI: 0.69-0.92) for the overall score of ASR were high. The correlational validity of the ASR with a published checklist was also high (r = 0.74, SE = 0.24). The item-specific test-retest reliability in SSD and SSM ranged from 0.58 to 1.00 with the associated SEs ranging from 0.01 to 0.32. The item-specific interrater reliability in SSD and SSM ranged from 0.17 to 1.00 with the associated SEs ranging from 0.01 to 0.22. Conclusions: This study suggested that it was feasible to assess statistical applications in AD publications.