Background While Alzheimer disease (AD) and vascular dementia (VaD) may be accelerated by hypercholesterolemia, the mechanisms underlying this association are unclear. We tested whether dysregulation of cholesterol catabolism, through its conversion to primary bile acids (BAs), was associated with dementia pathogenesis. Methods and findings We used a 3-step study design to examine the role of the primary BAs, cholic acid (CA), and chenodeoxycholic acid (CDCA) as well as their principal biosynthetic precursor, 7α-hydroxy-cholesterol (7α-OHC), in dementia. In Step 1, we tested whether serum markers of cholesterol catabolism were associated with brain amyloid accumulation, white matter lesions (WMLs), and brain atrophy. In Step 2, we tested whether exposure to bile acid sequestrants (BAS) was associated with risk of dementia. In Step 3, we examined plausible mechanisms underlying these findings by testing whether brain levels of primary BAs and gene expression of their principal receptors are altered in AD. • Step 1: We assayed serum concentrations CA, CDCA, and 7α-OHC and used linear regression and mixed effects models to test their associations with brain amyloid accumulation (N = 141), WMLs, and brain atrophy (N = 134) in the Baltimore Longitudinal Study of Aging (BLSA). The BLSA is an ongoing, community-based cohort study that began in 1958. Participants in the BLSA neuroimaging sample were approximately 46% male with a mean age of 76 years; longitudinal analyses included an average of 2.5 follow-up magnetic resonance imaging (MRI) visits. We used the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (N = 1,666) to validate longitudinal neuroimaging results in BLSA. ADNI is an ongoing, community-based cohort study that began in 2003. Participants were approximately 55% male with a mean age of 74 years; longitudinal analyses included an average of 5.2 follow-up MRI visits. Lower serum concentrations of 7α-OHC, CA, and CDCA were associated with higher brain amyloid deposition (p = 0.041), faster WML accumulation (p = 0.050), and faster brain atrophy mainly (false discovery rate [FDR] p = <0.001–0.013) in males in BLSA. In ADNI, we found a modest sex-specific effect indicating that lower serum concentrations of CA and CDCA were associated with faster brain atrophy (FDR p = 0.049) in males. • Step 2: In the Clinical Practice Research Datalink (CPRD) dataset, covering >4 million registrants from general practice clinics in the United Kingdom, we tested whether patients using BAS (BAS users; 3,208 with ≥2 prescriptions), which reduce circulating BAs and increase cholesterol catabolism, had altered dementia risk compared to those on non-statin lipid-modifying therapies (LMT users; 23,483 with ≥2 prescriptions). Patients in the study (BAS/ LMT) were approximately 34%/38% male and with a mean age of 65/68 years; follow-up time was 4.7/5.7 years. We found that BAS use was not significantly associated with risk of all-cause dementia (hazard ratio (HR) = 1.03, 95% confidence interval (CI) = 0.72–1.46, p = 0.88) or its subtypes. We found a significant difference between the risk of VaD in males compared to females (p = 0.040) and a significant dose–response relationship between BAS use and risk of VaD (p-trend = 0.045) in males. • Step 3: We assayed brain tissue concentrations of CA and CDCA comparing AD and control (CON) samples in the BLSA autopsy cohort (N = 29). Participants in the BLSA autopsy cohort (AD/CON) were approximately 50%/77% male with a mean age of 87/82 years. We analyzed single-cell RNA sequencing (scRNA-Seq) data to compare brain BA receptor gene expression between AD and CON samples from the Religious Orders Study and Memory and Aging Project (ROSMAP) cohort (N = 46). ROSMAP is an ongoing, community-based cohort study that began in 1994. Participants (AD/CON) were approximately 56%/36% male with a mean age of 85/85 years. In BLSA, we found that CA and CDCA were detectable in postmortem brain tissue samples and were marginally higher in AD samples compared to CON. In ROSMAP, we found sex-specific differences in altered neuronal gene expression of BA receptors in AD. Study limitations include the small sample sizes in the BLSA cohort and likely inaccuracies in the clinical diagnosis of dementia subtypes in primary care settings. Conclusions We combined targeted metabolomics in serum and amyloid positron emission tomography (PET) and MRI of the brain with pharmacoepidemiologic analysis to implicate dysregulation of cholesterol catabolism in dementia pathogenesis. We observed that lower serum BA concentration mainly in males is associated with neuroimaging markers of dementia, and pharmacological lowering of BA levels may be associated with higher risk of VaD in males. We hypothesize that dysregulation of BA signaling pathways in the brain may represent a plausible biologic mechanism underlying these results. Together, our observations suggest a novel mechanism relating abnormalities in cholesterol catabolism to risk of dementia.