Importance: In the absence of disease-modifying therapies for Alzheimer disease, there is a critical need to identify modifiable risk factors that may delay the progression of Alzheimer disease. Objective: To examine whether physical activity moderates the association of β-amyloid (Aβ) burden with longitudinal cognitive decline and neurodegeneration in clinically normal individuals and to examine whether these associations are independent of vascular risk. Design, Setting, and Participants: This longitudinal observational study included clinically normal participants from the Harvard Aging Brain Study. Participants were required to have baseline Aβ positron emission tomography data, baseline medical data to quantify vascular risk, and longitudinal neuropsychological and structural magnetic resonance imaging data. Data were collected from April 2010 to June 2018. Data were analyzed from August to December 2018. Main Outcomes and Measures: Baseline physical activity was quantified with a pedometer (mean steps per day). Baseline Aβ burden was measured with carbon 11-labeled Pittsburgh Compound B positron emission tomography. Cognition was measured annually with the Preclinical Alzheimer Cognitive Composite (PACC; median [interquartile range] follow-up, 6.0 [4.3-6.3] years). Neurodegeneration was assessed with longitudinal structural magnetic resonance imaging (2 to 5 scans per participant; median [interquartile range] follow-up, 4.5 [3.0-5.0] years), with a focus on total gray matter volume and regional cortical thickness. Physical activity and Aβ burden were examined as interactive predictors of PACC decline and volume loss in separate linear mixed models, adjusting for age, sex, education, apolipoprotein E ϵ4 status, and, where appropriate, intracranial volume. Secondary models adjusted for vascular risk and its interaction with Aβ burden. Results: Of the 182 included participants, 103 (56.6%) were female, and the mean (SD) age was 73.4 (6.2) years. In models examining PACC decline and volume loss, there was a significant interaction of physical activity with Aβ burden, such that greater physical activity was associated with slower Aβ-related cognitive decline (β, 0.03; 95% CI, 0.02-0.05; P <.001) and volume loss (β, 482.07; 95% CI, 189.40-774.74; P =.002). Adjusting for vascular risk did not alter these associations. In these models, lower vascular risk was independently associated with slower Aβ-related PACC decline (β, -0.04; 95% CI, -0.06 to -0.02; P <.001) and volume loss (β, -483.41; 95% CI, -855.63 to -111.20; P =.01). Conclusions and Relevance: Greater physical activity and lower vascular risk independently attenuated the negative association of Aβ burden with cognitive decline and neurodegeneration in asymptomatic individuals. These findings suggest that engaging in physical activity and lowering vascular risk may have additive protective effects on delaying the progression of Alzheimer disease.