Objective: To estimate the contribution of county-level contextual factors to differences in life expectancy in the United States. Methods: We used a counterfactual approach to estimate the years of life expectancy lost associated with 45 potentially modifiable county-level contextual characteristics in the United States in the year 2016. Contextual data and life expectancy data were obtained from the County Health Ranking Project and the U.S. Small-Area Life Expectancy Estimates Project, respectively. Results: Median census-tract–level life expectancy was 78.90 (interquartile range, 76.30-81.00) years, and the range across census tracts spanned 41.20 years. Large variations in life expectancy existed within and between states and within and between counties; the gap between counties was 20.30 years and gaps within counties ranged from 0 to 34.60 years. An array of 45 county-level factors was associated with 4.30 years of life expectancy loss. County-level adult smoking, food insecurity, adult obesity, physical inactivity, college education, and median household income were associated with 1.24-, 0.89-, 0.58-, 0.35-, 0.33-, and 0.14-year losses in life expectancy, respectively; and altogether were associated with a 3.53-year loss in life expectancy. The contribution of contextual factors to years of life expectancy lost varied among states and was more pronounced in states with lower life expectancy and in areas of increased socioeconomic deprivation and increased percentage of Black race. Conclusion: Substantial geographic variation in life expectancy was observed. Six county-level contextual factors were associated with a 3.53-year loss in life expectancy. The findings may inform and help prioritize approaches to reduce inequalities in life expectancy in the United States.