Importance: Exposure to early-life adversity alters the structural development of key brain regions underlying neurodevelopmental impairments. The association between prenatal exposure to adversity and brain structure at birth remains poorly understood. Objective: To examine whether prenatal exposure to maternal social disadvantage and psychosocial stress is associated with neonatal global and regional brain volumes and cortical folding. Design, Setting, and Participants: This prospective, longitudinal cohort study included 399 mother-infant dyads of sociodemographically diverse mothers recruited in the first or early second trimester of pregnancy and their infants, who underwent brain magnetic resonance imaging in the first weeks of life. Mothers were recruited from local obstetric clinics in St Louis, Missouri from September 1, 2017, to February 28, 2020. Exposures: Maternal social disadvantage and psychosocial stress in pregnancy. Main Outcomes and Measures: Confirmatory factor analyses were used to create latent constructs of maternal social disadvantage (income-to-needs ratio, Area Deprivation Index, Healthy Eating Index, educational level, and insurance status) and psychosocial stress (Perceived Stress Scale, Edinburgh Postnatal Depression Scale, Everyday Discrimination Scale, and Stress and Adversity Inventory). Neonatal cortical and subcortical gray matter, white matter, cerebellum, hippocampus, and amygdala volumes were generated using semiautomated, age-specific, segmentation pipelines. Results: A total of 280 mothers (mean [SD] age, 29.1 [5.3] years; 170 [60.7%] Black or African American, 100 [35.7%] White, and 10 [3.6%] other race or ethnicity) and their healthy, term-born infants (149 [53.2%] male; mean [SD] infant gestational age, 38.6 [1.0] weeks) were included in the analysis. After covariate adjustment and multiple comparisons correction, greater social disadvantage was associated with reduced cortical gray matter (unstandardized β = -2.0; 95% CI, -3.5 to -0.5; P =.01), subcortical gray matter (unstandardized β = -0.4; 95% CI, -0.7 to -0.2; P =.003), and white matter (unstandardized β = -5.5; 95% CI, -7.8 to -3.3; P <.001) volumes and cortical folding (unstandardized β = -0.03; 95% CI, -0.04 to -0.01; P <.001). Psychosocial stress showed no association with brain metrics. Although social disadvantage accounted for an additional 2.3% of the variance of the left hippocampus (unstandardized β = -0.03; 95% CI, -0.05 to -0.01), 2.3% of the right hippocampus (unstandardized β = -0.03; 95% CI, -0.05 to -0.01), 3.1% of the left amygdala (unstandardized β = -0.02; 95% CI, -0.03 to -0.01), and 2.9% of the right amygdala (unstandardized β = -0.02; 95% CI, -0.03 to -0.01), no regional effects were found after accounting for total brain volume. Conclusions and Relevance: In this baseline assessment of an ongoing cohort study, prenatal social disadvantage was associated with global reductions in brain volumes and cortical folding at birth. No regional specificity for the hippocampus or amygdala was detected. Results highlight that associations between poverty and brain development begin in utero and are evident early in life. These findings emphasize that preventive interventions that support fetal brain development should address parental socioeconomic hardships..