Purpose Racial/ethnic disparities exist in clinical clerkship grading, yet little is known about medical student and faculty perspectives on why these disparities occur. This study explored what happens during clerkships that might explain grading disparities. Method Medical students and clerkship teachers at 3 U.S. medical schools completed a demographic survey and semistructured interview. The constant comparative method was used to analyze transcripts by inductively developing codes; grouping codes in categories; and refining codes, descriptions, and group assignments to identify themes. Interpretations of and relationships among themes were iteratively discussed to develop a grounded theory. Results Fifty-nine participants (29 medical students, 30 teachers [28 clinical faculty, 2 residents]) were interviewed in 2020. The Social Milieu of Medical Education (relationships, fit, opportunities, and judgments in the clinical-learning setting) was the organizing theme, influenced by 5 additional themes: Societal Influence (experiences in society), Students' Characteristics and Background (personal characteristics and experiences outside medical school), Assessment Processes (collection of student performance data and how data inform grades), Learning Environment (resources available and messaging within the clinical setting), and Students' Interactions and Reactions (interactions with and reactions to peers and teachers). The grounded theory highlights complex, multilayered aspects of how the social milieu of medical education is shaped by and shapes students' experiences, relationships, and clerkship assessments and promotes clerkship-grading disparities. Conclusions Mitigating clerkship-grading disparities will require intervening on interrelated, contextual factors to provide equitable opportunities for students from diverse backgrounds and with varying styles of engagement in clinical-learning settings, along with attending to modifying assessment processes.