Objective The purpose of this analysis was to examine the correlates of the physical and psychosocial domains of quality of life (QOL) in a cohort of breast cancer survivors participating in a weight loss intervention trial. Methods Correlates of QOL and psychosocial functioning were examined in 692 overweight or obese breast cancer survivors at entry into a weight loss trial. QOL was explored with three measures: Short-form 36 (SF-36), Impact of Cancer scale (IOC), and the Breast Cancer Prevention Trial (BCPT) symptom scales. Available data included information on weight and physical activity, as well as demographic and medical characteristics. Multivariate analyses were used to identify associations adjusted for other characteristics. Results In multivariate analysis, younger age was associated with higher negative impact scores (p < 0.0001). Hispanic, African-American, and Asian women had higher positive IOC impact scores compared with White non-Hispanic women (p < 0.01). Increased number of comorbidities was associated with lower physical and mental QOL scores (p < 0.01). Body mass index was not independently associated with QOL measures. Physical activity was directly associated with physical and mental QOL and IOC positive impact, and inversely related to IOC negative impact and Breast Cancer Prevention Trial symptom scales. Conclusions Quality-of-life measures in breast cancer survivors are differentially associated with demographic and other characteristics. When adjusted for these characteristics, degree of adiposity among overweight or obese women does not appear to be independently associated with QOL. Among overweight or obese breast cancer survivors, higher level of physical activity is associated with higher QOL across various scales and dimensions.