Purpose The use of stereotactic body radiation therapy (SBRT) for prostate cancer has been reported predominantly from single institutional studies, although concerns for broader adoption exist. Methods and materials From 2011 through 2013, 66 men were accrued to a phase 2 trial at 5 centers. SBRT consisted of 5 fractions of 7.4 Gy to a total dose of 37 Gy using conventional linear accelerators. Electromagnetic transponders were used for motion management. Health-related quality of life (HRQOL) was evaluated via the Expanded Prostate Cancer Index Composite 26 questionnaire. Acute and late toxicities were collected according to Common Terminology Criteria for Adverse Events, version 4.0. Linear mixed modeling was performed to assess changes in HRQOL over time. Results Median follow-up was 36 months. All men had low- or intermediate-risk disease. There have been 0 biochemical recurrences. No grade 3 urinary or bowel toxicity was reported. Twenty-three percent of patients had acute grade 2 urinary toxicity, with 9% late grade 2 urinary toxicity. Four percent and 5% experienced acute or late grade 2+ bowel toxicity, respectively. Urinary bother and bowel HRQOL transiently decreased during the first 6 to 12 months post-SBRT, and then returned to baseline. In men with good erectile function at baseline, sexual HRQOL declined during the first 6 months and stabilized thereafter. On linear mixed modeling, the strongest predictor of sustained bowel and sexual HRQOL was baseline HRQOL. Conclusions In this multi-institutional phase 2 clinical trial using continuous real-time evaluation of prostate motion, prostate SBRT has excellent intermediate-term tumor control with mild and expected treatment-related side effects.