Antidepressants have previously been associated with better outcomes in patients hospitalized with COVID-19, but their effect on clinical deterioration among ambulatory patients has not been fully explored. The objective of this study was to assess whether antidepressant exposure was associated with reduced emergency department (ED) or hospital visits among ambulatory patients with SARS-CoV-2 infection. This retrospective cohort study included adult patients (N = 25 034) with a positive SARS-CoV-2 test performed in a non-hospital setting. Logistic regression analyses tested associations between home use of antidepressant medications and a composite outcome of ED visitation or hospital admission within 30 days. Secondary exposures included individual antidepressants and antidepressants with functional inhibition of acid sphingomyelinase (FIASMA) activity. Patients with antidepressant exposure were less likely to experience the primary composite outcome compared to patients without antidepressant exposure (adjusted odds ratio [aOR] 0.89, 95% CI 0.79–0.99, p = 0.04). This association was only observed with daily doses of at least 20 mg fluoxetine-equivalent (aOR 0.87, 95% CI 0.77–0.99, p = 0.04), but not with daily doses lower than 20 mg fluoxetine-equivalent (aOR 0.94, 95% CI 0.80–1.11, p = 0.48). In exploratory secondary analyses, the outcome incidence was also reduced with exposure to selective serotonin reuptake inhibitors (aOR 0.87, 95% CI 0.75–0.99, p = 0.04), bupropion (aOR 0.70, 95% CI 0.55–0.90, p = 0.005), and FIASMA antidepressant drugs (aOR 0.87, 95% CI 0.77–0.99, p = 0.03). Antidepressant exposure was associated with a reduced incidence of emergency department visitation or hospital admission among SARS-CoV-2 positive patients, in a dose-dependent manner. These data support the FIASMA model of antidepressants’ effects against COVID-19.