Objectives: The objective of this study was to assess the impact of unresected primary tumor, as well as extrahepatic metastasis, on the long-term prognosis of patients undergoing hepatic resection for non-functional neuroendocrine liver metastasis (NF-NELM). Methods: Patients who underwent hepatic resection for NF-NELM were identified from a multi-institutional database. Data on clinical and pathological details, as well as the long-term overall survival (OS) were obtained and compared. Propensity score matching was performed to generate matched pairs of patients. Results: Among the 332 patients with NF-NELM, 281 (84.6%) underwent primary tumor resection, while 51 (15.4%) did not. Patients who underwent primary resection were more likely to have a pancreatic primary and metachronous NELM. The long-term OS of patients who did and did not have the primary neuroendocrine tumor (NET) resected was comparable on both unmatched (10-year survival rate 66.8% vs. 54.0%, p = 0.192) and matched (10-year survival rate 75.7% vs. 60.4%, p = 0.271) analyses. In contrast, patients with NF-NELM and extrahepatic metastasis had a worse OS following resection compared with patients who had intrahepatic-only metastasis on unmatched (10-year survival rate 37.5% vs. 69.3%, p = 0.002) and matched (10-year survival rate 37.5% vs. 86.3%, p = 0.011) analyses. On multivariable analysis, while resection of the primary NET was not associated with OS (hazard ratio [HR] 0.7, 95% confidence interval [CI] 0.4–1.2, p = 0.195), the presence of extrahepatic metastasis was independently associated with long-term risk of death (HR 3.9, 95% CI 1.7–9.2, p = 0.002). Conclusions: While surgery should be considered for patients with NF-NELM who have an unresectable primary tumor, operative resection of NF-NELM may not be as beneficial in patients with extrahepatic disease.