Although the current obesity epidemic is of environmental origin, there is substantial genetic variation in individual response to an obesogenic environment. In this study, we perform a genome-wide scan for quantitative trait loci (QTLs) affecting obesity per se, or an obese response to a high-fat diet in mice from the LG/J by SM/J Advanced Intercross (AI) Line (Wustl:LG,SM-G16). A total of 1,002 animals from 78F16 full sibships were weaned at 3 weeks of age and half of each litter placed on high- and low-fat diets. Animals remained on the diet until 20 weeks of age when they were necropsied and the weights of the reproductive, kidney, mesenteric, and inguinal fat depots were recorded. Effects on these phenotypes, along with total fat depot weight and carcass weight at necropsy, were mapped across the genome using 1,402 autosomal single-nucleotide polymorphism (SNP) markers. Haplotypes were reconstructed and additive, dominance, and imprinting genotype scores were derived every 1cM along the F16 map. Analysis was performed using a mixed model with additive, dominance, and imprinting genotype scores, their interactions with sex, diet, and with sex-by-diet as fixed effects and with family and its interaction with sex, diet, and sex-by-diet as random effects. We discovered 95 trait-specific QTLs mapping to 40 locations. Most QTLs had additive effects with dominance and imprinting effects occurring at two-thirds of the loci. Nearly every locus interacted with sex and/or diet in important ways demonstrating that gene effects are primarily context dependent, changing depending on sex and/or diet.