On model selection criteria for climate change impact studies

Climate change impact studies inform policymakers on the estimated damages of future climate change on economic, health and other outcomes. In most studies, an annual outcome variable is observed, e.g. agricultural yield, along with a higher-frequency regressor, e.g. daily temperature. Applied researchers then face a problem of selecting a model to characterize the nonlinear relationship between the outcome and the high-frequency regressor to make a policy recommendation based on the model-implied damage function. We show that existing model selection criteria are only suitable for the policy objective if one of the models under consideration nests the true model. If all models are seen as imperfect approximations of the true nonlinear relationship, the model that performs well in the historical climate conditions is not guaranteed to perform well at the projected climate. We therefore propose a new criterion, the proximity-weighted mean squared error (PWMSE) that directly targets precision of the damage function at the projected future climate. To make this criterion feasible, we assign higher weights to historical years that can serve as “weather analogs” to the projected future climate when evaluating competing models using the PWMSE. We show that our approach selects the best approximate regression model that has the smallest weighted squared error of predicted impacts for a projected future climate. A simulation study and an application revisiting the impact of climate change on agricultural production illustrate the empirical relevance of our theoretical analysis.