Leveraging Loop Polarity to Reduce Underspecification in Deep Learning

Deep learning provides a set of techniques for detecting complex patterns in data and is a critical component of the burgeoning artificial intelligence revolution, enabling transformative advancement in a variety of fields. However, when the causal structure of the data-generating process is underspecified, deep learning models can be brittle, lacking robustness to shifts in data-generating distributions. In this paper, we demonstrate that methods and concepts familiar to system dynamics modelers can be used to address this problem of brittleness, thereby improving the efficacy of deep learning systems. Specifically, we turn to loop polarity analysis as a tool for specifying the causal structure of a data-generating process, in order to encode a more robust understanding of the relationship between system structure and system behavior within the deep learning pipeline. We use simulated epidemic data based on an SIR model to demonstrate how measuring the polarity of the accumulations of the different feedback loops that compose a system can lead to more robust inferences on the part of neural networks, improving out-of-distribution performance and infusing a system-dynamics-inspired approach into the deep learning pipeline. This case study provides one example of how to leverage an understanding of the causal structure of a data-generating process to extract low-dimensional summary statistics that in turn allow us to build more robust deep learning pipelines. Code for this paper is available at https://github.com/davidbkinney/loop_polarity_underspecification.