We have developed a new type of PET insert system that, when integrated into a whole-body PET scanner, offers the possibility of imaging the breast and chest wall region with higher resolution and sensitivity than a whole-body PET scanner alone. The focus of this paper is to describe a system model that can be used within a fully 3D iterative reconstruction algorithm to accurately model the measured data. We have modeled the most important components of the system matrix, includingthe spatially-variant geometric factor, inter-crystal penetration, and body attenuation. Due to the irregular sampling of the insert system, we compute the system matrix using the native detector geometry. Based on subdividing each detector volume into subvolumes, we show that this approach can model DOl crystals and non-DOl crystals under a common framework. We present some of our first results using data acquired by the physical PET insert system. More detailed modeling or normalization strategies will need to be employed to obtain the most benefit from the reconstruction algorithm.