Purpose: Traditional IMRT quality assurance (QA) methods are incapable of providing complete information about IMRT delivery. Most methods consist of a planar film or diode array measurement for relative dosimetry and 1 ‐2 ionization chamber (IC) measurements made in a homogeneous medium for absolute dosimetry. A new model‐ and measurement‐based IMRT QA tool makes 3D delivered‐dose verification possible through independent dose calculation and 2D fluence measurements, which combine to provide reconstructed dose on a 3D heterogeneous patient dataset. This work aims to commission this new IMRT QA tool and validate its performance relative to traditional QA methods. Methods: The COMPASS system (IBA Dosimetry) consists of a software package for dose calculation and visualization/analysis of measured data acquired using a gantry‐mounted MatriXX Evolution detector array (1020 ICs) and gantry angle sensor. Commissioning included modeling a Varian Truebeam 6 MV photon beam. The system computation algorithm was validated via direct comparison of the COMPASS and Pinnacle‐TPS calculated dose distributions for twelve previously‐treated patient plans (sites: lung, breast, pancreas, head and neck, prostate; delivery types: SMLC‐DMPO, SmartArc). These same plans were measured at the actual treatment beam angles using both the gantry‐mounted MatriXX detector and two ICs placed in a standard cuboidal IMRT solid‐ water phantom. The COMPASS reconstructed dose was validated by comparison of the average PTV dose to the average IC dose measurement, each relative to the TPS expected dose. Results: The COMPASS calculated dose differed from the TPS calculated dose by 0.71+\−0.37%. On average, the measured dose differed from the TPS expected dose by 2.04+\−0.93% for COMPASS and 1.45+\−0.94% for the ICs. Conclusion: The COMPASS computation algorithm is valid and the measured results are comparable with those of the traditional IMRT QA method, while providing more dosimetry information (absolute and relative) in a single measurement and taking patient tissue heterogeneity into account.