Purpose: To evaluate the accuracy of a real‐time automated method of performing dosimetric quality assurance using Eclipse DICOM files for patients receiving HDR‐brachytherapy and IMRT. Methods: GYN patients are treated with concurrent high‐dose rate brachtherapy and IMRT. The dosimetric parameters were obtained through an in‐house QA program developed using Matlab. The DICOM files containing DVH data for organsat‐risk (OAR) were analyzed Dosimetric data for 7 patients (total 42 fractions) were collected for bladder, rectum and sigmoid. The accuracy of the dosimetric parameters was estimated by comparing the parameters obtained from the DICOM based QA program and those in BrachyVision. Results: The maximal dose values (Dmax) for the OARs obtained using the DICOM‐based program are significantly smaller than those valued reported in BrachyVision by 36.2%–48.3%. The mean dose has a deviation from 1% – 2.4%. The dose for the volume of 2cc (D2cc) has a difference up to 7.6% for structures with the volume larger than 200 cc. The average difference of D2cc is 0.5% for structures less than 200 cc. We found that Eclipse BrachyVision only exports DVH data down to a volume equivalent to 1% of the maximum volume for a given structure. Therefore, the reported maximal dose values obtained from DICOM RT dose file do not accurately reflect the maximum dose in a treatment plan. This will also slightly affect the mean dose calculation and D2cc when the structure volume is larger than 200cc. Conclusions: The automatic QA tool based on DICOM files provides a quick retrieval of dose to organs‐at‐risk and coverage of targets. However, maximal dose to structures is not accurate due to the truncationof the DVH information contained in DICOM files.