Purpose: To introduce a new deformable image registration algorithm based on surface matching that accommodates organ delineation error in daily Cone‐beam CT images based on a priori knowledge of inter‐observer segmentation uncertainty. Methods: The dataset includes four prostate cancer patients who underwent primary external beam radiotherapy and had tumors that were confined to the prostate. All imaging was performed without intravenous contrast. Organ surface segmentation errors in a multiple observer‐contouring study on the pelvic organs in Fan‐beam CT (FBCT) and Cone‐beam CT (CBCT) were estimated from the training dataset. A novel deformable image registration algorithm is presented where the organ surface matching is penalized by this error. Portions of the organ surface that are delineated reliably are used to guide the registration whereas the portions that are highly uncertain are ignored. This approach reduces the impact of delineation errors in CBCT. An evaluation experiment compares three algorithms, namely intensity‐only registration (INT), equally‐weighted surface and image registration (EWSIR) and the proposed uncertainty‐ weighted surface and image registration. Results: The surface dissimilarity was reduced from 0.172 to 0.134, 0.043 and 0.044 respectively after registration. The Jacobian of the transformation found by the proposed method was closer to one than that of EWSIR in the prostate.Conclusions: In prostate external‐beam radiotherapy, slice‐by‐slice 2D manual contouring has variable spatial accuracy. For deformable image registration methods that match segmented surfaces, regions of high inaccuracy can misguide the registration. In contrast to the image registration methods where the FBCT and CBCT surfaces (or other features) are assumed to be exact, our method takes this uncertainty into account. Preliminary results show an improved registration performance suggesting a potential use in IGRT.