Purpose: To develop a validation framework for markerless tumor trajectory estimation algorithms and use this framework to optimize markerless tumor trajectory estimation in cone beam CT (CBCT) projections. Methods: Fiducial markers implanted in and near the tumor in six lung cancer patients were segmented in CBCT projections and the 3D marker trajectories were reconstructed. A correction based on the tumor position in the reconstructed CBCT, the tumor‐to‐marker displacement (T‐M) was added to the marker trajectory to generate the reference tumor trajectory. Four combinations of T‐M based on two marker segmentation methods and two tumor position measurement methods were evaluated. Since T‐M is expected to be fairly stable from day to day, optimal T‐M was evaluated based on day‐to‐day variability. For markerless tracking, three means of generating templates for 2D registration (a cylindrical template, expanded gross tumor volume (Ex‐GTV), and Ex‐GTV with padding to override non‐target edges (padded template)) were compared using the validation framework. A constraint method was also implemented to regularize the 2D registration. Results: The optimal T‐M was generated by planning CT to CBCT registration of the target volume along with marker segmentation from the projections. The mean (standard deviation) absolute registration error in five patients using the three different template methods were 5.3 (6.5) mm for the Ex‐GTV template, 6.1 (6.4) mm for the cylinder template, and 3.5 (5.0) for the padded template. The 90% error level was 12.4, 13.3, and 7.9 mm for the three methods, respectively. The mean registration error was reduced further to 2.1 (2.5) mm (90% error of 4.3 mm) with the constraint method. Conclusion: A validation framework for markerless trajectory measurement was optimized. The padded template method with constraints agreed best with the implanted marker trajectories. E Weiss receives research support from Varian Medical Systems.