Viral DNA genomes replicating in cells encounter a myriad of host factors that facilitate or hinder viral replication. Viral proteins expressed early during infection modulate host factors interacting with viral genomes, recruiting proteins to promote viral replication, and limiting access to antiviral repressors. Although some host factors manipulated by viruses have been identified, we have limited knowledge of pathways exploited during infection and how these differ between viruses. To identify cellular processes manipulated during viral replication, we defined proteomes associated with viral genomes during infection with adenovirus, herpes simplex virus and vaccinia virus. We compared enrichment of host factors between virus proteomes and confirmed association with viral genomes and replication compartments. Using adenovirus as an illustrative example, we uncovered host factors deactivated by early viral proteins, and identified a subgroup of nucleolar proteins that aid virus replication. Our data sets provide valuable resources of virus-host interactions that affect proteins on viral genomes.