RfaH is required for virulence in several Gram-negative pathogens including Escherichia coli and Klebsiella pneumoniae. Through direct interactions with RNA polymerase (RNAP) and ribosome, RfaH activates the expression of capsule, cell wall and pilus biosynthesis operons by reducing transcription termination and activating translation. While E. coli RfaH has been extensively studied using structural and biochemical approaches, limited data are available for other RfaH homologs. Here we set out to identify small molecule inhibitors of E. coli and K. pneumoniae RfaHs. Results of biochemical and functional assays show that these proteins act similarly, with a notable difference between their interactions with the RNAP β subunit gate loop. We focused on high-affinity RfaH interactions with the RNAP β’ subunit clamp helices as a shared target for inhibition. Among the top 10 leads identified by in silico docking using ZINC database, 3 ligands were able to inhibit E. coli RfaH recruitment in vitro. The most potent lead was active against both E. coli and K. pneumoniae RfaHs in vitro. Our results demonstrate the feasibility of identifying RfaH inhibitors using in silico docking and pave the way for rational design of antivirulence therapeutics against antibiotic-resistant pathogens.