CUL4B ubiquitin ligase belongs to the cullin-RING ubiquitin ligase family. Although sharing many sequence and structural similarities,CUL4Bplays distinct roles in spermatogenesis from its homologous protein CUL4A. We previously reported that genetic ablation of Cul4a in mice led to male infertility because of aberrant meiotic progression. In the present study, we generated Cul4b germ cell-specific conditional knock-out (Cul4bVasa), as well as Cul4b global knock-out (Cul4bSox2) mouse, to investigate its roles in spermatogenesis. Germ cell-specific deletion of Cul4b led to male infertility, despite normal testicular morphology and comparable numbers of spermatozoa. Notably, significantly impaired sperm mobility caused by reduced mitochondrial activity and glycolysis level were observed in the majority of the mutant spermatozoa, manifested by low, if any, sperm ATP production. Furthermore, Cul4bVasa spermatozoa exhibited defective arrangement of axonemal microtubules and flagella outer dense fibers. Our mass spectrometry analysis identified INSL6 as a novel CUL4B substrate in male germ cells, evidenced by its direct polyubiquination and degradation by CUL4B E3 ligase. Nevertheless, Cul4b global knock-out males lost their germ cells in an age-dependent manner, implying failure of maintaining the spermatogonial stem cell niche in somatic cells. Taken together, our results show that CUL4B is indispensable to spermatogenesis, and it functions cell autonomously in male germ cells to ensure spermatozoa motility, whereas it functions non-cell-autonomously in somatic cells to maintain spermatogonial stemness. Thus, CUL4B links two distinct spermatogenetic processes to a single E3 ligase, highlighting the significance of ubiquitin modification during spermatogenesis.