Systems composed of large ensembles of isolated or interacted dynamic units are prevalent in nature and engineered infrastructures. Linear ensemble systems are inarguably the simplest class of ensemble systems and have attracted intensive attention to control theorists and practionars in the past years. Comprehensive understanding of dynamic properties of such systems yet remains far-fetched and requires considerable knowledge and techniques beyond the reach of modern control theory. In this paper, we explore the classes of linear ensemble systems with system matrices that are not globally diagonalizable. In particular, we focus on analyzing their controllability properties under a Sobolev space setting and develop conditions under which uniform controllability of such ensemble systems is equivalent to that of their diagonalizable counterparts. This development significantly facilitates controllability analysis for linear ensemble systems through examining diagonalized linear systems.