Measuring political preferences

Theory: When analysts adopt surrogates of actors' political preferences for purposes unanticipated by the inventors of those measures, they often stretch (but not explicitly assess) the range of reliability and validity. Hypotheses: The consequences pushing measures beyond their intended purposes may significantly impact research findings, as well as the conclusions drawn from those findings. Methods: "Methodological audit" of measures developed by Segal and Cover (1989) to represent the political preferences of justices on the United States Supreme Court. Mainly regression analysis using the Segal/Cover scores and vote data drawn from the United States Supreme Court Judicial Database. Results: Analysts would be well advised to weigh carefully whether adequate tests have been performed before adopting others' preference measures for their own research. More specific conclusions are: 1) scholars should invoke the Segal/Cover scores in the set of circumstances indicated by their developers: aggregated individual-level decisions in civil liberties cases; and 2) students of the judicial process who seek to explore phenomena other than aggregated individual-level voting in civil liberties cases ought to give serious thought to devising new surrogates for judicial preferences.