From Schubert's the judicial mind to Spaeth's U.S. Supreme Court Judicial Data Base: A crossvalidation

Since 1990, when Spaeth made public his U.S. Supreme Court Judicial Data Base, scholars of courts and law have possessed a reproducible and reliable data set from which to conduct their analyses. Such was not always the case: many of the field's foundational studies relied on information that would not pass muster under current standards governing data collection. We crossvalidate a model from one of these studies (Epstein, Walker, and Dixon 1989), which relied heavily on data collected by Schubert (1976), with data derived from the Spaeth Data Base. The crossvalidation was only a partial success, with a key variable (prior behavior) failing to obtain statistical significance. While this finding may carry important implications for scholarship on Supreme Court decision making, the more general lesson of our effort is this: Simply because judicial specialists (or those in other fields for that matter) now have outstanding public data bases, it does not follow that they can ignore issues of measurement, reproducibility, reliability, and verification. Too many of the seminal studies and important constructs evolved from data bases that were something short of outstanding. This suggests the need for more crossvalidations of older work against data gathered in accord with contemporary standards.