TY - JOUR
T1 - So many questions, so little time
T2 - Integrating adaptive inventories into public opinion research
AU - Montgomery, Jacob M.
AU - Rossiter, Erin L.
N1 - Publisher Copyright:
© The Author(s) 2019. Published by Oxford University Press on behalf of the American Association for Public Opinion Research. All rights reserved.
PY - 2020/9/1
Y1 - 2020/9/1
N2 - One of the most difficult tasks facing survey researchers is balancing the imperative to keep surveys short with the need to measure important concepts accurately. Not only are long batteries prohibitively expensive but lengthy surveys can also lead to less informative answers from respondents. Yet, scholars often wish to measure traits that require a multi-item battery. To resolve these contradicting constraints, we propose the use of adaptive inventories. This approach uses computerized adaptive testing methods to minimize the number of questions each respondent must answer while maximizing the accuracy of the resulting measurement. We provide evidence supporting the utility of adaptive inventories through an empirically informed simulation study, an experimental study, and a detailed case study using data from the 2016 American National Election Study (ANES) Pilot. The simulation and experiment illustrate the superior performance of adaptive inventories relative to fixed-reduced batteries in terms of precision and accuracy. The ANES analysis serves as an illustration of how adaptive inventories can be developed and fielded and also validates an adaptive inventory with a nationally representative sample. Critically, we provide extensive software tools that allow researchers to incorporate adaptive inventories into their own surveys.
AB - One of the most difficult tasks facing survey researchers is balancing the imperative to keep surveys short with the need to measure important concepts accurately. Not only are long batteries prohibitively expensive but lengthy surveys can also lead to less informative answers from respondents. Yet, scholars often wish to measure traits that require a multi-item battery. To resolve these contradicting constraints, we propose the use of adaptive inventories. This approach uses computerized adaptive testing methods to minimize the number of questions each respondent must answer while maximizing the accuracy of the resulting measurement. We provide evidence supporting the utility of adaptive inventories through an empirically informed simulation study, an experimental study, and a detailed case study using data from the 2016 American National Election Study (ANES) Pilot. The simulation and experiment illustrate the superior performance of adaptive inventories relative to fixed-reduced batteries in terms of precision and accuracy. The ANES analysis serves as an illustration of how adaptive inventories can be developed and fielded and also validates an adaptive inventory with a nationally representative sample. Critically, we provide extensive software tools that allow researchers to incorporate adaptive inventories into their own surveys.
KW - Computerized adaptive testing
KW - Personality
KW - Survey methodology
UR - https://www.scopus.com/pages/publications/85097187702
U2 - 10.1093/jssam/smz027
DO - 10.1093/jssam/smz027
M3 - Article
AN - SCOPUS:85097187702
SN - 2325-0984
VL - 8
SP - 667
EP - 690
JO - Journal of Survey Statistics and Methodology
JF - Journal of Survey Statistics and Methodology
IS - 4
ER -