TY - JOUR
T1 - To BYOD or not
T2 - Are device latencies important for bring-your-own-device (BYOD) smartphone cognitive testing?
AU - Nicosia, Jessica
AU - Wang, Benjamin
AU - Aschenbrenner, Andrew J.
AU - Sliwinski, Martin J.
AU - Yabiku, Scott T.
AU - Roque, Nelson A.
AU - Germine, Laura T.
AU - Bateman, Randall J.
AU - Morris, John
AU - Hassenstab, Jason
N1 - Publisher Copyright:
© 2022, The Psychonomic Society, Inc.
PY - 2023/9
Y1 - 2023/9
N2 - Studies using remote cognitive testing must make a critical decision: whether to allow participants to use their own devices or to provide participants with a study-specific device. Bring-your-own-device (BYOD) studies have several advantages including increased accessibility, potential for larger sample sizes, and reduced participant burden. However, BYOD studies offer little control over device performance characteristics that could potentially influence results. In particular, response times measured by each device not only include the participant’s true response time, but also latencies of the device itself. The present study investigated two prominent sources of device latencies that pose significant risks to data quality: device display output latency and touchscreen input latency. We comprehensively tested 26 popular smartphones ranging in price from < $100 to $1000+ running either Android or iOS to determine if hardware and operating system differences led to appreciable device latency variability. To accomplish this, a custom-built device called the Latency and Timing Assessment Robot (LaTARbot) measured device display output and capacitive touchscreen input latencies. We found considerable variability across smartphones in display and touch latencies which, if unaccounted for, could be misattributed as individual or group differences in response times. Specifically, total device (sum of display and touch) latencies ranged from 35 to 140 ms. We offer recommendations to researchers to increase the precision of data collection and analysis in the context of remote BYOD studies.
AB - Studies using remote cognitive testing must make a critical decision: whether to allow participants to use their own devices or to provide participants with a study-specific device. Bring-your-own-device (BYOD) studies have several advantages including increased accessibility, potential for larger sample sizes, and reduced participant burden. However, BYOD studies offer little control over device performance characteristics that could potentially influence results. In particular, response times measured by each device not only include the participant’s true response time, but also latencies of the device itself. The present study investigated two prominent sources of device latencies that pose significant risks to data quality: device display output latency and touchscreen input latency. We comprehensively tested 26 popular smartphones ranging in price from < $100 to $1000+ running either Android or iOS to determine if hardware and operating system differences led to appreciable device latency variability. To accomplish this, a custom-built device called the Latency and Timing Assessment Robot (LaTARbot) measured device display output and capacitive touchscreen input latencies. We found considerable variability across smartphones in display and touch latencies which, if unaccounted for, could be misattributed as individual or group differences in response times. Specifically, total device (sum of display and touch) latencies ranged from 35 to 140 ms. We offer recommendations to researchers to increase the precision of data collection and analysis in the context of remote BYOD studies.
KW - Ambulatory assessment
KW - BYOD
KW - Remote assessment
KW - Smartphones
UR - http://www.scopus.com/inward/record.url?scp=85135780950&partnerID=8YFLogxK
U2 - 10.3758/s13428-022-01925-1
DO - 10.3758/s13428-022-01925-1
M3 - Article
C2 - 35953659
AN - SCOPUS:85135780950
SN - 1554-351X
VL - 55
SP - 2800
EP - 2812
JO - Behavior Research Methods
JF - Behavior Research Methods
IS - 6
ER -