TY - JOUR
T1 - Reinforcement biases subsequent perceptual decisions when confidence is low
T2 - A widespread behavioral phenomenon
AU - Lak, Armin
AU - Hueske, Emily
AU - Hirokawa, Junya
AU - Masset, Paul
AU - Ott, Torben
AU - Urai, Anne E.
AU - Donner, Tobias H.
AU - Carandini, Matteo
AU - Tonegawa, Susumu
AU - Uchida, Naoshige
AU - Kepecs, Adam
N1 - Funding Information:
This work was supported by the Wellcome Trust (grants 106101 and 213465 to A. L). N.U. was funded by NIH R01 MH110404 and Mind Brain Behavior (MBB) Faculty Award. A.K. was funded by NIH R01MH097061 and R01DA038209. M.C. was funded by the Wellcome Trust (205093). A.E.U. was funded by a German Academic Exchange Service (DAAD) doctoral scholarship. T.H.D. was funded by German Research Foundation (DFG) grants: DO 1240/2-1 and DO 1240/3-1. S.T. and E.H. were supported by RIKEN-CBS, JPB Foundation, and HHMI.
Publisher Copyright:
© 2020, eLife Sciences Publications Ltd. All rights reserved.
PY - 2020/4
Y1 - 2020/4
N2 - Learning from successes and failures often improves the quality of subsequent decisions. Past outcomes, however, should not influence purely perceptual decisions after task acquisition is complete since these are designed so that only sensory evidence determines the correct choice. Yet, numerous studies report that outcomes can bias perceptual decisions, causing spurious changes in choice behavior without improving accuracy. Here we show that the effects of reward on perceptual decisions are principled: past rewards bias future choices specifically when previous choice was difficult and hence decision confidence was low. We identified this phenomenon in six datasets from four laboratories, across mice, rats, and humans, and sensory modalities from olfaction and audition to vision. We show that this choice-updating strategy can be explained by reinforcement learning models incorporating statistical decision confidence into their teaching signals. Thus, despite being suboptimal from the experimenter’s perspective, confidence-guided reinforcement learning optimizes behavior in uncertain, real-world situations.
AB - Learning from successes and failures often improves the quality of subsequent decisions. Past outcomes, however, should not influence purely perceptual decisions after task acquisition is complete since these are designed so that only sensory evidence determines the correct choice. Yet, numerous studies report that outcomes can bias perceptual decisions, causing spurious changes in choice behavior without improving accuracy. Here we show that the effects of reward on perceptual decisions are principled: past rewards bias future choices specifically when previous choice was difficult and hence decision confidence was low. We identified this phenomenon in six datasets from four laboratories, across mice, rats, and humans, and sensory modalities from olfaction and audition to vision. We show that this choice-updating strategy can be explained by reinforcement learning models incorporating statistical decision confidence into their teaching signals. Thus, despite being suboptimal from the experimenter’s perspective, confidence-guided reinforcement learning optimizes behavior in uncertain, real-world situations.
UR - http://www.scopus.com/inward/record.url?scp=85084338705&partnerID=8YFLogxK
U2 - 10.7554/eLife.49834
DO - 10.7554/eLife.49834
M3 - Article
C2 - 32286227
AN - SCOPUS:85084338705
SN - 2050-084X
VL - 9
JO - eLife
JF - eLife
M1 - e49834
ER -