TY - JOUR
T1 - Evolving Images for Visual Neurons Using a Deep Generative Network Reveals Coding Principles and Neuronal Preferences
AU - Ponce, Carlos R.
AU - Xiao, Will
AU - Schade, Peter F.
AU - Hartmann, Till S.
AU - Kreiman, Gabriel
AU - Livingstone, Margaret S.
N1 - Funding Information:
This work was supported by NIH grants R01EY16187, R01EY25670, R01EY011379, P30EY012196, R01EY026025 and NSF STC award CCF-1231216 to the Center for Brains, Minds and Machines at MIT. We thank Richard Born, Shimon Ullman, and Christof Koch for comments. G.K. and W.X. conceived of the approach. W.X. implemented the network algorithm and did all the in silico experiments. C.R.P. and P.F.S. adapted the algorithm to neurophysiology. P.F.S. M.S.L. T.S.H. and C.R.P. collected and analyzed data. C.R.P. W.X. and M.S.L. wrote the manuscript, all authors revised the manuscript, and M.S.L. acquired funding. The authors declare no competing interests.
Funding Information:
This work was supported by NIH grants R01EY16187 , R01EY25670 , R01EY011379 , P30EY012196 , R01EY026025 and NSF STC award CCF-1231216 to the Center for Brains, Minds and Machines at MIT. We thank Richard Born, Shimon Ullman, and Christof Koch for comments.
Publisher Copyright:
© 2019 Elsevier Inc.
PY - 2019/5/2
Y1 - 2019/5/2
N2 - What specific features should visual neurons encode, given the infinity of real-world images and the limited number of neurons available to represent them? We investigated neuronal selectivity in monkey inferotemporal cortex via the vast hypothesis space of a generative deep neural network, avoiding assumptions about features or semantic categories. A genetic algorithm searched this space for stimuli that maximized neuronal firing. This led to the evolution of rich synthetic images of objects with complex combinations of shapes, colors, and textures, sometimes resembling animals or familiar people, other times revealing novel patterns that did not map to any clear semantic category. These results expand our conception of the dictionary of features encoded in the cortex, and the approach can potentially reveal the internal representations of any system whose input can be captured by a generative model. Neurons guided the evolution of their own best stimuli with a generative deep neural network.
AB - What specific features should visual neurons encode, given the infinity of real-world images and the limited number of neurons available to represent them? We investigated neuronal selectivity in monkey inferotemporal cortex via the vast hypothesis space of a generative deep neural network, avoiding assumptions about features or semantic categories. A genetic algorithm searched this space for stimuli that maximized neuronal firing. This led to the evolution of rich synthetic images of objects with complex combinations of shapes, colors, and textures, sometimes resembling animals or familiar people, other times revealing novel patterns that did not map to any clear semantic category. These results expand our conception of the dictionary of features encoded in the cortex, and the approach can potentially reveal the internal representations of any system whose input can be captured by a generative model. Neurons guided the evolution of their own best stimuli with a generative deep neural network.
KW - generative adversarial network
KW - inferotemporal cortex
KW - neural networks
UR - http://www.scopus.com/inward/record.url?scp=85064413111&partnerID=8YFLogxK
U2 - 10.1016/j.cell.2019.04.005
DO - 10.1016/j.cell.2019.04.005
M3 - Article
C2 - 31051108
AN - SCOPUS:85064413111
SN - 0092-8674
VL - 177
SP - 999-1009.e10
JO - Cell
JF - Cell
IS - 4
ER -