 |
Center for Nonlinear Studies |
In the past several decades, neuroscientists have been studying the response of the brain to sensory input and theorized that single neurons respond to individual concepts. In 2005, results on a study of epileptic patients demonstrated that some subjects had neurons that fired on a specific concept, and ignored other stimuli. For example, a woman had a neuron that fired when shown a picture of Jennifer Aniston, but not on other pictures of people, places, or things. Another patient had a neuron that fired when shown a picture of Halle Berry, as well as the text string "Halle Berry", demonstrating the invariance of a neuron to concepts and specific modalities. In our work, we sought to improve upon the standard feed-forward deep learning autoencoder by augmenting them with biologically inspired concepts of sparsity, top-down feedback, and lateral inhibition. While building and observing the behavior of our model, we were fascinated that multimodal, invariant neurons naturally emerged. Our experiments and results demonstrate the emergence of Halle Berry neurons, and we additionally show that our sparse representation of multimodal signals is qualitatively and quantitatively superior to the standard feed forward joint embedding in common vision and machine learning tasks.
Host: Garrett Kenyon, 505-667-1900, gkenyon@lanl.gov