Reverse-engineering neocortical intelligence

We are using detailed measurements of function and structure of mouse visual cortex to reverse engineer the inference and learning algorithms of the brain. We have recorded optically from 10⁵ neurons (thousands at a time) of a behaving mouse across all layers of cortex using 2- and 3-photon microscopy. We then used electron microscopy to reconstruct the nanoscale wiring of this circuit, and synthesize these diverse measurements in the context of probabilistic inference to relate distributed computations to algorithms. Finally, we are applying these new algorithms to real-world machine intelligence problems.

Computationally Constrained Control

Biological brains are able to interact efficiently with a complex world, and generalize far better than artificial agents to new situations while using vastly less energy than computer algorithms. Motivated by the strong performance of real brains in control tasks, we will generalize current theories of optimal control to include not only state rewards and action costs, but also the costs of thinking and the variability of neural machinery.

Inferring interactions between neurons, stimuli, and behavior

We are developing statistical tools to infer how large populations of neurons interact with each other and with the external world, using the large-scale data enabled by emerging neuroscience technologies. This is a large collaboration between statisticians, mathematicians, computational neuroscientists, machine learning practitioners, and experimental neuroscientists.

Distributed computations for complex tasks

Neuroscience has traditionally used simple, largely static tasks to peer inside the brain. Here with our collaborators we will ask animals to perform dynamic, complex, naturalistic tasks, in both virtual reality and enriched physical spaces. By including multiple nuisance variables, we will challenge the brain to untangle the representations of task-relevant variables in ways that simpler, highly controlled tasks do not. We will use this richness to drive and then model population responses to identify the distributed nonlinear processing that dynamically couples neural activity patterns between brain areas and thereby generates behavior.

Decoding language from human brains

Our collaborators have recorded neural activity from inside the skulls of well over 100 human patients performing language tasks that include speaking, reading, and listening. We are using rich models to explain these dynamic data, and to decode speech from neural activity from patients who are losing the ability to speak unaided.

Nonlinear population codes

Many task-irrelevant variables affect how neurons are tuned to task-relevant stimuli. To extract the relevant ones, the brain must transform its responses nonlinearly. Does it do this well? We have a new test for optimal nonlinear decoding and a method to extract the effective population decoding strategy.

Interactive Apps for Scientific Visualization

Many scientific concepts can be conveyed effectively by good interactive graphics. We are looking for a programmer-artist to develop interactive games both for teaching computational neuroscience, as well as for teaching concepts in color theory grounded in visual perception.