Neural representations of task rules are maintained in the prefrontal cortex, the part of the brain responsible for planning action.

Anne Trafton | MIT News Office  November 19, 2018

How the brain switches between different sets of rules

When you slow down after exiting the highway, or hush your voice in the library, you’re using this brain mechanism.

Image: MIT News -->

     Cognitive flexibility — the brain’s ability to switch between different rules or action plans depending on the context — is key to many of our everyday activities. For example, imagine you’re driving on a highway at 65 miles per hour. When you exit onto a local street, you realize that the situation has changed and you need to slow down.

     When we move between different contexts like this, our brain holds multiple sets of rules in mind so that it can switch to the appropriate one when necessary. These neural representations of task rules are maintained in the prefrontal cortex, the part of the brain responsible for planning action.

     A new study from MIT has found that a region of the thalamus is key to the process of switching between the rules required for different contexts. This region, called the mediodorsal thalamus, suppresses representations that are not currently needed. That suppression also protects the representations as a short-term memory that can be reactivated when needed.

     “It seems like a way to toggle between irrelevant and relevant contexts, and one advantage is that it protects the currently irrelevant representations from being overwritten,” says Michael Halassa, an assistant professor of brain and cognitive sciences and a member of MIT’s McGovern Institute for Brain Research.

     Halassa is the senior author of the paper, which appears in the Nov. 19 issue of Nature Neuroscience. The paper’s first author is former MIT graduate student Rajeev Rikhye, who is now a postdoc in Halassa’s lab. Aditya Gilra, a postdoc at the University of Bonn, is also an author.

Changing the rules

     Previous studies have found that the prefrontal cortex is essential for cognitive flexibility, and that a part of the thalamus called the mediodorsal thalamus also contributes to this ability. In a 2017 study published in Nature, Halassa and his colleagues showed that the mediodorsal thalamus helps the prefrontal cortex to keep a thought in mind by temporarily strengthening the neuronal connections in the prefrontal cortex that encode that particular thought.

     In the new study, Halassa wanted to further investigate the relationship between the mediodorsal thalamus and the prefrontal cortex. To do that, he created a task in which mice learn to switch back and forth between two different contexts — one in which they must follow visual instructions and one in which they must follow auditory instructions.

     In each trial, the mice are given both a visual target (flash of light to the right or left) and an auditory target (a tone that sweeps from high to low pitch, or vice versa). These targets offer conflicting instructions. One tells the mouse to go to the right to get a reward; the other tells it to go left. Before each trial begins, the mice are given a cue that tells them whether to follow the visual or auditory target.

“The only way for the animal to solve the task is to keep the cue in mind over the entire delay, until the targets are given,” Halassa says.

     The researchers found that thalamic input is necessary for the mice to successfully switch from one context to another. When they suppressed the mediodorsal thalamus during the cuing period of a series of trials in which the context did not change, there was no effect on performance. However, if they suppressed the mediodorsal thalamus during the switch to a different context, it took the mice much longer to switch.

     By recording from neurons of the prefrontal cortex, the researchers found that when the mediodorsal thalamus was suppressed, the representation of the old context in the prefrontal cortex could not be turned off, making it much harder to switch to the new context.

In addition to helping the brain switch between contexts, this process also appears to help maintain the neural representation of the context that is not currently being used, so that it doesn’t get overwritten, Halassa says. This allows it to be activated again when needed. The mice could maintain these representations over hundreds of trials, but the next day, they had to relearn the rules associated with each context.

     Sabine Kastner, a professor of psychology at the Princeton Neuroscience Institute, described the study as a major leap forward in the field of cognitive neuroscience.

     “This is a tour-de-force from beginning to end, starting with a sophisticated behavioral design, state-of-the-art methods including causal manipulations, exciting empirical results that point to cell-type specific differences and interactions in functionality between thalamus and cortex, and a computational approach that links the neuroscience results to the field of artificial intelligence,” says Kastner, who was not involved in the research.

Multitasking AI

     The findings could help guide the development of better artificial intelligence algorithms, Halassa says. The human brain is very good at learning many different kinds of tasks — singing, walking, talking, etc. However, neural networks (a type of artificial intelligence based on interconnected nodes similar to neurons) usually are good at learning only one thing. These networks are subject to a phenomenon called “catastrophic forgetting” — when they try to learn a new task, previous tasks become overwritten.

     Halassa and his colleagues now hope to apply their findings to improve neural networks’ ability to store previously learned tasks while learning to perform new ones.

1.      The research was funded by the National Institutes of Health, the Brain and Behavior Foundation, the Klingenstein Foundation, the Pew Foundation, the Simons Foundation, the Human Frontiers Science Program, and the German Ministry of Education.