Computational modelling of cognitive control over multiple trials

A key question of cognitive control research is: How does the brain select its actions, especially when there is conflict about what to do? This task can be taxing in our often uncertain and dynamic environment, especially when we have to cling to a goal over extended periods of time. In recent years, computational theories for goal-directed behaviour have been proposed, in particular reinforcement learning (RL) and hierarchical RL (HRL). Currently, in practice, different versions of RL models are used to explain different aspects of goal-directed behaviour. For example, it has been proposed that the brain uses two different controllers to reach incentivized goals: the so-called model-free and model-based controllers where an additional arbitration balances the two. It is a possibility that the brain runs two different controllers with two distinct computational mechanisms to reach a single goal; however a viable alternative is that the brain just uses a single controlling mechanism, which adapts its parameters to different environmental situations. Such a single-model approach would have several advantages, among them the possibility to predict new findings, as opposed to explaining away new findings with yet another RL model instantiation.
Therefore, the overarching goal of this project is to develop such a single computational model that explains how humans exert cognitive control, over extended time periods, to reach goals in uncertain and dynamic environments. To do this, we will build on a recently developed Bayesian approach, the so-called active inference (ActInf) framework. The ActInf framework has two advantages over RL and HRL models. First, it was specifically designed to account for the variability and uncertainty in our dynamic environment and is therefore better suited for addressing questions about resolving cognitive control dilemmas in ecologically valid experiments of goal-reaching under choice conflicts. Second, in comparison to HRL modelling, it is straightforward to build unifying hierarchical models with the ActInf framework. Such a hierarchical unification was hypothesized and identified before as an important research question for the model-free vs. model-based division but it was unclear with what model this may be achieved.
We expect four key outcomes of this project. First, the ActInf model will be able to explain human behaviour and physiological parameters as well or even better than conventional models and in a more efficient way. We will show this with two representative goal-reaching tasks, which require cognitive control. Second, the model can be used to test specific predictions about how goal-directed behaviour computations are implemented by the brain. Third, the work will be of fundamental importance for the CRC, because it will provide an explanation how specific control dilemmas are adaptively resolved without resorting to an additional (homunculus) arbitration mechanism. Fourth, the project will provide a novel way of analysing behavioural and neuroimaging data that has a powerful translational potential for many other types of cognitive control experiments, particularly those with ecologically valid stimulus conditions.

Project Members

Principal Investigators