Lifespan age differences in the arbitration of learning strategies

Many everyday tasks (such as commuting to work) can be pursued efficiently by relying on past experiences. However, such habits may be mal-adaptive if the environment changes unexpectedly (e.g. train got cancelled). Under these conditions we have to use knowledge about the structure of the environment for forward planning (e.g. find an alternative means of transportation). However, often the scenario is not that straightforward and the preferred train might be late rather than cancelled. In such a situation, we have to arbitrate between our habitual strategy (wait for the late train) and alternative courses of action (take the bus), that may involve forward planning (Balleine & O'Doherty, 2010; Dayan & Niv, 2008). One of the core hypotheses of the collaborative research center (CRC) is that adaptive behavior in dynamic environments depends on the ability to balance complementary control functions in a context-sensitive manner (Goschke, 2003; Gruber & Goschke, 2004). In this proposal we directly address this hypothesis in the domain of learning and decision-making by asking the question how the ability to arbitrate between learning strategies changes across the human lifespan. In line with recent theories (Daw, Gershman, Seymour, Dayan, & Dolan, 2011; O'Doherty, Lee, & McNamee, 2015), we assume that habitual decisions rely on model-free (experience-based) reinforcement learning; In contrast, we define goal-directed decisions as model-based mechanisms that involve learning the structure of the environment, which is used for forward planning. The central premise of the proposed project is that adaptive decision-making in dynamic environments depends on an arbitration mechanism that prioritizes these different learning strategies. Specifically, it is postulated that the arbitration between model-based and model-free learning strategies depends on the relative reliability with which these strategies lead to the desired goal (Daw, Niv, & Dayan, 2005b; Lee, Shimojo, & O'Doherty, 2014; Yoshida & Seymour, 2014). In the framework of the CRC reliability estimates of model-based and model-free learning strategies are conceptualized as meta-control parameters that are used by the arbitration mechanism to balance complementary control demands. Sub-optimality in the adjustments of these meta-control parameters either due to immaturity or aging (Eppinger, Haemmerer, & Li, 2011; Hämmerer & Eppinger, 2012) may compromise arbitration mechanisms and may underlie age-related deficits in adaptive behavior. These deficits are relevant for daily life when navigating in dynamic environments (e.g. in school or during grocery shopping) in which we have to rely on multiple learning strategies.

Project Members

Principal Investigators