Cost-benefit tradeoff mediates the transition from rule-based to memory-based processing during practice

The task involves training and test phases. In the training phase, participants were trained to learn the transition orders of 2 sequences (A and B) until they remembered transitions between tasks with greater than 90% accuracy (see Materials and methods). In the test phase, they were asked to select the task type of a sequence at a specific position (e.g., A4, indicating the fourth task of sequence A) within the scanner. For the test phase data, we conducted a model-free analysis. Since the rule-based strategy assumes a higher cost for later positions, whereas the memory strategy treats different cued positions equally, a transition from rule-based to memory-based strategy would predict a decreasing cost for later positions over time. To test this hypothesis, we analyzed trial-wise response times (RTs) and error rates (ERs) separately using linear mixed-effects (LME) models. Predictors included cued position (1–4), block (1–6), and their interaction. RT results (Fig 2A) showed a significant main effect of block (β = –0.33, SE = 0.03, 95% confidence interval (CI) = [–0.39, –0.28], t(33.3) = –11.41, p p² = 0.40), a significant main effect of position (β = 0.48, SE = 0.05, 95% CI = [0.38, 0.59], t(33.0) = 9.68, p p² = 0.58), and a significant interaction effect (β = –0.10, SE = 0.02, 95% CI = [–0.14, –0.07], t(59.8) = –5.73, p p² = 0.06). Post hoc comparisons showed that the linear effect of position in block 1 and 2 was larger than block 2–6 and 4–6, respectively (ps β = –0.20, SE = 0.05, 95% CI = [–0.29, –0.10], t(33.0) = –4.17, p p² = 0.06), position (β = 0.20, SE = 0.05, 95% CI = [0.11, 0.30], t(36.6) = 4.41, p p² = 0.06), and a significant interaction effect (β = –0.13, SE = 0.04, 95% CI = [–0.21, –0.04], t(40.4) = –3.09, p = 0.004, η_p² = 0.02). Post hoc comparisons showed that the linear effect of position in block 1 was larger than in blocks 3 and 6, in block 2 compared to blocks 3, 4, and 6, and in block 5 compared to block 6, ps S1 Text). Note that the slope of block 6 was not completely flat because not all participants had transitioned to memory strategy by that time. Overall, we observed 3 patterns: (1) performance, when collapsed across cued positions, improved over time; (2) performance, when collapsed across blocks, was worse when the later sequence positions were cued; and (3) this negative association between performance and cued position decreased with time.

We constructed several computational models to capture the dynamics of transitioning from rule- to memory-based strategies (Fig 2B and Materials and methods). Our first model (M0) is a quantitative implementation of the decision-selection model. It includes 3 main components: learning, decision, and prediction. The learning component updates the memory strength of the cue-task associations (i.e., a probabilistic distribution of the correct task, denoted as M) using a delta rule with learning rate α. The decision component makes winner-take-all decisions about which strategy to apply using cost-benefit analyses. The prediction component predicts trial-wise RT based on linear predictors of values driven by rule- (V_R) or memory-based (V_M) strategies, depending on the strategy used. The model was fit to trial-wise RT data to estimate model parameters and trial-wise predictions of the strategy used. On each trial, the model performs a cost-benefit analysis to determine which strategy to use based on their values, defined as a weighted sum of their costs and benefits. Conceptually, the rule-based strategy retrieves a task before the cued position (e.g., the first task of a sequence) and recalls later tasks using sequential memory until the cued position is reached. On the other hand, the memory-based strategy directly retrieves the cued task through cue-task associations. Thus, the rule-based strategy is more effortful (and thus costly) than the memory strategy. Early in the test phase, the cue-task association is weak, whereas the participants have strong sequential memory of the task order from the training phase. Thus, the rule-based strategy provides better value than the memory strategy due to its higher benefit (despite higher cost) and is favored. As participants receive feedback with practice, the cue-task association becomes stronger. Along with its smaller effort compared to the rule-based strategy, the memory-based strategy eventually achieves a higher value, leading to the strategy transition (Fig 1A, left panel).

Based on the trial-wise model prediction of which strategy to use, trials were categorized as rule-based and memory-based trials. For each participant, the model outputs were then linked to trial-wise RT using a linear model (M0, Table 1) with the following regressors: (1) rule implementation cost (C, number of steps from recalling start to cued position) for rule trials; (2) the strength of the cue-task association (S) for memory trials, quantified as the reversed entropy of the task distribution conditioned on the cued position; (3) trial type (i.e., rule trials (Type_R) versus memory trials (Type_M)); (4) the interaction between C and aggregated times of rule implementation since the beginning of the experiment (C*Times_R) reflecting the practice effects of rule implementation (i.e., more experience would accelerate sequence recall and reduce the effect of C on RT); (5) log-transformed trial number reflecting practice effect not involving strategy switch; and (6) response (dummy coded). We compared M0 to its reduced versions (Table 1, M1-M5) using likelihood ratio tests at the group level. Specifically, M1 assumes behavioral performance does not vary across the 2 trial types; M2 assumes there is no practice effect of rule implementation; M3 incorporates both assumptions from M1 and M2; M4 assumes only a rule-based strategy has been implemented throughout the task; and M5 assumes the model uses neither rule-based nor memory-based strategies. M0 outperformed all other models (ps Fig 2C). Of note, M4 and M5 were outperformed by all other models that assume 2 strategies (ps

We then compared M0 to 3 extended models, each examining the necessity of accounting for a specific factor. First, to test whether the behavioral performance considers the improved efficiency of rule-based strategy (i.e., increased instead of fixed values) in the cost-benefit analysis, we compared M0 to M6 (Table 1 and Note B in S1 Text). Comparison between unnested models was achieved through the Akaike information criteria (AIC). The results showed that M6 (AIC = –3,928) performed worse than M0 (AIC = –4,019), p p Table 1 and Note C in S1 Text). Third, we tested whether the cost-benefit analysis was probabilistic (i.e., strategy can switch back and forth) using M8, which assumes that both strategies influence trial-level performance in a weighted manner (see Materials and methods). M8 performed worse (AIC = –3,095) than M0 (Table 1, p

As M0 was the winning model, it was used for all subsequent analyses. Individual model parameters from M0 are shown in Fig 2D. The effect of C was significantly above 0 (β = 0.33, SE = 0.09, Cohen’s d = 0.64, 95% CI = [0.15, 0.51], t(32) = 3.70, p β = –0.09, SE = 0.04, Cohen’s d = –0.38, 95% CI = [–0.18, –0.01], t(32) = –2.20, p = 0.035), suggesting that sequence recall accelerated over time. The memory strength effect (S) was also significant (β = –0.61, SE = 0.17, Cohen’s d = –0.61, 95% CI = [–0.96, –0.26], t(32) = –3.52, p = 0.001), suggesting that participants responded faster when the model predicts a stronger cue-task association on memory trials. Based on this model, the predicted proportion of participants using the rule-based strategy decreased over time (Fig 2E), indicating that the rule-based strategy was replaced by the memory strategy with practice. We defined transition points as the moments immediately before the trial in which the value of the memory strategy first surpassed the rule-based strategy (S2A Fig). Transition points were defined for each cue except for cues at position 1 because for that position no transition was necessary. Among the remaining 192 trials for cued positions 2–5, the model predicted that a participant would apply the rule-based strategy on 94 ± 37 trials, varying from 21 to 166 trials.

To validate the transition points predicted by the model, we tested the strategy switch cost following the transition point [19]. Using an LME model, we observed a significant increase in RT after transition points (β = 0.40, SE = 0.15, 95% CI = [0.10, 0.69], t(80.5) = 2.64, p = 0.010, η_p² = 0.15, Fig 2F). To examine further whether this effect is reliably linked to the model-predicted transition points, we repeated this analysis 5,000 times using randomly selected transition points for each cue and each subject. The observed post-switch slowing was greater than all random ones (p Fig 2G). Notably, this effect is independent of the model fitting, which only assumes that the mean RT for memory and rule trials can be different (i.e., different intercepts), without any assumptions regarding the 2 adjacent trials surrounding the transition point. We further tested the alternative explanation that the structure of the computational model introduces biases leading to artificially reported post-transition slowing. To this end, we repeated the post-transition slowing analysis using randomly sampled model parameters while keeping the architecture of the model constant 5,000 times. The median of the resulting distribution was centered around 0, and significantly lower than the observed post-switch slowing, p = 0.005 (S2B Fig), thus ruling out the possibility that the model artificially introduced a post-switch slowing effect. The post-transition slowing results suggest that the model accurately predicted when strategy transition occurs.

As the behavior is not affected by the losing strategy, behavioral analyses cannot test whether the losing strategy is implemented as posited in the both-active model. On the other hand, the decision-selection model predicts that fMRI activation patterns selectively encode the rule- and memory-based strategies for rule and memory trials, respectively. We conducted fMRI analyses to assess the representational strength of the 2 strategies. For the rule-based strategy, we predicted that the similarity of fMRI activation patterns across 2 trials would reflect the number of shared recalled tasks (Fig 3A and 3B). For the memory-based strategy, we tested the cue effect such that the 2 trials’ activation patterns will be more similar if they share the same cue than if they do not, as the cue and its association with the task are used in the memory strategy (Fig 3A and 3C).

Results showed widespread brain regions with significant rule effects on rule trials, but not on memory trials (Fig 4A and 4B). By contrast, significant cue effects were observed in widespread regions on memory trials but only in limited visual and motor regions on rule trials (Fig 4D and 4E). To statistically test the strength of each effect across trial types while controlling for processing shared by both strategies (e.g., visual processing), we compared each effect between rule and memory trials. A stronger rule effect on rule trials compared to memory trials was observed in the striatum, frontoparietal, temporal, and occipital regions (Fig 4C, ps Fig 4F, ps Fig 4G, see S1 Table for a detailed list of brain regions and statistics). We replicated these findings after regressing out the difficulty of different conditions with a covariate of the RT difference in the regression model (see S6 Fig). The double dissociation showing stronger representations of the winning strategy than the losing strategy suggests that the winning strategy is selectively implemented or prioritized.

We then conducted a multivariate analysis of fMRI data to test the encoding of the key variables of the computational model [20]. A similar analysis using univariate fMRI methods is presented in Note F in S1 Text and S3 Fig. For the rule-based strategy, we tested the variable of cued position, as later cued position is associated with higher rule implementation cost (e.g., more tasks to sequentially recall, higher difficulty and higher working memory load). We found that this variable was encoded in the left IFSa, p Fig 5A and 5D). In contrast, for the memory-based strategy, we tested the encoding of the memory strength of cue-task associations (Eq 3) and found that memory strength was encoded in primary visual regions (bilateral V1, V2, right V3, right V4), sensorimotor regions (left primary motor area, left lateral 8B, right area 2), and temporoparietal junction regions (left PF complex, left intraparietal 2, right PFm complex, right medial intraparietal area), among others (left rostral area 6, left auditory 5 complex, right middle TE1, and right dorsal 32), all ps Fig 5B and 5D). In addition, given the importance of the posterior parietal cortex in memory retrieval [21], we also tested the memory strength effect in bilateral PFm, PGs, and PGi, and results showed that 4 of the 6 regions (bilateral PGs, right PFm, and right PGs) reached significance, ps

To test the presence of the cost-benefit analysis process predicted by the decision-selection model, we searched for brain regions encoding the decision variable, defined as the absolute value difference between the 2 strategies. We expected to observe this effect in the dorsomedial prefrontal area [16]. Consistently, the right 8BM and the right supplementary and cingulate eye field (SCEF) showed a significant decision variable effect, ps ps Fig 5C and 5D). The evidence supporting the encoding of the decision variable also argues against the both-active model, which assumes that there is no decision-making process to select the winning strategy prior to the race.

We next examined the explanatory power of the decision-selection model by investigating whether the model-fitting results were linked to the neural findings. The model fitting yielded a λ parameter that reflects the preference for the memory-based strategy over the rule-based strategy. A smaller λ value corresponds to a relatively higher value of the memory-based strategy (see Materials and methods). Therefore, we predicted that a smaller λ would correlate with stronger encoding of memory strength in the brain. To test this prediction, we calculated the average memory encoding strength across regions showing statistically significant representation of memory strength (Fig 5B). Our analysis revealed a negative correlation between λ (log-transformed) and encoding strength, r = –0.52, p = 0.001, one-tailed, uncorrected (Fig 3E). This negative correlation was observed in 3 out of 17 regions (bilateral V1 and left PF), ps r = –0.50, p = 0.026, one-tailed, FDR corrected (Fig 5F). These results suggest that the extent to which individuals prioritize the memory strategy is related to their neural sensitivity of memory encoding.

Our behavioral and modeling results suggest an abrupt shift of strategy when the value of memory retrieval outweighs the rule implementation (Figs 2F and S2A). We anticipated that there are neural substrates underlying this abrupt shift [5,22], distinct from the overall differences observed during the constant application of the 2 strategies (Fig 4). Therefore, we next aimed to identify the brain regions supporting the strategy switch. To this end, we treated a transition point as a boundary separating the 2 time periods using different strategies. We then tested the boundary effect (i.e., higher pattern similarity between trials on the same side of the boundary than different, termed as “within boundary” and “between boundary” conditions, respectively). To focus on the effects specific to the boundary, we compared pattern similarity of the trials immediately adjacent to the transition point with other temporally close trials (i.e., between 2 and 6 trials away from the transition point). We excluded trial pairs involving different cues to prevent potential confounding effects from intrinsic differences across cues (Fig 3C). We found that the right dorsolateral PFC (dlPFC region, area p9-46v) showed a statistically significant boundary effect (Fig 6, β = 0.0109, SE = 0.0028, 95% CI = [0.0054, 0.0165], t(1358.0) = 3.86, p = 0.023 corrected, η_p² = 0.012). No other regions survived the correction for multiple comparisons. This effect was stronger than its null distribution using shuffled transition points over 5,000 iterations (p = 0.024, Fig 6C), suggesting that the boundary effect is tied to the model-estimated transition points. To further rule out the possibility that this finding captured a generic trial-type effect (i.e., independent of the trials’ temporal distance from the transition points), we repeated the same analysis by replacing the temporally close trials with trials that were more than 6 trials away from the transition point and did not observe the boundary effect (t(30.7) = 0.84, p = 0.203 uncorrected). In other words, it appears that this effect only occurred on trials near strategy transition and hence supports the relation between dlPFC representation and strategy transition.

As the decision-selection model considers the improvement in memory strength as the driving factor of the strategy switch, we tested the increase in memory strength by analyzing the level of pattern-separation [23]. In other words, we predicted that pattern similarity between different cues would reduce with practice. To focus on strategy transition, trials were time-locked to their corresponding transition points. Trial positions containing too few (Fig 7A and 7B, t(52.4) = –3.93, p p = 0.048 corrected, η_p² = 0.23). No other regions survived the correction for multiple comparisons. Moreover, to explore the possibility that vmPFC displays a generic decrease of between-cue pattern similarity over time, we repeated this analysis 5,000 times with randomly selected transition points. These randomized data maintain the chronological order of trials. The generic decrease hypothesis predicts that the effects in the repeated analysis will be similar to the observed effect. However, results showed that the degree of pattern separation observed in the left vmPFC (i.e., the negative slope β) falls below all values in this distribution, p Fig 7C). These rules out a possibility of bias due to a generic temporal effect. Together, the results suggest that the vmPFC separates neural representations of different cue-task associations during task learning, potentially strengthening these associative memories via reducing interference between associations.