Robust measurement of habits with the forced-response task: A multiverse analysis
León, J. J., Martínez-López, P., Garre-Frutos, F., & Luque, D.
PsyArXivPreprint
Abstract
Instrumental behavior reflects a balance between two systems: an outcome-oriented, goal-directed system and a cognitively efficient habit system that gradually gains control with repeated instrumental responding in stable contexts. While overtraining reliably induces habits in animal models, human participants often retain goal-directed control when their performance is assessed using devaluation paradigms. To address this translational gap, the forced-response method has emerged as a promising approach for studying goal-directed and habitual behavior by constraining the time available to respond. This method imposes a dual task in which participants must respond accurately to an instrumental discriminative stimulus at a specific time signaled by a sound. Because the onset of the discriminative stimulus varies across trials, the time available for response preparation is systematically manipulated. Restricting this time can unmask stimulus-driven responses when the goal-directed system has insufficient time to override them. Although the forced-response method has become increasingly popular, its use has also been accompanied by growing flexibility in data analysis, creating a potential “garden of forking paths.” Using four independent datasets, we generated 48 a priori valid preprocessing pipelines to assess variability in two discrete measures of habitual behavior—the proportion of habits and the habit index—as well as their split-half reliability and the proportion of habits as a continuous function of the available response time, modeled using generalized additive models. Results indicated that both habit measures were generally robust across preprocessing pipelines. The proportion of habits was more sensitive to specific preprocessing choices, particularly accuracy and timing-synchrony exclusion criteria, but also showed greater reliability. In all cases, however, the resulting variability was small. Reliability analyses yielded acceptable estimates in most analytic universes. Overall, these findings support the forced-response method as a robust paradigm for studying habit formation and expression in humans.