Sebold M.a· Kiebel S.J.b· Smolka M.N.c· Heinz A.a· Deserno L.c,d,e

Author affiliations

aDepartment of Psychiatry and Neurosciences, Charité – Universitätsmedizin Campus Mitte, Berlin, Germany
bDepartment of Psychology, Technische Universität Dresden, Dresden, Germany
cDepartment of Psychiatry and Psychotherapy, Technische Universität Dresden, Dresden, Germany
dDepartment of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, University of Würzburg, Würzburg, Germany
eMax Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany

Log in to MyKarger to check if you already have access to this content.

Buy FullText & PDF Unlimited re-access via MyKarger Unrestricted printing, no saving restrictions for personal use read more

CHF 38.00 *
EUR 35.00 *
USD 39.00 *

Select

KAB

Buy a Karger Article Bundle (KAB) and profit from a discount!

If you would like to redeem your KAB credit, please log in.

Save over 20% compared to the individual article price.

Learn more

Access via DeepDyve Unlimited fulltext viewing Of this article Organize, annotate And mark up articles Printing And downloading restrictions apply

Select

Subscribe Access to all articles of the subscribed year(s) guaranteed for 5 years Unlimited re-access via Subscriber Login or MyKarger Unrestricted printing, no saving restrictions for personal use read more

Subcription rates

Select

* The final prices may differ from the prices shown due to specifics of VAT rules.

Article / Publication Details

First-Page Preview

Received: September 06, 2021
Accepted: August 14, 2022
Published online: October 20, 2022

Number of Print Pages: 18
Number of Figures: 1
Number of Tables: 0

ISSN: 0302-282X (Print)
eISSN: 1423-0224 (Online)

For additional information: https://www.karger.com/NPS

Abstract

Alcohol use disorder (AUD) is characterized by a combination of symptoms including excessive craving, loss of control, and progressive neglect of alternative pleasures. A mechanistic understanding of what drives these symptoms is needed to improve diagnostic stratification and to develop new treatment and prevention strategies for AUD. To date, there is no consensus regarding a unifying mechanistic framework that accounts for the different symptoms of AUD. Reinforcement learning (RL) and economic choice theories may be key to elucidating the underlying processes of symptom development and maintenance in AUD. These algorithms may account for the different behavioral and physiological phenomena and are suited to dissect mechanisms linked to different symptoms of AUD. We here review different RL and economic choice models and how they map onto three symptoms of AUD: (1) cue-induced craving, (2) neglect of alternative rewards, and (3) consumption despite adverse consequences. For each symptom and theory, we describe findings from animal and human studies. In humans, we focus on empirical studies that investigated RL models in the context of treatment outcome in AUD. The review indicates important gaps to be addressed in the future by highlighting the challenges in transferring findings from RL and economic choice studies to clinical application. We also critically evaluate the potential and pitfalls of a symptom-oriented approach and highlight the importance of elucidating the role of learning and decision-making processes across diagnostic boundaries.

© 2022 S. Karger AG, Basel

References Grant BF, Goldstein RB, Saha TD, Chou SP, Jung J, Zhang H, et al. Epidemiology of DSM-5 alcohol use disorder: results from the national epidemiologic survey on alcohol and related onditions III. JAMA Psychiatry. 2015;72(8):757–66. Griswold MG, Fullman N, Hawley C, Arian N, Zimsen SRM, Tymeson HD, et al. Alcohol use and burden for 195 countries and territories, 1990–2016: a systematic analysis for the Global Burden of disease study 2016. Lancet. 2018;392(10152):1015–35. Murray CJL, Aravkin AY, Zheng P, Abbafati C, Abbas KM, Abbasi-Kangevari M, et al. Global burden of 87 risk factors in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet. 2020;396(10258):1223–49. Nutt D, King LA, Saulsbury W, Blakemore C. Development of a rational scale to assess the harm of drugs of potential misuse. Lancet. 2007;369(9566):1047–53. van Amsterdam J, Nutt D, Phillips L, van den Brink W. European rating of drug harms. J Psychopharmacol. 2015;29(6):655–60. World Health Organization. ICD-10: International Statistical Classification of Diseases and Related Health Problems. 11th ed. World Health Organization; 2019. Heinz A, Kiefer F, Smolka MN, Endrass T, Beste C, Beck A, et al. Addiction research consortium: losing and regaining control over drug intake (ReCoDe)—from trajectories to mechanisms and interventions. Addict Biol. 2020;25(2):e12866. Lane SP, Sher KJ. Limits of current approaches to diagnosis severity based on criterion counts: an example with DSM-5 alcohol use disorder. Clin Psychol Sci. 2015;3(6):819–35. Heinz A. A new understanding of mental disorders: computational models for dimensional psychiatry: MIT Press; 2017. Joyce DW, Kehagia AA, Tracy DK, Proctor J, Shergill SS. Realising stratified psychiatry using multidimensional signatures and trajectories. Journal of Translational Medicine. 2017;15(1):15. Gillan CM, Kosinski M, Whelan R, Phelps EA, Daw ND. Characterizing a psychiatric symptom dimension related to deficits in goal-directed control. Elife. 2016;5:e11305. Rouault M, Seow T, Gillan CM, Fleming SM. Psychiatric symptom dimensions are associated with dissociable shifts in metacognition but not task performance. Biol Psychiatry. 2018;84(6):443–51. Martin CS, Chung T, Langenbucher JW. How should we revise diagnostic criteria for substance use disorders in the DSM-V? J Abnorm Psychol. 2008;117(3):561–75. Plana-Ripoll O, Pedersen CB, Holtz Y, Benros ME, Dalsgaard S, de Jonge P, et al. Exploring comorbidity within mental disorders among a Danish national population. JAMA Psychiatry. 2019;76(3):259–70. Lai HMX, Cleary M, Sitharthan T, Hunt GE. Prevalence of comorbid substance use, anxiety and mood disorders in epidemiological surveys, 1990-2014: a systematic review and meta-analysis. Drug Alcohol Depend. 2015;154:1–13. Castillo-Carniglia A, Keyes KM, Hasin DS, Cerdá M. Psychiatric comorbidities in alcohol use disorder. Lancet Psychiatry. 2019;6(12):1068–80. Zhou X, Menche J, Barabási A-L, Sharma A. Human symptoms: disease network. Nat Commun. 2014;5(1):4212. Rash CJ, Weinstock J, Van Patten R. A review of gambling disorder and substance use disorders. Subst Abuse Rehabil. 2016;7:3–13. Van den Eynde F, Koskina A, Syrad H, Guillaume S, Broadbent H, Campbell IC, et al. State and trait food craving in people with bulimic eating disorders. Eat Behav. 2012;13(4):414–7. Robbins TW, Gillan CM, Smith DG, de Wit S, Ersche KD. Neurocognitive endophenotypes of impulsivity and compulsivity: towards dimensional psychiatry. Trends Cogn Sci. 2012;16(1):81–91. Insel T, Cuthbert B, Garvey M, Heinssen R, Pine DS, Quinn K, et al. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry. 2010;167(7):748–51. Cuthbert BN, Insel TR. Toward the future of psychiatric diagnosis: the seven pillars of RDoC. BMC Med. 2013;11:126. Bernhardt N, Nebe S, Pooseh S, Sebold M, Sommer C, Birkenstock J, et al. Impulsive decision making in young adult social drinkers and detoxified alcohol-dependent patients: a cross-sectional and longitudinal study. Alcohol Clin Exp Res. 2017;41(10):1794–807. Ioannidis K, Hook R, Wickham K, Grant JE, Chamberlain SR. Impulsivity in Gambling Disorder and problem gambling: a meta-analysis. Neuropsychopharmacology. 2019;44(8):1354–61. Winstanley CA, Eagle DM, Robbins TW. Behavioral models of impulsivity in relation to ADHD: translation between clinical and preclinical studies. Clin Psychol Rev. 2006;26(4):379–95. Sebold M, Deserno L, Nebe S, Nebe S, Schad DJ, Garbusow M, et al. Model-based and model-free decisions in alcohol dependence. Neuropsychobiology. 2014;70(2):122–31. Wyckmans F, Otto AR, Sebold M, Daw N, Bechara A, Saeremans M, et al. Reduced model-based decision-making in gambling disorder. Sci Rep. 2019;9(1):19625. Voon V, Derbyshire K, Ruck C, Irvine MA, Worbe Y, Enander J, et al. Disorders of compulsivity: a common bias towards learning habits. Mol Psychiatry. 2015;20(3):345–52. Hogarth L. Addiction is driven by excessive goal-directed drug choice under negative affect: translational critique of habit and compulsion theory. Neuropsychopharmacology. 2020;45(5):720–35. Gillan CM, Kalanthroff E, Evans M, Weingarden HM, Jacoby RJ, Gershkovich M, et al. Comparison of the association between goal-directed planning and self-reported compulsivity vs obsessive-compulsive disorder diagnosis. JAMA Psychiatry. 2020;77(1):77–85. Fontanesi L, Gluth S, Spektor MS, Rieskamp J. A reinforcement learning diffusion decision model for value-based decisions. Psychon Bull Rev. 2019;26(4):1099–121. Pooseh S, Bernhardt N, Guevara A, Huys QJM, Smolka MN. Value-based decision-making battery: a Bayesian adaptive approach to assess impulsive and risky behavior. Behav Res Methods. 2018;50(1):236–49. Sutton RS, Barto AG. Introduction to reinforcement learning: MIT Press; 1998. Dayan P, Niv Y. Reinforcement learning: the good, the bad and the ugly. Curr Opin Neurobiol. 2008;18(2):185–96. Camerer CF. Behavioral economics. Curr Biol. 2014;24(18):R867–71. Huys QJM, Maia TV, Frank MJ. Computational psychiatry as a bridge from neuroscience to clinical applications. Nat Neurosci. 2016;19(3):404–13. Guest O, Martin AE. How computational modeling can force theory building in psychological science. Perspect Psychol Sci. 2021;16(4):789–802. Stephan KE, Schlagenhauf F, Huys QJM, Raman S, Aponte EA, Brodersen KH, et al. Computational neuroimaging strategies for single patient predictions. Neuroimage. 2017;145(Pt B):180–99. Maia TV, Frank MJ. From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci. 2011;14(2):154–62. Deserno L, Boehme R, Heinz A, Schlagenhauf F. Reinforcement learning and dopamine in Schizophrenia: dimensions of symptoms or specific features of a disease group? Front Psychiatry. 2013;4:172. Takahashi YK, Langdon AJ, Niv Y, Schoenbaum G. Temporal specificity of reward prediction errors signaled by putative dopamine neurons in rat VTA depends on ventral striatum. Neuron. 2016;91(1):182–93. Pan WX, Schmidt R, Wickens JR, Hyland BI. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J Neurosci. 2005;25(26):6235–42. Schad DJ, Rapp MA, Garbusow M, Nebe S, Sebold M, Obst E, et al. Dissociating neural learning signals in human sign- and goal-trackers. Nat Hum Behav. 2020;4(2):201–14. Stewardson HJ, Sambrook TD. Reward prediction error in the ERP following unconditioned aversive stimuli. Sci Rep. 2021;11(1):19912. Daw ND, Gershman SJ, Seymour B, Dayan P, Dolan RJ. Model-based influences on humans' choices and striatal prediction errors. Neuron. 2011;69(6):1204–15. Groman SM, Massi B, Mathias SR, Lee D, Taylor JR. Model-free and model-based influences in addiction-related behaviors. Biol Psychiatry. 2019;85(11):936–45. Deserno L, Huys QJM, Boehme R, Buchert R, Heinze HJ, Grace AA, et al. Ventral striatal dopamine reflects behavioral and neural signatures of model-based control during sequential decision making. Proc Natl Acad Sci U S A. 2015;112(5):1595–600. Walker SE, Peña-Oliver Y, Stephens DN. Learning not to be impulsive: disruption by experience of alcohol withdrawal. Psychopharmacology. 2011;217(3):433–42. Voon V, Irvine MA, Derbyshire K, Worbe Y, Lange I, Abbott S, et al. Measuring "waiting" impulsivity in substance addictions and binge eating disorder in a novel analogue of rodent serial reaction time task. Biol Psychiatry. 2014;75(2):148–55. Robinson ESJ, Eagle DM, Economidou D, Theobald DEH, Mar AC, Murphy ER, et al. Behavioural characterisation of high impulsivity on the 5-choice serial reaction time task: specific deficits in 'waiting' versus 'stopping. Behav Brain Res. 2009;196(2):310–6. Belin D, Mar AC, Dalley JW, Robbins TW, Everitt BJ. High impulsivity predicts the switch to compulsive cocaine-taking. Science. 2008;320(5881):1352–5. Mollick JA, Kober H. Computational models of drug use and addiction: a review. J Abnorm Psychol. 2020;129(6):544–55. Teufel C, Fletcher PC. The promises and pitfalls of applying computational models to neurological and psychiatric disorders. Brain. 2016;139(Pt 10):2600–8. Paulus MP, Huys QJM, Maia TV. A roadmap for the development of applied computational psychiatry. Biol Psychiatry Cogn Neurosci Neuroimaging. 2016;1(5):386–92. Huys QJM, Browning M, Paulus MP, Frank MJ. Advances in the computational understanding of mental illness. Neuropsychopharmacology. 2021;46(1):3–19. Huys QJM, Maia TV, Paulus MP. Computational psychiatry: from mechanistic insights to the development of new treatments. Biol Psychiatry Cogn Neurosci Neuroimaging. 2016;1(5):382–5. Browning M, Carter CS, Chatham C, Den Ouden H, Gillan CM, Baker JT, et al. Realizing the clinical potential of computational psychiatry: report from the banbury center meeting, february 2019. Biol Psychiatry. 2020;88(2):e5–e10. Konova AB, Lopez-Guzman S, Urmanche A, Ross S, Louie K, Rotrosen J, et al. Computational markers of risky decision-making for identification of temporal windows of vulnerability to opioid use in a real-world clinical setting. JAMA Psychiatry. 2020;77(4):368–77. Field M, Mogg K, Bradley BP. Craving and cognitive biases for alcohol cues in social drinkers. Alcohol Alcohol. 2005;40(6):504–10. Beck A, Wüstenberg T, Genauck A, Wrase J, Schlagenhauf F, Smolka MN, et al. Effect of brain structure, brain function, and brain connectivity on relapse in alcohol-dependent patients. Arch Gen Psychiatry. 2012;69(8):842–52. Tiffany ST, Conklin CA. A cognitive processing model of alcohol craving and compulsive alcohol use. Addiction. 2000;95(8 Suppl 2):145–53. Robinson TE, Berridge KC. The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Res Rev. 1993;18(3):247–91. Robinson TE, Berridge KC. Review. The incentive sensitization theory of addiction: some current issues. Phil Trans R Soc Lond B Biol Sci. 2008;363(1507):3137–46. Chaudhri N, Sahuque LL, Janak PH. Ethanol seeking triggered by environmental context is attenuated by blocking dopamine D1 receptors in the nucleus accumbens core and shell in rats. Psychopharmacology. 2009;207(2):303–14. Brodie MS, Shefner SA, Dunwiddie TV. Ethanol increases the firing rate of dopamine neurons of the rat ventral tegmental area in vitro. Brain Res. 1990;508(1):65–9. Schultz W. Predictive reward signal of dopamine neurons. J Neurophysiol. 1998;80(1):1–27. Yoder KK, Morris ED, Constantinescu CC, Cheng T-E, Normandin MD, O’Connor SJ, et al. When what you see isn’t what you get: alcohol cues, alcohol administration, prediction error, and human striatal Dopamine. Alcohol Clin Exp Res. 2009;33(1):139–49. Heinz A, Siessmeier T, Wrase J, Hermann D, Klein S, Grüsser SM, et al. Correlation between dopamine D(2) receptors in the ventral striatum and central processing of alcohol cues and craving. Am J Psychiatry. 2004;161(10):1783–9. Flagel SB, Clark JJ, Robinson TE, Mayo LM, Czuj A, Willuhn I, et al. A selective role for dopamine in stimulus-reward learning. Nature. 2011;469(7328):53–7. Flagel SB, Akil H, Robinson TE. Individual differences in the attribution of incentive salience to reward-related cues: implications for addiction. Neuropharmacology. 2009;56(Suppl 1):139–48. Tomie A, Aguado AS, Pohorecky LA, Benjamin D. Ethanol induces impulsive-like responding in a delay-of-reward operant choice procedure: impulsivity predicts autoshaping. Psychopharmacology. 1998;139(4):376–82. Lovic V, Saunders BT, Yager LM, Robinson TE. Rats prone to attribute incentive salience to reward cues are also prone to impulsive action. Behav Brain Res. 2011;223(2):255–61. Flagel SB, Watson SJ, Akil H, Robinson TE. Individual differences in the attribution of incentive salience to a reward-related cue: influence on cocaine sensitization. Behav Brain Res. 2008;186(1):48–56. Mayo LM, de Wit H. Acquisition of conditioned responses to a novel alcohol-paired cue in social drinkers. J Stud Alcohol Drugs. 2016;77(2):317–26. Oberlin BG, Dzemidzic M, Eiler WJA 2nd, Carron CR, Soeurt CM, Plawecki MH, et al. Pairing neutral cues with alcohol intoxication: new findings in executive and attention networks. Psychopharmacology. 2018;235(9):2725–37. Grüsser SM, Heinz A, Raabe A, Wessa M, Podschus J, Flor H. Stimulus-induced craving and startle potentiation in abstinent alcoholics and controls. Eur Psychiatry. 2002;17(4):188–93. Wrase J, Schlagenhauf F, Kienast T, Wüstenberg T, Bermpohl F, Kahnt T, et al. Dysfunction of reward processing correlates with alcohol craving in detoxified alcoholics. Neuroimage. 2007;35(2):787–94. Grüsser SM, Wrase J, Klein S, Hermann D, Smolka MN, Ruf M, et al. Cue-induced activation of the striatum and medial prefrontal cortex is associated with subsequent relapse in abstinent alcoholics. Psychopharmacology. 2004;175(3):296–302. Myrick H, Anton RF, Li X, Henderson S, Randall PK, Voronin K. EFfect of naltrexone and ondansetron on alcohol cue: induced activation of the ventral striatum in alcohol-dependent people. Arch Gen Psychiatry. 2008;65(4):466–75. Hammarberg A, Jayaram-Lindstrom N, Beck O, Franck J, Reid MS. The effects of acamprosate on alcohol-cue reactivity and alcohol priming in dependent patients: a randomized controlled trial. Psychopharmacology. 2009;205(1):53–62. Langosch JM, Spiegelhalder K, Jahnke K, Feige B, Regen W, Kiemen A, et al. The impact of acamprosate on cue reactivity in alcohol dependent individuals: a functional magnetic resonance imaging study. J Clin Psychopharmacol. 2012;32(5):661–5. Miranda R, Ray L, Blanchard A, Reynolds EK, Monti PM, Chun T, et al. Effects of naltrexone on adolescent alcohol cue reactivity and sensitivity: an initial randomized trial. Addict Biol. 2014;19(5):941–54. Bach P, Weil G, Pompili E, Hoffmann S, Hermann D, Vollstadt-Klein S, et al. Incubation of neural alcohol cue reactivity after withdrawal and its blockade by naltrexone. Addict Biol. 2020;25(1):e12717. Montague PR, Dayan P, Sejnowski TJ. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci. 1996;16(5):1936–47. Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275(5306):1593–9. McClure SM, Daw ND, Montague PR. A computational substrate for incentive salience. Trends Neurosci. 2003;26(8):423–8. McClure SM, Berns GS, Montague PR. Temporal prediction errors in a passive learning task activate human striatum. Neuron. 2003;38(2):339–46. Konova AB, Louie K, Glimcher PW. The computational form of craving is a selective multiplication of economic value. Proc Natl Acad Sci U S A. 2018;115(16):4122–7. Bornstein AM, Pickard H. Chasing the first high”: memory sampling in drug choice. Neuropsychopharmacology. 2020;45(6):907–15. Augier E, Barbier E, Dulman RS, Licheri V, Augier G, Domi E, et al. A molecular mechanism for choosing alcohol over an alternative reward. Science. 2018;360(6395):1321–6. Imperato A, Di Chiara G. Preferential stimulation of dopamine release in the nucleus accumbens of freely moving rats by ethanol. J Pharmacol Exp Ther. 1986;239(1):219–28. Koob GF, Volkow ND. Neurobiology of addiction: a neurocircuitry analysis. Lancet Psychiatry. 2016;3(8):760–73. Volkow ND, Wang GJ, Maynard L, Fowler JS, Jayne B, Telang F, et al. Effects of alcohol detoxification on dopamine D2 receptors in alcoholics: a preliminary study. Psychiatry Res. 2002;116(3):163–72. Heinz A, Siessmeier T, Wrase J, Buchholz HG, Gründer G, Kumakura Y, et al. Correlation of alcohol craving with striatal Dopamine synthesis capacity and D2/3 receptor availability: a combined [18F]DOPA and [18F]DMFP PET study in detoxified alcoholic patients. Am J Psychiatry. 2005;162(8):1515–20. Gleich T, Spitta G, Butler O, Zacharias K, Aydin S, Sebold M, et al. Dopamine D2/3 receptor availability in alcohol use disorder and individuals at high risk: towards a dimensional approach. Addict Biol. 2021;26(2):e12915. Sebold M, Spitta G, Gleich T, Dembler-Stamm T, Butler O, Zacharias K, et al. Stressful life events are associated with striatal dopamine receptor availability in alcohol dependence. J Neural Transm. 2019;126(9):1127–34. Di Chiara G, Imperato A. Drugs abused by humans preferentially increase synaptic dopamine concentrations in the mesolimbic system of freely moving rats. Proc Natl Acad Sci U S A. 1988;85(14):5274–8. Ahmed SH. Validation crisis in animal models of drug addiction: beyond non-disordered drug use toward drug addiction. Neurosci Biobehav Rev. 2010;35(2):172–84. Lenoir M, Serre F, Cantin L, Ahmed SH. Intense sweetness surpasses cocaine reward. PLoS One. 2007;2(8):e698. Hogarth L, Field M. Relative expected value of drugs versus competing rewards underpins vulnerability to and recovery from addiction. Behav Brain Res. 2020;394:112815. Zvorsky I, Nighbor TD, Kurti AN, DeSarno M, Naudé G, Reed DD, et al. Sensitivity of hypothetical purchase task indices when studying substance use: a systematic literature review. Prev Med. 2019;128:105789. Hardy L, Parker S, Hartley L, Hogarth L. A concurrent pictorial drug choice task marks multiple risk factors in treatment-engaged smokers and drinkers. Behavioural Pharmacology. 2018;29(8):716–25. Venniro M, Panlilio LV, Epstein DH, Shaham Y. The protective effect of operant social reward on cocaine self-administration, choice, and relapse is dependent on delay and effort for the social reward. Neuropsychopharmacology. 2021;46(13):2350–7. Venniro M, Russell TI, Zhang M, Shaham Y. Operant social reward decreases incubation of heroin craving in male and female rats. Biol Psychiatry. 2019;86(11):848–56. Venniro M, Zhang M, Caprioli D, Hoots JK, Golden SA, Heins C, et al. Volitional social interaction prevents drug addiction in rat models. Nat Neurosci. 2018;21(11):1520–9. Stitzer ML, Jones HE, Tuten M, Wong C. Community reinforcement approach and contingency management interventions for substance abuse. Handbook of motivational counseling: goal-based approaches to assessment and intervention with addiction and other problems. 2nd ed. 2011. p. 549–69. Redish AD. Addiction as a computational process gone awry. Science. 2004;306(5703):1944–7. Ahmed SH. Neuroscience. Addiction as compulsive reward prediction. Science. 2004;306(5703):1901–2. Panlilio LV, Thorndike EB, Schindler CW. Blocking of conditioning to a cocaine-paired stimulus: testing the hypothesis that cocaine perpetually produces a signal of larger-than-expected reward. Pharmacol Biochem Behav. 2007;86(4):774–7. Field M, Heather N, Murphy JG, Stafford T, Tucker JA, Witkiewitz K. Recovery from addiction: behavioral economics and value-based decision making. Psychol Addict Behav. 2020;34(1):182–93. Field M, Werthmann J, Franken I, Hofmann W, Hogarth L, Roefs A. The role of attentional bias in obesity and addiction. Health Psychol. 2016;35(8):767–80. Pedersen ML, Ironside M, Amemori KI, McGrath CL, Kang MS, Graybiel AM, et al. Computational phenotyping of brain-behavior dynamics underlying approach-avoidance conflict in major depressive disorder. PLoS Comput Biol. 2021;17(5):e1008955. Lee GA, Forsythe M. Is alcohol more dangerous than heroin? The physical, social and financial costs of alcohol. Int Emer Nurs. 2011;19(3):141–5. McClure SM, Bickel WK. A dual-systems perspective on addiction: contributions from neuroimaging and cognitive training. Ann N Y Acad Sci. 2014;1327:62–78. Daw ND, Niv Y, Dayan P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci. 2005;8(12):1704–11. Everitt BJ, Robbins TW. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat Neurosci. 2005;8(11):1481–9. Everitt BJ, Robbins TW. Drug addiction: updating actions to habits to compulsions 10 years on. Annu Rev Psychol. 2016;67(1):23–50. Zapata A, Minney VL, Shippenberg TS. Shift from goal-directed to habitual cocaine seeking after prolonged experience in rats. J Neurosci. 2010;30(46):15457–63. Deserno L, Moran R, Michely J, Lee Y, Dayan P, Dolan RJ. Dopamine enhances model-free credit assignment through boosting of retrospective model-based inference. Elife. 2021;10:e67778. Moran R, Keramati M, Dayan P, Dolan RJ. Retrospective model-based inference guides model-free credit assignment. Nat Commun. 2019;10(1):750. Dickinson A, Balleine B, Watt A, Gonzalez F, Boakes RA. Motivational control after extended instrumental training. Anim Learn Behav. 1995;23(2):197–206. de Wit S, Kindt M, Knot SL, Verhoeven AAC, Robbins TW, Gasull-Camos J, et al. Shifting the balance between goals and habits: five failures in experimental habit induction. J Exp Psychol Gen. 2018;147(7):1043–65. Luscher C, Robbins TW, Everitt BJ. The transition to compulsion in addiction. Nat Rev Neurosci. 2020;21(5):247–63. Ersche KD, Meng C, Ziauddeen H, Stochl J, Williams GB, Bullmore ET, et al. Brain networks underlying vulnerability and resilience to drug addiction. Proc Natl Acad Sci U S A. 2020;117(26):15253–61. Dickinson A, Balleine B. The Role of learning in the operation of motivational systems. Stevens' Handbook of Experimental Psychology; 2002. Lopez MF, Becker HC, Chandler LJ. Repeated episodes of chronic intermittent ethanol promote insensitivity to devaluation of the reinforcing effect of ethanol. Alcohol. 2014;48(7):639–45. Lesscher HMB, van Kerkhof LWM, Vanderschuren LJMJ. Inflexible and indifferent alcohol drinking in male mice. Alcohol Clin Exp Res. 2010;34(7):1219–25. Hopf FW, Bowers MS, Chang S-J, Chen BT, Martin M, Seif T, et al. Reduced nucleus accumbens SK channel activity enhances alcohol seeking during abstinence. Neuron. 2010;65(5):682–94. Mangieri RA, Cofresí RU, Gonzales RA. Ethanol seeking by long evans rats is not always a goal-directed behavior. PLoS One. 2012;7(8):e42886. Corbit LH, Nie H, Janak PH. Habitual alcohol seeking: time course and the contribution of subregions of the dorsal striatum. Biol Psychiatry. 2012;72(5):389–95. Barker JM, Zhang H, Villafane JJ, Wang TL, Torregrossa MM, Taylor JR. Epigenetic and pharmacological regulation of 5HT3 receptors controls compulsive ethanol seeking in mice. Eur J Neurosci. 2014;39(6):999–1008. Giuliano C, Puaud M, Cardinal RN, Belin D, Everitt BJ. Individual differences in the engagement of habitual control over alcohol seeking predict the development of compulsive alcohol seeking and drinking. Addict Biol. 2021;26(6):e13041. Hogarth L, Attwood AS, Bate HA, Munafò MR. Acute alcohol impairs human goal-directed action. Biol Psychol. 2012;90(2):154–60. Rose AK, Brown K, MacKillop J, Field M, Hogarth L. Alcohol devaluation has dissociable effects on distinct components of alcohol behaviour. Psychopharmacology. 2018;235(4):1233–44. Sjoerds Z, de Wit S, van den Brink W, Robbins TW, Beekman ATF, Penninx BWJH, et al. Behavioral and neuroimaging evidence for overreliance on habit learning in alcohol-dependent patients. Transl Psychiatry. 2013;3:e337. Luijten M, Gillan CM, de Wit S, Franken IHA, Robbins TW, Ersche KD. Goal-directed and habitual control in smokers. Nicotine Tob Res. 2020;22(2):188–95. Ersche KD, Gillan CM, Jones PS, Williams GB, Ward LHE, Luijten M, et al. Carrots and sticks fail to change behavior in cocaine addiction. Science. 2016;352(6292):1468–71. Ersche KD, Lim TV, Murley AG, Rua C, Vaghi MM, White TL, et al. Reduced Glutamate turnover in the Putamen is linked with automatic habits in human Cocaine addiction. Biol Psychiatry. 2021;89(10):970–9. van Timmeren T, Quail SL, Balleine BW, Geurts DEM, Goudriaan AE, van Holst RJ. Intact corticostriatal control of goal-directed action in alcohol use disorder: a Pavlovian-to-instrumental transfer and outcome-devaluation study. Sci Rep. 2020;10(1):4949. Hogarth L, Hardy L. Alcohol use disorder symptoms are associated with greater relative value ascribed to alcohol, but not greater discounting of costs imposed on alcohol. Psychopharmacology. 2018;235(8):2257–66. MacKillop J, Jackson J, Murphy J, Amlung M. Association between individual differences in alcohol's relative reinforcing value of alcohol and alcohol misuse: a meta-analysis: 122. Alcoholism: Clinical & Experimental Research; 2015. Pickard H. What we're not talking about when we talk about addiction. Hastings Cent Rep. 2020;50(4):37–46. Dayan P. Dopamine, reinforcement learning, and addiction. Pharmacopsychiatry. 2009;42(Suppl 1):S56–65. Huys QJM, Cruickshank A, Seriès P. Reward-based learning, model-based and model-free. In: Jaeger D, Jung R, editors. Encyclopedia of computational neuroscience. New York, NY: Springer New York; 2013. p. 1–10. Huys QJM, Deserno L, Obermayer K, Schlagenhauf F, Heinz A. Model-free temporal-difference learning and dopamine in alcohol dependence: examining concepts from theory and animals in human imaging. Biol Psychiatry Cogn Neurosci Neuroimaging. 2016;1(5):401–10. Voon V, Reiter A, Sebold M, Groman S. Model-based control in dimensional psychiatry. Biol Psychiatry. 2017;82(6):391–400. Sebold M, Nebe S, Garbusow M, Guggenmos M, Schad DJ, Beck A, et al. When habits are dangerous: alcohol expectancies and habitual decision making predict relapse in alcohol dependence. Biol Psychiatry. 2017;82(11):847–56. Nebe S, Kroemer NB, Schad DJ, Bernhardt N, Sebold M, Muller DK, et al. No association of goal-directed and habitual control with alcohol consumption in young adults. Addict Biol. 2018;23(1):379–93. Patzelt EH, Kool W, Millner AJ, Gershman SJ. Incentives boost model-based control across a range of severity on