Neurobiology of Decision Making: A Selective Review

REVIEW Neurobiology of Decision Making A Selective Review from a Neurocognitive and Clinical Perspective Monique Ernst and Martin P Paulus We present a temporal map of key processes that occur during decision making which consists of three stages 1 formation of preferences among options 2 selection and execution of an action and 3 experience or evaluation of an outcome This framework can be used to integrate findings of traditional choice psychology neuropsychology brain lesion studies and functional neuroimaging Decision making is distributed across various brain centers which are differentially active across these stages of decision making This approach can be used to follow developmental trajectories of the different stages of decision making and to identify unique deficits associated with distinct psychiatric disorders Key Words Anticipation anxiety choice selection development motivation schizophrenia D ecision making refers to the process of forming prefer ences selecting and executing actions and evaluating outcomes Here we define decision making as encom passing a wide range of behaviors having in common the basic generic structure of inputprocessoutputfeedback Input re fers to the presentation of separate stimuli each predicting a measurable rewarding or aversive outcome process refers to the appraisal of these stimuli and formation of preference output refers to the action carried out in response to the selected stimulus Feedback is the experience and evaluation of the outcome that follows the action perpetuated on the selected stimulus It is used for learning about the values of the stimuli The goal of this work is to provide a framework or generic template along which the various psychologic and neural pro cesses underlying decision making can be examined We show how findings from various fields of research can be integrated into this framework Decision making has received considerable attention from psychologists and economists Loewenstein et al 2001 Slovic et al 2002 Tversky and Kahneman 1975 neurologists and neuro psychologists Bechara 2004a Clark et al 2003 Damasio et al 1996 Lhermitte et al 1986 Shallice and Burgess 1991 psychia trists Ernst et al 2004 Paulus et al 2003 Rogers et al 1999 and neuroscientists Clark et al 2004 Glimcher 2002 Gold and Shadlen 2001 Platt and Glimcher 1999 Initial forays in the clinical realm of decision making came from the systematic examinations of patients with welldefined brain lesions for review see Bechara 2004a Damasio et al 1996 This unique body of work has not only identified brain regions essential for adaptive decision making but has also provided conceptual models of critical aspects of decision making eg the somatic marker theory Damasio et al 1996 Most important lesion studies have supplied experimental paradigms eg develop ment of the Gambling Task Bechara et al 1994 as well as hypotheses to the relatively new field of functional neuroimaging research Finally the integration of psychoeconomics that exam ines rules guiding choices Kahneman 1991 and neuroscience that establishes neural models of rewardmodulated behavior Schultz 2002 Schultz et al 1997 has pushed research on decision making to a new level of scrutiny This review focuses on biological processes keeping simple and constant the input component that is the presentation in a neutral environment of external cues defined by distinct physi cal features eg volume color shape that predict distinct measurable outcomes eg dollar amounts A large psychologic and social literature has examined the influence of context environment on decision making which operates at multiple levels sensory cognitive affective and social These influences could also be tracked along the various stages of decision making The model presented here is anchored on a neural systems framework primarily based on functional neuroanatomy Al though we do not address directly the neurochemical substrates of the various processes involved in decision making several neurotransmitter systems have been hypothesized to critically influence decision making For example dopamine is implicated in reward systems Di Chiara et al 2004 Wise 1996 and associative learning Schultz 2002 serotonin in impulsivity and emotion Hollander and Rosen 2000 acetylcholine in memory Gold 2003 and noradrenaline in attention and arousal Ber ridge and Waterhouse 2003 Robbins 1997 Interaction among these neurochemical modulators and the translation of their actions at the molecular level eg Nestler 2001 is an active area of research that is beyond the scope of this review Psychological Modulators and Neural Substrates of the Three Stages of Decision Making Decision making depends on three temporally and partially functionally distinct sets of processes 1 the assessment and formation of preferences among possible options 2 the selec tion and execution of an action and 3 the experience or evaluation of an outcome Figure 1 The analysis of these stages helps to distinguish which aspect of decision making may be differentially affected in various psychiatric disorders Although we address cognitive processes specific to each of these stages a number of psychologic constructs such as attention working memory motivation anticipation and impulsivity can be in volved in various degrees throughout these stages Stage 1 Forming Preferences Human and animal studies have strived to identify factors and rules that govern choices Identification of these rules have led From the Section of Developmental and Affective Neuroscience ME Na tional Institute of Mental Health National Institutes of Health Bethesda Maryland Laboratory of Biological Dynamics and Theoretical Medicine and Department of Psychiatry MPP University of California at San Diego San Diego and Veterans Affairs San Diego Healthcare System MPP San Diego California Address reprint requests to Monique Ernst MD PhD Section of Develop mental and Affective Neuroscience Mood and Anxiety Disorders Pro gram NIMHNIHHHS 15K North Drive Bethesda MD 20892 Email ernstmmailnihgov Received November 9 2004 revised March 28 2005 accepted June 3 2005 BIOL PSYCHIATRY 2005xxxxx 00063223053000 doi101016jbiopsych200506004 2005 Society of Biological Psychiatry ARTICLE IN PRESS 2 BIOL PSYCHIATRY 2005xxxxx M Ernst and MP Paulus Outcome Areas Assessment Execution Processing Cognitive DLPFC dACC SIPL et STG vACC ant Insula Amygdala vStriatum Other dStriatum ttt preSMA tire Stage 1 Assessment Stage 2 Execution Stage 3 Outcome processing Option A Execution and a oT Modification of menue Figure 1 Hypothetical model of the basic processes and brain areas involved in the different stages of decision making Decision making is divided into three stages 1 the assessment and formation of preferences among possible options 2 the selection and execution of an action and 3 the experience or evaluation of an outcome Table of neural circuitry top We propose that a distributed network of both cognitive and affective brain areas process these stages differentially Below is a possible decisionmaking scenario In this scenario the hypothesized neural substrates are involved in the three stages of decision making to varying degrees The degree of their involvement is reflected by the number of signs Conceivably certain types of decision making require relatively less emotional involvement whereas others require more cognitive involvement The balance between the engagements of these neural substrates is hypothesized to be altered in psychiatric disorders Taken together we predict that patients with different psychiatric disorders will exhibit stagedependent degrees of decisionmaking dysfunctions Decisionmaking schema bottom Stage 1 shows three available options A B and C among which one option must be selected Stage 2 is the stage during which the selected option option B is being executed Stage 3 is the stage during which the outcome of the action is being experienced and processed outcome B The fourth box represents processes involved in learning which occurs when the action outcome sequence is completed Learning modifies the value associated with each option of stage 1 the next time these options are presented Knowing outcome B not only influences the value of option B but also has a profound influence on the nonselected options Ant Insula anterior insula dACC dorsal anterior cingulate cortex DLPFC dorsolateral prefrontal cortex dStriatum dorsal striatum preSMA presupplementary motor area SIPL superior intraparietal lobule STG superior temporal gyrus vACC ventral anterior cingulate VLMPFC ventral lateralmedial prefrontal cortex vStriatum ventral striatum some to formalize mathematical models of choice behavior Most distinct neurochemical systems Some of these functional circuits prevalent psychologic theories and mathematical models applied are described later to the formation of choices include learning theories with Coding the probability or certainty of outcomes predicted by classical conditioning Pavlov 2005 operant conditioning Skin available options is specific to the process of forming prefer ner 1953 and a mathematical rendition of classical conditioning ences The parietal cortex has been shown to be involved in Rescorla and Wagner 1972 Matching law theory which posits computation Dehaene et al 1999 and in assessment of proba that over time the pattern of choices is a direct function of the bility Ernst et al 2004 Platt and Glimcher 1999 Shadlen and probability of outcomes Hernstein 1961 game theory which Newsome 2001 The anterior cingulate cortex ACC has been describes choice behavior in the context of several decision associated with processes of uncertainty Critchley et al 2001 makers setting a competitive or cooperative environment Elliott et al 1999 perhaps by integrating successes and errors Bernoulli 1954 Lewontin 1961 Nash 1953 and prospect the over time Carter et al 1999 ory which describes decisions under uncertainty Kahneman Editing options eg ignoring least attractive options pairing and Tversky 1979 options of similar values etc serves to simplify choices Tversky From a neural systems perspective the formation of values and Kahneman 1981 These operations can be mostly automatic involves both cognitive and emotional brain circuits A host or can involve conscious deliberative effort The right dorsolat of factors influence the development of preferences including eral and orbitofrontal cortex have been suggested to underlie physical features of the options characteristics of outcomes these processes Cummings 1995 Dias et al 1997 Reasoning predicted by the options such as valence positive negative part of deliberation has been proposed to be carried out by left salience intensity magnitude probability degree of certainty middle and inferior frontal gyri Goel et al 1998 and timing delay relative values and number of options to Affective appraisal of options also involves both automatic select from previous experience with these options and their and conscious processes Theories of emotions Cannon 1987 outcomes and external and internal context in which the deci Schachter and Singer 1962 have helped to shape cognitive sions are made eg social affective state Each of these factors neuroscience approaches to decision making Particularly the may be coded by specific neural circuits and modulated by JamesLange theory of emotion Cannon 1987 which underlies wwwsobporgjournal the role of physiologic and cognitive responses in the formation of emotion has paved the way to the contemporary somatic marker theory Bechara 2004a Damasio et al 1996 The affective attribute of an option is expected to recruit limbic regions such as amygdala insula orbitofrontal cortex and anterior cingulate An intermediate step in this operation is the production of somatic markers which signals the intensity salience of the valence negative or positive value of stimuli experienced by individuals Although the relative contribution of the somatic markers in decision making continues to be debated Heims et al 2004 Hornak et al 2003 Maia and McClelland 2004 it remains a central aspect of emotional tagging of stimuli Structures involved in the somatic marker model comprise the orbitofrontal cortex amygdala and ventral striatum This model described later also applies to the assessment of outcome stimuli in stage 3 The amygdala belongs to a network of structures which includes the insula anterior cingulate gyrus and medial prefron tal cortex This network helps to identify the emotional signifi cance of a stimulus generate an affective response and regulate the affective state Phillips et al 2003 The insula has afferent and efferent connections to medial and orbital prefrontal cortex ACC and several nuclei of the amygdala Augustine 1996 Together with the amygdala the insula underlies the generation of somatic markers autonomic changes such as skin conduc tance blood pressure heart rate or the activation of the representations of somatic markers Bechara 2004a These so matic markers in turn send feedback signals to cortical struc tures particularly to insulasomatosensory and orbitofrontal cortices and perhaps ACC The insular cortex appears to be important for subjective feeling states and interoceptive aware ness Craig 2002 Critchley et al 2004 Finally the emotional intensity salience carried by stimuli has been associated with enhanced activation of ventral striatum particularly nucleus accumbens Zink et al 2004 Stage 2 Execution of Actions The goal of this stage is to initiate perform and complete an action according to the preferences established during the first stage Cognitively competing actions have to be suppressed or inhibited and sequences of actions have to be implemented appropriate subgoals have to be monitored correction of errors has to take place and timing of actions has to be planned The general model of control of actions formulated by Shallice et al 1989 could be best articulated at this juncture although it refers more specifically to the planning and execution of complex multitasks This stage engages the neural systems supporting initiation monitoring and completion of actions The ACC has been consistently found to be recruited in error monitoring Carter et al 1998 Holroyd and Coles 2002 and in conflict detection Van Veen et al 2004 The lateral prefrontal cortex may also contribute to the monitoring of action through its interaction with the ACC during error monitoring Mathalon et al 2003 and in guiding compensatory actions Gehring and Knight 2000 Motivation is functionally defined as the determinant of the direction and the energy of an action The nucleus accumbens a component of the ventral striatum has been shown to modulate the motivational aspects of an action Ernst et al 2002 2004 Knutson et al 2001 Mogenson and Yang 1991 Salamone and Correa 2002 The amygdala and the sublenticular extended amygdala of the basal forebrain Breiter and Rosen 1999 and ventrolateral prefrontal cortex Taylor et al 2004 may also contribute to this process Thus far it has been difficult to separate motivation from arousal For example larger activation in premotor cortex with greater incentives Roesch and Olson 2004 could reflect enhanced arousal rather than enhanced motivation A number of abnormalities including prematurely initiated actions eg impulsivity incomplete actions eg behavioral fragmentation or delayed and insufficiently motivated actions eg psychomotor retardation can be observed during this stage The stage 2 multiprocesses that is action selection online monitoring of performance accuracy motivation to act and anticipation of outcome interact in a manner not yet fully understood Thus not surprisingly this complex equilibrium is often perturbed in psychiatric disorders Stage 3 Experiencing the Outcome The outcome of the selected action is experienced or con sumed at this stage Like during Stage 1 values are attributed to the outcome experience Thus assessment processes such as coding physical and emotional characteristics of stimuli occur in both stage 1 and stage 3 The somatic marker theory Damasio 1996 is also operative during this last step Stage 1 and stage 3 however differ critically in their ultimate function the function of stage 1 is to form preference based on expected values and that of stage 3 is to consume and learn the actual values of option stimuli for the supreme goal of adaptive behavior A number of factors that are specific to stage 3 influence the formation of actual values For example experienced outcome strongly depends on counterfactual possibilities that is what might have happened if a different choice had been made in stage 1 Shepperd and Mcnulty 2002 Zeelenberg et al 1996 Regret and disappointment profoundly influence future behavior Zeelenberg 1998 The degree of surprise associated with the outcome experience is also tantamount to the computation of the actual value Surprise can emerge from earlier than expected time of occurrence or from the nature of the expected outcome By definition surprise infers a difference between actual value and expected value In daily experience outcome or actual values coded during stage 3 often differ from the option or expected values coded during stage 1 Kahneman and Snell 1990 A number of factors may contribute to the difference between expected and actual values such as the contrast between imagined and experienced event Mellers and McGraw 2001 or the adjustment of the expected value as a function of the time interval between the two stages Ainslie 1992 This value difference is critical to learning processes Electro physiologic work in monkeys has demonstrated that dopamine neurons code the value difference between the expected and actual value of outcomes and this value difference serves as a learning signal that permits behavior to become adaptive Schultz 2002 The larger the difference the more unexpected the outcome and the greater the learning signal This prediction is supported by behavioral Coughlan and Connolly 2001 Mellers et al 1997 neuroimaging Berns et al 2001 and neurophysiologic studies Schultz 1998 all showing greater emotional and neural impact with unexpected outcome than with expected outcomes Processing the difference between the expected and observed outcomes is central to the temporal difference model Functional neuroimaging experiments have shown that ventral striatum Pagnoni et al 2002 and orbitofrontal cortex ODoherty et al M Ernst and MP Paulus BIOL PSYCHIATRY 2005xxxxx 3 wwwsobporgjournal ARTICLE IN PRESS 2003b are involved in generating this difference signal in humans McClure et al 2003 In addition to the already mentioned regions implicated in emotion processing amygdala nucleus accumbens orbitofron tal cortex and insula the medial prefrontal cortex particularly within Brodmann area 10 seems to be uniquely involved in feedback processes Knutson et al 2003 The ventral medial prefrontal cortex including the orbitofrontal cortex receives sensory inputs from several modalities and provides the major cortical output to visceromotor structures of the hypothalamus and brainstem Ongur and Price 2000 The medial prefrontal cortex has been implicated in assessment of pleasurability Mit terschiffthaler et al 2003 tracking of rewarding outcomes Knut son et al 2003 and formation of hedonic associations Passing ham et al 2000 Finally associative learning is triggered when events occur repeatedly in close temporal proximity Specifically if feedback occurs close enough to stimulus presentation or to the action associative learning is initiated The amygdala and the nucleus accumbens have been critically involved in this process Baxter and Murray 2002 Cardinal et al 2002 Gabriel et al 2003 Salamone and Correa 2002 Schoenbaum and Setlow 2003 In conclusion psychologic and neural correlates of decision making can be anchored on a cognitiveaffective neuroscience framework that will permit a more systematic approach to developmental milestones of decision making and perturbations of motivated behaviors in distinct psychiatric disorders Clinical Applications Neurodevelopment The cognitive and affective components that contribute to decision making reviewed in the previous section are all subject to developmental changes These developmental changes occur at a biological and environmental level There is a large neuro psychologic literature addressing agerelated changes in cogni tive affective and social domains Spear 2000 although few studies have focused directly on decision making Byrnes 2002 Most work has focused on economic perspectives of decision making in adults but none of this work has been conducted in children Normative neurodevelopmental investigations in hu mans are beginning to emerge particularly since the advent of noninvasive functional neuroimaging At present however only three neuroimaging studies address specifically decisionmaking processes in young people Bjork et al 2004 Ernst et al 2005 May et al 2004 These studies have explored in adolescents the neural substrates of motivation for action stage 2 and response to feedback stage 3 From an ontogenic perspective decision making seems to be first under the primacy of emotional controls and then evolves toward a progressively larger involvement of cognitive function to bring the decisionmaking process to a mature level of optimizing goal achievement This evolving balance between affective and cognitive com ponents of decision making can be conceptualized along the framework of two putative parallel decisionmaking systems a fast mostly automatic system and a slow deliberate system described by DenesRaj and Epstein 1994 The fast more rudimentary system is present early in life and the second system develops progressively with age and at times competes with the older system In addition brain lesion studies suggest that the initial formation of emotional tags attached to stimuli depend on the integrity of the amygdala and that the represen tation of the affective tags are accessed through the ventromedial frontal cortex for review see Bechara 2004b Early dysfunction in these regions and associated networks could compromise significantly the development of adaptive decision making Another formulation particularly applicable to adolescence relates to the balance between reward seeking approach behav ior and harm avoidance avoidance behavior Both appetitive and aversive stimuli are found to be processed by the same structures including amygdala ventral striatum and orbitofron tal cortex suggesting that these structures can carry opposite functions based on different modulatory controls affecting neu ronal output This imbalance may be most influential on the incentive value of stimuli presented in stage 1 and the experience of outcome in stage 3 of decision making Such hypothesis can be tested behaviorally and in the functional magnetic resonance imaging environment using appropriate decisionmaking para digms Adolescence is a transition period that is marked by changes in behavior reflecting a distinct pattern of decision making Byrnes 2002 Chambers and Potenza 2003 Larson et al 2002 Spear 2000 This pattern of decision making underlies risk taking and noveltyseeking behaviors which confer a high level of morbidity and mortality to adolescents Grunbaum et al 2004 The heightened fascination for novelty during this period may represent an evolutionary adaptive motivational force that facil itates learning and the move toward independence It is accom panied by a sense of invulnerability which has not yet been examined from a neuroscience perspective Risk taking implies the prominence of sensation seeking over harm avoidance suggesting a distinct balance within the neural systems involved in these processes In support of this model adolescents have been found to be more sensitive to the rewarding effects of illicit substances as evidence by high incidence rates of substance abuse and to be less aware of negative consequences of events Clayton 1992 The balance between approach and avoidance may be translated differently at the various stages of decision making delineated in this review Bjork et al 2004 Ernst et al 2005 Substance Use Disorder Several altered decisionmaking patterns have been observed in substancedependent subjects First these individuals show a propensity to select actions associated with large shortterm gains and longterm losses preferentially to those associated with small shortterm gains and longterm gains Bechara and Damasio 2002 Grant et al 2000 Second they are more likely to select risky options Lane and Cherek 2000 and show an altered temporal horizon of risks and benefits ie a steeper temporal discounting function Madden et al 1999 Petry et al 1998 Third these subjects do not value appropriately the probability and magnitude of potential outcomes Rogers et al 1999 Rogers and Robbins 2001 Fourth they generate perseverative responses when making a prediction and select actions that are more stimulus bound and less dependent on changes in the frequency of prediction errors Paulus et al 2002 2003 It is unclear whether these altered decisionmaking patterns reflect dysfunction in a single or several processes that contribute to decision making Monterosso et al 2001 Several investigators have shown an increased activation of the inferior medial and lateral prefrontal cortex in substancedependent subjects in response to cues that elicit craving responses Breiter et al 1997 Childress et al 1999 Grant et al 1996 Wang et al 1999 This altered activation pattern could reflect an increased valuation of the drugrelated stimuli and therefore fundamentally affect 4 BIOL PSYCHIATRY 2005xxxxx M Ernst and MP Paulus wwwsobporgjournal ARTICLE IN PRESS stage 1 the formation of preferences of the decisionmaking process Specifically an option which is associated with sensi tized stimuli may have acquired an overwhelming weight which results in an altered decisionmaking pattern Several neuroimaging studies have revealed dysfunctions of the ventromedial ventrolateral and dorsolateral prefrontal cor tex in stimulantdependent subjects London et al 2000 Paulus et al 2002 Volkow and Fowler 2000 Based on their pattern of decision making just described stimulantdependent individuals are expected to show a lack of flexible association of outcomes with advantageous actions attenuated trend detection The inferior prefrontal cortex including orbitofrontal cortex has been shown to play an essential role in this process This is consistent with studies that found altered inferior prefrontal activation at baseline and during decision making in stimulant dependent subjects Bolla et al 2003 Volkow and Fowler 2000 Dysfunction of the anterior insula may also be involved in substance abuse Paulus et al 2003 reported a close correlation between risky responses harm avoidance and insula activation a finding that is consistent with the insulas role in punishment Critchley et al 2001 ODoherty et al 2003a Substancedepen dent subjects may show attenuated insula activation which is associated with increased risk taking It is unclear however whether this process occurs at a particular stage of decision making or whether attenuated processing of aversive values occurs throughout the decisionmaking process A key question is whether decisionmaking dysfunctions and their underlying neural substrates are a preexisting condition and contribute to the initiation of drug use or are a consequence of the repeated use of these drugs Altered processing of the value of available options during stage 1 which affects prediction of outcome may represent preexisting deficits Alternatively defi cient processing of the outcome value which can lead to poorer acquisition of advantageous over disadvantageous actions may result from altered dopaminergic signaling secondary to a resid ual error signal as a consequence of substance use Redish 2004 Some investigators have suggested that the develop ment of drug dependence may require the presence of both altered drug initiating and drug maintaining behaviors Ken dler 2001 Thus perturbed decision making in drugdepen dent individuals may reflect both a preexisting alteration of assessment of options and a substancerelated attenuation of outcome processing Schizophrenia Experimental evidence supports the general hypothesis that schizophrenia patients may exhibit dysfunctions during forma tion of preference execution and outcome evaluation Kraepe lin Kraepelin and Robertson 1919 conceptualized schizophre nia as a disorder of volition rather than one of intellect which refers to the ability to make and carry out conscious decisions Zec 1995 and to the capacity for motivation to act stage 2 A large body of literature evidences cognitive deficits in schizo phrenia affecting attention and executive functioning ie work ing memory and planning We limit our discussion to the findings directly applicable to the decisionmaking model A number of data relevant to decisionmaking processes in schizophrenia concern the stage 1 of formation of preference These patients seem to request less information before reaching a decision as evidenced in a probability inference task Garety et al 1991 although they take longer to make their decisions Hutton et al 2002 Aspects of learning that is use of previous outcome experiences to make appropriate decisions seem to be impaired Schizophrenia patients are more ready to change their estimates of the likelihood of an event when confronted with potentially disconfirmatory information Garety et al 1991 and they show deficits on measures of risk adjustment Hutton et al 2002 They also fail to show a priming effect that is facilitation of performance based on previous exposure to stimulus Passe rieux et al 1997 Vinogradov et al 1992 Other cognitive processes seem to contribute to poor decision making for example inadequate discrimination of old items from new insufficient distinction between selfgenerated items and externally generated items and poor recognition of the modality in which an event was presented Brebion et al 1998 These various abnormalities may point toward a mixture of assessment and executive dysfunctions Several investigators have proposed a relationship between semantic processing and decision making Schizophrenia patients may show an impair ment of action selection because they do not benefit from the automatic retrieval of processing information about the options available Baving et al 2001 Thus far no neuroimaging studies have investigated the different stages of decision making in this population Neuropsy chologic and clinical observations suggest the deficient integra tion of assessment and action selection processes stage 1 and stage 2 Accordingly an inadequate formation of values of options would result in a poorly formed internal model to guide the selection of action in a decisionmaking situation Studies using an experimental probe that can manipulate each compo nent process could assess each process separately and isolate the ones most significantly disrupted in schizophrenia patients As with substance dependence schizophrenia has been as sociated with dopaminergic dysfunction perhaps secondary to glutamatergic deficits Laruelle et al 2003 In view of the central role of dopamine in learning and reward processes its contribu tion to behavioral symptoms and neuroimaging findings in schizophrenia needs to be further examined In the same vein the influence of antipsychotic medications on decision making needs further evaluation Kapur 2004 Anxiety Disorders To our knowledge characteristics of decision making in anxiety disorders have not yet been systematically examined however a number of investigations report on cognitive sub strates of anxiety the most widespread substrate being atten tional bias toward threat Mogg and Bradley 1999 An obvious difficulty in the study of anxiety is the heterogeneity of disorders placed under the umbrella of anxiety disorders Nonetheless several theoretical models of generic anxiety have been pro posed that focus on the interaction between cognition affect physiology and behavior for review Wilken et al 2000 The association of stimuli with adverse affective experiences is a critical determinant of hyperarousal Dowden and Allen 1997 and anxious apprehension Nitschke et al 1999 which occur across anxiety disorders Accordingly the neural substrates engaged in the processing of aversive stimuli have been impli cated in the pathophysiology of anxiety These include limbic amygdala ventral striatum and paralimbic structures orbito frontal cortex insula ACC For example subjects with obsessivecompulsive disorder show increased errorrelated activity in the ACC Ursu et al 2003 which could hypothetically affect stage 2 error monitoring during execution and stage 3 error detection during feedback of decision making Posttraumatic stress disorder has been associated with dysfunction of medial prefrontal cortex and ACC M Ernst and MP Paulus BIOL PSYCHIATRY 2005xxxxx 5 wwwsobporgjournal ARTICLE IN PRESS Liberzon et al 2003 which could underlie impaired feedback processing stage 3 Recently a riskasfeelings hypothesis which highlights the role of affect experienced at the moment of decision making has been proposed Loewenstein et al 2001 Accordingly antici pated outcomes are translated into different body states based on previous experiences This process critically depends on the orbitofrontal cortex insula amygdala and ACC Given the importance of hyperarousal and related autonomic changes in anxiety anxious patients may show an altered pattern of aversive somatic markers during the assessment stage of decision making stage 1 as well as during the experience of outcome stage 3 A number of processes can contribute to disturbed assessment for example appraisal processes Mogg and Bradley 1999 encoding and recall biases Pury and Mineka 2001 Reidy and Richards 1997 Russo et al 2001 expectancy changes Chan and Lovibond 1996 or increased sensitivity to punishment Corr et al 1997 In conclusion the systems neuroscience framework based on distinct stages of decision making can provide a road map to determine which component of decision making is dysfunctional in psychiatric populations This line of investigation can prove to be critical for the development and testing of new interventions aimed to improve decision making and ensuing quality of life in impaired populations Craig 2003 Ainslie G 1992 Picoeconomics The strategic interaction of successive moti vational states within the person New York Cambridge University Press Augustine JR 1996 Circuitry and functional aspects of the insular lobe in primates including humans Brain Res Brain Res Rev 22229244 Baving L Wagner M Cohen R Rockstroh B 2001 Increased semantic and repetition priming in schizophrenic patients J Abnorm Psychol 1106775 Baxter MG Murray EA 2002 The amygdala and reward Nat Rev Neurosci 3563573 Bechara A 2004a The role of emotion in decisionmaking Evidence from neurological patients with orbitofrontal damage Brain Cogn 553040 Bechara A 2004b Disturbances of emotion regulation after focal brain lesions Int Rev Neurobiol 62159193 Bechara A Damasio AR Damasio H Anderson SW 1994 Insensitivity to future consequences following damage to human prefrontal cortex Cognition 50715 Bechara A Damasio H 2002 Decisionmaking and addiction part I Im paired activation of somatic states in substance dependent individuals when pondering decisions with negative future consequences Neuro psychologia 4016751689 Bernoulli D 1954 Exposition of a new theory on the measurement of risk Econometrica 222336 Berns GS McClure SM Pagnoni G Montague PR 2001 Predictability mod ulates human brain response to reward J Neurosci 2127932798 Berridge CW Waterhouse BD 2003 The locus coeruleusnoradrenergic system modulation of behavioral state and statedependent cognitive processes Brain Res Brain Res Rev 423384 Bjork JM Knutson B Fong GW Caggiano DM Bennett SM Hommer DW 2004 Incentiveelicited brain activation in adolescents Similarities and differences from young adults J Neurosci 2417931802 Bolla KI Eldreth DA London ED Kiehl KA Mouratidis M Contoreggi C et al 2003 Orbitofrontal cortex dysfunction in abstinent cocaine abusers performing a decisionmaking task Neuroimage 1910851094 Brebion G Smith MJ Amador X Malaspina D Gorman JM 1998 Word recog nition discrimination accuracy and decision bias in schizophrenia Associ ation with positive symptomatology and depressive symptomatology JNervMentDis 186604609 Breiter HC Rosen BR 1999 Functional magnetic resonance imaging of brain reward circuitry in the human Ann N Y Acad Sci 877523547 Breiter HC Gollub RL Weisskoff RM Kennedy DN Makris N Berke JD 1997 Acute effects of cocaine on human brain activity and emotion Neuron 19591611 Byrnes JP 2002 The development of decisionmaking J Adolesc Health 31208215 Cannon WB 1987 The JamesLange theory of emotions A critical exami nationandanalternativetheoryByWalterBCannon1927AmJPsychol 100567586 Cardinal RN Parkinson JA Hall J Everitt BJ 2002 Emotion and motivation The role of the amygdala ventral striatum and prefrontal cortex Neuro sci Biobehav Rev 26321352 Carter CS Botvinick MM Cohen JD 1999 The contribution of the anterior cingulate cortex to executive processes in cognition Rev Neurosci 10 4957 Carter CS Braver TS Barch DM Botvinick MM Noll D Cohen JD 1998 Anterior cingulate cortex error detection and the online monitoring of performance Science 280747749 Chambers RA Potenza MN 2003 Neurodevelopment impulsivity and adolescent gambling J Gambling Stud 195384 Chan CK Lovibond PF 1996 Expectancy bias in trait anxiety J Abnorm Psychol 105637647 Childress AR Mozley PD McElgin W Fitzgerald J Reivich M OBrien CP 1999 Limbic activation during cueinduced cocaine craving Am J Psy chiatry 1561118 Clark L Manes F Antoun N Sahakian BJ Robbins TW 2003 The contribu tions of lesion laterality and lesion volume to decisionmaking impair ment following frontal lobe damage Neuropsychologia 4114741483 ClarkLCoolsRRobbinsTW2004Theneuropsychologyofventralprefron tal cortex Decisionmaking and reversal learning Brain Cogn 554153 Clayton R 1992 Transitions in drug use Risk and protective factors In GlantzMPickensReditorsVulnerabilitytoDrugAbuseWashingtonDC American Psychological Association 1552 Corr PJ Pickering AD Gray JA 1997 Personality punishment and proce dural learning A test of JA Grays anxiety theory J Pers Soc Psychol 73337344 Coughlan R Connolly T 2001 Predicting affective responses to unex pected outcomes Organ Behav Hum Decis Process 85211225 Craig AD 2002 How do you feel Interoception The sense of the physio logical condition of the body Nat Rev Neurosci 3655666 Craig AD 2003 A new view of pain as a homeostatic emotion Trends Neurosci 26303307 Critchley HD Mathias CJ Dolan RJ 2001 Neural activity in the human brain relating to uncertainty and arousal during anticipation Neuron 29537 545 Critchley HD Wiens S Rotshtein P Ohman A Dolan RJ 2004 Neural sys tems supporting interoceptive awareness Nat Neurosci 7189195 Cummings JL 1995 Anatomic and behavioral aspects of frontalsubcorti cal circuits Ann N Y Acad Sci 769113 Damasio AR 1996 The somatic marker hypothesis and the possible func tions of the prefrontal cortex Philos Trans R Soc Lond B Biol Sci 3511413 1420 Damasio AR Damasio H Christen Y Neurobiology of decisionmaking Berlin and New York Springer Verlag 1996 Dehaene S Spelke E Pinel P Stanescu R Tsivkin S 1999 Sources of math ematical thinking Behavioral and brainimaging evidence Science 284 970974 DenesRaj V Epstein S 1994 Conflict between intuitive and rational pro cessing When people behave against their better judgment J Pers Soc Psychol 66819829 Di Chiara G Bassareo V Fenu S De Luca MA Spina L Cadoni C et al 2004 Dopamine and drug addiction the nucleus accumbens shell connec tion Neuropharmacology 47suppl 1227241 Dias R Robbins TW Roberts AC 1997 Dissociable forms of inhibitory con trol within prefrontal cortex with an analog of the Wisconsin Card Sort Test Restriction to novel situations and independence from online processing J Neurosci 1792859297 Dowden SL Allen GJ 1997 Relationships between anxiety sensitivity hy perventilation and emotional reactivity to displays of facial emotions J Anxiety Disord 116375 Elliott R Rees G Dolan RJ 1999 Ventromedial prefrontal cortex mediates guessing Neuropsychologia 37403411 Ernst M Bolla K Mouratidis M Contoreggi C Matochik JA Kurian V et al 2002 Decisionmaking in a risktaking task A PET Study Neuropsycho pharmacology 26682691 Ernst M Nelson EE Jazbec S McClure EB Monk CS Leibenluft E Blair J Pine DS 2005 Amygdala and nucleus accumbens in responses to receipt 6 BIOL PSYCHIATRY 2005xxxxx M Ernst and MP Paulus wwwsobporgjournal ARTICLE IN PRESS and omission of gains in adults and adolescents Neuroimage 2512791291 Ernst M Nelson EE McClure EB Monk CS Munson S Eshel N et al 2004 Choice selection and reward anticipation An fMRI study Neuropsycho logia 4215851597 Gabriel M Burhans L Kashef A 2003 Consideration of a unified model of amygdalar associative functions Ann N Y Acad Sci 985206217 Garety PA Hemsley DR Wessely S 1991 Reasoning in deluded schizo phrenic and paranoid patients Biases in performance on a probabilistic inference task J Nerv Ment Dis 179194201 Gehring WJ Knight RT 2000 Prefrontalcingulate interactions in action monitoring Nat Neurosci 3516520 Glimcher P 2002 Decisions decisions decisions Choosing a biological science of choice Neuron 36323332 Goel V Gold B Kapur S Houle S 1998 Neuroanatomical correlates of human reasoning J Cogn Neurosci 10293302 Gold JI Shadlen MN 2001 Neural computations that underlie decisions about sensory stimuli Trends Cogn Sci 51016 Gold PE 2003 Acetylcholine modulation of neural systems involved in learning and memory Neurobiol Learn Mem 80194210 Grant S Contoreggi C London ED 2000 Drug abusers show impaired performance in a laboratory test of decision making Neuropsychologia 3811801187 Grant S London ED Newlin DB Villemagne VL Liu X Contoreggi C et al 1996 Activation of memory circuits during cueelicited cocaine crav ing Proc Natl Acad Sci U S A 931204012045 Grunbaum JA Kann L Kinchen S Ross J Hawkins J Lowry R et al 2004 Youth risk behavior surveillanceUnited States 2003 MMWR Surveill Summ 53196 Heims HC Critchley HD Dolan R Mathias CJ Cipolotti L 2004 Social and motivational functioning is not critically dependent on feedback of au tonomic responses Neuropsychological evidence from patients with pure autonomic failure Neuropsychologia 4219791988 HernsteinRJ1961Relativeandabsolutestrengthofresponseasafunction of frequency of reinforcement J Exp Anal Behav 4267272 Hollander E Rosen J 2000 Impulsivity J Psychopharmacol 14S39S44 Holroyd CB Coles MG 2002 The neural basis of human error processing reinforcement learning dopamine and the errorrelated negativity Psy chol Rev 109679709 Hornak J Bramham J Rolls ET Morris RG ODoherty J Bullock PR Polkey CE 2003 Changes in emotion after circumscribed surgical lesions of the orbitofrontal and cingulate cortices Brain 12616911712 Hutton SB Murphy FC Joyce EM Rogers RD Cuthbert I Barnes TR et al 2002 Decision making deficits in patients with firstepisode and chronic schizophrenia Schizophr Res 55249257 Kahneman D Tversky A 1979 Prospect theory An analysis of decision under risk Econometrica 47263291 Kahneman D 1991 Judgment and decision making A personal view Psy chol Sci 2142145 Kahneman D Snell J 1990 Predicting utility In Hogarth RM editor Insights in decision making A tribute to Hillel J Einhorn Chicago University of Chicago Press pp 295310 Kapur S 2004 How antipsychotics become antipsychoticfrom dopa mine to salience to psychosis Trends Pharmacol Sci 25402406 Kendler KS 2001 Twin studies of psychiatric illness An update Arch Gen Psychiatry 5810051014 Knutson B Fong GW Adams CM Varner JL Hommer D 2001 Dissociation of reward anticipation and outcome with eventrelated fMRI Neurore port 1236833687 Knutson B Fong GW Bennett SM Adams CM Hommer D 2003 A region of mesial prefrontal cortex tracks monetarily rewarding outcomes Charac terization with rapid eventrelated fMRI Neuroimage 18263272 Kraepelin E Robertson GM 1919 Dementia praecox and paraphrenia Edin burgh Scotland Livingstone Lane SD Cherek DR 2000 Analysis of risk taking in adults with a history of high risk behavior Drug Alcohol Depend 60179187 Larson RW Moneta G Richards MH Wilson S 2002 Continuity stability and change in daily emotional experience across adolescence Child Dev 7311511165 Laruelle M Kegeles LS AbiDargham A 2003 Glutamate dopamine and schizophrenia From pathophysiology to treatment Ann N Y Acad Sci 1003138158 Lewontin RC 1961 Evolution and the theory of games J Theor Biol 1382 403 Lhermitte F Pillon B Serdaru M 1986 Human autonomy and the frontal lobes Part I Imitation and utilization behavior A neuropsychological study of 75 patients Ann Neurol 19326334 Liberzon I Britton JC Luan PK 2003 Neural correlates of traumatic recall in posttraumatic stress disorder Stress 6151156 LoewensteinGFWeberEUHseeCKWelchN2001RiskasfeelingsPsychol Bull 127267286 London ED Ernst M Grant S Bonson K Weinstein A 2000 Orbitofrontal cortex and human drug abuse Functional imaging Cereb Cortex 10 334342 Madden GJ Bickel WK Jacobs EA 1999 Discounting of delayed rewards in opioiddependent outpatients Exponential or hyperbolic discounting functions Exp Clin Psychopharmacol 7284293 Maia TV McClelland JL 2004 A reexamination of the evidence for the somatic marker hypothesis what participants really know in the Iowa gambling task Proc Natl Acad Sci U S A 1011607516080 Mathalon DH Whitfield SL Ford JM 2003 Anatomy of an error ERP and fMRI Biol Psychol 64119141 May JC Delgado MR Dahl RE Stenger VA Ryan ND Fiez JA Carter CS 2004 Eventrelatedfunctionalmagneticresonanceimagingofrewardrelated brain circuitry in children and adolescents Biol Psychiatry 55359366 McClure SM Berns GS Montague PR 2003 Temporal prediction errors in a passive learning task activate human striatum Neuron 38339346 Mellers BA McGraw AP 2001 Anticipated emotions as guides to choice Curr Dir Psychol Sci 10210214 MellersBASchwartzAHoKRitovI1997DecisionaffecttheoryEmotional reactions to the outcomes of risky options Psychol Sci 8423429 Mitterschiffthaler MT Kumari V Malhi GS Brown RG Giampietro VP Bram mer MJ et al 2003 Neural response to pleasant stimuli in anhedonia An fMRI study Neuroreport 14177182 Mogenson Gj Yang CR 1991 The contribution of basal forebrain to limbic motor integration and the mediation of motivation to action Adv Exp Med Biol 295267290 Mogg K Bradley BP 1999 Some methodological issues in assessing atten tional biases for threatening faces in anxiety A replication study using a modified version of the probe detection task Behav Res Ther 37595 604 Monterosso J Ehrman R Napier KL OBrien CP Childress AR 2001 Three decisionmaking tasks in cocainedependent patients Do they measure the same construct Addiction 9618251837 Nash JF 1953 Two person cooperative games Econometrica 21128140 Nitschke JB Heller W Palmieri PA Miller GA 1999 Contrasting patterns of brain activity in anxious apprehension and anxious arousal Psychophys iology 36628637 ODoherty JP Critchley HD Deichmann R Dolan RJ 2003a Dissociating valence of outcome from behavioral control in human orbital and ven tral prefrontal cortices J Neurosci 2379317939 ODoherty JP Dayan P Friston K Critchley H Dolan RJ 2003b Temporal difference models and rewardrelated learning in the human brain Neu ron 38329337 Ongur D Price JL 2000 The organization of networks within the orbital and medial prefrontal cortex of rats monkeys and humans Cereb Cortex 10206219 Pagnoni G Zink CF Montague PR Berns GS 2002 Activity in human ventral striatum locked to errors of reward prediction Nat Neurosci 59798 Passerieux C Segui J Besche C Chevalier JF Widlocher D HardyBayle MC 1997 Heterogeneity in cognitive functioning of schizophrenic patients evaluated by a lexical decision task Psychol Med 2712951302 Passingham RE Toni I Rushworth MF 2000 Specialisation within the pre frontal cortex The ventral prefrontal cortex and associative learning Exp Brain Res 133103113 Paulus MP Hozack N Frank L Brown GG Schuckit MA 2003 Decision making by methamphetaminedependent subjects is associated with errorrateindependent decrease in prefrontal and parietal activation Biol Psychiatry 536574 PaulusMPHozackNEZauscherBEFrankLBrownGGBraffDLSchuckitMA 2002 Behavioral and functional neuroimaging evidence for prefrontal dysfunction in methamphetaminedependent subjects Neuropsy chopharmacology 265363 Pavlov IP 2005 Conditioned Reflexes London Oxford University Press M Ernst and MP Paulus BIOL PSYCHIATRY 2005xxxxx 7 wwwsobporgjournal ARTICLE IN PRESS Petry NM Bickel WK Arnett M 1998 Shortened time horizons and insensi tivity to future consequences in heroin addicts Addiction 93729738 Phillips ML Drevets WC Rauch SL Lane R 2003 Neurobiology of emotion perception I The neural basis of normal emotion perception Biol Psychi atry 54504514 Platt ML Glimcher PW 1999 Neural correlates of decision variables in parietal cortex Nature 400233238 Pury CLS Mineka S 2001 Differential encoding of affective and nonaffec tive content information in trait anxiety Cogn Emotion 15659693 Redish AD 2004 Addiction as a computational process gone awry Science 30619441947 Reidy J Richards A 1997 Anxiety and memory A recall bias for threatening words in high anxiety Behav Res Ther 35531542 Rescorla RA Wagner AR 1972 A theory of Pavlovian conditioning Varia tions in the effectiveness of reinforcement and nonreinforcement In Black AH Prokasy WF editors Classical conditioning II Current research and theory New York AppletonCenturyCrofts 6499 Robbins TW 1997 Arousal systems and attentional processes Biol Psychol 455771 Roesch MR Olson CR 2004 Neuronal activity related to reward value and motivation in primate frontal cortex Science 304307310 Rogers RD Everitt BJ Baldacchino A Blackshaw AJ Swainson R Wynne K et al 1999 Dissociable deficits in the decisionmaking cognition of chronic amphetamine abusers opiate abusers patients with focal dam age to prefrontal cortex and tryptophandepleted normal volunteers evidence for monoaminergic mechanisms Neuropsychopharmacology 20322339 Rogers RD Robbins TW 2001 Investigating the neurocognitive deficits associated with chronic drug misuse Curr Opin Neurobiol 11250257 Russo R Fox E Bellinger L NguyenVanTam DP 2001 Moodcongruent free recall bias in anxiety Cogn Emotion 15419433 Salamone JD Correa M 2002 Motivational views of reinforcement Impli cations for understanding the behavioral functions of nucleus accum bens dopamine Behav Brain Res 137325 Schachter S Singer JE 1962 Cognitive social and physiological determi nants of emotional state Psychol Rev 69379399 Schoenbaum G Setlow B 2003 Lesions of nucleus accumbens disrupt learning about aversive outcomes J Neurosci 2398339841 Schultz W 1998 Predictive reward signal of dopamine neurons J Neuro physiol 80127 Schultz W 2002 Getting formal with dopamine and reward Neuron 36 241263 Schultz W Dayan P Montague PR 1997 A neural substrate of prediction and reward Science 27515931599 Shadlen MN Newsome WT 2001 Neural basis of a perceptual decision in the parietal cortex area LIP of the rhesus monkey J Neurophysiol 86 19161936 Shallice T Burgess PW 1991 Deficits in strategy application following frontal lobe damage in man Brain 114Pt 2 727741 Shallice T Burgess PW Schon F Baxter DM 1989 The origins of utilization behaviour Brain 11215871598 Shepperd JA Mcnulty JK 2002 The affective consequences of expected and unexpected outcomes Psychol Sci 138588 Skinner BF 1953 Science and Human Behavior New York Macmillan Slovic P Finucane M Peters E MacGregor DG 2002 The affect heuristic In Gilovich T Griffin D editors Heuristics and biases The psychology of intu itive judgment New York Cambridge University Press pp 397420 Spear LP 2000 The adolescent brain and agerelated behavioral manifes tations Neurosci Biobehav Rev 24417463 Taylor SF Welsh RC Wager TD Phan KL Fitzgerald KD Gehring WJ 2004 A functional neuroimaging study of motivation and executive function Neuroimage 2110451054 Tversky A Kahneman D 1981 The framing of decisions and the psychology of choice Science 211453458 Tversky A Kahneman D 1975 Judgment under uncertainty Heuristics and biases Catalog Selected Documents Psychol 5182 Ursu S Stenger VA Shear MK Jones MR Carter CS 2003 Overactive action monitoring in obsessivecompulsive disorder Evidence from functional magnetic resonance imaging Psychol Sci 14347353 van Veen V Holroyd CB Cohen JD Stenger VA Carter CS 2004 Errors without conflict Implications for performance monitoring theories of anterior cingulate cortex Brain Cogn 56267276 Vinogradov S Ober BA Shenaut GK 1992 Semantic priming of word pro nunciation and lexical decision in schizophrenia Schizophr Res 8171 181 Volkow ND Fowler JS 2000 Addiction a disease of compulsion and drive involvement of the orbitofrontal cortex Cereb Cortex 10318325 Wang GJ Volkow ND Fowler JS Cervany P Hitzemann RJ Pappas NR et al 1999 Regional brain metabolic activation during craving elicited by recall of previous drug experiences Life Sci 64775784 Wilken JA Smith BD Tola K Mann M 2000 Trait anxiety and prior exposure to nonstressful stimuli Effects on psychophysiological arousal and anx iety Int J Psychophysiol 37233242 Wise RA 1996 Neurobiology of addiction Curr Opin Neurobiol 6243251 Zec RF 1995 Neuropsychology of schizophrenia according to Kraepelin Disorders of volition and executive functioning Eur Arch Psychiatry Clin Neurosci 245216223 Zeelenberg M Beattie J van der Plight J de Vries NK 1996 Consequences of regret aversion Effects of expected feedback on risky decision mak ing Org Behav Hum Decision Process 65148158 Zeelenberg M van Dijk WW van der Pligt J Manstead ASR van Empelen P Reinderman D 1998 Emotional reactions to the outcomes of decisions The role of counterfactual thought in the experience of regret and disappointment Organ Behav Hum Decis Process 75117141 Zink CF Pagnoni G MartinSkurski ME Chappelow JC Berns GS 2004 Human striatal responses to monetary reward depend on saliency Neu ron 42509517 8 BIOL PSYCHIATRY 2005xxxxx M Ernst and MP Paulus wwwsobporgjournal ARTICLE IN PRESS