University of Southern California Dissertations and Theses

Behavioral and neural evidence of incentive bias for immediate rewards relative to preference-matched delayed rewards

(USC Thesis Other)

Behavioral and neural evidence of incentive bias for immediate rewards relative to preference-matched delayed rewards

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

BEHAVIORAL AND NEURAL EVIDENCE OF INCENTIVE BIAS FOR
IMMEDIATE REWARDS RELATIVE TO PREFERENCE-MATCHED DELAYED
REWARDS

by
Shan Luo

A Thesis Presented to the
FACULTY OF THE GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
MASTER OF ARTS
(PSYCHOLOGY)

December 2009

Copyright 2009 Shan Luo
ii

Acknowledgements

I am grateful to my advisor Dr. John Monterosso for leading me to the path of
studying underlying mechanisms of human self-control. His unconditional support and
guidance made me finish this project; to our collaborator Dr. George Ainslie for helpful
advice and input; to my project assistants Lisa Giragosian, Xochitl Cordova and Jodi
Stone for their help with data collection; to Dr. Gui Xue for his teaching how to use FSL
and thoughtful input for experimental design and imaging data analysis.

iii

Table of Contents

Acknowledgements ii

List of Tables iv

List of Figures v

Abstract vi

Chapter 1: An argument against dual valuation system competition 1
When maximization stopped satisfying behavioral science 1
Intertemporal choice and rational choice theory 2
What does intertemporal choice have to do with impulsivity and
self-control? Synchronic multiple selves models 6
Neuroimaging research on intertemporal choice: The case for
dual valuation 9
Neuroimaging research on intertemporal choice: The case
against dual valuation 13
Framing effects and self-signaling: The basics of a diachronic
multiple self model 15
Distinguishing specified contingencies from prevailing
contingencies 21
The future of interteporal choice neuroimaging research:
Enough with the money already 22

Chapter 2: Delay discounting for anticipated rewards 26
Introduction 26
Materials and methods 27
Results 37
Discussion 48
Conclusions 52

References 55

iv

List of Tables

Table 1: Anatomical Region of Interest Analysis Result 40

Table 2: Whole Brain Analysis Result 43

v

List of Figures

Figure 1: Reaction Time Data in Monetary Incentive Delay Task 39

Figure 2: 95% Confidence Interval of the Effect Sizes for the
Magnitude Effect and Immediacy Effect 41

Figure 3: Contrast Maps for Immediate-Delayed Rewards 42

Figure 4: Contrast Map for High-Low Rewards 44

Figure 5: Parametric Analysis Result 45

Figure 6: Conjunction Maps for High-Low and Immediate-Delayed 46

Figure 7: Connectivity Analysis Result 47

vi

Abstract
There are two classes of self-control models in behavioral science: “synchronic
multiple selves” and “diachronic multiple selves”. The former class conceptualizes self-
control struggle as competition between distinct motivation systems (economists call
these “subagents” within the person.). The latter conceives self-control struggle as based
on systematic inconsistency in preference over time (retaining the self as a unitary agent
at any point in time). My master thesis comprises two papers. The first one reviews
recent functional Magnetic Resonance Imaging (fMRI) work in the domain of
intertemporal choice, specifically considering “synchronic multiple selves” hypothesis
that delay discounting is determined by the competition between an evolutionarily older
system that discounts precipitously with delay (System 1) and a newer system that
exhibits very little discounting (System 2). I argue that neuroimaging evidences do not
support this separate and competing valuation systems. Rather than acting as a competing
value system, I argue that sophisticated cognitive capacities instantiated in the neocortex
affect intertemporal choice through mediation of a single valuation system, instantiated
within the extended limbic system.
The second paper is an empirical investigation of a prediction that is common to
any theory of self-control that posits that one or more processes engaged specifically
during decision-making results in increased attention to more far-sighted consequences.
Specifically, I tested (and found support for) the hypothesis that intertemporal decisions
are more farsighted than would be predicted by the incentive associated with immediate

vii
and delayed rewards encountered outside a decision context (that is, outside a context
where self-control is engaged).
1

Chapter 1: An argument against dual valuation system competition
When maximization stopped satisfying behavioral science
Rational Choice Theory (RCT) dominated the behavioral sciences throughout
most of the 20
th
century (Arrow, 1951). RCT models the individual as maximizing her
utility based on a stable set of preferences (Becker, 1976). From the perspective of
neuroscience, it is astonishing that this model holds as well as it does; somehow the
immeasurable complexity of the living human brain achieves functionality that is not all
that far from the rational pursuit of a stable set preferences. That said, systematic
irrationality in intertemporal choice and other domains has been well documented and
there is growing dissatisfaction with RCT. Of course, demonstration of systematic
irrationality is insufficient basis for abandoning RCT; as economist Milton Friedman
noted:
Complete realism is clearly unattainable, and the question whether a theory is
‘realistic enough’ can be settled only be seeing whether it yields predictions that
are good enough for the purpose in hand or that are better than predictions from
alternative theories (Friedman, 1971, page 51).

Attempts to find an alternative that makes better predictions than RCT have
generally tried in one way or another to increase the psychological realism of choice
models (Edwards, 1953; Kahneman & Tvesky, 1979; Simon, 1967). These efforts
incorporate functional constructs intermediate to the contingencies in the environment
and the individual’s choice (e.g., emotions and cognitions) that serve as psychological
gears within the human decision-making apparatus. The trend has been towards increased
collective confidence that the payback in modeling accuracy justifies this massive
2

complication. The rapid emergence of “neuroeconomics” reflects the possibility that we
are on cusp of a new development in this program; grounding behavioral theories in
neural realism. Perhaps consideration of the physical gears within the human decision-
making apparatus will lead to better behavioral science, at least in those areas in which
RCT is most faulty. In this chapter, I look specifically at this issue with respect to
intertemporal choice, giving special consideration to the widely discussed idea of dual
value systems (Loewenstein, 1996; McClure, Laibson, Loewenstein, & Cohen, 2004;
McClure, Ericson, Laibson, Loewenstein, & Cohen, 2007;). Whether or not the dual-
valuation systems hypothesis is correct, I think it is illustrative of how the neural systems
perspective could, in principle, contribute something important to behavioral science.
Intertemporal choice and rational choice theory
There has been intense investigation of the relationship between immediacy of
consequences and motivation (the term “intertemporal choice” is primarily used in
neuroscience, “delay discounting” is primarily used in behavioral psychology, and “time
preference” is primarily used in economics.). Research on “intertemporal choice”
generally requires participants to express preference between a narrow set of alternatives
(generally 2 alternatives) that form a conspicuous “sooner smaller” versus “later larger”
contrast (e.g., $5 today or $10 in a month). Rationality with respect to delay has been
conceived in two ways; the first, which I think is in line with common sense, conceives
the perfectly rational agent as unaffected by delay per se (though of course opportunity
costs and any associated uncertainty are rational considerations by any reasonable

3

standard.). As Jevons put it while characterizing the rational ideal, “all future pleasures or
pains should act upon us with the same force as if they were present.”(Jevons, 1871, page
76). The second conception allows that the rational actor devalue delayed outcomes (a
preference for immediacy of good outcomes and delay of bad outcomes), provided that
the devaluation occurs at a fixed rate per unit of time (Friedman, 1963). Although such
“exponential discounting” can be of any rate and is thus, I think, inconsistent with the
common sense idea of a rational orientation to delay, it is attractive to rational choice
theorists because it allows delay to affect motivation, while arguably introducing no
violation of RCT’s axioms. Present value for a stream of consumptions (c1, c2, . . .)
discounted exponentially can be represented as :
Equation 1

where u is the utility function, with lower values of the discount parameter δ indicating
steeper exponential discounting.
Exponential discounting introduces no dynamic inconsistency in preferences. The
exponential discounter can make a plan with the expectation that she will follow it unless
the circumstances upon which she based her plan shift in an unanticipated way. Her past
and future selves harmoniously pursue preferences in an unbroken chain; their success
limited only by imperfect information or power. But human and non-humans decision-
making does not conform to this standard. On the contrary, there are conditions under
which most subjects reverse their initial preference for larger later rewards when smaller
4

sooner alternatives are near at hand. Furthermore, when given the opportunity, subjects
choose to lock in their larger later preference while it is still the most appealing
(dominant) option in order to protect against an anticipated future self with alternate
preferences (Ainslie, 1974; Schelling, 1978; Shefrin & Thaler, 1992; Strotz, 1956). In
this way, behavioral discounting data look systematically non-exponential, although
opinions differ as to the precise alternative functional form that best fits behavioral data
(Green & Myerson, 2004). One widely considered alternatives is hyperbolic discounting
(discounting that is proportional to delay) (Strotz 1956; Ainslie, 1975, 1992, 2001).
Present value for a stream of consumptions (c1, c2, . . .) discounted hyperbolically can be
represented as:
Equation 2

where u is the utility function, with higher values of the discount parameter k indicating
steeper hyperbolic discounting. A second proposal to account for preference reversals is
quasi-hyperbolic discounting. In quasi-hyperbolic discounting, a ballistic devaluation
occurs in the presence of any delay, but the effect of additional delay is modeled
exponentially (Laibson, 1997). Present value for a stream of consumptions (c1, c2, . . .)
discounted quasi-hyperbolically can be represented as:
Equation 3

5

where u is the utility function, and parameters β and δ both serving as discount
parameters. Note that β does not affect the immediate period in the stream of
consumption; lower values of β indicate a greater devaluation of all outcomes that are not
immediate. The additional effect of delay beyond this discontinuity (δ) is exponential.
In both, hyperbolic and quasi-hyperbolic discounting, the difference in value
between immediacy and some delay, d is proportionally greater than the difference
between some positive delay, X and X + d. As a result, both introduce time
inconsistency, though for quasi-hyperbolic discounting, this inconsistency is restricted to
situations involving immediacy relative to situations without immediacy. As I will
consider below, the discontinuity inherent in quasi-hyperbolic discounting lends itself
well to a dual-process account of the neural basis of temporal discounting.
The lack of resolution regarding modeling alternatives is related to the enormous
variability in discounting observed within individuals across contexts (Chapman, 2000),
between individuals in the same context (Ainslie & Monterosso, 2003a), and across
different research studies (Frederick, 2002). As a result, there is now considerable
skepticism that any model of delay discounting will be consistently adequate. As
Roelofsma and Read put it:
The study of intertemporal choice is currently undergoing a change in emphasis,
as has already occurred in the study of decision making under risk and
uncertainty. Rather than searching for the holy grail of a single utility function,
researchers now take the more pragmatic view that preferences are constructed
based on the circumstances of their expression (Roelofsma & Read, 2000, page
172 ).

I will return later to the issue of endemic variability in intertemporal choice behavior.

6

What does intertemporal choice have to do with impulsivity and self-control? Synchronic
multiple selves models
I think that much of the interest in delay discounting relates to the underlying
expectation that it bears importantly on self-control problems including overspending,
overeating, under-exercising, and addiction. But given the emotionally ‘cool’ nature of
the typical intertemporal choice assessment (would you prefer $5 today or $10 in a
month?), it is not self-evident that they would be associated with real self-control
phenomena. The hypothesized association between intertemporal choice measures and
self-control turns on the researcher’s conceptualization of the nature of self-control
struggle. With rare exception (Becker & Murphy, 1988), behavioral scientists that model
self-control problems either 1) drop the RCT modeling assumption that the relevant agent
is a whole unified person and instead model her as the product of conflicting sub-person
agents(e.g., Shefrin & Thaler, 1992) or 2) drop the modeling assumption that the agent is
unified over time (Ainslie, 1992, 2001; Ross, 2005; Schelling, 1978, 1980, 1984; Strotz,
1956) and instead model her as the product of a temporal series of agents, sometimes
engaged in strategic opposition to each other. Following Ross (Ross, In press) I refer in
turn to these two classes of self-control models as “synchronic multiple selves” and
“diachronic multiple selves” models. I presently focus on synchronic multiple-self
models, and will return later diachronic models.
Purely “synchronic” models conceptualize self-control struggle as occurring
between distinct ‘sub-agents’ within the person, that differentially value the same
outcome. Although there is no reason why synchronic multi-self models are restricted to
7

two sub-person agents, most in fact do include just two, sometimes referring to a “System
1” that is evolutionarily older, that is instantiated largely in limbic and paralimbic brain
structures, that works faster and without requiring attention, and that is more stimulus-
bound and present-focused, and a “System 2” that is evolutionarily recent, that is
instantiated in the neocortex, that is generally slow and demanding of attention, and that
utilizes more abstract representations of the environment and so is capable of future-
oriented behavior (Evans, 2008).
Freud’s ego and id were so conceived, with each operating by its own principles
and often at odds. Thaler and Shefrin proposed to capture self-control phenomena with a
model of the individual as including “two sets of coexisting and mutually inconsistent
preferences: one concerned with the long run, and the other with the short run.” (Shefrin
& Thaler, 1992, page 291).
1
Loewenstein’s “visceral motivations” model is also in this
spirit, since visceral motivations sit apart from non-visceral motivations, and so imply
synchronic multiple selves (Loewenstein, 1996). Metcalf and Mischel’s analysis of the
“dynamics of willpower” posits multiple agents; one the manifestation of a cool,
cognitive "know" system, the other a hot emotional "go" system (Metcalfe & Mischel
1999). The strength-model of self-control (Baumesiter, Vohs, & Tice, 2007) implies a

1
Indeed they argued that their theory was, “roughly consistent with the scientific
literature on brain function... The planner in our model represents the prefrontal cortex.
The prefrontal cortex continually interfaces with the limbic system, which is responsible
for the generation of emotions. The doer in our model represents the limbic system. It is
well known that self-control phenomena center on the interaction between the prefrontal
cortex and the limbic system (Restak, 1984)." (Shefrin & Thaler, 1992, page 291).
8

dual system as well, since there is a presumed set of motivations that the “willpower
muscle” preferences, and an alternative set that it does not.
From the perspective of these synchronic multi-agent models, delay discounting is
generally viewed as one relevant factor, but neither as the primary source of impulsivity,
nor as the key to understanding mechanisms of self-control. Instead, in purely synchronic
models, time inconsistency is one manifestation of the underlying competition between
sub-person agents with competing preferences, and self-control maps directly as the
outcome of that competition. In a purely synchronic model of self-control an act entails
that a certain sub-agent or sub-agents prevails over others (Freud’s “ego” prevailing over
“id”, Thaler’s “planner” over “doer”, Loewenstein’s “non-visceral” over “visceral”
motivations, Metcalf and Mischel’s “know-system” over “go-system”, the strength
model’s “ideals” over “passions.”).
From the standpoint of neuroeconomics, the possibility that there are two separate
valuation systems within the brain is attractive. In addition to creating clear intervention
targets (e.g., intervening to shift the balance between the systems to treat self-control
problems), it suggests that models of individual behavior could be dramatically improved
by finding a way to develop two sets of utility functions for the individual. This would
not, I think, be mere “neuro-fication” of the timeless rational versus emotional-self
model, since it would suggest incorporating neuroscience methodologies to efficiently
solve for the set of parameters in whatever underlying dual-system utility function was
used. For instance, if the dual-system discounting function included a weighting
parameter to capture the strength of the contribution of each system, the weighting
9

parameter might be informed by gross neural markers like metabolism, connectivity,
gray-matter density, tonic and phasic striatal dopamine, receptor availability, etc.
Neuroimaging research on intertemporal choice: The case for dual valuation
There have been a series of reports combining an intertemporal decision-making
task with fMRI. Thus far, the highest impact papers among them (Kable & Glimcher,
2007; McClure, et al., 2004; McClure, et al., 2007) have focused on assessing the
validity of a synchronic multiple self perspective, and I will discuss these in some detail.
McClure et.al 2004 (McClure, et al., 2004) conducted the first study pairing the typical
methodology of intertemporal choice investigation with fMRI. In their study, the rewards
at stake for participants were gift certificates for Amazon.com, ranging from $5 to $40.
The smaller sooner option varied in delay between the same day ("today") and 4-week
delay; the larger later option was always either 2 or 4 weeks after the smaller sooner
option. Subjects received one of their certificates at the specified delay chosen, but did
not find out which one until after the end of the test. The analytic approach taken was
directed at assessing whether the modeling duality of quasi-hyperbolic discounting
(Equation 3) might correspond to an underlying duality in neural substrates of valuation.
Recall that Quasi-hyperbolic discounting models behavior with two fit parameters, β
(beta) which represents a ballistic devaluation with any delay, and δ (delta), which
represents additional exponential discounting that is continuous over delay. The authors’
were interested in the possibility that these two parameters relate to two separate
valuation systems:

10

Our key hypothesis is that the pattern of behavior that these two parameters
summarize, β, which reflects the special weight placed on outcomes that are
immediate, and δ, which reflects a more consistent weighting of time periods,
stems from the joint influence of distinct neural processes, with β mediated by
limbic structures and δ by the lateral prefrontal cortex and associated structures
supporting higher cognitive functions (McClure, et al., 2004, page 504).

Rather than seeking to identify regions that reflected value at the time of choice, the
analytic approach was taken to identify regions that were preferentially recruited during
choices that included an immediate alternative. Consistent with their hypothesis, this
pattern was observed in a set of regions that included parts of limbic and paralimbic
systems (“beta regions”). In contrast, lateral prefrontal cortex and posterior parietal
cortex were active while subjects were making choices irrespective of delay (and so
labeled “delta–regions”). They also observed greater activity in limbic/paralimbic regions
associated with choosing smaller sooner rewards, whereas greater activity in fronto-
parietal regions was associated with choices of larger later rewards. Thus the authors
viewed their results as suggesting not only the neural basis of delay discounting’s
functional form, but also a more general synchronic multiple-selves account of self-
control, arguing that behavior is “… governed by a competition between lower level,
automatic processes that may reflect evolutionary adaptations to particular environments,
and the more recently evolved, uniquely human capacity for abstract, domain-general
reasoning and future planning.”(McClure, et al., 2004, page 506).
There is something on its face odd, however, about the idea of a valuation system
only concerned with immediate reward that is recruited by the prospect of an immediate
gift certificate to Amazon.com. The participant’s stream of consumption was not likely to
change any time soon by obtaining an immediate gift certificate. And the idea that the
11

non-exponential component of discounting is a function of a categorical devaluation of
anything that is not immediate is inconsistent with behavioral data (Green & Myerson,
2004) and conceptually problematic since all goal-directed behavior, by definition, entails
delayed reward. The fact that the shortest delay used in the study was 2 weeks makes it
difficult to discern a continuous effect of delay if discounting is relatively steep (Ainslie
& Monterosso, 2004).
In order to address these concerns, they conducted a follow-up study (McClure, et
al., 2007) that used primary rewards with clear points of consumption; thirsty subjects
were asked to decide between receiving smaller sooner and larger later amounts of fruit
juice (or water), with delays ranging from 0 to 25 minutes. In addition to quasi-
hyperbolic discounting, the researchers considered a “continuous-time generalization”
variant of quasi-hyperbolic discounting in which value in the beta system was
exponentially related to delay, but at a steeper rate than the delta-system. Present value
for a stream of consumptions (c1, c2, . . .) discounted accordingly can be represented as
the double-exponential discount function:

Where u is the utility function, and discount parameter β and δ are bounded
between 0 and 1, with lower values indicating steeper exponential discounting for each of
the hypothesized systems, and a weighting parameter w, also bounded between 0 and 1,
which serves to parameterizing the relative contribution of beta and delta systems.

12

Although discounting is exponential within system, the aggregated result is non-
exponential, since the differential rates of the two systems means the net effect of a unit
of delay is not uniform across delays. Again they reported fronto-parietal activation that
was present during decision-making regardless of immediacy, and limbic and paralimbic
activation when an immediate or near immediate reward was possible. However there
was little voxel-level overlap in the identified beta-regions across the studies which may
be related to different reward types used, or alternatively could relate to timing
differences across the studies (Lamy, 2007). Their analysis indicated that the identified
beta system did not respond to juice that was delayed by 5 or more minutes, which they
point out is odd given that the gift cards that recruited beta-system activity in their 2004
study were not even physically received until an hour after the experiment (and of course,
impacted consumption much later). They conjectured that 1) primary rewards are
generally discounted more steeply than secondary rewards (an idea with considerable
empirical support as reviewed in Ainslie and Monterosso 2003 (Ainslie & Monterosso,
2003b) )and 2) secondary rewards are likely to be more subject to framing effects than
primary rewards, such that framing the Amazon gift card as immediate recruits the beta
system even though the realization of the reward is not literally immediate. Although I
think this is plausible, I believe that the idea that framing effects are reflected in beta
system activity hints at an integration between delta and beta systems that contrasts with
the dual valuation model as articulated by the authors. I will return to this idea
subsequently. Unlike in the previous study, the investigators did not find evidence that

13

activity in beta and delta systems predicted behavior, when controlling for the content of
the alternatives, but nevertheless conclude that their findings support the idea that
intertemporal choice, and behavior more generally, “reflects the interaction of
qualitatively different systems” (McClure et al., 2007, page 5802).
Neuroimaging research on intertemporal choice: The case against dual valuation
A critical objection raised with regard to McClure et al. 2004’s interpretation of
their data relates to the issue of value (Kable & Glimcher, 2007). In McClure et al. 2004,
the authors interpret greater activity recruited by choices involving an immediate
alternative as evidence that the region is part of the beta system. However, the alternative
single value-system account also predicts that choices that include an immediate
alternative will recruit more activity than those that do not in any region associated with
value. Since the absence of delay results in higher valuation for the same monetary
amount, without taking additional steps to offset the discounting effect, pairs with an
immediate alternative will tend to have higher value than pairs without. Thus theorists
favoring a single value system would suspect that the observed “beta regions” are
actually all regions associated with value, and observed delta regions are those related to
processes apart from valuation that are involved in performing the task (Kable &
Glimcher, 2007). Presumably it was with this objection in mind that a secondary analysis
was included in McClure et al. 2004, in which the primary contrast was repeated after
controlling for estimated value, with the main results unchanged (see footnote 29).
However, value in this secondary analysis was estimated using an exponential discount

14

rate (7.5% per week). If the actual discounting with delay was hyperbolic, their procedure
would tend to leave positive residual value in choice pairs that included an immediate
alternative. And so their re-analysis did not rule out the most plausible single-valuation
system model.
Kable and Glimcher 2007 reported on another intertemporal choice fMRI study,
in which the results did not suggest any duality of valuation (Kable & Glimcher, 2007).
In this study, the immediate reward was $20 now in all trials, and delayed options were
constructed using one of six delays (6h-180d) and one of six amounts ($20.25-$110).
Delays were the same for every subject but changed across sessions; amounts were
individually chosen for each subject based on the previous behavioral sessions to ensure
an approximately equal number of immediate and delayed options were chosen. The
authors took a “psychometric-neurometric” comparison approach in which they assessed
whether delay and amount (external variables) influence both psychophysical and
neurobiological measurements in a similar manner (Kable & Glimcher, 2007).
Participant’s behavioral data were well-captured by hyperbolic discounting model
(Equation 2). Moreover, when neuroimaging data were compared to value inferred from
behavior and modeled using Equation 2, there was a striking correspondence. Increases in
activity in limbic and paralimbic regions including the ventral striatum, medial prefrontal
cortex and posterior cingulate cortex (the beta-system regions of McClure et al. 2004,
2007) appeared to track subjective value. The authors conclude that their findings, falsify
the hypothesis that the limbic and paralimbic regions form an impulsive neural system
that contributes one of two inputs relevant in overall valuation; “beta regions do not even
15

primarily value immediate rewards, as the value implied by neural activity is not more
impulsive than the person's behavior, as the beta-delta hypothesis requires.” (Kable &
Glimcher, 2007, page 1631). Rather these data appear to show that the “beta system”
identified in McClure et al. 2004 track subjective value at all delays. Further, although
not directly measured in these data, I believe these data suggest greater integration
between the limbic system and the higher-order associative cortical structures that
support good decision-making in human (Damasio, 1994). This hypothesis can be more
directly examined using connectivity analysis (Hare, Camerer & Rangel, 2009).
Framing effects and self-signaling: The basics of a diachronic multiple self model
The “value-integration” alternative to dual-valuation is compatible with the
possibility that there are important dissociations related to neural systems that will inform
behavioral science. However, value-integration entails that those dissociations feed
ultimately into a single motivational system that ultimately selects action (Montague &
Berns, 2002). Here I briefly develop a diachronic multiple-selves model, and suggest
some implications for neuroeconomics.
In contrast to synchronic multiple self models, purely diachronic multiple self
models do away only with the idea that the agent is consistent over time. As a result of
non-exponential discounting the individual’s preferences at any moment in time may be
threatened by her future self’s foreseeably conflicting preferences (Ainslie, 1992, 2001;
Ross, 2005; Schelling, 1978, 1980, 1984; Strotz, 1956). She can at one moment prefer to
lose weight, while at the same time know that her future self may choose to overeat. She
may take “precommiting” action to secure her current preference, for instance spending
16

her money on a wardrobe two sizes too small, or undergoing gastrointestinal bypass
surgery. But such actions are last resorts – the approach that is likely tried many times
before precommitment is to simply resolve to a particular course, such as eating less.
“Resolving” is not merely planning, in that it entails something more – something
directed at the foreseen uncertainty that one’s own future self will abide. I think it is not
possible to develop a reasonable program of research for neuroeconomics in the area of
intertemporal choice and of self-control without simultaneously developing a functional
account of “resolving”. I believe there is a compelling account (Ainslie 1975, 1992,
2001; Benabou & Tirole, 2004; Bodner & Prelec, 2001) to which I now turn.
Recall that the findings of McClure et al. 2004 suggest that framing makes a
difference in the effect that delay has on motivation; Amazon gift cards available
“immediately” recruited substantially more limbic activity than those available a few
days subsequently. What if a similar experiment were conducted that framed immediate
gift-certificates as ‘prizes that could be received in the mail in a few days’ and what if
that experiment also included other prizes that could be selected immediately from a lab
store? Based on previously reported effects in the behavioral discounting literature
(Lowenstein, 1988) and of fMRI research looking at framing effects related to Prospect
Theory (Breiter, Aharon, Kahneman, Dale, & Shizgal, 2001), I expect that the
“immediate gift certificate” would illicit less activity in regions associated with reward
value, which is to say I agree with the interpretation offered in McClure et al. 2007 that
their earlier findings related to the framing of some rewards as immediate. Framing
effects that violate RCT are easy to generate (Kahneman & Tvesky, 1979). Importantly,
17

framing effects need not be imposed upon the individual. As elegantly demonstrated in
work on “mental accounts” (Thaler, 1980), the individual actively generates her own
framing effects. For example an individual may decide that she will treat one source of
money as untouchable, and another source as “fun money”, despite the reality that money
is fungible.
Self-generated framing effects take us one step in the direction of a diachronic
multiple-selves model of self-control. Given the realization that one’s future self may
thwart current preference, self-generated framing effects can be a strategic gambit. For
example, someone struggling to save money may cultivate a “penny saved is a penny
earned” orientation by keeping track of money saved relative to her former careless
spending habits. Strategic framing effects can also capitalize on the fact that goal-directed
behavior can be conceived on a continuum from the molecular (e.g., trying to hit a nail
with a hammer) to the molar (e.g., trying to build a house) (Rachlin, 1995). Indeed self-
control conflict can be conceived as a clash between preference at molecular and molar
levels (Barnes, 1984). For example, one may want this particular cigarette (a molecular
framing), but prefer being a non-smoker to a smoker (a molar framing). While it has been
argued that the distinction is essentially cognitive, with molar level conception something
that is learned (Rachlin, 2000), a more plausible alternative is that in a time where
temptation is low, the individual may make an effort to “stamp-in” a global frame for her
future self. For example, a mother trying to quit smoking, in a moment when the desire to
quit is strong, may decide to frame abstinence as an expression of her love for her child.
In so doing, she is attempting to cast the contingencies her future self experiences in a
18

highly global frame. Resolutions can be viewed this way; a resolution to quit smoking
places any subsequent particular cigarette (which alone carries minimal health cost) into a
context in which, if the frame sticks, the goal of being a non-smoker is implicated by the
particular decision of whether to smoke the next tempting cigarette (Monterosso &
Ainslie, 1999).
But what is the force that serves to make a strategic self-generated framing effect
stick? What prevents the fore-mentioned mother from deciding to reframe the craved
cigarette at a molecular level (just this one)? Often this is, I think, precisely what
happens. But sometimes resolutions succeed. Momentum may play a role (Nevin &
Grace, 2000; Rachlin, 2000), but the potential reward for abandoning a molar frame may
be very high, and so there needs to be more force to explain how resolutions ever
succeed. The key idea, as formulated by Ainslie (Ainslie, 1975, 1992, 2001) is that the
behavior cannot help but be treated by the individual as a test-case. If I have, as Mark
Twain quipped, quit smoking hundreds of times, and I recently resolved to never smoke
again, is it credible that I will quit tomorrow if I break yesterday’s resolution and smoke
today? Having strategically framed the contingency in the past, the individual may be
saddled with the conception that current behavior holds importance to the molar goal.
Economists have incorporated this idea into formal modeling under the label of “self-
signaling” ( Benabou & Tirole, 2004; Bodner & Prelec, 2001). The idea is that, in part as
a function of one’s history of resolution-making, the individual case takes on additional
meaning and utility. In addition to the benefits and the cost of one cigarette, my own
behavior serves as a signal that informs my expectation of whether I will be a smoker in
19

the long-run. The probabilistic signal, multiplied with the utility of whatever the signal
refers to, thereby contributes to the overall utility. So if one’s framing of not smoking as
an act of motherly love prevails, then smoking takes on considerable additional disutility.
A complication here is, I think, that the cost of the frame is not independent of its
likelihood of prevailing. When there is a high reward to be gained by molecular framing
of the contingencies, there is a greater chance that molecular framing will prevail. And
so the model includes an internal feedback between framing and valuation that is difficult
to model. While unfortunate for behavioral scientists, this cannot be taken as evidence
against its correctness.
From this point of view, there are at least two important places to consider in
developing the neuroeconomics of self-control. There is 1) behavior at the time of
resolving, and there is 2) behavior at the time at which prior strategic behavior may or
may not have its effect (that is, the time of temptation). I think both of these are
dependent upon the sort of sophisticated cognitive capacities that get classified as
“executive functions” and that depend heavily upon multimodal associative cortex. Any
resolution depends on sophisticated cognitions that if not linguistic, are at least related to
the abstraction that makes language possible. A resolution requires abstracting a
category (e.g., smoking), holding a preference with regard to the category, and foreseeing
its vulnerability. And of course, all strategic reframing deployed at the time of making a
resolution is not equal – some may even backfire (Ainslie, 1992). At the time of
temptation, I think again executive functions are likely to play a critical role. Molar and
molecular frames are likely asymmetrical with respect to their dependence upon high-
20

order cognition, because molecular frames (e.g., this one cigarette) are more directly
connected to the stimulus (e.g., the cigarette in view), and so are boosted by bottom-up
associations. Molar frames (e.g., interpreting smoking as related to one’s commitment to
motherhood) get less help from the immediate stimulus and so depend upon
remembering, and perceiving the applicability of past resolutions. In addition, there may
be an important role for interference control, whereby top-down processes bias attention
to prevent a currently dominant molar framing of the contingencies from being
overturned by the local framing that would otherwise be suggested by bottom-up
associations. It was this that William James had in mind when he wrote:
Effort of attention is thus the essential phenomenon of will. Every reader must
know by his own experience that this is so, for every reader must have felt some
fiery passion's grasp. What constitutes the difficulty for a man laboring under an
unwise passion of acting as if the passion were wise? Certainly there is no
physical difficulty.... The difficulty is mental: it is that of getting the idea of the
wise action to stay before our mind at all (James, 1890, page 565).

The above account of self-control makes many predictions that are in line with
some of those made by the beta-delta account. On the above, self-control is expected to
relate in part to one’s capacity for high-level cognition (a proposition that is supported
both by the developmental literature and by the individual difference literature).
Moreover, taxing of high-order cognitive capacity with a dual task manipulation or
through fatigue (Baumeister & Heatherton, 1996), or through chronic drug use (Goldstein
& Volkow, 2002) would be expected to lead to self-control lapses. But while the
predictions are highly overlapping with those of the beta-delta System 1 versus System 2
models, the present model maintains that higher-level processes have their impact by
21

brokering reward within a single motivation marketplace. I think this account is
consistent with the exciting recent finding reported by Hare, Camerer and Rangel 2009
(Hare, Camerer, & Rangel, 2009) that health considerations only sometimes affected
choice and signal change in the VMPFC, and whether or not they did was at least
statistically mediated by the presence of more activity in the DLPFC.
Distinguishing specified contingencies from prevailing contingencies
The perspective on self-control outlined above suggests a methodological
problem for behavioral scientists using the delay-discounting construct. As discussed
above, the enormous variability within and between subjects has suggested to some that
there is no single function that can represent a subject’s basic discount rate (Loewenstein
& Prelec, 1992). Human delay discounting varies markedly from one domain to another
in the same individual, so that, for example, an individual’s level of discounting money
scarcely predicts her level of discounting of health outcomes (Chapman, 1996). This
undercuts the idea that estimates of delay discounting derived from a monetary choice
procedure (or any other single reward procedure) provide a general index of impulsivity
that can be used to predict behavior in other domains (Roelofsma & Read, 2000). The
framing effects discussed above may be the underlying basis of the problem. Among
humans the contingencies perceived to be at stake go well beyond the outcome of the
choice at hand. Just as resolutions and other strategic framing effects may lead to the
formation of rules about smoking, they can lead to tacit or explicit rules about monetary
trade-offs between delay and amount, choices between short term and long term personal
relationships, and even judgments about whether an impulse is consistent with the
22

person’s perception of her character. When a person structures her choices in relation to
resolutions or more subtle strategic reframing, she can be expected to express different
preferences than she would if she were making a choice just on the basis of its own
merits, and these preferences are apt to differ among categories of reward, according to
their temporal distribution, emotional relevance, impulse control history, and doubtless
many other factors. The experimenter is blind to all this, and the computation of k (or
whatever parameters are included in the investigator’s model) assumes unrealistically that
the prevailing contingencies are precisely those specified by the alternatives presented to
the participant. A subject’s choice of $10 delayed by a month over $8 today may carry
the additional utility of signaling to the self that she is a financially responsible person,
but there is no easy way to incorporate this framing effect, and so the researcher is left
inferring that her discounting must occur in a way that leaves $10 retaining more than
80% of its value given 1 month delay. The reality may be that in the absence of framing
effects, the immediate alternative would be more valued. Related, in our lab we are
currently looking at the possibility that valuation assessed in a non-choice context (where
presumably, such self-signaling is not applicable) shows greater delay discounting than
appears when inferred based on choices. That is, we are interested in the possibility that,
for instance, signal change in brain regions that respond in step with value might indicate
greater value for the immediate $8 than the delayed $10 in the same participant that
choses the delayed $10.
The future of interteporal choice neuroimaging research: Enough with the money already

23

I know of 11 papers relating intertemporal choice behavior to brain activity as
indexed by fMRI. In brief, some other findings of note include associations between
variability of activity on individual trials and choice, with 1) greater neural activity in
posterior parietal cortex, dorsal prefrontal cortex and right parahippocampal gyrus
associated with choosing more immediate rewards (Boettiger, et al., 2007) while greater
activity in LOFC (Boettiger, et al., 2007) and insula (Wittmann, Leland, & Paulus, 2007)
were associated with choosing delayed rewards. Between subject variability in task-
related activity was also reported to be associated with between-subject variability in
discounting; those that discounted more steeply exhibited more recruitment in the ventral
striatum in response to gains and losses (Hariri, et al., 2006). In another study, individual
degree of delay discounting was associated with recruitment in rostral anterior cingulate
cortex when rating the current versus future self (Ersner-Hershfield, Wimmer, &
Knutson, 2008). An inverse correspondence between individual differences in delay
discounting and signal change during intertemporal choice was reported in inferior
frontal cortex (Wittmann, et al., 2007; Monterosso et al., 2007b), left anterior prefrontal
cortex(Shamosh, et al., 2008), and neucleus accumbens, dorsoleral prefrontal cortex,
posterior cingulated cortex and medial prefrontal cortex (Ballard & Knutson, 2009).
Other than McClure et al. 2007, all the above papers used money as reward. And
without exception, all were based on decision-making between a set of 2 alternatives that
formed a conspicuous “sooner-smaller” versus “later-larger” set (e.g., $5 today or $10 in
a month). If, as I think sensible, one considers all situations in which contingencies
involve varying delays to be “intertemporal choices,” then the topic is far broader than
24

this. Consider for example, the idea of “social capitol” -- the potentially rich benefits
associated with the network of social relationships that humans maintain (Bourdieu &
Richardson, 1986). Every interaction, or avoidance of interaction, affects the individual’s
social capital. Does she inhibit annoyance when feeling irritated? Does she indulge in the
immediate gratifications of bragging? Does she lie for immediate protection at the risk of
a long lasting cost to reputation? In general, although the trade-offs are vague and
uncertain (what exactly is the cost of being thought a braggart, and when is that cost
paid?), the individual’s orientation to delay is de facto incorporated into her behavior. At
the extreme, an individual with no concern for the future will have no reason to exercise
the restraint required to maintain social relationships. The conspicuous “sooner-smaller”
versus “later-larger” choice is, I think a relatively rare case. And while monetary rewards
may make life easier on researchers and may bring particular elements of function to
light, it is not likely that experiments based on monetary trade-offs are inclusive of all the
factors relevant in intertemporal choice more broadly conceived, and they may well be
idiosyncratic. Indeed, $8 now versus $10 in one month may be treated as a math problem
for some subjects; intertemporal choices that do not involve well specified contingencies
within a single domain (such as whether or not to refuse an inconvenient request from a
friend) could not be so treated. I suspect that this accounts for the disconnect between the
neuroeconomic literature on delay discounting and the clinical lesion literature which
implicates the ventral medial prefrontal cortex as a critical substrate of far-sighted
behavior (Bechara, Damasio, & Damasio, 1994; Bechara, Damasio, Tranel, & Damasio,
1997; Damasio, 1994). Again, I think the recent work by Hare and colleagues (Hare, et
25

al., 2009) in which appetite and interest in health were used as the contingency domains
provides a compelling example of where investigation of intertemporal decision-making
will move in the near future. With respect to the dual-valuation theory, I think the current
evidence suggests that although self-control relies on sophisticated cognitive processes
supported by evolutionarily recent associative cortex, that its effect is realized within a
single underlying market place of valuation(Ainslie & Monterosso, 2004).
26

Chapter 2: Delay discounting for anticipated rewards
Introduction
Relative to other species, human exhibit extraordinary willingness to forgo
smaller sooner rewards in order to obtain larger later ones. This may be related to distinct
mechanisms of self-control that are engaged during decision-making. One account links
self-control to a hypothesized recursive feedback loop between one’s present choices and
their anticipated future behavior (Ainslie, 1992), an idea that has been formalized in
economic modeling (Benabou & Tirole, 2004; Bodner & Prelec, 2001) but that
hypothesizes no neurophysiological substrate. Another account equates self-control with
the intentional suppression of goal-inappropriate prepotent responses, which is
hypothesized to depend on inhibitory pathways between the lateral prefrontal cortex and
the basal ganglia (Jentsch & Taylor, 1999; Barkley, 1997; Goldstein & Volkow, 2002).
Still another hypothesis suggests that self-control involves the alteration of value signals
that results from effortful processing of long-term contingencies, and that is primarily
dependent upon the dorsolateral prefrontal cortex (Hare, Camerer & Rangel, 2009).
Common to these accounts is the notion that self-control involves distinct
processes engaged during decision-making that, in the case of intertemporal choice, may
attenuate the tendency to discount delayed rewards. Concretely, if self-control processes
specific to decision-making affect choice, then it follows that the individual might
choose, for example, $50 delayed by four months over an immediate $40, despite it being
the case that when the two expectancies are evaluated individually outside of a decision
context, the valuation of the immediate $40 is the higher of the two alternatives. If this is
27

in fact the case, it entails an important challenge that must be met by any neuroeconomic
model of intertemporal choice.
There are at least three ways that incentive may be measured in a non-choice
context. First, individuals can be asked to introspect on the motivation associated with
single rewards. Second, a behavioral correlate of incentive (e.g., speed on a reaction time
(RT) task) can be measured while single rewards are pursued. Third, neural correlates of
incentive can be measured during the pursuit of single rewards. In the present study I
included each of these approaches. I was particularly interested in testing the hypothesis
that when immediate and delayed rewards are encountered in isolation, behavioral and
neural markers would indicate greater incentive for the immediate, even when those
alternatives are matched for value as inferred from preferences. Such a demonstration
would be consistent with conceptions of self-control as involving processes that occur
during decision-making, and that result in reduced delay discounting. Moreover, if a
discrepancy between choice and non-choice incentive value were quantifiable, it could
provide a starting point for operationally defining the extent to which self-control affects
choices for specific individuals and specific decisions.
Materials and methods
Subjects Forty-three healthy volunteers participated in the study. All subjects
gave written informed consent, and the experiment was approved by the Institutional
Review Board of the University of Southern California. Before enrolling, volunteers were
screened for physical and neurological disorders (using self-report questionnaires) and for
current Axis I psychiatric disorders, including substance abuse and dependence (as
28

assessed by the Mini International Psychiatric Interview). Of the forty-three subjects,
three were excluded from analysis due to failure to reach stability criteria in the adaptive
delay discounting task (described below). In addition, one subject was excluded
subsequent to scanning due to history of stroke (not reported during initial screening),
one subject was excluded due to a neurological abnormality observed (severe ventricular
enlargement) and another subject was excluded due to an operational error during
scanning. Among the thirty-seven subjects included in analyses, 19 were female. Ages
ranged from 21 to 44 (mean 32.1 + 6.9).
Subjects were informed that they could win bonuses up to $160 during the course of
their participation and these bonuses would be awarded in the form of Visa gift cards that
they would receive at the end of the session. They were further instructed that some
available bonus earnings would be delayed, and if they won delayed bonus earnings, the
Visa gift cards would not be activated until the specified date. Finally, subjects were
informed that if they won delayed earnings and lost the Visa gift cards before the
specified date, they would be provided with a replacement card.
All subjects first completed a computerized version of the Monetary-Choice
Questionnaire developed by Kirby et al. (1999). Subjects were presented with a fixed set
of twenty-seven choices between smaller immediate rewards (ranging from $11 to $80)
and larger delayed rewards (ranging in amount from $20 to $85 and in delay from 7 to
186 days). These responses were used to derive an initial estimate of the subject’s level of
delay discounting using the hyperbolic discount function,
Equation 4
29

V= A/(1+k*D)
in which V is value, A is amount, D is delay in days, and k is the fit parameter that
quantifies level of discounting, with k = 0 indicating no delay discounting and higher
values of k indicating steeper discounting (for details of the estimation procedure utilized,
see Monterosso et al., 2007a). This model makes the simplifying assumption that value
scales linearly with amount. In the present context in which participants choose between
sooner smaller and later larger rewards, unmodeled concavity in the actual association
between amount and value results in inflation of best-fit values for the k parameters (see
Pine et al, 2009).
The individual estimate of discounting from the Monetary-Choice questionnaire
was used as the starting value in a second computer-administered delay discounting
choice task, this one employing adaptive questioning in order to gain precision in
determining indifference pairs of rewards. On each trial, subjects were presented with a
choice between a larger later reward (LL) and a smaller sooner reward (SS). Participants
were informed that one trial would be selected from this procedure and that they would
receive the alternative they selected on that trial. The delay of the LL was always 120
days and the SS was always zero delay. For half the trials, the LL was $28 + $7 (“Low”)
and for half the trials the LL was $53 + $7 (“High”). The magnitude of the SS was
initially generated by computing what would be an amount of equal value to the LL
based on the fit parameter for the subject (k-value) derived from responses on the
Monetary-Choice questionnaire, as modeled using Equation 4. On each trial, if the
participant chose the SS alternative, then the k-parameter associated with that reward was
30

adjusted upward a quarter step on a log
10
scale and consequently, the SS on the next trial
in which that LL appeared was lower. Conversely, if the participant chose the LL, then
the k-parameter associated with that reward was adjusted downward (again, a quarter step
on a log
10
scale) resulting in a higher SS on the next trial in which that LL appeared. This
adjusting procedure continued until subjects reached stability for both reward pairs, with
stability operationalized as a window of eight trials in which k-values did not deviate by
more than two steps. Participants who did not reach this criterion after eight minutes (n =
3) were excluded from the study. The final indifference pairs were then generated using
the geometric mean of the k-values for the eight trials during which stabilization was
achieved (separately for the Low and High LL amounts). In this way, two indifference
pairs were established for further investigation during fMRI. For analyses that required a
single discount parameter estimate for each participant, I used the geometric mean of
these two k-parameter estimates.
Monetary Incentive Delay Task In the next step of the study, participants
performed a variant of the Monetary Incentive Delay (MID) task (Knutson, Adams, Fong
& Hommer, 2001a). Prior to the scan, participants were trained to associate each of the
two SSs and two LLs comprising the derived indifference pairs with each of four colored
shapes (LLs were always $28 in four months and $53 in four months, and SSs were
individualized to the participant’s performance.). The pairing of the colored shapes with
rewards was counterbalanced across participants. Subjects completed a computerized
memory training program in which the four pairs of colored squares and corresponding
“prizes” briefly appeared on the screen, one at a time, and were instructed to memorize
31

the pairs. Next, subjects completed a memory test, during which the colored squares
flashed on the screen, and subjects reported the corresponding prizes. Subjects were
asked to provide the prize in the appropriate format (e.g., amount followed by delay, or
vice versa depending on counterbalanced order) and received feedback about their
responses. Upon satisfactory completion of the memory test, subjects completed a
practice version of the task, similar to the fMRI version.
Each trial of the MID task began with the appearance of a colored shape that
indicated which of the four rewards was available on that trial. After an anticipation
period of between 4 and 4.5 seconds subsequent to presentation of the available reward
stimulus, either a target appeared (the character “+”) or the words “no target” appeared
(50% of the time). If the target appeared, the participant was required to respond as
quickly as possible with a button press in order to win on the trial. Participants were
instructed that their likelihood of winning would be greater if they responded faster,
although in reality, outcome was predetermined in order to optimize orthogonality
between anticipation and outcome periods (with the exception of RTs > 500 ms which
were always scored as too slow, to avoid suspicion). In order to optimize power, an
exponential distributed inter-trial-interval with mean 2s was used. Unlike the standard
MID task, the colored shape remained on the screen until the target (or “no target”
message) appeared. The critical epoch for analyses was the 4 - 4.5 seconds during which
the participant was cued to the possible reward and was readying to try to obtain it.
Following prior work with the task (Knutson et al., 2001a), I refer to this throughout as
the “anticipation period.” In each run of the task, each of the four targets was presented
32

16 times. Each participant completed two runs of the task. Participants were instructed
that one target trial would be selected from each run, and any money that they won on
these trials would be paid, again using Visa cards, with credit activated at the specified
date. If their response was not sufficiently fast on the target trial selected, then they did
not win a bonus for that run of the task.
fMRI Acquisition fMRI data were collected using 3T Siemens MAGNETOM
Tim/Trio scanner with a standard birdcage head-coil in the Dana and David Dornsife
Cognitive Neuroscience Imaging Center at University of Southern California. For each
participant, sagittal images (256×256×176) with 1×1×1mm
3
resolution were obtained by
a T1-weighted 3D MPRAGE (magnetization prepared rapid gradient echo) sequence
(TI=900 ms, TR=1950 ms, TE=2.26 ms, flip angle=90°). Functional scanning used Echo
Planar Imaging (EPI) sequence (TR=2000ms, TE=30ms, flip angle=90°, FOV=192, in-
plane resolution=64×64) with PACE (prospective acquisition correction) which helps
reduce head motion during data acquisition. Thirty- two axial slices were used to cover
the whole cerebral cortex with no gap and the slices were positioned along anterior
commissure-posterior commissure plane.
fMRI Analysis fMRI data processing was conducted using FEAT (FMRI Expert
Analysis Tool) Version 5.98, part of FSL (FMRIB's Software Library,
www.fmrib.ox.ac.uk/fsl). The first four volumes before the task were automatically
discarded by the scanner for T1 equilibrium. For preprocessing, the head movement
which was not captured by PACE was corrected in three dimension by MCFLIRT
(Jenkinson, Bannister, Brady & Smith, 2002). Six motion parameters were added into the
33

general linear model (GLM) in order to explain variance in signal related to head motion.
Data were temporally filtered by a high-pass filter with 100s cut-off and spatially
smoothed by a Gaussian kernel of full-width-half-maximum (FWHM) 5mm. The
preprocessed data were then submitted to a GLM which was used to analyze the
contributions of experimental factors to blood oxygen level dependent (BOLD)
responses. All within-subject statistical analyses were performed in native image space,
and then the statistical maps were transformed into standard space before high-level
(group) analysis. The transformation into standard space was performed in two steps: EPI
images were first aligned to the participant’s own MPRAGE structural scan, and then the
image was normalized into standard space (Montreal Neurological Institute (MNI)) using
affine transformation (Jenkinson & Smith, 2001).
My primary analyses targeted brain signal changes during the anticipation period.
Since this period ended at the onset of the target stimulus or the onset of the stimulus
indicating no-target, all trials could be used in analysis of the anticipation period,
irrespective of the trial outcome, resulting in good statistical power. The subsequent
variation relating to the target and the outcome of the trial allowed us to better isolate
anticipation period effects by reducing covariation with effects related to subsequent
events. I carried out two variants of this analysis. In the first variant of this analysis, there
were thirteen events modeled: four events during the anticipation period including High-
Immediate, Low-Immediate, High-Delay, Low-Delay, and nine events during the
feedback period including High-Immediate-Win/Loss, Low-Immediate-Win/Loss, High-
Delay-Win/Loss, Low-Delay-Win/Loss and No-Target. Each event was convolved with
34

double-gamma hemodynamic response function and temporal derivatives were added as a
covariate of no interest in order to improve statistical sensitivity. Null events were not
modeled. In the second variant of this analysis, the anticipation epoch was modeled using
three parameters: Value (High or Low), Discount Fraction (the participant-specific
denominator in Equation 4) and Amount (the undiscounted monetary amount available).
Both of the latter variables were orthogonalized to Value, and Amount was additionally
orthogonalized to Discount Fraction. It is important to note that the Value parameter is
inferred from preference data. If there is a shift in value in the non-choice context that is a
function of delay (as hypothesized), regions tracking this shift would be associated with
the Discount Fraction rather than Value parameter.
For both above models, a Cross-Run High-level analysis was performed using a
fixed effects model by forcing the random effects variance to zero in FLAME
(Beckmann, Jenkinson & Smith, 2003). Results were input to group-level analysis using
FLAME stage 1 (Beckmann, et al., 2003; Woolrich, Behrens, Beckmann, Jenkinson &
Smith, 2004; Woolrich, 2008).
Region of Interest (ROI) Analyses ROI analyses were carried out. Regions were
selected based on prior findings associated with the MID task (e.g., Knutson, et al.,
2001a, 2005). Four of the selected ROIs (bilateral putamen, thalamus, caudate, and
nucleus accumbens) were defined based on the automated segmentation tool FIRST,
which is specifically designed to classify subcortical structures (Patenaude et al., 2007).
Since this tool is not applicable to the additional ROIs selected (the left and right anterior
insula, the midbrain and the supplementary motor area) I adopted an alternative strategy
35

for these regions, drawing 6 mm spheres around the peek coordinates within the region,
reported in a prior study that included a contrast isolating sensitivity to reward magnitude
(Knutson et al., 2005; coordinates converted into MNI space). In order to examine
whether reward magnitude and immediacy affected MR signal change in these regions
during the anticipated period, extracted beta-values for each of the four rewards were
subjected to repeated measure analysis with magnitude (high versus low rewards), delay
(immediate versus four months delay), and brain region included as within-subject
variables. In order to examine the possibility that particular regions might show more
sensitivity to delay, and others might show more sensitivity to magnitude, I repeated the
analysis, with Magnitude and Delay recoded as two levels of a within subject variable
("Dimension”), each of which in turn included two levels (High and Low, and Immediate
and Delayed). An interaction between Dimension and Region would provide evidence of
differential sensitivity to magnitude and delay.
Whole Brain Analyses The primary whole-brain analysis was based on the above
model in which the anticipation epoch was modeled with four separate events
corresponding to the four rewards. In this, I compared preference-matched immediate and
delayed rewards (Immediate – Delayed, and Delayed – Immediate), thresholded using
cluster detection statistics, with a height threshold of Z > 2.3 and a cluster probability of
p<.05 corrected for search space (Worsley, 2001). Contrasts were also performed
isolating sensitivity to reward magnitude (High versus Low, and Low versus High) across
the entire brain. Exploratory conjunction analyses were carried out examining the overlap
between High – Low contrasts (this time without correction for multiple comparisons)
36

and both Immediate – Delay and Delay – Immediate contrasts (without search space
correction). Because visualization related to this analysis was not corrected for multiple
comparisons, I do not use it to support hypothesis testing, but include it as the observed
pattern with this less conservative thresholding is, I think, informative.
A parametric analysis was also performed, as described above, in which Value
(High or Low), Discount Fraction (the participant-specific denominator in Equation 4)
and Amount (the actual monetary amount available) were used to model change in signal
during anticipation. In this analysis, I was particularly interested in whether the Discount
Fraction predicted response during anticipation after variance related to Value was
modeled.
Subjective Ratings Subsequent to the completion of the task, subjects were
instructed to rank the four rewards according to the following instructions, “Rank the
prizes in terms of how each made you feel at the moment you were going for them during
the game. If there are any that are exactly tied, you can give them the same rank. This is a
little different than asking you which one you would choose. Don’t worry about which
prize you would choose if you compared them, just think about how each made you feel
at the moment you were going for the prizes.”
Post-MID Choice Task In order to test for the presence of systematic drift in
discounting, a subset of participants (N=15) completed a choice task after the MID task.
These participants were presented with 24 choice trials in which alternatives were value-
matched based on the discount parameter previously derived in the adaptive choice
procedure and 24 choice trials that were mismatched based on the same previously
37

derived discount parameter. These mismatched trials were generated by creating
indifference pairs based on a k fit parameter estimate (Equation 4) that was one log unit
larger (50% of trials) or one log unit smaller (50% of trials) than the participant’s actual
fit parameter estimate.
Results
Intertemporal Choice Task Across participants, the immediate reward amounts
that formed indifference pairs with $28 delayed by four months ranged from $3 to $25,
with a median of $13 (corresponding to k = .010, Equation 4). Across participants, the
reward amounts that formed indifference pairs with $53 delayed by four months ranged
between $6 and $52, with a median of $28 (corresponding to k = .007).
Behavioral Results for the MID Task Median RT data for each subject on the MID
task were modeled using repeated-measures analysis of variance with magnitude and
delay included as within-subject independent variables. Magnitude was coded as High for
both $53 delayed by four months and for the immediate amount that was equally
preferred to it (individualized to the participant). Magnitude was coded as Low for the
$28 delayed by four months and for the immediate amount that was equally preferred to
it (also individualized to the participant). RT was faster for the High magnitude trials than
the Low magnitude (F(1,36) = 4.38, p < .05) and faster for the Immediate reward trials
than for the (preference-matched) Delayed reward trials (F(1,36) = 9.1, p < .01; see
Figure 1).
Neuroimaging ROI Analyses Beta-values were extracted during anticipation for
twelve anatomically defined ROIs for each of the four rewards (High Immediate, High
38

Delayed, Low Immediate, Low Delayed). These values were subjected to a repeated
measure analysis, with Magnitude, Delay, and Region included as within-subject
variables. Significant main effects were observed for amount (F(1, 36) = 6.46, p < .05),
delay (F(1,36) = 4.32, p < .05), and for region (F (11, 26) = 14.7, p < .001). When
Magnitude and Delay were recoded as two levels of the variable Dimension (as described
above) no interactions were observed between Dimension and Region. Difference scores
highlighting the immediacy effect (beta values for the two Immediate rewards minus
beta-values for the two Delayed rewards) and the magnitude effect (beta-values for the
two High rewards minus that for the two Low rewards) are presented for each ROI in
Table 1 (see also Figure 2). As it can be seen, all difference scores were positive
(indicating higher beta-values for immediate relative to delayed rewards, and higher beta-
values for high magnitude rewards relative to low magnitude). In six of the twelve
anatomical ROIs, difference scores were greater than zero at least at a trend level (2-
tailed, α =.1) for both amount and delay. Two additional regions reached this threshold
only for amount, and two more regions, only for delay. In general, difference scores
across different regions were highly correlated for both Delay – Immediate and High –
Low (Chronbach’s alpha = .95 and .92 respectively).
Neuroimaging Whole Brain Analyses I also carried out a whole-brain analysis
comparing MR-signal during anticipation of immediate rewards with signal during
anticipation of the preference-matched delayed rewards. I used cluster detection statistics
with the height threshold of Z > 2.3, p <.05 cluster-level correction for search space. As
shown in Figure 3, immediate rewards recruited greater signal change than delayed
39

Figure 1: Reaction Time Data in Monetary Incentive Delay Task

Mean and standard error of individual median RT by condition on target trials (computed
as distance from the overall median for each participant). Based on repeated measures
ANOVA, RTs were significantly faster for both the High versus Low pair, and for the
Immediate versus Delayed rewards.

rewards in left caudate, putamen (bilateral), insula (bilateral), left pallidum,
supramarginal gyrus (bilateral), anterior cingulate cortex and supplementary motor area
(see Immediate > Delayed section of Table 2). Immediate rewards recruited significantly
less activation in clusters within the precuneus and occipital cortex.
I also contrasted high and low rewards in a whole brain analysis. As shown in
Figure 4, cluster-level significance was reached in the right caudate, thalamus (bilateral),
lateral occipital cortex, and occipital pole (see High > Low section of Table 2). Higher
rewards did not recruit significantly less activity than lower rewards in any region. As an
40

Table 1: Anatomical Region of Interest Analysis Result

Magnitude Effect (High –
Low)
Delay Effect (Immediate
– Delay)
beta values p -value beta values p -value
L Caudate 4.97 + 22.06 .18. 4.51 + 21.64 .21
R Caudate 7.48 + 20.35 .03 4.71 + 20.11 .16
L Putamen 4.43 + 14.48 .07 7.77 + 19.92 .02
R putamen 5.19 + 14.23 .03 6.26 + 18.4 .04
L Thalamus 6.13 + 16.9 .03 5.80 + 19.9 .08
R Thalamus 7.74 + 16.79 .008 6.15 + 20.52 .08
L Nucleus Accumbens 4.61 + 25.29 .28 4.27 + 30.43 .40
R Nucleus Accumbens 7.05 + 23.23 .07 1.62 + 27.32 .72
L Insula 6.53 + 23.96 .11 9.21 + 23.83 .02
R Insula 4.34 + 19.36 .18 5.66 + 16.58 .04
Midbrain 7.96 + 19.03 .02 8.29 + 26.05 .05
Supplementary Motor Area 10.49 + 31.00 .047 14.80 + 35.98 .017

alternative approach, I also modeled the data during anticipation parametrically, as
described above. The Value and Discount Fraction predictor variables used in this
additional model yielded similar activation maps to those associated with Immediate –
Delay, and High – Low contrasts (see Figure 5).
41

Figure 2: 95% Confidence Interval of the Effect Sizes for the Magnitude Effect
and Immediacy Effect

95% confidence interval of the effect sizes for the magnitude effect (High – Low, shown
in full lines) and immediacy effect (Immediate – Delayed, shown in dashed lines) of all
ROI’s (bilateral ROI’s collapsed).

For exploratory purposes, a conjunction analysis was carried out that identified all
voxels that evidenced greater activity (Z > 2.3 voxel height threshold, without cluster
correction) in both 1) High > Low and 2) either Immediate > Delay (Figure 6, shown in
red) or Delay > Immediate (Figure 6 shown in blue.) The overlap between voxels
identified in High – Low and Immediate – Delay was widespread within subcortical
structures previously associated with incentive on the MID task (including brainstem,
42

Figure 3: Contrast Maps for Immediate-Delayed Rewards

Contrast maps for Immediate – Delayed (warm colors indicate Immediate > Delayed,
cool colors Delayed > Immediate). All clusters based on whole-brain analysis with voxel
threshold of Z = 2.3, and cluster level correction of p < .05. Findings for Immediate >
Delayed include left caudate, putamen (bilateral), insula (bilateral), left pallidum,
supramarginal gyrus (bilateral), anterior cingulate cortex and supplementary motor area.
Findings for Delayed > Immediate include clusters within the precuneus and occipital
cortex.

right caudate, bilateral pallidum, bilateral putamen, right thalamus, bilateral insula, and
left supramarginal gyrus). There was one large cluster in the visual cortex in which Delay
> Immediate overlapped High > Low voxels. I hypothesized that this activity in visual
cortex was likely related to visual attention which might be heightened for different
reasons in the two contrasts. To explore this hypothesis, a functional connectivity
analysis was carried out in which we used activity in the cluster that overlapped both
43

Table 2: Whole Brain Analysis Result

Whole brain x, y, z Max Z
R Caudate 12,14,4 2.83
R Thalamus 8,-4,10 3.15
L Thalamus -12,-22,-4 3.34
Lateral Occipital
Cortex
-18,-84,12 3.5
High-Low
Occipital Pole -8,-98,10 3.37
R Insula 44,12,-4 4.17
R Putamen 26,14,-2 3.39
R Supramarginal
Gyrus
64,-34,28 3.37
R Inferior Frontal
Gyrus
54,18,4 2.86
L Insula -40,16,-4 4.03
L Putamen -16,8,-8 3.54
L Caudate -10,12,2 2.9
L Pallidum -16,4,2 3.0
L Supramarginal
Gyrus
-60,-34,28 3.12
Anterior Cingulate
Cortex
0,22,32 2.81
Immediate-
Delayed
Supplementary
Motor Cortex
0,2,56 4.24
Occipital Cortex 0,-94,10 4.07 Delayed-
Immediate Precuneous Cortex 0,-56,24 3.26

High > Low and Delayed > Immediate contrasts as a seed to predict activity throughout
the rest of the brain, contrasting connectivity with the seed in the Immediate versus
Delayed trials. As shown in Figure 7, a significant differential functional connectivity
effect was observed (p < .05, cluster corrected for search space) in regions overlapping
with those previously associated with incentive on the task (putamen, anterior insula,
thalamus, pallidum). Specifically, there was significantly greater association with the
44

Figure 4: Contrast Map for High-Low Rewards

Contrast maps for High – Low rewards (red indicates High > Low; no clusters were
observed for Low > High). Findings for High > Low include right caudate, thalamus
(bilateral), lateral occipital cortex, and occipital pole.

occipital cortex seed region when rewards were immediate relative to when rewards were
delayed.
Post-hoc correlational analysis indicated no correlation between individual
variance in delay discounting as measured by the choice procedure and either variance in
the effect of immediacy on RT (r(37) = - .19, p = .26), or of immediacy on MR signal in
regions of interest (r(37) = - .10 , p = .56 ). The immediacy effect on RT was significantly
correlated with the effect of immediacy on MR signal in a priori ROIs (r(37) = .35, p <
.05).
Beta values in the occipital cortex for Delayed > Immediate were not correlated
with discounting (r(37) = -.14, p=.40), or with the immediacy effect on RT (r(37) = -.08,
p = .62), but were inversely correlated with beta values within clusters identified in the
Immediate – Delayed contrast (r(37) = -.49, p = .002).
Subjective Ranking Questionnaire The mean ranking for the larger immediate
45

Figure 5: Parametric Analysis Result

Parametric analysis with anticipation modeled with three parameters: Value (High or
Low), Discount Fraction (the participant-specific denominator in Equation 4) and
Amount (the actual monetary amount available). Both of the latter variables were
orthogonalized to Value, and Amount was additionally orthogonalized to Discount
Fraction. Areas associated with Value (shown in red) include right caudate, and thalamus
(bilateral). Areas associated with Discount Fraction (shown in yellow and blue) in the
direction of greater activity in association with the greater immediacy include, right
putamen, insula (bilateral), supramarginal gyrus (bilateral), and right inferior frontal
gyrus. Conversely, greater delay was associated with greater activity in clusters within
the precuneus and occipital cortex. Finally, higher amounts were associated with higher
activity in clusters within the occipital cortex displayed in green.

reward was 1.56 + .84, for the larger delayed reward was 1.58 + .77, for the smaller
immediate reward was 3.14 + .90, and for the smaller delayed reward was 3.28 + .78.
When subjected to a repeated measures ANOVA, magnitude (High versus Low) was a
46

Figure 6: Conjunction Maps for High-Low and Immediate-Delayed

Conjunction whole brain maps for High – Low and Immediate – Delayed (red indicates
Immediate-Delayed, blue indicates Delayed-Immediate p < .05, uncorrected for each).

highly significant predictor of ranking (F(1,35) = 214.0 , p <.001) and delay was not a
significant predictor of ranking (F(1,35) = .16, p=.69). Although there was not an
indication of a group level effect of immediacy on ranking for the preference-matched
pairs, we examined the relation between individual differences in ranking based on
immediacy and individual variance in the effect of immediacy on RT and on MR signal
47

Figure 7: Connectivity Analysis Result

Connectivity analysis using the cluster common to High > Low and Delayed > Immediate
(which was within occipital cortex) as the seed region. Green areas were significantly
more associated with seed region during Immediate reward trials than during Delayed
reward trials. Clusters in putamen, anterior insula, thalamus, pallidum were all
significant, p < .05, controlling for whole-brain search space.

change during anticipation. To do this, I computed a difference score for the mean
ranking of delayed rewards minus the ranking of immediate rewards. Although this index
of the subjective effect of immediacy (given matched preference) did not predict variance
in the effect of immediacy on MR signal difference (r(36) = .27, p = .11), the association
with the effect of immediacy on RT was significant, and in the anticipated direction
(r(36) = .36, p < .05).
Post fMRI Delay Discounting Choice Reassessment In the post MID task
reassessment of delay discounting that was administered to a subset of participants
(N=15), in trials generated to be at the participant’s indifference point, the mean
percentage choice of the SS was 50.3% (+ 12.7%). Among trials generated to be
48

mismatched in value, participants chose the option generated to be of higher value (based
on their individual estimated discount function) on 93.5% (+ 8.5%) of trials.
Discussion
I compared behavior and brain response during anticipation of long delayed (4
months) and smaller, equally preferred immediate monetary rewards (“indifference
pairs”). The data unequivocally indicate observable differences between the conditions;
when anticipating immediate rewards relative to delayed rewards, responses to target
stimuli were faster and neural activity was greater in a network of regions previously
implicated in incentive during the task. This was especially evident in the superior
portion of the anterior insula (contiguous with frontal operculum) and putamen, where
findings were bilateral, and evident in anatomical ROI and whole-brain analyses. In
addition to its established association with feeling states (Damasio et al., 2000), the
anterior insula is implicated in executive function tasks (Wager, Jonides & Reading,
2004a) and may be associated with affective signals accompanying mental effort (Wager
& Barrett, 2004b). A similar locus of activation was reported using the MID task in
contrasts identifying sensitivity to the presence (Knutson, Fong, Adams, Varner &
Hommer, 2001b) and magnitude (Knutson, Taylor, Kaufman, Peterson & Glover, 2005)
of reward. The putamen is implicated in reinforcement learning (Packard & Knowlton,
2002) and especially in preparation of motor responses (Alexander & Crutcher, 1990).
Using the same task, activity in the putamen has also been repeatedly reported during
anticipation of rewards (e.g., Knutson, et al., 2001a) and appears to be preferentially
recruited during positive incentive (Knutson, et al., 2005). In addition, evidence of
49

differential activity during immediate relative to delayed reward trials was observed in
most regions previously implicated as sensitive to reward during the MID task, including
the brainstem, pallidum, caudate, supplementary motor area, supermarginal gyrus, and
anterior cingulate cortex. The fact that participants who completed a choice task
subsequent to the MID task preferred the SS on 50.3% of trials designed to be at the
participant’s indifference point indicates findings were not the product of drift towards
greater discounting during the experiment.
While these data demonstrate that there was something different during
anticipation of preference-matched immediate versus delayed rewards, several reasonable
interpretations warrant consideration. One possible basis is that subjects, on average,
valued the immediate rewards more than the delayed rewards, despite their having been
matched based on revealed preference. This is consistent with conceptions of self-control
that posit distinct mechanisms engaged during decision-making that generally shift
preference towards greater future orientation. Accordingly, the more immediate reward
within a decision-based “indifference pair” would be, on average, of higher value than
the more delayed reward if each was encountered in a context where self-control was not
operative, as arguably is the case in the MID task. Before returning to this interpretation,
I consider two alternative accounts.
Alternatively, it could be hypothesized that the differences in MR signal result
from differences in the representation of subcomponents of expected value; regions more
active in the immediate reward condition might be specifically sensitive to the dimension
of immediacy rather than a difference in overall incentive value. To maintain this as an
50

explanation of observed findings, differential activation should be absent in regions
sensitive to overall value. The lack of differences observed in the nucleus accumbens and
ventromedial prefrontal cortex could be viewed as supportive. However, in the repeated
measures analysis of signal in ROIs implicated in incentive in prior work with the MID
task, I found no evidence of specificity in sensitivity to reward magnitude versus reward
immediacy. That is, I observed no statistical evidence of divergence within the examined
network in sensitivity to the two dimensions. Furthermore, differential activation was
observed in regions that are important to the execution of the experimental task, but
which are not plausibly substrates representing sub-components of expected value (e.g.,
the supplementary motor area). If some of the observed findings based on the Immediate
– Delay imaging contrast are related to subcomponents that contribute to value, but that
imply no overall divergence in value across the conditions, then an additional explanation
would have to be provided to explain activation differences in regions that are unlikely
related to subcomponents of expected value, as well as to the observed difference in RTs
to the targets. The correlation between the immediacy effect across brain regions
(chronbach’s α = .95) and between fMRI data and reaction time (r=.35, p < .05) make
this prospect less convincing. It may be important to note that the MID task utilizes a
small set (here four) of highly familiar rewards; it is possible that neural recruitment
associated with value differs here from situations in which novel rewards are
encountered.

51

A second alternative account is that observed findings reflect a conditioned
response, whereby stimuli signaling immediacy potentiate a motor response (rather than
discrepant valuation). Perhaps because of semantic overlap or learning history, cues of
immediacy could prime action, and cues of delay prime inaction. The widespread overlap
between voxels sensitive to amount and delay (Figure 6) and the absence of any statistical
evidence of an interaction involving these predictor variables, fails to lend support. And
while visual inspection of the thresholded images invites conjecture that the imbalance in
incentive observed between immediate and delayed rewards has some particular
association with the potentiation of action, signal change data across ROIs (Table 1,
Figure 2) does not suggest regional specificity in sensitivity to the two orthogonal
independent variables of Magnitude and Immediacy. It is also worth noting that subjects
that reported more favorable subjective rankings for immediate rewards tended to
demonstrate greater RT superiority during immediate reward trials (r = .36, p < .05).
This association would not be predicted based on the conditioned response interpretation
of our findings.
Finally, it could reasonably be suggested that the primary findings might not be
related to choice per se, but rather to the mere juxtaposition of multiple alternatives.
Perhaps having another option for consideration, irrespective of whether it has to be
chosen against, increases focus on amount rather than on delay. With respect to this
possibility, I note that the four rewards we used in the MID task were presented
continually in close temporal proximity and so did naturally form a frame for
comparison. Also, I know of no a priori basis for supposing juxtaposition would
52

differentially shift attention to the amount dimension over the immediacy dimension.
Nevertheless, the possibility that concurrent juxtaposition shifts valuation towards
amount and away from immediacy cannot be ruled out (for evidence that choice itself is a
context which can change the value of options, in their case, enduringly, see Sharot et al.,
2009).
On balance, I believe the data are best explained as the result of on-average higher
incentive value for the immediate rewards relative to the equally preferred delayed
rewards. However, one aspect of the data appears inconsistent at first blush. Differential
activity was observed in a cluster in the visual cortex for High relative to Low, and also,
Delayed relative to Immediate reward trials. The results of our functional connectivity
provide a clue. Functional connectivity to this occipital cortex cluster (plausibly related
to heightened visual attention) was significantly diminished when rewards were delayed
in a network of regions implicated in incentive during the task (putamen, anterior insula,
thalamus, pallidum). While High relative to Low rewards might differentially recruit
visual attention because of differential incentive value (David et al., 2005), the reduced
connectivity to basal ganglia activity during delayed rewards suggests heightened visual
attention during the delay condition related to something other than value. So while the
basis for greater activation in this region for delayed relative to immediate rewards
remains unclear, connectivity results suggest it is not based on heightened incentive
value. It should be noted that unlike the standard MID task, the incentive stimulus was
visually displayed in the present study throughout the anticipation period.
Conclusions
53

My findings are consistent with theories that posit that self-control includes
engagement of processes during decision-making (as opposed to during valuation more
generally). On this interpretation, the heightened incentive for the immediate rewards
relative to the preference-matched delayed rewards reflects removal of the typically
moderating influence that self-control processes had on the tendency to devalue delayed
rewards (or the tendency to exhibit concavity in the association between amount and
value, which in this context would similarly result in greater preference for sooner
smaller over later larger). This suggests that two factors contribute to response to delay
during intertemporal choices. The first factor is the direct effect delay has on incentive
value, which I expect is itself complexly determined, and subject to framing effects (e.g.,
whether the delay is expressed as a waiting period, or by specifying the day the reward
will be received, as in Read et al., 2005). The second factor consists of self-control
processes engaged during decision making, which tend to push preference towards later,
larger rewards. Bringing this dissociation under quantitative analysis may be
illuminating. For example, to the extent that farsighted choices are based on self-control
processes engaged during explicit decision-making (second of the above factors), the
individual may be more vulnerable to short-sightedness in the presence of anything that
selectively undermines higher cognitive functions, such as, fatigue or distraction.
Alternatively, some framing manipulations could selectively influence the direct
incentive value of rewards (first of the above factors) but have influence over high level
decision-making, if the decision-making process explicitly ignores the frame. Although
the present work does not investigate the mechanisms underlying self-control, by
54

examining valuation outside of a decision-context, these data provide support for the
conceptions of self-control as entailing one or more processes that are engaged during
decision-making, and that generally result in less shortsightedness in behavior than would
be predicted by the individual incentive value of immediate and delayed rewards.
55

References
Ainslie, G. (1974). Impulsive control in pigeons. Journal of the Experimental Analysis of
Behavior, 21, 485-489.

Ainslie, G. (1975). Specious reward: a behavioral theory of impulsiveness and impulsive
control. Psychol Bull, 82(4), 463-496.

Ainslie, G. (1992). Picoeconomics. New York: Cambridge University Press.

Ainslie, G. (2001). Breakdown of will. New York: Cambridge University Press.

Ainslie, G., & Monterosso, J. (2003a). Hyperbolic discounting as a factor in addiction: A
critical analysis. In R. Vuchinich & N. Heather (Eds.), Choice, Behavioral
Economics, and Addiction. Oxford: Elsevier.

Ainslie, G., & Monterosso, J. (2003b). Will as intertemporal bargaining: Implications for
rationality. University of Pennsylvania Law Review, 151(3), 825-862.

Ainslie, G., & Monterosso, J. (2004). A Maketplace in the Brain? Science, 306.

Alexander, G.E., & Crutcher, M.D. (1990). Preparation for movement: neural
representations of intended direction in three motor areas of the monkey. J
Neurophysiol, 64(1), 133-150.

Arrow, K. (1951). Alternative approaches to the theory of choice in risk-taking situations.
Econometrica, 19, 404-437.

Ballard, K., & Knutson, B. (2009). Dissociable neural representations of future reward
magnitude and delay during temporal discounting. NeuroImage, 45, 143-150.

Barkley, R. A. (1997). ADHD and the Nature of Self-Control. London: Guilford Press.

Barnes, J. (1984). The Complete Works of Aristotle. Princeton: Princeton University
Press.

Baumeister, R. F., & Heatherton, T. F. (1996). Self-regulation failure: An overview.
Psychological Inquiry, 7(1), 1-15.

Baumesiter, R., Vohs, K., & Tice, D. (2007). The strength model of self-control. Current
directions in psychological science, 16(6), 351-355.

56

Bechara, A., Damasio, A. R., & Damasio, H. (1994). Insensitivity to future consequences
following damage to human prefrontal cortex. Cognition, 50, 7-15.

Bechara, A., Damasio, H., Tranel, D., & Damasio, A. R. (1997). Deciding
advantageously before knowing the advantageous strategy. Science, 275, 1293-
1295.

Becker, G. S. (1976). The Economic Approach to Human Behavior. Chicago: University
of Chicago Press.

Becker, G. S., & Murphy, K. M. (1988). A Theory of Rational Addiction. Journal of
Political Economy, 96, 675-700.

Beckmann, C., Jenkinson, M., & Smith, S.M. (2003). General multi-level linear
modelling for group analysis in FMRI. NeuroImage, 20, 1052-1063.

Benabou, R., & Tirole, J. (2004). Will-power and Personal Rules. Journal of Political
Economy, 112(4), 848-886.

Bodner, R., & Prelec, D. (2001). Self-Signaling and Diagnostic Utility in Everyday
Decision Making. In I. Brocas & J. D. Carrillo (Eds.), Psychological of Economic
Decisions. Oxford: Oxford University Press.

Boettiger, C., Mitchell, J., Tavares, V., Robertson, M., Joslyn, G., & D’Esposito, M., et
al. (2007). Immediate Reward Bias in Humans: Fronto-Parietal Networks and a
Role for the Catechol-O-Methyltransferase 158Val/Val Genotype. Journal of
Neuroscience, 27(52), 14383-14391.

Bourdieu, P., & Richardson, J. (1986). The forms of capital. Handbook of Theory and
Research for the Sociology of Education. New York: Greenwood New York.

Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., & Shizgal, P. (2001). Functional
imaging of neural responses to expectancy and experience of monetary gains and
losses. Neuron, 30(2), 619-639.

Chapman, G. B. (1996). Temporal discounting and utility for health and money. Journal
of Experimental Psychology: Learning, Memory, & Cognition, 22(3), 771-791.

Chapman, G. B. (2000). Preferences for improving and declining sequences of health
outcomes. Journal of Behavioral Decision Making, 13(2), 203-218.

Damasio, A. R. (1994). Descartes’error: Emotion, reason, and the human brain. New
York: GP Putnam’s Sons.

57

Damasio, A.R., Grabowski, T.J., Bechara, A., Damasio, H., Ponto, L.L.B., Parvizi, J., &
Hichwa, R.D. (2000). Subcortical and cortical brain activity during the feeling of
self-generated emotions. Nature Neuroscience, 3(10), 1049-1056.

David, S., Munafò, M., Johansen-Berg, H. S., Rogers, R., Matthews, P., & Walton, R.
(2005) Ventral Striatum/Nucleus Accumbens Activation to Smoking-Related
Pictorial Cues in Smokers and Nonsmokers: A Functional Magnetic Resonance
Imaging Study. Biological Psychiatry, 58(6), 488-494.

Edwards, W. (1953). Probability-Preferences in Gambling. The American Journal of
Psychology, 66(3), 349-364.

Ersner-Hershfield, H., Wimmer, G. E., & Knutson, B. (2008). Saving for the future self:
Neural measures of future self-continuity predict temporal discounting. Social
Cognitive and Affective Neuroscience, 4(1), 85-92.

Evans, J. S. B. T. (2008). Dual-Processing Accounts of Reasoning, Judgement, and
Social Cognition. Annual Review of Psychology, 59, 255-278.

Frederick, S. (2002). Time Discounting and Time Preference: A Critical Review. Journal
of Economic Literature, 40, 351-401.

Friedman, M. (1963). Windfalls, the "Horizon," and Related Concepts in the Permanent-
Income Hypothesis Measurement in economics. Standford: Standford University
Press.

Friedman, M. (1971). Essays in positive economics. Chicago: University of Chicago
Press.

Goldstein, R. Z., & Volkow, N. D. (2002). Drug Addiction and Its Underlying
Neurobiological Basis: Neuroimaging Evidence for the Involvement of the
Frontal Cortex. American Journal of Psychiatry, 159, 1642-1652.

Green, L., & Myerson, J. (2004). A Discounting Framework for Choice With Delayed
and Probabilistic Rewards. Psychol Bull, 130, 769-792.

Hare, T., Camerer, C., & Rangel, A. (2009). Self-contrl in decision-making invovles
modulation of the vmPFC valuation system. Science, 324, 646 - 648

Hariri, A., Brown, S., Williamson, D., Flory, J., Wit, H., & Manuck, S. (2006).
Preference for Immediate over Delayed Rewards Is Associated with Magnitude of
Ventral Striatal Activity. Journal of Neuroscience, 26(51), 13213-13217.

58

Jentsch, J. D., & Taylor, J. R. (1999). Impulsivity resulting from frontostriatal
dysfunction in drug abuse:implications for the control of behavior by reward-
related stimuli. Psychopharmacology, 146(4), 373-390.

Jevons, W. (1871). A Theory of Political Economy. London & New York: Macmillan
and Co.

Kable, J., & Glimcher, P. (2007). The neural correlates of subjective value during
intertemporal choice. Nature Neuroscience, 10(12), 1625-1633.

Kahneman, D., & Tvesky, A. (1979). Prospect theory: An Analysis of Decision under
Risk. Econometrica, 47(2), 263-291.

Kirby, K.N., Petry, N.M., & Bickel, W.K. (1999). Heroin addicts have higher discount
rates for delayed rewards than non-drug-using controls. Journal of Experimental
Psychology: General, 128(1), 78-87.

Knutson, B., Adams, C., Fong, G., Hommer, D. (2001a). Anticipation of Increasing
Monetary Reward Selectively Recruits Nucleus Accumbens. Journal of
Neuroscience, 21(RC159), 1-5.

Knutson, B., Fong, G.W., Adams, C.M., Varner, J.L., & Hommer, D. (2001b).
Dissociation of reward anticipation and outcome with event-related fMRI.
Neuroreport, 12(17), 3683-3687.

Knutson, B., Taylor, J., Kaufman, M., Peterson, R., Glover, G. (2005). Distributed Neural
Representation of Expected Value. Journal of Neuroscience, 25(19), 4806-4812.

Laibson, D. I. (1997). Golden Eggs and Hyperbolic Discounting. Quarterly Journal of
Economics, 112, 443-477.

Lamy, M. (2007). For Juice or Money. Journal of Neuroscience, 27(45), 12121-12122.

Lowenstein, G. (1988). Frames of Mind in Intertemporal Choice. Management Science,
34(2), 200-214.

Loewenstein, G., & Prelec, D. (1992). Anomalies in Intertemporal Choice: Evidence and
an Interpretation. Quarterly Journal of Economics, 107(2), 573-597.

Loewenstein, G. (1996). Out of control: Visceral influences on behavior. Organizational
Behavior & Human Decision Processes, 65(3), 272-292.

James, W. (1890). The Principles of Psychology. H. Holt and Company.

59

Jenkinson, M., & Smith, S.M. (2001). A Global Optimisation Method for Robust Affine
Registration of Brain Images. Medical Image Analysis, 5(2), 143-156.

Jenkinson, M., Bannister, P., Brady, M., & Smith, S. (2002). Improved optimisation for
the robust and accurate linear registration and motion correction of brain images.
NeuroImage, 17(2), 825-841.

McClure, S., Laibson, D., Loewenstein, G., & Cohen, J. (2004). Separate Neural Systems
Value Immediate and Delayed Monetary Rewards. Science, 306, 503-507.

McClure, S., Ericson, K., Laibson, D., Loewenstein, G., & Cohen, J. (2007). Time
Discounting for Primary Rewards. Journal of Neuroscience, 27(21), 5796-5804.

Metcalfe, J., & Mischel, W. (1999). A hot/cool-system analysis of delay of gratification:
Dynamics of willpower. Psychological Review, 106, 3-19.

Montague, P. R., & Berns, G. S. (2002). Neural economics and the biological substrates
of valuation. Neuron, 36(2), 265-284.

Monterosso, J., & Ainslie, G. (1999). Beyond discounting: possible experimental models
of impulse control. Psychopharmacology, 146(4), 339-347.

Monterosso, J., & Ainslie G (2007a). The behavioral economics of will in recovery from
addiction. Drug Alcohol Depend, 90 (Suppl 1), 100-111.

Monterosso, J., & Ainslie, G., Xu, J., Cordova, X., Domier, C., & London, E. (2007b).
Frontoparietal Cortical Activity of Methamphetamine-Dependent and Comparison
Subjects Performing a Delay Discounting Task Human Brain Mapping, 28, 383-
393.

Nevin, J. A., & Grace, R., C. (2000). Behavioral momentum and the law of effect.
Behavioral & Brain Sciences, 23, 73-130.

Packard, M.G., & Knowlton, B.J. (2002). Learning and memory functions of the basal
ganglia. Annu Rev Neurosci, 25, 563-593.

Patenaude, B., Smith, S., Kennedy, D., & Jenkinson, M (2007) FIRST - FMRIB's
integrated registration and segmentation tool. In Human Brain Mapping
Conference.

Pine, A., Seymour, B., Roiser, J.P., Bossaerts, P., Friston, K., Curran, H.V., & Dolan,
R.J. (2009). Encoding of marginal utility across time in the human brain. Journal
of Neuroscience 29(30), 9575–9581.

60

Read, D., Frederick, S., Orsel, B., & Rahman, J. (2005). Four Score and Seven Years
from Now: The Date/Delay Effect in Temporal Discounting Management
Science, 51(9), 1326-1335.

Rachlin, H. (1995). Self-control: Beyond commitment. Behavioral & Brain Sciences,
18(1), 109-159.

Rachlin, H. (2000). The Science of Self-Control. Cambridge: Harvard University Press.

Restak, R. (1984). The brain. New York : Bantam.

Roelofsma, P. H. M. P., & Read, D. (2000). Intransitive intertemporal choice. Journal of
Behavioral Decision Making, 13(2), 161-177.

Ross, D. (2005). Economic Theory and Cognitive Science:Microexplanation. Cambridge:
MIT Press.

Ross, D. (In press). Economic Models of Pathological Gambling in “What is Addiction”
Cambridg: MIT Press

Schelling, T.C. (1978). Economics, or the art of self-management. American Economic
Review, 68, 290-294.

Schelling, T. C. (1980). The intimate contest for self-command. Public Interest, 60, 94-
118.

Schelling, T. C. (1984). Self-command in practice, in policy, and in a theory of rational
choice. American Economic Review, 74, 1-11.

Shamosh, N., DeYoung, C., Green, A., Reis, D., Johnson, M., & Conway, A., et al.
(2008). Individual Differences in Delay Discounting: Relation to Intelligence,
Working Memory, and Anterior Prefrontal Cortex. Psychological Science, 19(9),
904-911.

Sharot, T., De Martino, B., & Dolan, R. J. (2009). How choice reveals and shapes
expected hedonic outcome. Journal of Neuroscience, 29, 3760–3765.

Shefrin, H. M., & Thaler, R. (1992). Mental accounting, saving and self-control. In J.
Elster & G. Loewenstein (Eds.), Choice Over Time. New York: Russell Sage
Foundation.

Simon, H. (1967). Motivational and emotional controls of cognition. Psychological
Review, 74, 29-39.

61

Strotz, R. (1956). Myopia and inconsistency in dynamic utility maximization. Review of
Economic Studies, 23, 165-180.

Thaler, R. (1980). Toward a positive theory of consume choice. Journal of Economic
Behavior and Organization, 1(1), 39-60.

Wager, T.D., Barrett, L.F. (2004a). From affect to control: functional specialization of the
insula in motivation and regulation. PsycExtra (online).

Wager, T.D., Jonides, J., & Reading, S. (2004b). Neuroimaging studies of shifting
attention: a meta-analysis. Neuroimge, 22(4), 1679-1693.
Wittmann, M., Leland, D., & Paulus, M. (2007). Time and decision making: differential
contribution of the posterior insular cortex and the striatum during a delay
discounting task Exp Brain Res, 179, 643-653.

Woolrich, M.W., Behrens, T.E.J., Beckmann, C.F., Jenkinson, M., & Smith, S.M. (2004).
Multi-level linear modelling for FMRI group analysis using Bayesian inference.
NeuroImage, 21(4), 1732-1747.

Woolrich, M.W. (2008). Robust Group Analysis Using Outlier Inference. NeuroImage,
41(2), 286-301.

Worsley, K.J. (2001). Statistical analysis of activation images. In P.Jezzard,
P.M.Matthews & S.M.Smith (Eds.), Functional MRI: An Introduction to
Methods. Oxford: Oxford Unviersity Press.

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Behabioral and neural evidence of state-like variance in intertemporal decisions

PDF

Sequential decisions on time preference: evidence for non-independence

PDF

Homeostatic imbalance and monetary delay discounting: effects of feeding on RT, choice, and brain response

PDF

Validation of a neuroimaging task to investigate decisions involving visceral immediate rewards

PDF

Reward substitution: how consumers can be incentivized to choose smaller food portions

PDF

The acute impact of glucose and sucralose on food decisions and brain responses to visual food cues

PDF

Visual and audio priming of emotional stimuli and their relationship to intertemporal preference shifts

PDF

The evolution of decision-making quality over the life cycle: evidence from behavioral and neuroeconomic experiments with different age groups

Asset Metadata

Creator Luo, Shan (author)

Core Title Behavioral and neural evidence of incentive bias for immediate rewards relative to preference-matched delayed rewards

School College of Letters, Arts and Sciences

Degree Master of Arts

Degree Program Psychology

Degree Conferral Date 2009-12

Publication Date 10/21/2009

Defense Date 09/21/2009

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag decision-making,fMRI,intertemporal choice,OAI-PMH Harvest,Self-control

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Monterosso, John R. (committee chair), Bechara, Antoine (committee member), Lu, Zhong-Lin (committee member)

Creator Email luoshanbest@gmail.com,shanluo@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-m2679

Unique identifier UC1153862

Identifier etd-Luo-3335 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-266711 (legacy record id),usctheses-m2679 (legacy record id)

Legacy Identifier etd-Luo-3335.pdf

Dmrecord 266711

Document Type Thesis

Rights Luo, Shan

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Repository Name Libraries, University of Southern California

Repository Location Los Angeles, California

Repository Email cisadmin@lib.usc.edu