University of Southern California Dissertations and Theses

Mechanisms of stress effects on learning and decision making in younger and older adults

(USC Thesis Other)

Mechanisms of stress effects on learning and decision making in younger and older adults

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

MECHANISMS OF STRESS EFFECTS ON LEARNING AND DECISION MAKING

IN YOUNGER AND OLDER ADULTS

by

Nichole Renee Lighthall

A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(GERONTOLOGY)

August 2012

Copyright 2012 Nichole Renee Lighthall

ii

DEDICATION
This dissertation is dedicated to my grandfather, Larry F., for showing me the
stars in his telescope and encouraging me to do “research” for our dinner table debates.
These moments contributed to my enduring curiosity about the big questions in life and
my belief that I could find answers to those questions.

iii

ACKNOWLEDGEMENTS
My thanks to my committee members, Mara Mather, Elizabeth Zelinski,
Bosco Tjan, and Antoine Bechara. In particular, I would like to thank Dr. Zelinski for
her unwavering support and assistance in my professional development, Dr. Tjan for his
excellent advice on neuroimaging methods and analysis, and Dr. Bechara for his
constructive criticism and demand for conducting meaningful research. Most of all, I
must thank my advisor Dr. Mather, without whom, my research abilities and career
aspirations would not be what they are today. This dissertation work was supported by
the National Institute on Aging R21AG030758, R01AG038043, T32AG0037 and
F31AG038137.

iv

TABLE OF CONTENTS
Dedication ii
Acknowledgements iii
List of Tables v
List of Figures vi
Abstract vii
Introduction 1
Chapter 1: Gender Differences in Reward-Related Decision Processing Under 6
Stress
Chapter 2: Stress Modulates Reinforcement Learning in Younger and Older 34
Adults
Chapter 3: Effects of Stress on Reinforcement Learning: Age Differences and 66
Neural Mechanisms
Chapter 4: Conclusion 95
References 102
Appendix A: Chapter 1. Supplementary Tables 123
Appendix B: Chapter 1. Supplementary Independent Component Analysis 125
Methods and Results
Appendix C: Chapter 3. Feedback Word List for Audio Clips 129
Appendix D: Chapter 3. Computational Model Fitting 130
Appendix E: Chapter 3. Criteria for Removal of Noise Components with 131
Melodic ICA
Appendix F: Chapter 3. Response to Learning Parameters Across Groups 132

v

LIST OF TABLES
Table 1.1: Regions with activation differences by gender and stress conditions 24
Table 2.1: Group characteristics 39
Table 2.2: Stress response and reinforcement learning outcomes by group 49
Table 3.1: Feedback distributions by cue pair and slot machine type 74
Table 3.2: Regions responding to learning components across groups 87

vi

LIST OF FIGURES
Figure 1.1: Two versions of the fMRI-adapted BART 12
Figure 1.2: Cortisol levels by stress and gender group 17
Figure 1.3: Gender-by-stress effects on behavior and earnings in the BART 19
Figure 1.4: Effort ratings by stress and gender group 21
Figure 1.5: Gender–stress interactions for the active vs passive contrast 23
Figure 2.1: Probabilistic selection task 43
Figure 2.2: Mean cortisol levels by stress condition across sample intervals 48
Figure 2.3: Stress-by-valence interaction for learning performance in younger 53
and older adults
Figure 3.1: Reinforcement-learning task 74
Figure 3.2: Subjective response to the cold pressor and control task 79
Figure 3.3: Alterations to self-reported motivation with stress 81
Figure 3.4: Interaction of age, stress and feedback valence for cue selection 82
accuracy
Figure 3.5 Plot of slot machine selections in choice trials 85
Figure 3.6 Group differences for response to learning components 88

vii

ABSTRACT
Stress is common in daily life and often present when making important decisions
that involve risk or that require learning from the positive and negative outcomes of past
choices (reinforcement-based learning). An emerging literature indicates that stress can
strongly influence these types of learning and decision making, but the mechanisms of
these stress effects are not yet clear. Stress affects numerous brain regions and networks
involved in motivated learning and decision making including those subserving reward
processing, attentional control, perception, and integration hubs for cognitive, affective,
and sensory information.
The first aim of this dissertation was to determine whether and how these neural
networks may mediate stress effects on learning and decision making (Study 1 and Study
3). Findings from Study 1 indicated that stress affects the involvement of the dorsal
striatum and anterior insula in young adults during risky decision making involving
monetary reward, but stress effects are opposite for men and women. Results from Study
3 revealed that stress affects the involvement of attentional control and visual perception
regions during social reinforcement learning.
A second aim of this dissertation was to examine age differences the impact of
stress on reinforcement learning (Study 2 and Study 3). Aging is associated with declines
in reinforcement learning abilities and changes to the neural networks involved in
reinforcement learning and stress response. Thus, stress may affect reinforcement
learning differently in younger and older people. Given older adults’ already
compromised functioning in this domain, it is important to determine whether stress

viii

enhances or impairs reinforcement learning in older age. Study 2 indicated that
reinforcement-based learning for positive cue-outcome associations is similarly enhanced
by acute stress in younger and older adults, but Study 3 found that only younger adults
had this stress-related enhancement. These mixed findings highlighted the potential
importance of task difficulty, level of stress arousal, and type of reinforcement in
determining when age differences in stress effects will be observed for reinforcement
learning. Finally, for brain activation, Study 3 found no significant stress-by-age effects,
but did find age differences during social reinforcement learning, indicative of an age-
related bias for positive social feedback. In sum, this dissertation provides insight into
how motivated learning and decision making are affected by short-term stress and how
these stress effects may depend on age. Findings presented here provide information
about young and older adults’ ability to manage risk- and reward-related decisions and
may inform interventions targeted at addressing age-specific cognitive vulnerabilities
during stress.

1

INTRODUCTION
An accumulating body of research suggests that acute stress alters learning and
decision making – particularly when potential rewards or incentives are involved (Mather
& Lighthall, 2012; Starcke & Brand, 2012). In particular, stress appears to affect risk
taking (Lighthall et al., 2009; Mather et al., 2009; Putman et al. 2010; Starcke et al. 2008)
and learning about from the frequency and magnitude of positive and negative outcomes
(reinforcement-based learning; Cavanagh, Frank, & Allen, 2010; Petzold et al., 2010;
Preston et al., 2007; van den Bos et al., 2009). There is, however, a substantial gap in this
literature, as previous studies used primarily behavioral methods and, thus, have not
determined how stress affects learning and decision processing in the brain. The current
dissertation aims to address this issue by using functional magnetic resonance imaging
(fMRI) to test specific hypotheses about the neural mechanisms of stress effects. This
research may have implications for public health as stress appears to be a strong trigger
for risky, reward-driven behaviors like substance abuse relapse (Brewer et al., 1998) and
numerous studies have found that neurochemical stress responses affect brain regions
involved in reward response to drugs (Jacobsen et al., 2001). Development of treatment
for people in vulnerable populations will benefit from an understanding of how stress and
motivated decision making interact in the brain.
Despite their vulnerabilities in the domains of learning and decision making,
previous research on stress and decision making has neglected to study older adults. It is
clear that at least a subset of non-demented older adults suffer from poor decision
making, some even facing long-term problems of addiction. For example, prescription

2

drug abuse is expected to persist in aging baby boomers (Dowling et al., 2008) and
pathological gambling rates are similar across adulthood when confounding variables are
controlled (Welte et al., 2001). Beyond risky behavior, older adults appear to be more
vulnerable to fraud and are disproportionately targeted for telemarketing scams
(American Association for Retired Persons, 1996). Furthermore, disadvantageous
decision making was observed in 35-40% of healthy, community-dwelling older adults
(Denburg et al., 2007) using the Iowa Gambling Task (IGT), which involves making
decisions about financial rewards under uncertainty. Performance on the IGT is thought
to depend on bodily sensations that signal potential consequences of an action and help
guide decision making (Bechara et al., 2005). As older adults with impaired performance
show normal physiological response to “good” and “bad” choices, age-related
impairments on the IGT are thought to arise from dysfunction in the integration of
emotional and somatic cues in decision processing (Denberg et al., 2007). These findings
suggest the possibility of age differences in how stress signals influence motivated
learning and decision making, but neither behavioral nor neuroimaging studies have been
conducted to determine whether age differences exist. A second aim of this dissertation is
to address this shortcoming, by including both younger and older adults and directly
testing for age differences.
Several lines of research have indicated that acute stress affects the function of
brain regions involved in motivated learning and decision making, and central to this
dissertation, these brain regions are differentially affected by normal aging. In particular,
acute stress is appears to affect several dopaminergic brain regions in the “reward

3

network” including the ventral (nucleus accumbens) and dorsal (putamen) aspects of the
striatum, as well as the medial prefrontal cortex (mPFC). Animal research and positron
emission tomography (PET) studies have found that acute stress increases dopamine
activity in the striatum and PFC (e.g., Abercrombie et al., 1989; Feenstra et al., 2000;
Kalivas & Duffy, 1995; Scott et al., 2006). Further, research indicates that stress releases
corticotropin releasing factor in the ventral tegmental area (VTA) which can trigger
reward seeking behavior by elevating glutamate release and thereby activating VTA
dopamine neurons (Wang et al. 2005). Relevant to reinforcement learning, dopamine
serves to “stamp-in” stimulus-outcome associations (linking the value of some behavior),
presenting the possibility that acute stress will alter reinforcement learning via effects on
dopaminergic neurons (Cohen and Frank, 2009 for review). Risk taking may also be
affected by stress via the reward network. For example, the lure of rewards and reward-
related brain activation in the medial PFC appears to be altered by even mild stressors
(Ossewaarde et al., 2011).
Stress may also affect motivated learning and decision making by way of brain
regions that serve as “integration hubs” for cognition, motivation, and affect. For
example, these three processes would co-occur and require integration if one were to
complete a memory task involving some reinforcement, while feeling stress about the
potential outcome (e.g., a final exam with the grade as the reinforcer). Brain regions that
serve to integrate these processes include the dorsolateral prefrontal cortex (DLPFC) and
anterior insula, and stress appears to alter functioning in both of them (e.g., Wang et al.,
2007; Qin, Hermans, van Marle, Luo, & Fernández, 2009). In addition, risky decision

4

making and reinforcement learning also rely on more basic processes such as attentional
control and vision perception. In particular, motivation to obtain some reward can bias
these regions such that they respond more strongly as motivation to execute the task
increases (Pessoa & Engelmann, 2010; Serences, 2008). The impact of acute stress on
these executive control and vision networks is not well studied; however, there is some
evidence that stress alters brain activation in these networks during tasks with emotional
stimuli (Henckens, Hermans, Pu, Joëls, & Fernández, 2009; van Marle, Hermans, Qin, &
Fernández, 2009).
Normal aging affects these brain regions and networks to different extents. The
PFC sustains the most age-related atrophy but substantial decline is also found in the
striatum (Raz, 2004) and insula (Good et al., 2001; Resnick et al., 2003). In particular,
there is dramatic decline in the mesocorticolimbic and nigrostriatal dopamine reward
system with advancing age (Kaasinen & Rinne, 2002; Rollo, 2009). The volume of brain
regions in attentional control (e.g., the anterior cingulate) and vision (occipital cortex) are
less affected by aging (Raz, 2004). In addition, there is evidence that aging is associated
with a shift in recruitment of these regions (i.e., as compared to younger adults) such that
frontal and parietal regions are recruited more, and occipital regions less with advancing
age (Davis, Dennis, Daselaar, Fleck, & Cabeza, 2008). In sum, while it is presently
difficult to make directional hypotheses about how stress will affect the neural correlates
of motivated cognition in older adults versus younger adults, the available research
suggests that age differences may be present.

5

To address the aims of this dissertation, three studies were conducted. The study
in Chapter 1 examined the impact of acute stress on behavior and brain activation in
younger men and women during a risky-decision task in which performance was
rewarded with monetary compensation. Chapter 2 describes a behavioral study conducted
with younger and older adults that examined the impact of age and acute stress on
reinforcement learning. The study in Chapter 3 used a social version of the
reinforcement-learning task from Study 2 and was conducted with fMRI to examine the
neural correlates of age and stress effects on social reinforcement learning. Each of these
studies manipulated stress between subjects using the cold pressor stress task, which was
conducted prior to the cognitive tasks. Together, these studies provide insight into the
mechanisms of stress effects on risky decision making and learning involving incentives
in early and late adulthood.

6

CHAPTER 1: GENDER DIFFERENCES IN REWARD-RELATED DECISION
PROCESSING UNDER STRESS
1

Nichole R. Lighthall, Michiko Sakaki, Sarinnapha Vasunilashorn,

Lin Nga, Sangeetha Somayajula, Eric Y. Chen, Nicole Samii,

and Mara Mather

Recent experimental studies reveal stress-induced alterations to motivated
decision making: stress alters reward learning (Cavanagh et al., 2010; Petzold et al.,
2010), risk taking (Lighthall et al., 2009; Porcelli and Delgado, 2009; Preston et al., 2007;
Starcke et al., 2008; van den Bos et al., 2009), reward responsivity (Bogdan and
Pizzagalli, 2006; Ossewaarde et al., 2011), and decision-making speed (Porcelli and
Delgado, 2009; van den Bos et al., 2009). Furthermore, several studies have observed
gender-dependent effects of stress, including our previous work (Lighthall et al., 2009),
which examined the impact of cold pressor stress (Lovallo, 1975) on subsequent decision
behavior for the Balloon Analogue Risk Task (BART; Lejuez et al., 2002). The BART is
a risky decision task that involves pumping up a series of computerized balloons in order
to earn reward. Balloons may be “cashed out” to collect earnings at any time, with larger
balloons yielding greater earnings. However, each additional pump increases the risk of
an explosion that eliminates earnings for that balloon. Thus, earnings on the BART are
optimized by balancing some risk-taking to earn reward while avoiding too many

1
This chapter is a pre-published paper. Lighthall, N. R., Sakaki, M., Vasunilashorn, F.,
Nga, L., Somayajula, S., Chen, E. Y, Samii, N., & Mather, M. (2011). Gender differences
in reward-related decision processing under stress. Social Cognitive and Affective
Neuroscience, doi: 10.1093/scan/nsr026. Nichole Lighthall holds the copyright to this
publication. Minor modifications have been made from the published version to fit within
the organization of the dissertation.

7

explosions. Our previous study revealed that men and women had similar BART
behavior and earnings under control conditions but diverged with stress (Lighthall et al.,
2009). Specifically, stress increased earnings and risk taking under uncertainty in males,
but decreased earnings and risk taking for females. Studies using psychological stressors
and the Iowa Gambling Task report consistent findings (Preston et al., 2007; van den Bos
et al., 2009). In men, psychological stress led to more high-risk disadvantageous choices.
In women, increased stress led to more low-risk advantageous choices, with some decline
in females’ performance at the highest levels of stress response (van den Bos et al.,
2009). In a study of men alone, pharmacologically elevated stress hormone levels
(cortisol) resulted in increased risk-taking behavior (Putman et al., 2010). Thus, at least in
males, activation of the hypothalamic-pituitary-adrenal (HPA) axis appears to influence
risk-related decision making. In addition, stress from threat of shock has been found to
decrease women’s reward responsiveness (Bogdan and Pizzagalli, 2006). Thus, stress
may decreases women’s risk taking by diminishing the lure of rewards that could be
gained thorough risky behavior. Supporting this proposition, a recent study found that
exposure to mild psychological stress (aversive movie clips) resulted in diminished
reward-related activation of the medial prefrontal cortex (PFC) in women during a
monetary incentive task (Ossewaarde et al., 2011). However, as males were not included
in this study, it is unclear how gender may modulate stress effects on neural response to
reward.

Little is known about the neural underpinnings of these gender-specific stress
effects. To our knowledge, the present study is the first fMRI study to examine gender

8

differences in response to decision making among individuals exposed to stress. We used
the BART as our decision task as we have previously observed gender differences in
stress effects with this task (Lighthall et al., 2009). Given the dearth of research on neural
mechanisms of gender-stress interactions in decision processing, hypotheses for the brain
regions mediating these interactions were derived by identifying brain regions that have
been independently associated with 1) gender-related differences in response to stress,
and 2) decision making on the BART.
Brain regions involved in decision making are also affected by acute stress
(Dedovic et al., 2009 for review), and show gender differences in stress response. For
example, Wang and colleagues (2007) exposed males and females to varying levels of a
psychological stressor with mental arithmetic during a perfusion fMRI scan. Stress
increased cerebral blood flow in the right PFC and decreased blood flow in the left
orbitofrontal cortex in men, but increased blood flow responses in limbic structures
including the insula, cingulate cortex, ventral striatum, and dorsal striatum (putamen) in
women. Further, cortisol reactivity predicted neural response to stress more in men than
in women. Gender differences in neural response to visceral stress have also been
observed, with greater stress responses in the ventromedial PFC, right anterior cingulate,
and left amygdala in women, but greater activation of the insula and right dorsolateral
PFC in men (Naliboff et al., 2003). Presenting negative pictures to elicit stress in men and
women revealed stronger amygdala and anterior cingulate responses in males compared
to females, with the magnitude of gender differences depending on menstrual cycle phase
(Goldstein et al., 2010). Several brain regions showing gender-specific stress responses

9

have also been associated with decision behavior on an fMRI-adapted BART (Rao et al.,
2008). In this study, voluntary risk taking was found to rely on the striatum, anterior
insula, midbrain, dorsolateral PFC, and anterior cingulate/medial PFC. Thus, the common
regions mediating BART behavior and gender-stress interactions appear to include the
striatum, anterior insula, and PFC, leading us to hypothesize that one or more of these
regions would moderate gender differences in stress effects on decision making in our
fMRI-adapted BART.
In the current study, healthy males and females were exposed to either the cold
pressor stress task or a control task prior to playing an fMRI-adapted version of the
BART, which included real monetary outcomes. Salivary cortisol was collected to
confirm an elevated HPA axis response during decision processing among stressed
participants. Based on our previous behavioral findings and those of others, we predicted
different effects of stress on behavior and brain activation for males and females. More
specifically, we predicted that stress would enhance risk taking in males but diminish risk
taking in females. Further, we hypothesized that, depending on gender, stress would alter
neural decision responses to the BART in one or more of our regions of interest. Given
the lack of previous studies examining gender-stress interactions in neural response to
decision making, we did not make directional hypotheses for effects on brain activation.
Methods
Participants
Twenty-three females (age 18-31, M
age
= 21.8 ± 3.6, 11 stressed) and 24 males
(age 18-33, M
age
= 23.0 ± 3.6, 12 stressed) participated in the study after providing

10

written informed consent approved by the University of Southern California Institutional
Review Board. One stressed female was excluded from the original group of 12 due to
protocol inconsistency. All were right-handed, non-smokers who did not use hormone
birth control, corticosteroid medications or beta-adrenergic agonists. Participants did not
have any chronic illnesses, history of head trauma or neurological disorders, were not
pregnant, and did not have any MRI contraindications. In order to observe stable cortisol
levels, all participants avoided eating and exercising within 1 hr of the study and avoided
sleeping within 2 hrs of the study. There were no differences by gender or stress group in
self-reported education, hours of sleep the previous night, or baseline measures of stress
(Daily Inventory of Stressful Events; Almeida et al., 2002), affect (Positive and Negative
Affect Scale; Watson et al., 1988), or depression (The Center for Epidemiologic Studies-
Depression Scale; Radloff, 1977). Participants were paid $15 in addition to their earnings
from the decision task.
Protocol
The study was conducted from 2-5 pm to reduce diurnal variations in cortisol
levels. Participants completed psychosocial questionnaires and drank 8 oz of water
(completed ≥ 10 min before the first saliva sample). Participants received scan- and task-
related instructions and practiced two abbreviated trials of each task condition. Baseline
saliva samples were then collected followed by either cold pressor stress or a control task
outside the scanner. Next, participants entered the scanner and a brief structural scan was
conducted. The decision task with fMRI followed, beginning ~24 min after the start of
the stress manipulation. To measure HPA axis response to the cold pressor or control

11

task, saliva samples were taken immediately before and after the decision task, while
participants were in the scanner. At the end of the session, participants provided post-
experiment ratings of stress experienced during the hand immersion task and fMRI scan
(7-point Likert scale; 1 = no stress, 7 = a great deal of stress) as well as ratings of effort
put into the BART (7-point Likert scale; 1 = no effort, 7 = a great deal of effort).
Stress Induction
The cold pressor task was used to induce a stress response in half of the
participants of each gender. For the stress task, participants held their non-dominant hand
in a pitcher of ice water at 0-5° C for as long as they could up to three minutes. No
participants quit the cold pressor before 60 s passed. For the control task, participants
held their non-dominant hand in a pitcher of warm water at 37-40° C for three minutes.
Stress conditions were randomly assigned and participants did not know their stress
condition until administration of the hand immersion task. To increase the strength of the
stress manipulation, participants were told they might be asked to repeat their assigned
hand immersion task at the end of the session. Four female and two male stress subjects
quit the cold pressor task before three minutes elapsed (range: 60-120 s); however, there
were no significant gender differences in duration of hand immersion in the cold pressor
group, F
1,21
= 1.78, P = .20.
Salivary Biomarkers
Before the manipulation, participants provided 1 ml of saliva (s1) for the
assessment of baseline cortisol. Immediately before (s2) and after (s3) the decision task
(21 min and 35 min after the start of the manipulation, respectively), saliva samples for

12

cortisol assay were collected while participants lay in the scanner bore. Based on prior
research (Dickerson and Kemeny, 2004, see also Schwabe et al., 2008), cortisol
responses to stress were expected to be at their peak during the decision task. Post-stress
samples were collected using sorbettes (Salimetrics, LLC, State College, PA). The
difference between s1 (baseline) cortisol levels and the average of s2 and s3 levels was
used to measure cortisol change. Post-experiment, samples were stored in a laboratory
freezer at -30°C and later transported frozen to a CLIA-certified laboratory (Salimetrics,
LLC, State College, PA) and stored frozen at -80°C until assayed. Samples were
centrifuged on the day of assay at 3000 rpms for 15 min to remove mucins and
biomarkers were duplicate tested.
Decision Task

13

The Balloon Analogue Risk Task (BART; Lejuez et al., 2002) was adapted to
allow for a blocked design and programmed using MATLAB software (The Mathworks,
Inc., Natick, MA). The task included four “active” blocks and four “passive” blocks.
During active blocks, the words PLAY GAME appeared at the top of the screen. Also on
the screen was a red balloon, a Money Earned box, a Cash out $$$ button, and a Click to
pump balloon button (Figure 1.1 A). Passive blocks included the same visual and
auditory stimuli except that the words KEEP CLICKING appeared at the top of the screen
and the Cash out $$$ button was disabled (Figure 1.1 B).
In active blocks, participants used the pump button (right hand) to increase the
size of balloons presented one at a time, accumulating money in a temporary bank for
each pump. To encourage participants to pump balloons to larger sizes, they were told
that the amount they could earn per pump was larger for bigger balloons. Pumps 1-20
earned $.03 each while pumps over 20 earned $.07 each; however, the payout values and
cutoffs were not disclosed to participants. Participants collected earnings for individual
balloons by pressing the cash out button (left hand), which transferred earnings from their
temporary bank to their permanent bank (the Money Earned box). Cash outs triggered a
slot machine payout sound and explosions involved a balloon popping sound.
Subsequently participants received a new, uninflated balloon. Every balloon was set to
explode at a random point from pump 1-90; with each pump, the likelihood of a balloon
explosion increased. Explosions resulted in a loss of earnings in the temporary bank but
not the permanent bank. As participants did not know the probability of balloon
explosions on any given pump, active blocks involved making reward-related decisions

14

under uncertainty. Participants received their full earnings at the end of the experiment
(actual earnings ranged from $3.94 to $45.15).
In passive blocks, participants pressed the pump button as balloons appeared at
random sizes; however, they could not cash out or earn money. Passive blocks also
included cash out and explosion sounds played at random, at the approximate frequency
experienced in active blocks. In this way, sensory experiences were similar in the passive
and active blocks, but in passive blocks no rewards were gained or lost.
The length of active blocks was ≥ 60 s (range 60-82 s) such that participants were
allowed to complete the balloon currently on the screen 60 s after the start of an active
block. Active block time exceeding 60 s was subtracted from the subsequent passive
block, which also had a base time of 60 s (i.e., passive blocks ≤ 60 s, range 38-60 s). For
example, if a participant spent an additional 4 s to finish their last balloon in an active
block (total block time = 64 s), the following passive block would be 56 s (60 s – 4 s).
This allowed for observation of each balloon’s outcome in active blocks (explosion or
cash out). Fixation periods lasting 10 s occurred before and after each active or passive
block.
Imaging Data Acquisition
Imaging was done using a 3T Siemens MAGNETOM Trio scanner with a 12-
channel matrix head coil at the USC Dornsife Cognitive Neuroscience Imaging Center.
Functional scans were acquired in a single 9.5 min run, with a repetition time of 2000 ms
in a T2*-sensitive echo-planar imaging sequence (echo time, 25 ms; flip angle, 90°).
Volumes included 31 slices at 3.5-mm thickness (in-plane resolution, 3 x 3 mm; no gap;

15

matrix size = 64 x 64) extended axially from the temporal lobe to the top of the skull.
Prior to the functional scan, high-resolution structural scans were acquired using a T1-
weighted MPRAGE sequence (resolution, 1 x 1 x 1 mm; repetition time, 1950 ms; echo
time, 2.26 ms; flip angle, 7°).
Whole-brain analysis
Whole-brain analyses were conducted with FMRIB's Software Library (FSL;
www.fmrib.ox.ac.uk/fsl) using FSL FEAT v. 5.98. Preprocessing included: motion
correction with MCFLIRT, spatial smoothing with a Gaussian kernel of full-width half-
maximum 5 mm, high-pass temporal filtering equivalent to 140 s, and skull stripping of
structural images with BET. MELODIC ICA (Beckmann and Smith, 2004) was used to
remove noise components. Components with normalized time-courses that contained a
spike in the component-related activity during one TR for which the absolute value of the
peak of that spike minus the peak of the next largest spike during the whole scan was
three standard deviations or larger were removed from the data. In addition, components
that visually encompassed whole slices or formed rings around the brain were also
removed. Registration was performed with FLIRT; each functional image was registered
to both the participant’s high-resolution brain-extracted structural image and the standard
Montreal Neurological Institute (MNI) average of 152 brains (with 2-mm voxel
resolution) using an affine transformation with 12 degrees of freedom.
The individual time-series statistical analysis was carried out using FILM
(Woolrich et al., 2001) with local autocorrelation correction. Both explanatory variable
regressors (active, passive), convolved with a double-gamma hemodynamic response

16

function, and their temporal derivates were used to model data. The primary lower level
contrasts were conducted for active – passive and its inverse. Higher level mixed effects
analysis was carried out using FMRIB’s local analysis of mixed effects (FLAME 1+2;
Beckman et al., 2003). The general linear model (GLM) included two between-subject
conditions, each with two levels: gender (male, female) and stress (cold pressor, control
task). Unequal variance among the four gender/stress groups was assumed. The GLM
was used to test for main effects of stress and gender and their interaction for the two
lower-level contrasts. In these lower-level and higher-level analyses, Z (Gaussianised
T/F) statistic images were corrected for multiple comparisons with clusters determined by
Z > 2.3 voxel-wise thresholding and a family-wise error-corrected cluster significance
threshold of P < 0.05 (Worsley, 2001). To facilitate communication of results including
clusters spanning several brain regions (revealed by the whole brain analysis at Z > 2.3,
cluster threshold P < 0.05), more stringent thresholds were applied in some cases (see
Appendix A for Supplementary Tables).
Region-of-interest (ROI) analysis
The whole-brain analyses described above test for gender-by-stress interactions,
but do not indicate which group showed more task-related activation. As post-hoc tests to
characterize the direction and relative magnitude of gender-specific stress effects, ROIs
were created. These were based on the significant clusters of activation revealed in the
whole-brain gender-by-stress interactions that were hypothesized to mediate gender-
stress interactions. For significant clusters spanning multiple brain regions, anatomical
borders for ROIs were structurally defined using masks from FSL’s MNI structural atlas

17

(based on probabilistic map; probability at .5). Average percent signal change values
were determined for each ROI. To determine the relationship between individual
differences in ROI activation and other outcome measures, Pearson’s correlations were
conducted across and within groups.
Results
Salivary cortisol
The cold pressor nearly doubled mean cortisol levels in stress subjects while
cortisol levels did not change in controls, F
1,43
= 25.56, P = .000002 (Figure 1.2).
Although mean cortisol levels were higher in males overall, F
1,43
= 5.15, P = .03, there
were no main effects of gender on cortisol change, F
1,43
< 1, nor was there a gender-by-
stress condition interaction, F
1,43
< 1. The effect of the stressor on cortisol elevation
remained highly significant after excluding participants from the analysis who did not

18

complete the full 3-min cold pressor challenge (4 females, 2 males), F
1,37
= 22.73, P =
.000003, and cortisol change in response to the stress condition did not differ for males
and females after excluding these participants, F
1,37
< 1. These results indicate that the
cold pressor reliably elevated cortisol levels without significant gender-specific effects.
Subjective stress: Hand immersion task
Post-experiment ratings of stress resulting from the hand immersion task indicated
that subjective stress experienced by participants in the cold pressor group was greater
than that experienced by the control group, F
1,43
= 231.75, P < .000001 (M
stress
= 5.11 ±
1.41; M
control
= 1.04 ± .21). Subjective responses to the cold pressor were greater in
women than men, F
1,43
= 4.57, P = .04 (M
women
= 5.73 ± .91; M
men
= 4.50 ± 1.57), with no
gender differences in the control group (M
women
= 1.0 ± .00; M
men
= 1.08 ± .29), resulting
in a gender-by-stress interaction, F
1,43
= 6.00, P = .02. Additional analyses indicated that
gender differences in stress-modulated decision processing were not simply the result of
greater subjective stress response in females compared to males.
Specifically, excluding participants who completed less than 3 min of the cold
pressor challenge did not alter observed group differences. Consistent with whole sample
results, ratings were higher in the stress group, F
1,37
= 212.28, P < .000001(M
stress
= 4.81
± .40; M
control
= 1.04 ± .33), in females versus males,F
1,37
= 4.89, P = .03 (M
female
= 3.21 ±
.39; M
male
= 2.64 ± .35), and stressed females compared to stressed males (gender by
stress effect), F
1,37
= 6.42, P = .02 (M
stress_female
= 5.43 ± .62; M
stress_male
= 4.20 ± .52;
M
control_female
= 1.00 ± .47; M
control_male
= 1.08 ± .47).

19

Consistent with previous
reports (Dickerson and Kemeny,
2004), however, subjective
distress ratings did not correlate
with cortisol responses to the
stressor. That is, higher
subjective stress ratings for the
cold pressor did not predict
cortisol response to the stressor
across gender groups, R
23
= .11,
P = .64, or within stressed male
and female groups individually
(R
12
= .42, P = .18 and R
11
= -
.40, P = .23, respectively).
To address the possibility
that gender differences in
behavioral and neural outcomes
were the result of differential
stress in males and females,
additional analyses for primary
outcome measures were conducted in which males and females did not differ
significantly in their subjective stress ratings of the cold pressor. These analyses excluded

20

the three females with the highest subjective stress ratings for the cold pressor (7 out of 7)
and the two male subjects with the lowest rating for the cold pressor (2 out of 7). Within
the remaining sample, there was still a strong main effect of the stress condition on
subjective stress ratings, F
1,38
= 448.84, P < .000001 (M
stress
= 5.13 ± .30; M
control
= 1.04 ±
.31), but ratings no longer differed for males and females in the stress condition, F
1,38
<
1(M
stress_male
= 5.00 ± .39; M
stress_female
= 5.25 ± .44). Importantly, differences in cortisol
response due to the stress condition also remained highly significant after excluding
subjects to control for gender differences in subjective stress, F
1,38
= 26.43, P = .000009
(M
stress
= .19 ± .06; M
control
= -.01 ± .05). And, as in the full sample, there was no gender
by condition interaction for cortisol change when subjective ratings were similar in
stressed males and females, F
1,38
< 1.
Behavioral data
Consistent with our previous behavioral findings (Lighthall et al., 2009), BART
behavior and earnings were similar for males and females under control conditions, but
diverged with stress, leading to a gender-by-stress interaction. As displayed in Figure 1.3,
stress increased gender differences in reward collection rate (mean number of balloons
“cashed out”), F
1,43
= 4.82, P = .03, decision speed (button pressing intervals), F
1,43
=
4.79, P = .03, and total earnings, F
1,43
= 7.93, P = .007. Examination of confidence
intervals indicated no gender differences in these outcome measures in the control
condition, only in the stress condition. While gender-by-stress interactions were observed
for these measures, individual cortisol change values were not correlated with any of the
behavioral outcomes across conditions or within any groups (Ps > .05). Regarding our

21

measure of risk taking (mean number of pumps per balloon for non-exploding balloons)
we did not find any differences by stress condition, F
1,43
< 1, gender, F
1,43
< 1, nor was
there a gender-by-stress interaction, F
1,43
< 1. Relative to the number of pumps possible
per balloon (maximum = 90), the average number of pumps was fairly low across groups
(M = 19.39 ± 2.38), indicating low risk taking overall. As the likelihood of losses
(“explosions”) during the BART
increased with the number of
pumps per balloon, the average
number of explosions
experienced per block was also
low across conditions (M = 2.59
± .31). There were no significant
group differences or interactions
in the number of explosions per block (Ps > .05); as may be expected given the low
number of pumps per balloon on average.
While risk taking was not modulated by gender or stress in this study, our fMRI-
adapted BART differed from the original task (Lejuez et al., 2002) in that the number of
balloons played was only limited by the duration of active blocks. This introduced a
potentially successful strategy – not present in the original BART – of playing as many
balloons as quickly as possible in order to earn more money. Notably, Pearson’s
correlations confirmed that balloon count and decision speed were related to earnings
(R
47
= .33, P = .03; R
47
= -.76, P < .000001, respectively). Direction of stress effects by

22

gender indicated that, from an earnings standpoint, stress led to more profitable decision
behavior in males but less profitable behavior in females.
Subjective stress: Scan session
Post-experiment ratings of stress experienced during the brain scan (with
concurrent decision task) were similar across stress conditions, F
1,43
< 1, gender groups
F
1,43
< 1, and these two factors did not interact to affect stress ratings, F
1,43
< 1. Based on
the 7-point rating scale, ratings of stress experienced during this period appeared to be
moderately low (M = 2.49 ± .31). Thus, despite apparent differences in task performance
and strategy by males and females in the stress condition, participants in these groups, as
well as those in the control condition, rated their level of effort similarly.
BART effort
Post-experiment ratings of the amount of effort participants put into playing the
decision game did not differ by stress condition F
1,43
< 1, gender F
1,43
< 1, nor was there
an interaction between these factors, F
1,43
< 1 (Figure 1.4). Based on the 7-point rating
scale, ratings of expended effort across participants were moderately high (M = 4.98 ±
.42). Thus, despite gender differences in task performance and strategy in the stress
condition, males and females in these groups, as well as those in the control condition,
rated their level of effort similarly.
Whole-brain analyses
As expected, decision-related activation (active – passive contrast) across groups
was observed in regions associated with motivation and decision making. In particular,
the decision task resulted in robust activation of the thalamus, putamen, caudate, anterior

23

cingulate, dorsolateral and
ventrolateral PFC, insula,
inferior parietal lobe, and
inferior frontal gyrus.
Significant clusters were
also apparent in
sensorimotor and visual
structures (Appendix A,
Supplementary Table 1).
To address the possibility
that dorsal striatum
activation differences
merely reflected
differences in motor
movements, Z-scores were calculated for button pressing in both active and passive tasks
(standardized across all participants). The difference between active button pressing and
passive button pressing (Z
active
– Z
passive
) was calculated for each participant representing
motor activity for the contrast of interest in our fMRI analysis (active – passive).
Including button pressing difference from active to passive tasks as a covariate did not
alter the interaction of gender and stress for activation in the putamen (F
1,42
= 22.16, P =
.0001).

24

Of primary interest, group level analysis of decision-related activation revealed
gender-by-stress interactions in motivation and decision regions – most notably in the left
dorsal striatum (putamen) and left anterior insula (Figure 1.5; Table 1.1). Gender-stress
interactions were also apparent in sensorimotor and visual regions.
ROI analyses
We anticipated gender-
stress interactions for brain
activation response to reward-
related decision processing in the
insula, PFC, and striatum. The
whole-brain analysis revealed
gender-by-stress interactions in
the dorsal striatum (putamen) and
anterior insula (Figures 1.5 A,
1.5 B, and 1.5 C). We examined the direction of effects in these ROIs by extracting mean
percent signal change values by group for the lower-level contrast of active – passive. An
ANOVA was performed on signal change values with gender and stress condition as
between-subject factors and significant interactions were found for both the left dorsal
striatum, F
1,43
= 22.51, P < .0001, and the left anterior insula, F
1,43
= 6.88, P < .05.
Examination of group means revealed that stress increased activation in the dorsal
striatum and anterior insula for males during decision making but decreased activation in
these regions for females (Figures 2.5 D and 2.5 E). Dorsal striatum activation did not

25

appear to be the result of differences in motor movements alone between the active and
passive conditions (see Results above). Further, an Independent Component Analysis
(ICA; Calhoun et al., 2001) was conducted to identify functional networks in the brain
that were differentially involved in the active BART depending on one’s gender and
stress status (see Appendix B for Supplementary Methods and Results). The ICA results
largely confirm results from the whole brain GLM and ROI analyses, suggesting that
males and females generally relied on the same network of brain regions to complete the
BART; but under stress, there were gender differences in the involvement of the putamen
and insula in this network.
Correlations were determined for cortisol change and activation in dorsal striatum
and insula ROIs across and within conditions. For males only, change in cortisol
predicted activation in the dorsal striatum ROI during decision processing (R
24
= .55, P =
.005; all other Ps > .05); change in cortisol did not predict insula activation for any group
(Ps > .05). Thus, in males, higher levels of physiological stress response were associated
with enhanced dorsal striatum response to reward-related decision making.
Behavior and ROI Correlations
Across participants, dorsal striatum ROI activation was positively correlated with
number of balloons cashed (i.e., reward collection; R
47
= .64, P = .000001) and total
money earned (R
47
= .38, P = .008), and negatively with risk taking as measured by
number of pumps per balloon (R
47
= -.40, P = .005). We also observed a correlation
between anterior insula ROI activation and number of balloons cashed (R
47
= .30, P =
.04).

26

Analyses conducted separately for the stress and control groups revealed a
significant positive relationship between dorsal striatum ROI activation and number of
balloons cashed (R
23
= .87, P < .000001) and a negative relationship with activation in
this region and risk taking (R
23
= -.60, P = .002) in the stress group. A positive correlation
was also observed in the stress group for anterior insula ROI activation and number of
balloons cashed (R
23
= .60, P = .002). In control participants, the only significant
correlation observed was between decision speed and activation of the dorsal striatum
ROI (R
24
= -.43, P = .03).
Breaking groups into gender-stress cells revealed that significant relationships
between dorsal striatum ROI activation and behavioral outcome measures were primarily
driven by stressed males. Specifically, a significant positive correlation was observed for
dorsal striatum ROI activation and number of balloons cashed (R
12
= .90, P < .000001)
and a negative correlations between dorsal striatum ROI activation and risk taking (R
12
=
-.75, P = .005). No other significant relationships were observed within gender-stress
groups. However, as a relationship between anterior insula activation ROI activation and
reward collection was observed for the stress group as a whole, but not for stressed males
and females alone, we tested for a statistical difference in non-significant correlations for
these two groups (Blalock, 1972), but found that there was none (P > .05). This indicated
that the relationship between anterior insula ROI activation and reward collection was
similar in stressed males and females despite group differences in activation of this
region.
Discussion

27

Exposure to cold pressor stress resulted in differential reward-related decision
processing on a risky decision task in males and females. Specifically, stress exposure
affected behavior and brain activity during the decision task in opposite ways for men
and women. Across conditions the risky decision task elicited robust responses in reward-
and decision-associated regions including the thalamus, striatum, anterior cingulate,
insula, as well as prefrontal and parietal regions (for reviews, see Clark, 2010; Ernst and
Paulus, 2005; Haber and Knutson, 2010; Taylor et al., 2007). The decision task was
timed to occur during HPA axis response to cold stress (~24 min post-stress). Salivary
hormone measures confirmed a significant elevation in cortisol for the stress group
during decision making with no gender differences in cortisol response. We also
observed no gender differences in decision behavior or neural response to decision
making under control conditions. With stress, however, decision behavior diverged for
males and females for several outcome measures – but not risk taking, which was low
across groups. This finding runs counter to our prediction that risk taking would increase
in males and decrease in females with stress, but task design limitations may explain this
finding as discussed below.
We did observe gender-dependent stress effects for number of balloons “cashed
out” (reward collection rate), decision speed, and total money earned. Relative to
controls, males exposed to cold stress exhibited more profitable decision behavior which
included faster decision responses and more cashed balloons, while stressed females had
slower decision responses and fewer cashed balloons resulting in diminished earnings.
Notably, these behavioral differences were associated with group and individual

28

differences in brain activation. The fMRI results presented here are the first we know of
to demonstrate that exposure to an acute stressor affects brain activity during motivated
decision making differently for healthy men and women. Consistent with our predictions,
the striatum and insula were associated with gender-dependent stress effects on decision
processing. Specifically, exposure to cold stress increased neural response to the risky
decision task in the dorsal striatum (putamen) and anterior insula among men, but
decreased response in these regions among women.
Differences in correlations between activation of these ROIs and behavior by
group shed light on the underlying mechanisms of gender-dependent stress effects in this
decision task. For participants exposed to cold stress, activation of the anterior insula ROI
was associated with reward collection rate. Relevant to this finding, the anterior insula
has been associated with risk-less choices and behavioral switching from risky to safe
choices (Kuhnen and Knutson, 2005). Risk taking was low overall but with each
additional pump of the balloon, the chance of losses (“explosions”) increased. Thus,
making the decision to stop inflating a balloon, and collect its earnings, reflects a switch
from taking risk to making a “safe” decision. The relationship between insula ROI
activation and reward collection rate was similar across stressed males and females,
suggesting that the anterior insula mediated cash-out choices under stress for both
genders.
Stress did not alter risk taking in men or women as previously observed (Lighthall
et al., 2009; Preston et al., 2007; van den Bos et al., 2009). In fact, participants displayed
low risk-taking behavior (number of pumps per balloon) across groups. The absence of

29

group differences in risk taking is likely due to changes made to the BART for
compatibility with fMRI analysis. In particular, the original BART (Lejuez et al., 2002)
limited the number of balloons, while our version of this risky decision task did not limit
the balloon number but instead limited the total time available to play the game. Thus, in
our task, participants could increase earnings by increasing decision speed and keeping
balloons relatively small to reduce risk of losses. While this design choice may have
limited our ability to observe differences in risk processing, a notable strength of our
design was that participants could respond flexibly in a way that captured the impact of
stress on motivated decision processes.
Indeed, because multiple strategies could be used to increase profits in our
decision task, the present study provides new insight into the conditions under which men
and women may become more risk seeking or risk averse. That is, this version of the
BART presented two potentially profitable strategies: a) fast decision speed with greater
risk taking and longer reward delays (large balloons, cashed intermittently), or b) fast
decision speed with less risk taking and shorter reward delays (small balloons, cashed
frequently). Our results revealed that when an alternative low-risk option was present that
provided rapid delivery of small rewards during the entire active block, males were
biased toward this option under stress. This stress-related shift in behavior appeared to be
related to activation of the dorsal striatum as, in stressed males alone, activation in this
ROI was significantly associated with an increased reward collection rate and less risk
taking. Further, stress-related effects in males included increased decision speed,
consistent with more automatic processing (Porcelli and Delgado, 2009). In addition,

30

among males only, cortisol change from baseline to the decision task was positively
correlated with decision-related activation in the dorsal striatum. This finding is
consistent with previous reports of stronger relationships between cortisol and neural
response to stress in males (Wang et al., 2007) and further suggests that males and
females differ in the degree to which acute fluctuations in cortisol predict neural response
to motivated decision making. Relatedly, some recent evidence suggests that male
traders’ cortisol responses to volatile financial markets may result in exaggerated market
movements (Coates and Herbert, 2008). An important avenue for future research will be
to determine whether real-life financial decisions, including stock trading, are
differentially affected by physiological stress responses in men and women.
Compared with female controls, stressed females in our study exhibited decreased
dorsal striatum activation, slower decision speeds and fewer reward collections. Our
findings are consistent with other reports of decreased reward responsiveness in stress-
exposed females (Bogdan and Pizzagalli, 2006; Ossewaarde et al., 2011); as, in our study,
stressed females tended to collect their earnings less frequently (i.e., decreased drive for
small rewards) while stress did not affect risk taking. Furthermore, in contrast to males,
exposure to cold stress led to slower decision speed in females, perhaps indicative of
more deliberative processing under stress in females. These stress effects in women are in
line with a previously observed trend towards greater explicit knowledge about game
contingencies in females with increasing stress response, but opposite patterns in males
(Preston et al., 2007). Together with stress effects we observed in males, these findings
support the conclusion that the dorsal striatum mediated gender-stress interactions in

31

level of automatic and reward-driven processing for the BART. The dorsal striatum is
thought to integrate sensorimotor, cognitive, and motivational, as well as emotional
signals (Balleine et al., 2007). In decision making, this region appears to play a role in
obtaining predictable rewards (Doya, 2008). For example, single-cell recordings with
monkeys show activation of the dorsal striatum during execution of well-learned
behaviors resulting in a juice reward (Miyachi et al., 2002). In our study, dorsal striatum
activation was associated with reward-motivated behavior that carried little risk. That is,
cashing out many smaller balloons quickly to accumulate small – but predictable –
rewards. This behavior is also consistent with the proposed role of the dorsal striatum as
the “instrumental actor” that maintains information action-reward associations
(O’Doherty et al., 2004).
While this study provides new information about gender-stress interactions in
motivated decision processing, further research is needed to better understand interaction
mechanisms. In particular, future studies may use more controlled fMRI tasks to examine
stress effects on specific decision components (e.g., Bolla et al., 2004; Rao et al., 2008;
Xue et al., 2010). For example, studies may test stress effects on level of automatic
processing or reward responsiveness among men and women. Implementation of
different stressors may also help to determine the precise mechanisms of gender
differences in decision making under stress. We chose to use the cold pressor stress task,
which resulted in equivalent and sustained cortisol responses in men and women. Some
gender differences in subjective stress response to the cold pressor were observed, which
may have reflected real differences in psychological stress and/or other factors, such as

32

gender-related social norms about expressing pain-related stress. With respect to social
factors, our study included a female experimenter at each session, which has been related
to underrerported pain, unpleasantness, and arousal in males exposed to a thermal
stressors; even when their physiological response is similar to female subjects (Aslaksen
et al., 2007; Levine and De Simone, 1991). Although our behavioral and fMRI findings
were largely unaffected after we controlled for gender differences in subjective stress
response, important insights can be gained from studies that specifically examine the
relationship between levels of psychological distress and decision processing. In
particular, we did not find gender-stress interactions in PFC response to decision making
as hypothesized, but it is possible that our choice of stressor impacted our ability to
observe group differences in this region. This proposal is supported by research
indicating that exposure to a psychological stressor (aversive movie clips) altered PFC
response to reward processing among females (Ossewaarde et al., 2011). Finally, from an
earnings perspective, more deliberative processing among stressed females was not
beneficial in our risky decision task due to time constraints. A full understanding of
gender-stress interactions in decision making requires consideration of decision tasks in
which optimal behavior is associated with thoughtful and rational processing. It may be
in these situations that women perform best under stress.
In sum, the current study found that cold pressor stress altered motivated decision
making on a risky decision making task, and did so in a gender-specific manner.
Behavioral results indicated that risk taking was not altered by stress when an alternative
option was present that offered rapid delivery of small rewards under a time constraint. In

33

addition, neural substrates of reward-motivated decision making for this task, including
the dorsal striatum and anterior insula, were differentially altered by stress exposure in
males and females. The present study also found differences in decision speed between
men and women only after stress exposure, which raises the possibility that stress leads to
gender differences in level of processing. While the current study contributes to our
understanding of the neural mechanisms of these gender-stress interactions, it also begs
the larger question about why such interactions exist. Addressing this question is likely to
require consideration of individual effects of social environment, genetics, sex hormones,
development and their interactions.

34

CHAPTER 2: STRESS MODULATES REINFORCEMENT LEARNING IN
YOUNGER AND OLDER ADULTS
2

Nichole R. Lighthall, Marissa A. Gorlick, Andrej Schoeke, Michael J. Frank,
and Mara Mather

Good choices require predicting how each option is likely to turn out. These
predictions are often based on past experience with the same or similar options.
Experiencing stress while making choices and learning about options is not uncommon;
an estimated 80% of Americans report at least moderate levels of stress on a daily basis
(American Psychological Association, 2008). In addition, learning about decision
options can be stressful (e.g., comparing complex health care plans) and high-stake
choices (e.g., which job offer to accept) may elicit stress given the weight of their
potential outcomes. An emerging literature indicates that a brief episode of stress affects
subsequent decision behavior. In particular, behavioral studies suggest that stress alters
reward-motivated decision making (Cavanagh, Frank, & Allen, 2010; Lighthall et al.,
2011; Lighthall, Mather, & Gorlick, 2009; Mather, Gorlick, & Lighthall, 2009; Petzold,
Plessow, Goschke, & Kirschbaum, 2010; Preston, Buchanan, Stansfield, & Bechara,
2007; Starcke, Wolf, Markowitsch, & Brand, 2008; van den Bos, Harteveld, & Stoop,
2009).

2
This chapter is a paper submitted for publication. Lighthall, N. R., Gorlick, M. A.,
Schoeke, A., Frank, M. J., & Mather, M. (in revision). Stress modulates reinforcement
learning in younger and older adults. Minor modifications have been made from the
submitted version to fit within the organization of the dissertation.

35

In the real world, the outcomes of decision options are often uncertain – requiring
that people make choices based on the perceived likelihood of positive and negative
results. Thus, decisions often rely on previous learning. For instance, reinforcement
learning involves making associations between choices and subsequent outcomes based
on experience. Laboratory studies with young adults indicate that acute stress may
improve learning about potential rewarding outcomes, but impair learning about potential
aversive outcomes. For example, experiencing a psychosocial stressor impaired learning
which cues were more frequently associated with negative feedback but did not impair
learning which cues were more frequently associated with positive feedback (Petzold et
al., 2010). Using the same probabilistic selection task, another study found that social
stress was associated with better positive feedback learning, but poorer negative feedback
learning, in young adults with low punishment sensitivity compared to those with high
punishment sensitivity (Cavanagh et al., 2010). In addition, a study with college-age
adults found an association between higher basal stress hormone levels (cortisol) and
reward dependency (greater monetary gains) on the Iowa Gambling Task (van Honk,
Schutter, Hermans, & Putman, 2003).
These stress effects on reinforcement learning may be related to stress and stress-
hormone-related enhancements to dopamine in key reward processing regions
(Abercrombie, Keefe, DiFrischia, & Zigmond, 1989; Anstrom & Woodward, 2005;
Feenstra, Botterblom, & Mastenbroek, 2000; Imperato, Puglisi-Allegra, Casolini, &
Angelucci, 1991; Kalivas & Duffy, 1995; Piazza et al., 1996). Stress has also been found
to alter synaptic plasticity in dopamine neurons (Saal, Dong, Bonci, & Malenka, 2003),

36

increase drug craving (Sinha, 2008), and, in positron emission tomography (PET) studies,
enhance striatal dopamine responses to painful stressors (Scott, Heitzeg, Koeppe, Stohler,
& Zubieta, 2006) and psychological stressors in stress-vulnerable individuals (Pruessner,
Champagne, Meaney, & Dagher, 2004; Soliman et al., 2008). In sum, these findings
support the hypothesis that stress can affect reward-related processing by increasing
dopamine activity (Mather & Lighthall, 2012).
Dopamine appears to promote reward learning by facilitating dopamine-
dependent plasticity in corticostriatal circuits, but impair aversion learning by preventing
dips in dopamine that are necessary for learning to avoid negative feedback (Frank &
O’Reilly, 2006; Frank, Santamaria, O’Reilly, & Willcutt, 2007; Frank, Seeberger, &
O’Reilly, 2004; Waltz, Frank, Robinson, & Gold, 2007, see also Cohen & Frank, 2009
for review). Central to the current study, increases in striatal dopamine support slow,
habitual learning by determining the long-term probability of positive and negative
outcomes with experience and related changes to synaptic plasticity (for review Doll &
Frank, 2009). Thus by increasing dopamine activity in the striatum, stress may
strengthen long-term memory for positive stimulus-feedback associations but impair
memory for negative stimulus-feedback associations. Different stress effects may be
expected for components of reinforcement learning primarily supported by the prefrontal
cortex (PFC). In particular, the PFC affects sensitivity to recent feedback as it supports
rapid response to outcomes through maintenance of recent reinforcement history (Doll &
Frank, 2009) and prefrontal dopamine modulates such working memory-based functions
(Seamans & Yang, 2004). Previous research has found stress-related impairments to

37

working memory (Schoofs, Preuss, & Wolf, 2008; Schoofs, Wolf, & Smeets, 2009),
which appear to be mediated by stress effects on the PFC (Qin, Hermans, van Marle,
Luo, & Fernández, 2009). Thus, stress may affect specific components of reinforcement
learning differently depending on their neural mechanisms.
In this study, we were particularly interested in how stress affects reinforcement
learning for older adults compared with younger adults. Dopamine function and related
cognitive processes decline as people age (Bäckman, Lindenberger, Li, & Nyberg, 2010;
Bäckman, Nyberg, Lindenberger, Li, & Farde, 2006) and attenuated dopamine responses
to stress have been observed in older versus younger rats (Del Arco et al., 2011). These
declines suggest the possibility that, in late life, dopaminergic systems will be less
responsive to reward outcomes as well as to the effects of acute stressors, leading to a
reduction in stress effects on reinforcement learning with advancing age.
However, there are some suggestions from previous research that reward
processing is relatively well-maintained in aging. Older adults show robust brain
activation in striatal regions in response to reward outcomes, indicating that processing
rewards engages dopaminergic pathways in older adults (Cox, Aizenstein, & Fiez, 2008;
Mell et al., 2009; Samanez-Larkin et al., 2007; Samanez-Larkin, Kuhnen, Yoo, &
Knutson, 2010; Schott et al., 2007). Enhanced incidental memory for pictures seen with,
or following, positive feedback has been observed in younger and older adults, with no
age differences in the strength of the feedback effect (Mather & Schoeke, 2011).
However, some previous findings also suggest that stress may have as much or more of
an effect on behavior in late life as it does earlier. For example, older rats show similar

38

impairing effects of stress on working memory despite attenuated dopamine responses to
stress (Del Arco et al., 2011), and dopamine agonist administration affects memory and
related brain activation more in older adults than in young adults (Morcom et al., 2010).
Thus, even if stress-related enhancements to dopamine are diminished in older adults,
they may still show robust effects of stress on dopamine-dependent cognitive processing.
These findings highlight the possibility that younger and older adults may exhibit similar
stress effects on reinforcement learning.
This study is the first to examine age differences in stress effects on reinforcement
learning in humans. Specifically, the present study investigated performance on a
probabilistic reinforcement-learning task after healthy younger and older adults
experienced the cold pressor stress task (Lovallo, 1975) or a control task. Cortisol levels
and ratings of stress and pain were used to evaluate reactivity to the stressor.
Importantly, the cold pressor has been shown to result in similar cortisol and subjective
stress responses in younger and older adults (e.g., Mather et al., 2009). The current study
was designed to achieve two primary goals: 1) test the hypothesis that cold pressor stress
would result in improved reward learning and impaired avoidance learning on a
probabilistic selection task, and 2) determine whether stress-related alterations to
reinforcement learning are similar or different for younger and older adults.
Method
Participants
The present study included 96 healthy adults; 48 younger adults age 18-34 (23
female, M
age
= 23.12 yrs, SD = 4.7, M
Edu
= 14.44) and 48 older adults age 65-85 years (25

39

female, M
age
= 72.58, SD = 5.7, M
Edu
= 16.44). Basic demographics and measures of
cognitive function are included in Table 2.1. Participants were recruited for a study of
stress and cognition from the University of Southern California (USC) campus and Los
Angeles community via flyers, online and print advertising, snowball recruitment, and
from the USC Healthy Minds Participant Pool (http://healthyminds.matherlab.com/).
Individuals were not recruited if they reported using hormone birth control, sex hormone
replacement medications, corticosteroidThe present study included 96 healthy adults; 48

40

younger adults age 18-34 (23 female, M
age
= 23.12 yrs, SD = 4.7, M
Edu
= 14.44) and 48
older adults age 65-85 years (25 female, M
age
= 72.58, SD = 5.7, M
Edu
= 16.44). Basic
demographics and measures of cognitive function are included in Table 2.1. Participants
were recruited for a study of stress and cognition from the University of Southern
California (USC) campus and Los Angeles community via flyers, online and print
advertising, snowball recruitment, and from the USC Healthy Minds Participant Pool
(http://healthyminds.matherlab.com/). Individuals were not recruited if they reported
using hormone birth control, sex hormone replacement medications, corticosteroid
medications, beta-blocker medications, or cigarettes. None of the younger female
participants reported having been pregnant within the previous six months and all
reported having regular menstrual cycles (every 26-30 days). To control for the possible
influence of sex or menstrual cycle on stress effects (Diamond, 2007), the number of men
and women was approximately balanced, and younger female participants also reported
on their current day of the menstrual cycle. Young women were then categorized as early
follicular (days 1-7), late follicular (8-13), or mid-luteal (18-24). Potential participants
were not recruited if they had any of the following medical conditions: coronary artery
disease, angina, arrhythmia, peripheral vascular disease, diabetes, Raynaud’s
phenomenon, cryoglobulinemia, vasculilitis, lupus, tingling or numbness in hands and/or
feet, previous stroke, or Alzheimer’s disease. The Mini Mental Status Exam (MMSE;
Folstein, Folstein, & McHugh, 1975) was administered to older participants to assess
their global cognitive functioning. All older participants in the present study met a cutoff
score of 24 points on the MMSE (see Table 2.1 for means).

41

Protocol
Study sessions were conducted between 2 and 6 pm. The study was restricted to
these times to reduce the impact of diurnal variations in cortisol (Kudielka, Schommer,
Hellhammer, & Kirschbaum, 2004). In terms of cognitive performance, this time of day
may have been more ideal for younger than older adults overall (Yoon, May, & Hasher,
2000), but study time was consistent across stress conditions and age groups to minimize
confounding effects of test time on potential age differences in stress effects. Eligibility
was determined over the phone. After completing informed consent, participants drank
an 8-oz bottle of water to rinse their mouths and improve hydration. Next, participants
completed questionnaires for at least 10 min, then provided baseline pain ratings and
saliva samples (t0). Immediately after the baseline saliva sample, participants completed
either the stressor or control task followed by a second pain rating (worst pain
experienced during the hand immersion task). Thereafter, participants completed a 5-min
paced breathing task (results not included here) and a 9-min selective working memory
task (adapted from Gazzaley, Cooney, Rissman, & D’Esposito, 2005; results not included
here), followed by a second saliva sample (t1; M = 20.42 min, SD = 2.77 from
manipulation start), then the learning task, and finally a third saliva sample (t2; M = 44.40
min, SD = 10.35 from manipulation start). At the end of the session, participants
provided ratings of psychological stress experienced during the stress manipulation.
Stress Manipulation
Participants were randomly assigned to either the stress or control condition and
did not know their condition assignment until the start of the manipulation. The

42

percentage of participants in each condition by demographic group was similar (stress:
24.4% younger female, 24.4% younger male, 26.7% older female, 24.4% older male;
control: 23.5% younger female, 27.5% younger male, 25.5% older female, 23.5% older
male). The cold pressor (Lovallo, 1975) is a widely used laboratory stress manipulation
in which participants are exposed to a cold stimulus for a few minutes. In the present
study, cold pressor participants were asked to hold their non-dominant hand in ice water
(0-5°C) for as long as possible up to 3 min. An experimenter was present during the
hand immersion task and let participants know when 3 min had elapsed. All participants
in the stress condition completed the cold pressor task for at least 60 sec. The control
task was conducted in the same manner with warm water at 37-40° C.
Salivary Biomarker Collection and Assay
Salivary cortisol was used as our primary measure of physiological response to
the stressor. To reduce variation in cortisol, participants were asked to refrain from
exercise and food within 1 hr, sleep within 2 hrs, caffeine within 3 hrs, and alcohol within
24 hrs prior to the experiment. Passive drool samples (1 ml) were collected at three time
points during the test session and each was assayed for cortisol. One sample was
collected immediately before the stress manipulation (t0), a second sample was collected
immediately before the learning task (t1), and a third sample was collected immediately
after the learning task (t2). Samples were frozen immediately after testing at -30°C.
Levels of cortisol found in saliva are considered good estimates of unbound levels
(bioavailable) in plasma (Kirschbaum & Hellhammer, 1994). Samples were transported
frozen to CLIA-certified analytical laboratories where cortisol levels were determined

43

with high-sensitivity enzyme immunoassay kits (Salimetrics, LLC, State College, PA).
Duplicate assays were conducted for each sample interval and the mean of the two values
was included in the final analysis. For three sample interval values, hormone values
could only be determined for one of the two duplicate assays. In these cases, the single
available value was included in the final analysis.
Pain and Subjective Stress Ratings
Participants completed pain ratings on a visual analogue scale just before and
immediately after the manipulation, reporting on their present pain level and the worst
pain they experienced during the hand immersion task, respectively. Ratings were made
on a line 10-cm long, such that 0 cm represented no pain and 10 cm represented worst
possible pain. Pain response to the hand immersion task was measured by change from
baseline pain (e.g., 7
cm hand immersion
task – 1 cm baseline).
At the end of the
session, participants
indicated the amount
of stress they
experienced during
the hand immersion
task on a 7-point
Likert scale, such that

44

1 represented no stress at all and 7 represented a great deal of stress.
Probabilistic Learning Task
Participants completed a probabilistic learning task (Frank et al., 2004; Frank &
Kong, 2008) involving three pairs of symbols. For each pair, they had to learn which
symbol (Japanese Hiragana characters) more often predicted positive feedback and which
more often predicted negative feedback (Figure 2.1). After meeting a learning criterion
or completing six training blocks, participants completed a test without feedback. In the
test, they were instructed to select the symbol they thought had the highest chance of
leading to a positive outcome for novel pair combinations and previously learned symbol
pairs. Symbols appeared in black 72-point font on a white background (Figure 2.1). The
participants pressed a designated key on the right or the left side of the keyboard to
indicate which stimulus they chose to be “correct.” After each choice, “Correct!”
appeared in blue text or “Incorrect” in red text. If no response was made within 4 sec, the
words “no response detected” appeared.

During training, three stimulus pairs were presented (Figure 2.1); each pair was
presented 20 times per training block. Valenced outcomes followed stimulus selections
in a probabilistic manner (AB pair = 80%-correct and 20%-incorrect for A and 20%-
correct, 80%-incorrect for B; CD pair = 70%-correct and 30%-incorrect for C, 30%-
correct and 70%-incorrect for D; EF pair = 60%-correct and 40%-incorrect for E, 40%-
correct and 60%-incorrect for F). With training, participants typically learn to choose
symbols with higher probability of positive outcomes (A, C, and E) and to avoid selecting
symbols with higher probability of negative outcomes (B, D, or F). Participants who met

45

the performance criteria after any 60-trial training block advanced to the test. The
criterion differed for each pair in the following manner: 65% A in AB, 60% C in CD,
40% E in EF. In the EF pair, stimulus E is correct 60% of the time, but this is especially
difficult to learn. Therefore, a 40% criterion was used for this pair (simply to ensure that
participants do not have a robust preference for the incorrect stimulus). Outcome
measures from the training phase were number of training blocks completed (training
required to meet learning criterion for all symbol pairs) and sensitivity to recent feedback
during initial training (trial-by-trial adjustments to feedback in the first block of
training).
3
This measure evaluated tendency to select the same symbol following a win
(win-stay) and select the alternative symbol following a loss (lose-shift). As symbol pairs
are presented randomly, the reoccurrence of a given pair is usually not immediate. For
this reason, selection of a previously rewarded cue or avoidance of a previously punished
cue is thought to require
maintenance of recent reinforcement experiences in working
memory. Note that, although win-stay/lose-shift behavior is not optimal in the long run,
it is a commonly implemented strategy when uncertainty about stimulus-outcome
contingencies is high (e.g., in the first training block) that provides information about the
influence of feedback in a short-term context.
The test to assess learning from positive and negative feedback was administered
immediately after the training period. In this phase, participants were tested with the

3
Performance accuracy during the training phase cannot be used to differentiate reward
and aversion learning per se, as every trial can be responded to correctly either on the
basis of positive-outcome learning or negative-learning learning. Even the measure of
whether people make the same or a different choice after getting positive or negative
feedback is not a clear measure of learning, as some proportion of trials involve incorrect
feedback that contradicts prior learning.

46

same symbols in familiar and novel combinations with pairs presented at random (4 times
per pair). In this way, the test phase examined generalization of the learned stimulus
values. Participants were told they would not receive feedback during this phase and
should select the symbol that “feels” more correct based on what they learned. Positive
feedback learning was assessed by preference for the most positive stimulus A when
paired with more neutral stimuli (C, D, E, F) which have on average a neutral (50%)
value, whereas negative learning was assessed by avoidance of the most negative
stimulus B when paired with those same neutral stimuli.
Statistical Analysis
Group effects for cortisol levels, subjective pain and stress ratings, and learning
outcomes were assessed with ANOVAs that included stress condition (cold pressor,
control), age (younger, older), and sex (male, female) as between-subject factors.
Valence of feedback (positive, negative) was included as a within-subject factor for
sensitivity to recent feedback (win-stay, lose-shift behavior) and the learning
generalization test (Choose-A, Avoid-B acccuracy). Cortisol values were measured in
ug/dL. To determine the impact of between-subject factors on cortisol levels, we
conducted repeated-measures ANOVAs for individual cortisol levels with sample
interval at three levels: t0, t1, and t2 (corresponding with baseline, post-manipulation and
pre-task, and post-manipulation and post-task). Values for each sampling interval were
tested for skewed distributions against a normal distribution using one-sample
Kolmogorov-Smirnov (K-S) tests. Based on these tests, normal distributions could not be
assumed for cortisol values (one-sample K-S tests, all three sample intervals ps < .05).

47

Thus, log-transformed values were used for all cortisol analyses; however, raw values
were included in results and figures to allow for comparisons across studies. Cortisol
change scores were also calculated by subtracting individual log-transformed t0 levels
from the mean of t1 and t2 levels. Partial eta squared (η
p
2
) values were reported to
provide measures of effect size and means are presented with their standard errors (SEM).
Greenhouse-Geisser values were reported for any analyses in which the homogeneity of
variance assumption was not met. Additional analyses of stress effects on learning were
conducted with younger females alone that included menstrual cycle phase as a covariate.
Correlations were used to determine the relationship between behavioral outcome
measures and cortisol change values.
Results
Cortisol
All cortisol analyses were conducted on log-transformed cortisol values. Baseline
(t0) cortisol values were higher in younger adults compared with older adults, F(1,88) =
4.69, p = .03, η
p
2
= .05, but there were no other significant group differences or
interactions in t0 cortisol values (ps > .05). We observed a main effect of stress condition
on cortisol across the three sample intervals, F(1,88) = 6.80, p = .01, η
p
2
= .07, such that
cortisol levels were higher in stressed participants. Importantly, we found a significant
interaction of stress with sample interval, F(2,176) = 8.12, p = .001, η
p
2
= .09 (Table 2.2,
Figure 2.2), confirming that the stress manipulation affected cortisol levels. This two-
way interaction of stress and sample interval was not further qualified by any interactions
with age or sex group, thus, the impact of stressor on cortisol was similar for younger and

48

older adults.
4
Consistent with the repeated-measures analysis, a main effect of stress
group was found for cortisol change ([mean of cortisol at t1 and t2] – t0 cortisol), F(1,88)
= 11.03, p = .001, η
p
2
= .11, with no significant interaction between age and stress group,
F(1,88) < 1. Examination of means indicated that cortisol increased in the s tress group
(M = .11, SEM = .06) and decreased in the control group (M = -.15, SEM = .06). The lack
of a significant age-by-stress interaction on cortisol change observed here is consistent
with a previous research eliciting salivary cortisol responses with the cold pressor
(Mather et al., 2009) and
laboratory psychosocial stress
tasks (Kudielka, Schmidt-
Reinwald, Hellhammer, &
Kirschbaum, 1999; Kudielka,
Schmidt-Reinwald, Hellhammer,
Schürmeyer, & Kirschbaum,
2000; Nicolson, Storms, Ponds,
& Sulon, 1997; Rohleder,
Kudielka, Hellhammer, Wolf, & Kirschbaum, 2002).

4
There were significant differences by age group in delay before the second sample (t1)
F(1,92) = 45.17, p < .001, η
p
2
= .33 (M
young
= 20.85 min, SD = 1.99; M
old
= 23.98 min, SD
= 2.54). However, when we examined cortisol values at baseline and just before the
learning task (t0 and t1) in a repeated-measures ANOVA, there was still a significant
interaction of stress condition by sample interval when delay to the t1 sample was
included as a covariate, F(1,87) = 12.79, p = .001, η
p
2
= .13. Likewise, the three-way
interaction between stress condition, sample interval and age remained non-significant
with the covariate, F(1,87) = .60, p = .44, η
p
2
= .007, suggesting that a difference in delay
to the t1 sample between age groups of about 3 min did not significantly influence stress
effects on cortisol response in younger and older adults.

49

Tests for stress and control groups separately indicated a linear decrease in
cortisol values for the control group across samples, F(1,47) = 10.47, p = .002, η
p
2
= .18,
but a quadratic increase in cortisol values for the stress group such that mean cortisol
levels were highest at the pre-learning task sample (t1), F(1,41) = 6.00, p = .02, η
p
2
= .13.
Within the stress group alone, we found no age differences in cortisol change from t1 to
t2, indicating that there was no significant difference in cortisol trajectories between

50

stressed younger and older adults from the beginning to end of the learning task.
5
Further
examination of cortisol responses indicated that 46.7% participants in the stress group
had a cortisol response ≥ 10% of their baseline level, while 25.4% of participants in the
control group experienced such a cortisol change. This difference in the percentage of
responders by group was significant, χ
2
(1, N = 96) = 7.82, p < .01. Stress group
differences did not interact with age or sex (ps > .05). These results indicate that, across
age and sex groups, cortisol levels were higher in the cold pressor group than in the
control group during the learning task.
There was also an interaction of age and sample interval, F(2,176) = 8.17, p =
.001, η
p
2
= .09. To better characterize the nature of this interaction, repeated-measures
ANOVAs were conducted on younger and older adult groups separately. There was a
linear decrease in cortisol values for younger adults across stress conditions, F(1,44) =
12.30, p = .001, η
p
2
= .22 (M
t0
= .13, SEM = .01; M
t1
= .12, SEM = .01; M
t2
= .11, SEM =
.01), but the effect of sample interval was not significant in older adults across stress
conditions, p > .05 (M
t0
= .11, SEM = .01; M
t1
= .12, SEM = .01; M
t2
= .12, SEM = .01).
While the reason for this age difference is unclear, it may be related to a stronger diurnal
decline in younger adults than older adults (e.g., Deuschle et al., 1997; Heaney, Phillips,

5
We conducted a post-hoc test for age differences in cortisol change from t1 to t2 in the
stress group to provide information about whether there were age differences in
stability/change in cortisol during the reinforcement-learning task within this group.
Post-manipulation cortisol change scores were calculated by subtracting t2 cortisol from
t1 cortisol; this outcome measure was included in an ANOVA with age as between-
subject factor. The analysis revealed no significant age differences, F(1,41) = 3.32, p =
.08, η
p
2
= .08 (M
younger
= .13, SEM = .06; M
older
= -.03, SEM = .06). These results indicate
that cortisol change was similar in younger and older adults in the stress group from the
beginning to end of the learning task.

51

& Carroll, 2010; VanCauter, Leproult, & Kupfer, 1996),
or it may be due to age
differences in how stressful participants found the general experimental context or the
selective working memory task (before the stress manipulation).
Pain and Subjective Stress
The stressor increased ratings of physical pain relative to baseline ratings, F(1,87)
= 155.85, p < .001, η
p
2
= .64 (Table 2.2). There were no other main effects or
interactions by group (ps > .05). Post-experiment ratings of psychological stress
experienced during the hand immersion task were higher for those in the stress condition
than for those in the control condition, F(1,87) = 95.44, p < .001, η
p
2
= .52 (Table 2.2),
without any interactions of age or sex with stress condition.
Number of Training Blocks Completed during Learning Acquisition
Older adults required more training blocks to learn the stimulus-outcome
associations (M = 3.88, SEM = .28) than did younger adults (M = 2.85, SEM = .29),
F(1,88) = 6.50, p = .01, η
p
2
= .07. Consistent with stress effects observed in a study of
only younger adults (Petzold et al., 2010), exposure to the stressor did not modulate the
number of training blocks completed. We also found no interactions of stress condition
with age or sex (ps > .05). A separate analysis of younger females alone, which included
cycle phase as a covariate, also found no effect of stress condition on learning acquisition
(p > .05). Due to age differences in the amount of training completed, post-hoc tests for
generalization of learning (test phase performance) were conducted that included number
of training blocks was included as a covariate, but inclusion of this covariate did not alter
the nature of any result (i.e., significance status or direction of mean differences).

52

Sensitivity to Recent Feedback: Win-stay and Lose-Shift Behavior
For trial-to-trial behavior in the first training block, participants were more likely
to select previously correct stimuli (win-stay) than to avoid selecting previously incorrect
stimuli (lose-shift), F(1,88) = 198.00, p < .001, η
p
2
= .69 (M
win-stay
= .72, SEM = .02; M
lose-
shift
= .39, SEM = .01). Additionally, stress reduced sensitivity to recent feedback, F(1,88)
= 17.70, p < .001, η
p
2
= .17 (M
stress
= .52, SEM = .01; M
control
= .59, SEM = .01), but did
not interact with age, sex, or feedback valence (ps > .05; Table 2.2). That is, stress had a
similar impairing effect on win-stay and lose-shift performance. We found no other
significant main effects or interactions (ps > .05). We conducted follow-up analysis to
determine whether stress effects on the tendency to stay with a previously selected cue
depended on whether the previous choice was the better option in the long-term (i.e.,
accuracy; selected A) or not. There was an interaction between the accuracy of the
previous choice and feedback received with the previous choice for stay behavior,
F(1,88) = 4.91, p = .03, η
p
2
= .05, such that staying with a previously selected cue was
most likely for positive cues that yielded positive feedback that last time it was selected,
(M
pos cue-pos

feedback
= .74, SEM = .02; M
pos cue-neg

feedback
= .61, SEM = .02; M
neg cue-pos

feedback

= .61, SEM = .02 M
neg cue-neg

feedback
= .56, SEM = .02); however, stress did not modulate
this interaction (p > .05). Thus, during initial training, stress appeared to reduce behavior
reflecting sensitivity to recent feedback regardless of whether the previously selected cue
was associated with long-term positive or negative contingencies. These results suggest
that stress effects on sensitivity to recent feedback during initial training were not isolated
to the selection of just positive or negative cues.

53

Generalization of Learning: Choose-A (positive) and Avoid-B (negative) Accuracy

There was no main effect of stress condition on cue selection in the test phase
across feedback valence types, F(1,88) = 1.89, p = .17, η
p
2
= .02; however, there was a
significant interaction of stress condition and feedback valence for cue-selection
accuracy, F(1,88) = 9.93, p = .002, η
p
2
= .10. Age did not modulate the effect of stress,
F(1,88) = .11, p = .74, η
p
2
< .001, nor did it modulate the stress-by-valence interaction,
F(1,88) = .10, p = .76, η
p
2
< .001. Separate ANOVAs for Choose-A and Avoid-B
confirmed that stress enhanced Choose-A performance, F(1,88) = 10.23, p = .002, η
p
2
=
.10, but did not significantly affect Avoid-B performance, F(1,88) = 1.63, p = .21, η
p
2
=
.02. Age and sex did not significantly interact with stress condition to influence Choose-
A or Avoid-B performance in these separate ANOVAs (Table 2.2; Figure 2.3). In
addition, the stress-by-valence effect on learning was significant in young women alone
whether menstrual cycle phase was included as a covariate, F(1,20) = 4.55, p = .045, η
p
2

54

= .19, or not, F(1,21) = 4.63, p = .04, η
p
2
= .18. Differences in performance by feedback
valence were significant in ANOVAs conducted with stress, F(1,41) = 5.40, p = .03, η
p
2
=
.12, and control groups individually, F(1,47) = 4.60, p = .04, η
p
2
= .09; such that
performance was better for positive feedback in stressed participants but better for
negative feedback in controls. Age and sex did not interact with valence type in these
additional analyses, nor were there any three-way interactions of these factors. Although
older adults had lower accuracy across feedback types, F(1,88) = 10.09, p = .002, η
p
2
=
.10, stress effects on learning about positive and negative outcomes were similar in
younger and older adults. Providing further support for this assertion, significant
interactions of stress and feedback valence – with similar effect sizes – were obtained for
both age groups when their data was analyzed separately, F(1,44)
younger
= 5.33, p = .03,
η
p
2
= .11; F(1,44)
older
= 4.60, p = .04, η
p
2
= .10. No other significant effects or
interactions were found (all ps > .05); indicating that stress selectively enhanced
probabilistic learning involving positive outcomes in both younger and older adults.
Additional analyses were conducted on control participants alone to address the
possibility that a lack of age differences in stress effects were the result of valence-
specific changes to feedback learning in aging (e.g., to examine the possibility that
reward learning is less affected by aging than avoidance learning). These analyses
examined whether there were age differences in reinforcement learning in non-stressed
conditions, and whether these age differences depended on feedback valence. While
performance was lower in older controls overall, F(1,47) = 5.90, p = .02, η
p
2
= .11 (Table
2.2 for means), there was no interaction of age group and feedback valence (p > .05).

55

Consistent with a lack of age-by-feedback valence effects, Pearson’s correlations did not
yield any significant associations between age in years and any of the valenced outcome
measures within the older adult group across stress conditions or in the control condition
alone (ps > .05).
Correlations for Cortisol Change and Learning Performance
Correlations were conducted for log-transformed cortisol change ([mean of
cortisol at t1 and t2] – t0 cortisol) and our primary learning performance measures, which
included number of training blocks completed and accuracy measures for the win-stay,
lose-shift, Choose-A, and Avoid-B components of the task. Across groups, there were no
significant relationships between cortisol change and these learning outcomes (ps > .05);
however, there was a marginal positive correlation between cortisol change and Choose-
A performance. We then examined relationships between cortisol change and learning
performance in individual groups. There were no significant correlations for separate
analyses of the stress group, control group, and younger adult group (ps > .05). In older
adults, we observed a significant positive relationship between cortisol change and
Choose-A performance across stress conditions, r(46) = .36, p = .01. This result
indicated that older adults who experienced greater increases in cortisol from baseline
learned better from positive reinforcement. Although not significant, the relationship
between cortisol change and Choose-A performance was also positive for younger adults,
r(46) = .17, p = .26 (all other correlations, p > .05). A follow-up test for differences
between correlations for independent samples confirmed that these correlations were not
significantly different (Fisher’s r-to-Z transformation, Z = .97, two-tailed p = .33). Thus,

56

the positive relationship between cortisol change and positive reinforcement learning was
similar across age groups, but only significant in older adults. In general though,
reinforcement learning performance appeared to be more sensitive to stress exposure (as
reflected in the stress group differences) than to inter-individual differences in cortisol
change.
Discussion
Previous research indicates that stress increases dopamine levels in reward-
processing regions of the brain and increases cravings and the likelihood of relapse
among drug addicts (Mather & Lighthall, 2012). Such findings suggest that stress
amplifies the impact of rewarding cues. The present study provides further evidence
along these lines by showing that stress enhances learning about associations with
positive outcomes, but does not enhance learning about associations with negative
outcomes.
In this study, we investigated whether inducing stress with the cold pressor task
would alter subsequent reinforcement learning and whether effects of stress would
depend on age. During the initial block of learning acquisition, when uncertainty about
stimulus-outcome associations was greatest, stress exposure resulted in reduced
sensitivity to recent feedback (i.e., lower win-stay and lose-shift accuracy) and this effect
was similar across age groups. Stress also altered performance on the post-learning test,
such that selection of cues associated with positive feedback was enhanced by stress in
younger and older adults. There was no age difference in this effect (Figure 2.3);
selection accuracy for stimuli with a high probability of positive outcomes increased an

57

average of 23% in younger adults and 28% in older adults. Older adults’ enhanced
learning from positive reinforcement under stress could not be explained by an overall
age-related decline or maintenance in positive reinforcement learning, as in the control
condition there were no age-by-valence interactions in learning. Further, while we must
exercise caution in interpreting a lack of age-dependent stress effects on learning, the
absence of such effects did not appear to be attributable to age differences in stress
response, as the magnitude of stress responses to the cold pressor was similar across age
groups. We also found that, although individual cortisol responses were not strongly
related to performance, there was a significant correlation between cortisol change from
baseline and positive reinforcement learning in older adults, with young adults exhibiting
a similar, albeit non-significant, correlation. Together these findings suggest that, in
early and late adulthood alike, acute stress can enhance learning which cues are most
associated with positive feedback.
This study is the first to test the effects of acute stress on reinforcement learning
in older adults, but similar effects of stress on feedback-based learning have been
observed in studies with young adults that involved cortisol administration or exposure to
a psychosocial stressor. For example, our finding that cold pressor stress enhanced
positive feedback learning is consistent with an earlier study with college-age adults that
found an association between higher basal cortisol levels and reward dependency (greater
monetary gains) on the Iowa Gambling Task (van Honk et al., 2003). In addition, using
the same probabilistic selection task implemented in the current study, others have found
that psychosocial stress leads to impairments in negative feedback learning in young

58

adults (Petzold et al., 2010). Using the same task, another recent study found that social
stress was associated with better positive feedback learning, but poorer negative feedback
learning, in young adults with low punishment sensitivity compared to young adults with
high punishment sensitivity (Cavanagh et al., 2010). Thus, across studies, stress
enhances positive feedback learning and impairs negative feedback learning, although
which effect is stronger varied depending on the study.
Although the neural underpinnings of stress effects on reinforcement learning
have yet to be clarified, animal research and nuclear imaging studies with humans are in
agreement that acute stress increases dopamine activity in the striatum and PFC (e.g.,
Abercrombie et al., 1989; Feenstra et al., 2000; Kalivas & Duffy, 1995; Scott et al.,
2006). Dopamine action in the striatum appears to be critically important for
generalization of stimulus-outcome learning on this task (i.e., test phase performance; see
Doll & Frank, 2009 for review). For example, increasing phasic dopamine release in the
striatum enhances selection of reward-associated cues but impairs selection of
punishment-associated cues (Frank & O’Reilly, 2006), and enhances striatal response to
prediction error during feedback-based learning, with increased activation predicting
better performance in a later test phase (Jocham, Klein, & Ullsperger, 2011). Dopamine
treatment also increases the difference in ventromedial PFC response to positive versus
negative feedback cues after learning acquisition (Jocham, et al., 2011). Thus, by
increasing dopamine action in the striatum and PFC, stress may have opposite effects on
learning from positive versus negative outcomes.

59

In contrast with the slow accumulation of learning about cues that relies on striatal
regions (Yin & Knowlton, 2006), trial-to-trial win-stay and lose-shift decisions appear to
depend more on prefrontal working memory processes, as they require maintenance of
recent selections and outcomes during intervening trials. The dorsolateral PFC appears to
support win-stay behavior in decision making when potential outcomes are unpredictable,
while the anterior cingulate is important in mediating lose-shift behavior at both high and
low levels of predictability (Paulus, Hozack, Frank, & Brown, 2002). Although age
differences have not been examined, stress appears to alter brain activation in both of
these regions (Pruessner et al., 2008; Wang et al., 2005), and stress-related impairments
to working memory have been linked to dorsolateral PFC disruption (Qin et al., 2009).
Accordingly, we found that stress impaired participants’ initial win-stay and lose-shift
performance during the learning phase of the probabilistic selection task with no
differences in stress effect by feedback valence. Others have found similar stress-related
impairment to working memory tasks without reinforcement (e.g., Schoofs et al., 2008;
Schoofs et al., 2009). The current study adds the novel finding that older adults also
experience stress-related impairments to working-memory-related performance during
reinforcement learning acquisition; however, these effects do not appear to be valence
specific. Given these earlier findings, our results suggest that acute stress may interfere
with initial reinforcement learning in younger and older adults by altering maintenance of
stimulus-outcome associations, perhaps via PFC disruption.
The current study found no age differences in physiological responses to stress
(cortisol reactivity) or in stress effects on reinforcement learning between healthy

60

younger and older adults. The present finding suggests that, despite age-related declines
in various aspects of the dopamine system, healthy older adults still experience robust
effects of stress on reward learning. Other studies have observed greater stress-induced
changes to risk taking (Mather et al., 2009) and an age-related increase in sensitivity to
pharmacological dopamine manipulation on episodic memory (Morcom et al., 2010).
These earlier human studies, taken with the findings from the current study, present the
possibility that the magnitude of stress effects and other dopamine-modulators on
cognition and underlying neural circuitry may be maintained or increase in normal aging.
This may be due to adaptations of the system to lower levels of dopamine, such that less
of a perturbation is needed to have the same impact. However, given that we did not
measure dopamine in our study, we must also leave open the possibility that the stress
effects on learning about positive outcomes operate via mechanisms unrelated to
dopamine. Such other mechanisms may include stress effects on information processing
related to emotional goals (e.g., regulating mood by focusing on rewarding outcomes).
With respect to cortisol, we found that change in cortisol was weakly related to
positive reinforcement learning in older adults. The correlation between cortisol change
and positive reinforcement learning was also positive, although not significant, in young
adults alone, and the correlations for the older and younger group were not statistically
different from each other. As these correlations were weak and observed across stress
conditions, we cannot make strong claims that inter-individual differences in cortisol
change affected performance. However, the lack of age differences in the relationship
between cortisol change and memory effects is consistent with a previous study reporting

61

similar effects of cortisol administration on younger and older adults’ declarative
memory (Wolf et al., 2001). We did not find any significant relationships between
cortisol change and any of our other cognitive outcomes.
A caveat to the current study is that the design may not have been optimal for
observing age differences in stress effects on reinforcement learning. For example, it is
possible that the cold pressor can elicit age-by-stress interactions for some cognitive tasks
(e.g., risk tasks under time pressure, Mather et al., 2009) but not others. The cold pressor
effect is moderate (the effect size of the cortisol difference between stress groups was η
p
2

= .09); it may be that more powerful stressors or direct pharmacological manipulation of
stress hormone levels will elicit age differences in the impact of stress on reinforcement
learning. It is also possible that older adults experienced additional stress from the
selective working memory task (completed prior to the reinforcement-learning task).
However, any age differences in stress elicited by the selective working memory task
would have been present across stress conditions. Thus, differences in reinforcement
learning between the cold pressor stress condition and the control condition are due
specifically to the stress induced by the cold pressor, not the selective working memory
task. A lack of power may have also prevented observation of age differences in stress-
by-feedback valence effects; a limitation that can be addressed in future studies.
However, the conclusion that the interaction of stress and feedback valence does not
differ for older and younger adults is supported by the very small effect size obtained for
the age-by-stress-by-feedback valence interaction (η
p
2
= .001). Power calculations were
conducted with G-Power version 3.1.2 for a repeated-measures design with 2 levels, 4

62

groups, and a correlation between repeated measures of .01 (for Choose-A and Avoid-B
scores), and the effect size for the age-by-stress-by-feedback valence interaction of .001
(the latter two parameters obtained from the current study). For α = .05 and power = .90,
a sample size of 7012 subjects would be required to detect an interaction of age, stress
condition and feedback valence. Thus, it seems that stress exerts different effects on
learning about positive compared to negative feedback, and these stress effects are
similar in early and late adulthood.
A remaining issue, however, is that the present study could not determine whether
effects of stress are linear or not, as we included a moderate stressor and did not
manipulate the intensity of the stress induction. Some previous research suggests that
cognitive performance and stress, as well as cognitive performance and dopamine, have
inverted-U relationships (see Bäckman et al., 2006; Lupien, McEwen, Gunnar, & Heim,
2009). Future studies may help determine whether stronger manipulations of stress lead
to a decline in reward learning performance, consistent with an inverted-U relationship.
With regard to stressor type, the two previous studies (Cavanagh et al., 2010; Petzold et
al., 2011) examining effects of stress on the probabilistic selection task used psychosocial
stressors and observed significant impairing effects of stress on aversion learning
(however, only in those with low-punishment sensitivity in Cavanagh et al., 2010),
whereas we only found trends in that direction for aversion learning. One possibility is
that psychosocial stressors have a greater impact on affect and emotion regulation goals
than physiological stressors like the cold pressor. Specifically, psychological stressors
like the Trier Social Stress Test (Kirschbaum, Pirke, & Hellhammer, 1993) involve

63

negative social feedback. Aftereffects may include a motivation to avoid attending to
negative feedback – resulting in poorer aversion learning. Furthermore, psychological
and painful stressors affect dopamine transmission in different aspects of the striatum
(Scott et al., 2006), which may result in different behavioral biases (e.g., toward reward
versus away from punishment). PET studies conducted with healthy populations may
help determine the specific neural mechanisms of stress effects on learning from positive
and negative feedback, and whether there are age differences in these mechanisms.
Although not the main focus of our study, it is also interesting to compare the
relative influence of positive versus negative feedback on learning in younger and older
adults in the control condition. Research on this topic is mixed (for review, Eppinger,
Hämmerer, & Li, 2011), with some previous studies finding better learning from positive
than negative feedback in older adults relative to younger adults (Nashiro, Mather,
Gorlick, & Nga, 2011; Wood, Busemeyer, Koling, Cox, & Davis, 2005), others finding
no age differences in learning from positive and negative feedback in younger and older
adults (Eppinger, Herbert, & Kray, 2010; Pietschmann, Endrass, Czerwon, & Kathmann,
2011), and still others finding better learning from negative relative to positive feedback
for older adults compared with young adults (Eppinger & Kray, 2011; Hämmerer, Li,
Müller, & Lindenberger, 2011) and for older compared with younger seniors (Frank &
Kong, 2008). Recent studies have also observed age differences in brain blood oxygen
level-dependent activation during loss anticipation (Samanez-Larkin et al., 2007), and in
the electrophysiological correlates of feedback-related negativity signals (Eppinger, Kray,
Mock, & Mecklinger, 2008; Hämmerer, Li, Müller, & Lindenberger, 2011). The current

64

findings are consistent with those studies reporting no significant relationship between
age and whether learning was better from positive or negative outcomes. Although age is
associated with dopamine declines and lower dopamine levels are associated with better
learning from negative than positive feedback, there are some reasons older adults may
not exhibit this pattern of avoidance learning dominance. First, striatal response to
rewarding feedback is maintained in older adults (Cox et al., 2008; Mell et al., 2009;
Samanez-Larkin et al., 2007; Samanez-Larkin et al., 2010; Schott et al., 2007), indicating
that the dopamine system continues to respond to rewarding feedback in older age. And
second, aging is associated with an increased focus on positive information (Mather &
Carstensen, 2005), which may selectively improve memory for positive stimulus-
outcome associations in the face of dopamine declines. Further research is needed to sort
out when age differences in valence bias for reinforcement learning will be seen and
when they will not. For example, neuroimaging research may reveal whether structural
integrity of the corticostriatal pathway mediates age differences in reinforcement
learning.
In summary, this study revealed that acute stress enhanced learning involving
positive feedback in both younger and older adults. Stress is present in everyday life, and
thus, often accompanies everyday decision making. These findings suggest that stress
may affect abilities to quickly adapt to positive and negative feedback when learning out
choice outcomes. In addition, choices made under stress may be biased toward
associations with positive or rewarding experiences – a bias which may hold implications
for decisions about finances, social interactions, and health behaviors. Decisions in these

65

domains can significantly impact well being and must be made from early adulthood until
very late in life; thus, further research on the mechanisms driving the effect of stress on
reinforcement learning is likely to have application potential across adulthood.

66

CHAPTER 3: EFFECTS OF STRESS ON REINFORCEMENT LEARNING: AGE
DIFFERENCES AND NEURAL MECHANISMS
The current fMRI study was conducted to examine the neural mechanisms of
valence-dependent stress effects on reinforcement learning we observed in younger and
older adults in a previous behavioral study (Lighthall et al., in revision; Chapter 2). The
earlier behavioral study found that cold pressor stress enhanced the generalization of
learning for cues associated with positive, but not negative feedback, and this effect was
similar in younger and older adults. Within this earlier paper, we hypothesized that stress
effects on reinforcement learning are mediated by dopaminergic reward processing
regions, including the striatum and PFC – particularly the mPFC and DLPFC regions. We
also hypothesized that the neural mechanisms of stress-by-valence effects in
reinforcement learning may depend on age given age-related changes to the structure and
function of these regions. These two hypotheses were developed from previous literature
and the previous study’s findings (see Chapter 2, Introduction and Discussion). In
addition, aging is associated with a bias towards positive emotional information, which is
thought to stem from a shift in goals towards prioritizing emotional well being more with
age (Mather & Carstsensen, 2005). Study 2 did not find a positivity bias (i.e., positive >
negative feedback-learning) in older adults’ performance relative to younger adults’, but
the feedback in this earlier study did not have strong social-emotional salience. The
current study used feedback with greater social-emotional salience (emotional faces and
words), increasing the likelihood of observing a positivity bias in older adults’ learning.
The current study tested these hypotheses by using fMRI to investigate effects of cold

67

pressor stress on neural response to a feedback-based probabilistic learning task
involving positive and negative social outcomes.
Although the present study is a follow-up to the previous behavioral study, it
differed from the previous in several important ways. First, as mentioned above, the
current study used social feedback. Social feedback was included to increase the
ecological validity of the study and to enhance participants’ engagement with the task
given the large number of trials required for an fMRI investigation. Second, cue pairs in
the current fMRI study included one “valenced” cue (e.g., 72% positive, 28% neutral)
and one “mean-neutral” cue (positive 33.3%, negative 33.3%, neutral 33.3%). Previous
stress and reinforcement learning studies including cue pairs with one positive and one
negative cue (Cavanagh et al., 2010; Lighthall et al., in revision; Petzold et al., 2010)
could not determine whether stress differentially affected learning acquisition about
positive and negative cues, as each pair included a highly positive and highly negative
cue. In contrast, cue pairs in the current study allowed us to examine potential differences
in stress effects on learning acquisition about positive versus negative cue-outcome
associations. Third, the current study included only female participants. Although stress-
by-gender effects were not observed in our previous reinforcement learning study
(Lighthall et al., in revision; Chapter 2), we did observe stress-by-gender effects in
behavior and brain activation with a risky decision task (Lighthall et al., 2009; Chapter
1). This study also found larger effects of stress on neural response to the risky-decision
task in young women than in young men. Furthermore, others have observed gender
differences in response to social and monetary reinforcers (Spreckelmeyer et al., 2009)

68

and to acute stress (Goldstein et al., 2010 Naliboff et al., 2003; Wang et al., 2007). Thus,
the study was conducted with only women to increase our power to detect significant
effects of stress on reinforcement learning signals in the brain.
As discussed in Chapter 2, previous research suggests that stress and stress
hormones affect brain regions in the reward/valuation network (Abercrombie, Keefe,
DiFrischia, & Zigmond, 1989; Anstrom & Woodward, 2005; Feenstra, Botterblom, &
Mastenbroek, 2000; Imperato, Puglisi-Allegra, Casolini, & Angelucci, 1991; Kalivas &
Duffy, 1995; Piazza et al., 1996; Saal, Dong, Bonci, & Malenka, 2003), which includes
mesolimbic and mesocortical dopamine systems (e.g., structures including the striatum
and mPFC; Wise & Ropre, 1989, for review). As reinforcement learning depends on
these brain regions, one possibility is that the reward network mediates stress effects on
reinforcement learning behavior.
Another possibility is that stress alters reinforcement learning via networks
involved in task-relevant cognition and perception regions. In particular, previous
research has indicated that fronto-parietal regions involved in attentional control (e.g.,
frontal eye fields, anterior cingulate cortex, intraparietal sulcus, temporo-parietal
junction) and occipito-temporal vision regions (e.g., fusiform gyrus, area around the
calcarine fissure) are recruited during motivated cognition including during choice
selection tasks (Pessoa and Engelmann, 2010). During such tasks, reward-related
motivation appear to bias these neural networks to facilitate task performance and
increase the chances of obtaining the desired reward or outcome. Importantly, acute stress
has been found to alter brain activation in these attention control and early visual regions

69

during tasks involving emotionally salient stimuli (Henckens, Hermans, Pu, Joëls, &
Fernández, 2009; van Marle, Hermans, Qin, & Fernández, 2009). In terms of volume,
brain regions in the attentional control and vision networks (e.g., the anterior cingulate
and occipital cortex, respectively) are relatively spared by normal aging (Raz, 2004).
However, aging is also associated with an increased recruitment (i.e., task-related
functional activation) of parietal regions and reduced recruitment of occipital regions
(Davis, Dennis, Daselaar, Fleck, & Cabeza, 2008). Thus, assuming stress effects are
mediated by fronto-parietal and/or occipito-temporal networks, it is difficult to determine
whether stress effects will be similar or different in younger and older adults.
In sum, the current study addresses two important issues, examining the
mechanisms driving stress-induced alterations to reinforcement-based learning, and
investigating age-related changes to these stress effects. To address these objectives, the
current study included healthy younger and older adults and used fMRI to investigate
effects of cold pressor stress on neural response to a feedback-based probabilistic
learning task involving positive and negative outcomes.
Methods
Participants
This study included healthy, community-dwelling, younger and older women
from the Los Angeles metropolitan area, recruited for a brain imaging study on stress and
decision making. The study used a 2 X 2 design with stress condition (cold pressor,
control) and age (young, old) as between-subject factors. Participants were randomly
assigned to stress and control groups upon arriving to the lab, but their group assignment

70

was not revealed until the stress manipulation took place. The sample included 27
younger women (age range: 18-34 years; M = 23.11; SD = 4.68; 14 stress) and 24 older
women (age range = 60-74 years; M = 64.42; SD = 4.43; 12 stress). Participants were
recruited primarily from University of Southern California (USC) student and alumni
population, advertisements, and through the USC Healthy Minds Volunteer Corps
(https://healthyminds.matherlab.com). All participants were paid $15/hr for participation
with an additional $10 reimbursement for travel expenses if they drove or took public
transit to the USC campus to participate.
Only young women with regular menstrual cycles were recruited (cycle every 24-
32 days). Participants were right handed, had normal or corrected vision, normal hearing,
and were fluent in English. Individuals were not recruited if they smoked cigarettes, were
nursing, pregnant, or planned to become pregnant, had conditions or medical
devices/implants that preclude MRI participation (e.g., claustrophobia, non-MRI
compatible surgical implants), had a diagnosis of heart disease, peripheral vascular
disease, diabetes, Raynaud’s phenomenon, cryoglobulinemia, vasculilitis, lupus,
Parkinson’s disease or cancer, took hormone birth control, beta-adrenergic antagonists,
oral contraceptives, steroid medications, or psychoactive drugs. Potential older
participants also completed the modified Telephone Interview for Cognitive Status
(TICS; Brandt, Spencer, & Folstein, 1988) with a score ≥ 30 to exclude individuals with
questionable cognitive function or dementia (Welsh, Breitner, & Magruder-Habib, 1993).
Procedure

71

To reduce variations in stress hormone levels between participants, the study was
conducted between 1:30 and 5pm and participants were instructed to abstain from food
and exercise within 1 hr of the study, as well as, sleep within 2 hrs and caffeine or alcohol
within 3 hrs. Prior to entering the scanner, participants completed questionnaires, visited
a mock scanner to review scan procedures, reviewed instructions for computerized tasks,
practiced the reinforcement learning task on a laptop, and completed baseline measures
of subjective stress and pain. Immediately before entering the scanner, participants
completed the hand immersion task (either the cold pressor or warm-water control task)
and then subjective ratings of stress and pain experienced during the hand immersion
task. After entering the scanner, participants completed a reaction speed task (results not
included here). Then, they completed the reinforcement learning task with concurrent
fMRI. The learning task began ~19 min after the completion of the hand immersion task.
This delay period allowed for the reinforcement-learning task to occur during the peak of
the hypothalamic-pituitary-adrenal axis response to stress (Dickerson and Kemeny,
2004). The reinforcement-learning task was divided into four runs of 37 trials each (each
lasting ~6.5 min). A brief structural MRI task followed (~4 min). At the end of the study,
participants completed a post-experiment questionnaire and received their payment.
Stress Manipulation: Hand Immersion in Water
Participants were randomly assigned to either the cold pressor task (Lovallo,
1975) or a warm-water control. For the cold pressor (stress condition), participants were
asked to submerge their non-dominant hand in ice water (0-3 degrees C) for 3 min. They
were told that they could remove their hand at any point if it became too painful. For the

72

warm-water control task, participants were asked to submerge their non-dominant hand in
warm water (37-40 degrees C) for 3 min. Participants who could not keep their hand in
the ice water for at least 60 sec were excused from the study.
Manipulation Check: Subjective Stress and Pain Ratings
On separate visual analogue scales, participants reported on their current stress
and pain levels just before the hand immersion task (baseline). Immediately following the
hand immersion task, they provided separate ratings of the worst stress and pain they
experienced during the hand immersion task. Ratings were made on a line 10-cm long,
such that 0 cm represented no stress/pain and 10 cm represented worst possible
stress/pain. Subjective stress and pain response values were determined from the change
from baseline divided by the maximum of the scale (e.g., [7 cm hand immersion task – 1
cm baseline] / 10 cm). In this way a value of 1 represented greatest possible subjective
stress response, .5 represented a moderate response, and 0 represented no response.
Task Stimuli
Feedback for the reinforcement-learning task consisted of pictures of faces and
audio clips of single words. For positive social feedback participants saw happy faces and
heard complimentary words (e.g., superb), for negative feedback they saw angry faces
and heard insulting words (e.g., moron), and for neutral feedback they saw neutral faces
and heard neutral words (e.g., chair). Pictures were drawn from the FACES database,
which has been validated for emotional expression identification by a sample of younger,
middle age and older adults (Ebner, Riediger, & Lindenberger, 2011). For each feedback
valence category, images included equal numbers of faces of young men and women

73

(models M = 24.3 years, SD = 3.5; age range, 19–31) and older men and women (models
M = 73.2 years, SD = 2.8; age range, 69–80). Specifically, the number of faces in each
gender/age category that may be seen by a given participant was equal, balanced across
experimental runs, and determined prior to experimental sessions. A total of 384 pictures
were included in the study.
Face pictures were accompanied by an audio clip of a single word matched to the
emotional valence of the facial expression, as well as, to the gender and age of the
photographed person. The author recorded audio feedback clips with four volunteers (one
from each gender-age group). Pilot data was collected with an independent sample to
determine that the audio clips were rated appropriately in terms of valence (positive,
negative, or neutral), arousal (emotional, non-emotional), and could be understood by
participants (e.g., good sound quality). There were 12 words in each valence category
(see Appendix C for a full list of word stimuli).
Behavioral Task
The reinforcement-learning task used in the current study was adapted from a
previous fMRI study (Lin, Adolphs, & Rangel, 2011), and involved learning to select
visual cues (slot machine images) that were more likely to result in positive social
feedback, and avoid selecting cues that were more likely to result in negative social
feedback. There were a total of 148 trials. As shown in Figure 3.1 each trial began with a
fixation cross (for 1-6 s jittered), followed by a presentation of slot machine pairs and
choice selection (2.5 s), then a delay (1-5 s jittered), and then the probabilistic outcome
was presented (1.5 s). Trials were broken up into 4 runs of 37 trials each (run duration

74

~6.5 min). Each run involved learning the feedback contingencies of two pairs of slot
machines. After the second run was completed, two new pairs of slot machines replaced

75

the previous pair to avoid a plateau in learning and to keep participants engaged in the
task.
Slot machines had one of three outcome distributions: “mean-positive”, “mean-
negative” or “mean-neutral” (Table 3.1). In a given run, there was always one positive
slot machine pair (mean-positive and mean-neutral) and one negative pair (mean-negative
and mean-neutral). Selecting a mean-positive slot machine resulted in positive feedback
72% of the time (neutral 28%), while selecting a mean-negative slot machine resulted in
negative feedback 72% of the time (neutral 28%), and mean-neutral slot machine was
equally likely to give positive, negative and neutral feedback. Feedback probabilities
were not disclosed to participants; they were expected to use trial and error to learn cue-
outcome associations. Specifically, participants were told that over time, they should try
to learn which slot machines are more likely to result in positive, negative and neutral
feedback, and that their goal was to get as much positive feedback, and as little negative
feedback, as possible.
The task included two trial types: choice (select from two different slot
machines) and no-choice (select from two identical slot machines). In both cases, the
resulting outcome followed the selected machine’s outcome distribution; however,
inclusion of the two trial types allowed for a choice versus no-choice comparison (i.e., to
control for blood-oxygen-level dependent response to the visual cues). Across the
experiment, there were 100 choice trials (50 with mean-positive/mean-neutral pairs, 50
with mean-negative/mean-neutral pairs) and 48 no-choice trials (balanced across all slot
machine stimuli).

76

Participants completed a practice session outside of the scanner that included at
least 37 trials. During the practice, game contingencies were identical to the task, but
participants saw different slot machine stimuli and saw text feedback instead of social
feedback. Feedback text included the words “correct” (positive), “incorrect” (negative),
and “neutral” (neutral). If performance on either the positive or negative pair was not ≥
60%, participants repeated the practice block for another 37 trials. If they did not meet
the performance criterion within the next 37 trials, another practice block was completed
up to a maximum of four practice blocks. If performance was not ≥ 60% for at least one
of the pairs by the fourth practice block, the participant was excluded.
Computational Model and Fitting
6

We fit a reinforcement learning (RL) model to subjects’ choices. RL is a
framework for learning from rewards and punishments by trial and error (Sutton & Barto,
1998). For our task, the RL model estimates the expected value of each cue and chooses
appropriate actions based on these expected values.
On each trial, the model presented with two cues (from four possible cues in each
session), and . Then the model chooses one of them, say , and receives a
feedback, . The feedback could be a social positive stimulus, , a social negative
stimulus, , or a neutral stimulus, . The model uses feedback to update its
estimate of expected value of the chosen cue:

6
Collaborator Payam Piray provided text and tables (see Appendix D) describing the
implemented computational model (from Sutton & Barto, 1998) and fitting.
q
t
! q
t
q
t
t
r r
t
=1
r
t
=!1 r
t
=0
Q
t+1
(q
t
)=Q
t
(q
t
)+!"
t

77

Here, is the expected value (EV) for , where is the index of trial. is the
prediction error (PE) signal, which is the difference between actual outcome and
expected value of the relevant cue:

Finally, is learning rate, which determines the degree that recent feedbacks
affect expected value (e.g., for = 0: feedback not taken into account; = 1: only the
most recent feedback influences the next choice). The probability of choosing is
computed using softmax rule:

where is inverse temperature, which determines the degree of exploration for new
actions (e.g., higher is associated with more exploitation). represents the effect
of choices sequence, independent of reward history, on subjects’ choices and
determines the degree that affects choosing . Recent studies have shown that
is an effective parameter on subjects’ choices (Rutledge et al., 2009; Schönberg,
Daw, Joel, & O’Doherty, 2007). Thus, on th trial, if is also the cue
selected at its last presentation, otherwise . So, for model tends to
switch its choices and for , the model tends to preserve on previous choices. For
, the sequence of choices has no effect on model choices and the action-selection
rule reduces to basic softmax. Finally, and are also updated simply on each
trial:
Q
t
(q
t
) q
t
t !
t
!
t
=r
t
!Q
t
(q
t
)
00q
t
p(q
t
)=
1
1+exp(!(Q( " q
t
)#Q(q
t
))+$(C( " q
t
)#C(q
t
)))
!
! C(q
t
)
!1"! "1
C(q
t
) q
t
C(q
t
)
t C(q
t
)=1 q
t
0!C(q
t
)<1 ! <0
! >0
! =0
C(q
t
) C( ! q
t
)

78

where is a constant, which determines the degree that recent choices affect
decision making in next trials. Notably, since participants receive feedback in forced
trials, we assumed learning occurs in these trials.
We used maximum likelihood to fit the model to participants’ choices
individually. For the th subject, the vector containing four parameters of the model,
, is estimated using participant’s choices:
(see Appendix D for measures of model fitting by group.)
It should also be noted that although we fitted the parameters of the model for each
subject, we used a common set of parameters for each group, i.e., the mean of individual
fitted parameters in that group, to generate regressors for neural analysis. This is because
noise in estimation of each subject’s parameters reduces statistical power in a subsequent
second-level fMRI analysis, mainly due to rescaling of regressors between subjects,
which has been reported in many studies used similar algorithms and methods (Daw,
2011; Daw et al., 2006; Schönberg et al., 2007; Schonberg et al., 2011; see Appendix D
for parameters used for generating regressors by group).
Imaging Data Acquisition
Whole brain functional scans were conducted using a sequence designed to
optimize blood-oxygen-level dependent signals in medial prefrontal regions (Deichmann
et al., 2003). The fMRI data acquisition was conducted with the following parameters:
echo time (TE) = 25 ms, slice thickness = 2.5 mm, in-plane resolution = 2 x 2 mm, no
C(q
t
)=1
C( ! q
t
)=dC( ! q
t
)
0!d!1
i
!
i
= ["
i
,#
i
,$
i
,d
i
% ]
!
i
= argmax log p(q
t
i
t
!
)

79

gap, repetition time (TR) = 2.75 sec, flip angle = 90°, and 48 axial slices and a tilted
acquisition sequence at 30° to the AC-PC line. On each test day, 144 volumes were
collected during each of four functional runs lasting ~6.5 min each. Thereafter, a 4-min
T1-weighted structural scan was collected to be co-registered with associated T2*-
weighted images. Anatomical scans had the following parameters: MPRAGE sequence,
resolution = 1 x 1 x 1 mm; TE = 2.26 ms, TR = 1950 ms, echo time, flip angle = 7°).
Brain Imaging Data Analysis
Whole-brain analyses were conducted with FMRIB's Software Library (FSL;
www.fmrib.ox.ac.uk/fsl) using FSL FEAT v. 5.98. Skull stripping was conducted with
BET (Smith, 2002), motion correction with MCFLIRT (Jenkinson, Bannister, Brady, &
Smith, 2002), spatial smoothing with a Gaussian kernel (5mm full-width half-maximum),
and registration with FLIRT (Jenkinson et al., 2002) such that each functional image was
registered to individual participants’ brain-extracted structural image and the standard
Montreal Neurological Institute (MNI) average of 152 brains using an affine
transformation with 12
degrees of freedom.
MELODIC ICA
(Beckmann and Smith,
2004) was used to remove
noise components
(Appendix E for removal
criteria). Further correction for motion artifact was conducted by including subject-

80

specific realignment values from the original fMRI series data as nuisance regressors.
Intensity normalization and high-pass filtering (100 s cutoff) were also applied.
We estimated a general linear model (GLM) on first-level (time-series) with
FILM (Woolrich et al., 2001). Two time points were modeled for each trial: cue and
feedback presentation (excluding no-response trials). For cue presentation, separate
regressors were created for the two trial conditions (choice, no-choice). For feedback
presentation, separate regressors were created for the three valence conditions (positive,
negative, neutral). For the analysis of positive feedback response, we conducted a
positive > neutral feedback presentation contrast across choice and no choice trials, and
likewise for negative feedback response. Trial-by-trial estimates of EV and PE generated
by the computational model were used as parametric modulators of the cue and outcome
time points, respectively. Modulators for the primary regressors that had more than one
modulator were orthogonalized. All regressors were convolved with a double-gamma
hemodynamic response function. Second-level analysis (across runs) was conducted with
FLAME 1 and third-level mixed effects analysis (across subjects) with FLAME 1+2
(Beckman et al., 2003). The third-level GLM included two between-subject factors: stress
condition (cold pressor, control) and age (younger, older). Unequal variance among the
four stress/age groups was assumed. For each level of analysis, Z (Gaussianised T/F)
statistic images were corrected for multiple comparisons with clusters determined by Z >
2.3 voxel-wise thresholding and a family-wise error-corrected cluster significance
threshold of p < 0.05 (Worsley, 2001).
Results

81

Stress Manipulation Check
Separate univariate ANOVAs were conducted to examine group differences in
subjective stress and pain responses to hand-immersion task. Participants exposed to the
cold pressor had higher subjective stress responses, F(1,47) = 69.14, p < .001, η
p
2
= .60,
and pain responses, F(1,47) = 169.10, p < .001, η
p
2
= .78, compared to those who
submerged their hand in warm water (the control task; Figure 3.2). Notably, stress effects
did not differ by age for subjective stress or pain (stress-by-age interaction ps > .05).
These results indicate that the cold pressor was effective in increasing feelings of stress
and pain in younger and older participants, and these stress effects were similar in
magnitude across age groups.
Self-reported Motivation
In a post-experiment questionnaire, participants provided separate ratings of their
motivation to obtain positive feedback and avoid negative feedback during the learning
task. A repeated-measures ANOVA was conducted with motivation valence as the
within-subject factor (obtain positive feedback, avoid negative feedback), and stress

82

condition and age as between-subject factors. Overall motivation ratings (across types)
did not differ significantly by stress group, age, nor was there a stress-by-age interaction.
There was, however, a significant stress-by-motivation valence interaction, F(1,47) =
5.84, p = .02, η
p
2
= .11 (Figure 3.3). This interaction was not modulated by age (three-
way interaction p > .05). Under stress conditions, there was a bias toward wanting to
obtain positive feedback over wanting to avoid negative feedback, but the opposite
pattern was observed in the control group.
7

Learning Accuracy

7
Post-hoc repeated-measure ANOVAs were conducted for the stress and control groups
separately, with the two motivation categories as within-subject factors. These analyses
yielded only marginal differences between ratings for the two motivation types (p = .08
for both ANOVAs). Paired sample t-tests for individual stress/age groups found only a
marginal difference between negative and positive motivations in stressed young adults,
t(13) = -1.81, p = .09; all other groups ns.

83

A repeated-measures ANOVA was conducted for overall selection accuracy for
choice trials (i.e., select mean-positive, avoid mean-negative) with cue pair valence as the
within-subject factor (positive, negative), and stress condition and age as between-subject
factors. Results are displayed in Figure 3.4. Significant main effects were observed for
age group, F(1,47) = 4.19, p = .046, η
p
2
= .08 (lower performance in older adults) and cue
pair valence, F(1,47) = 8.80, p = .005, η
p
2
= .16 (better accuracy for positive slot machine
pairs), but not for stress condition, F(1,47) < 1. There was also a two-way interaction of
age and cue pair valence, F(1,47) = 5.91, p = .02, η
p
2
= .11, indicating that, across stress
conditions, older adults performed best on positive pairs but accuracy on positive and
negative pairs did not differ for younger adults. However, stress modulated this
interaction as we found a three-way interaction of age, cue pair valence, and stress
condition, F(1,47) = 5.91, p = .02, η
p
2
= .11. Specifically, in young adults alone stress
affected accuracy depending on the cue pair valence, F(1,25) = 4.77, p = .04, η
p
2
= .16,
but, when older adults were analyzed alone, the interaction of stress and cue pair valence
was not significant, F(1,22) = 1.54, p = .23, η
p
2
= .07. In young adults, overall accuracy
for negative slot machines was similar across stress conditions, but was marginally
increased by stress accuracy for positive slot machines.
8
Whereas in older adults,
performance was best on positive pairs relative to negative pairs, regardless of stress
condition. Post-hoc t-tests were conducted to determine if performance differed for
positive and negative pairs within stress/age groups. Results revealed better performance
for positive relative to negative cue selection in stress-group younger adults, t(13) = 2.46,

8
A post-hoc t-test for the difference in performance on positive pairs alone for young
controls versus stressed subjects narrowly missed significance, t(25) = -2.04, p = .05.

84

p = .03, and in control-group older adults t(11) = 4.70, p = .001. There were no valence-
based learning biases for the other two groups (ps > .05).
Correlations between Learning Accuracy and Motivation Ratings
Pearson’s correlations were computed to examine inter-individual differences in
the relationship between self-reported motivation for valenced feedback (obtain positive,
avoid negative) and learning accuracy (positive, negative). Bivariate correlations for
these four measures were conducted separately for younger and older adults. In the young
adults, we observed a strong correlation between motivation to obtain positive feedback
and selection accuracy for positive cues, r(25) = .69, p < .001. This correlation fits well
with the stress effects observed in younger adults at the group level, in which stress
appeared to enhance the desire for positive feedback and accuracy in selecting the most
positive cues. In contrast, greater motivation to avoid negative feedback was marginally
associated with poorer performance on avoiding negative cues in younger adults, r(25) =
-.38, p = .05 (all other ps > .05). A different picture emerged for older adults. In this
group, there was a significant association between motivation to obtain positive feedback
and avoid negative feedback, r(22) = .46, p = .02. In addition, motivation to avoid
negative feedback was associated with accuracy for selecting positive cues, r(22) = .41, p
= .049 (all other ps > .05). The latter result may suggest that the observed positivity bias
in older adults’ learning was at least partly driven by a desire to avoid negative outcomes,
rather than to obtain positive outcomes.
Learning by Block

85

We also examined
whether learning by trial
block was modulated by
stress or age. Each participant
was presented with two
positive and two negative slot
machine pairs during the
experiment; one of each type
(positive, negative) during
the first half of the
experiment and one of each
type in the second half of the
experiment. We collapsed
accuracy scores across the
first and second half of the
experiment to examine
learning by number choice
trials for each pair type.
Mean accuracy values were divided into blocks of five trials each (e.g., trials 1-5, 6-10).
Separate repeated-measures ANOVAs were conducted for mean accuracy for positive
and negative pairs during choice trials. This analysis included trial block (1-5, 6-10, 11-

86

15, 16-20, 21-25) as a within-subject factor, and stress condition and age as between
subject factors.
Accuracy increased linearly by block for both positive, F(1,47) = 35.25, p < .001,
η
p
2
= .43, and negative pairs, F(1,47) = 15.28, p < .001, η
p
2
= .25 (Figure 3.5). For
positive pairs, there was also a linear stress-by-age-by-block interaction, F(1,47) = 17.00,
p = .01, η
p
2
= .13 (Figure 3.5 A). Examination of means and confidence intervals
indicated that accuracy was similar across groups for the first block, but increased
significantly from the 1
st
to 2
nd
block only for younger stressed and older control
participants – performance in these two groups tracking together from the 1
st
to last
block. In contrast, we observed slower, steady increases in control younger adults, but no
gains in stressed older adults – the latter group having similar performance from the 1
st
to
last block. A different pattern was observed for negative pairs, such that stress did not
significantly affect learning by trial block. The only interaction observed for negative pair
learning was a 4
th
-order age-by-block interaction, F(1,47) = 4.66, p = .04, η
p
2
= .09
(Figure 3.5 B). Examination of means and confidence intervals revealed that, across
stress conditions, younger adults selected the negative slot machines significantly less
from the 1
st
to 2
nd
block, but older adults’ gains in avoiding negative machines did not
differ from one block to the next (i.e., means and confidence intervals were overlapping
for consecutive blocks).
In sum, the behavioral results indicate that stress affected cue learning more when
associated outcomes were positive than when they were negative, but the nature of stress
effects depended on age. Furthermore, different relationships between motivation and

87

learning performance were observed in younger and older adults, such that these
measures tracked together for positive feedback in younger adults but not older adults.
Neural Response to Reinforcement Learning Parameters Across Groups
Our results focus on group differences rather than group-mean responses to
learning parameters; however, details for the observed neural response to EV, PE, and
feedback by valence type across groups are displayed in Table 3.2. and discussed in
Appendix F.

88

Learning-related Activation: Stress Effects
Stress effects were observed for activation related to negative EVs in the occipital
fusiform area and precuneus such that there was a parametric increase in response to
negative EVs with stress relative to control (Figure 3.6 A). Similarly, the stress group had
significantly greater response to negative PEs in the precuneus/superior parietal lobe,
lateral occipital cortex, and DLPFC (Figure 3.6 B). No stress effects were observed for
positive EVs or PEs. The only stress effect observed for a positive learning component
was for positive feedback (Figure 3.6 C). Control subjects had a stronger response to
positive feedback (positive > neutral) in the occipital pole. Thus, across age groups, stress
appeared to enhance response to learning parameters with a negative valence in the

89

fronto-parietal, occipito-temporal visual regions and the DLPFC, but reduce response to
positive feedback in the occipital cortex.
Learning-related Activation: Age Effects
An age difference for negative EVs was observed such that younger adults had a
stronger response than older adults in the occipital fusiform gyrus and superior frontal
gyrus/medial PFC (Figure 3.6 D). Conversely, for response to positive EVs, there was an
age difference in the visual cortex (occipital pole) with older adults showing greater
activation in this region than younger adults (Figure 3.6 E). No significant age
differences were found for response to PEs or positive and negative feedback. The
observed age effects are in line with an age-related positivity effect (Mather &
Carstensen, 2005), such that younger adults had a stronger neural response to negatively
valenced EVs, while older adults had stronger response to positively valenced EVs.
Learning-related Activation: Age-dependent Stress Effects
We did not observe any significant stress-by-age effects for neural response to the
examined learning parameters.
Discussion
The current fMRI study examined the impact of cold pressor stress on subsequent
learning of cue-outcome associations in a probabilistic selection task with social feedback
(emotional faces and words). This study adds to a growing list of findings indicating that
effects of stress on reinforcement learning depend on whether reinforcers are positive or
negative (Mather & Lighthall, 2012), and makes the unique contribution of identifying
neural and motivational mechanisms – as well as age-related differences – for these stress

90

effects. Consistent with previous behavioral findings (Cavanagh et al., 2010; Lighthall et
al., in revision; Petzold et al., 2010), the present study found that acute stress led to
enhanced cue-selection performance for positive cues relative to negative cues in younger
adults. We also observed a stress-related enhancement in participants’ self-reported
desire to receive positive feedback over avoiding negative feedback during the task –
with no difference in this effect by age.
Counter to our prediction and previous research (Lighthall et al., in revision;
Chapter 2), older adults in the current study did not exhibit enhanced positive feedback
learning with stress. Instead, across stress conditions, their behavior reflected a positivity
effect (Mather & Carstensen, 2005), such that they performed best at selecting cues with
the greatest probability of leading to positive social feedback. This divergence from our
previous findings may be related to the current learning task, which differed from our
previous task (Lighthall et al., in revision), in that feedback was social. The importance of
positive social-emotional experiences appears to increase with age (Carstensen,
Isaacowitz, & Charles, 1999) and older adults performed at the level of younger adults
for positive cue-selection accuracy in the current study (82% in younger and older). Thus,
one possibility is that stress could not enhance older adults’ performance for positive cue
learning as they were already at their peak level. Another possibility is that stress-by-
valence effects may be observed in older adults when the contingencies for choice
options are more distinct. For example, in the current study, the probability of positive
feedback for the most positive cue was .72, and the alterative choice option resulted
positive, negative and neutral feedback with equal likelihood. In our previous study

91

(Lighthall et al., in revision; Chapter 2), the probability of positive feedback from the
most positive cue was .80, and the alternative option had a .80 chance of resulting in
negative feedback. An age difference in the relationship between self-reported motivation
and performance was also observed. In younger adults, motivation to obtain positive
feedback was correlated with performance on positive cue selection, and motivation to
avoid negative feedback was correlated with negative cue avoidance. In older adults, it
was motivation to avoid negative feedback that predicted performance for positive cues.
This finding highlights the possibility that older adults’ motivation to avoid negative
information made them focus more on positive cue pairs, resulting in better performance
for positive. It is also possible of that completing the learning task in the scanner
environment changed the nature of cold pressor stress effects for older adults. We did not
measure subjective stress related to the scanner environment, but it is possible that the
experience was more novel and/or stressful for older adults. Thus, older controls may
have been experiencing mild/moderate stress during the task, while older adults exposed
to the cold pressor and scanner environment were experiencing high levels of stress,
perhaps pushing the latter group to the downward slope of the inverted-U function of
stress effects on cognition (Lupien, McEwen, Gunnar, & Heim, 2009). This proposition
is consistent with older adults’ behavioral results indicating significantly better reward
learning relative to aversion learning in the control condition, but no significant
difference between the two learning types in the stress condition.
The primary contribution of the current study is that it provides the first findings
on the neural correlates of stress effects on reinforcement learning using fMRI, and

92

furthermore examines age differences in these neural correlates. We investigated stress-
related differences in brain activation correlating with positive and negative expected
stimulus value (value weights for cues based on past experiences), prediction error
(degree to which the actual outcome differs from the expected outcome), and feedback
type (positive, negative). We hypothesized that stress effects would be observed in
dopaminergic reward regions in the basal ganglia and prefrontal cortex. These results
were not observed, however, we did find stress effects on other regions and networks that
support motivated choice and learning. Specifically, across age groups, stress enhanced
brain activation corresponding with negatively valenced expected values for feedback
cues and prediction error in fronto-parietal regions and occipito-temporal regions, and
diminished response to positive feedback in the occipital pole. Increased activation of
these regions during value-based decision making or motivated selection tasks is thought
to reflect attentional (fronto-parietal) and sensory processing (vision regions) biases
(Pessoa & Engelmann, 2010; Serences, 2008). That is, effective selection a stimulus that
will lead to reward among competing stimuli requires attentional control functions, such
as holding the rewarding stimulus in mind, and goal-directed vision functions, such as
scanning for the most rewarding stimuli. Importantly, however, as stress enhanced
learning from positive feedback but not negative feedback, patterns of increased
activation in attention and perception regions during learning about negative feedback
may represent suboptimal or compromised processing under stress (e.g., increased effort
or interference during negative feedback learning relative to positive).

93

We also found age differences in the involvement of fronto-parietal and vision
networks during reinforcement learning. Specifically, greater activation of these networks
was observed in younger than older adults for negatively valenced expected values, but
involvement of these networks was greater in older adults for positively valenced
expected values. Given that older adults’ learning accuracy was biased in favor of
positive cue pairs, and younger adults performed better than older adults on negative
pairs across stress conditions, the observed age differences in neural response to expected
values may well reflect different feedback-related priorities and associated attentional and
sensory biases in younger and older adults.
The observed stress and age differences for response to expected value, prediction
error and social feedback were not qualified by any interactions of stress and age,
indicating effects of stress on neural response to these learning parameters did not differ
significantly for younger and older adults. Given the stress-by-age effects observed for
behavior, however, we must consider the possibility that our sample size was not
sufficient to detect interaction effects in the fMRI data. This limitation may be addressed
by larger studies in the future, or with region of interest analysis, which can help to
overcome challenges present in detecting age differences using whole brain analysis of
fMRI data (Samanez-Larkin & D'Esposito, 2008). Another potential limitation of the
current study is that we did not observe learning parameter-related signals in the classic
dopaminergic reward-processing regions (e.g., striatum, ventromedial PFC), which may
have been the result of our study design, particularly the use of social stimuli. Use of
social feedback in the current study holds value in that everyday reinforcement learning

94

often involves social feedback (e.g., choosing social partners, making decisions about
trust). However, previous research using a similar task design and stimuli found that,
compared to monetary feedback, social feedback-based learning signals are weaker and
more difficult to observe in the basal ganglia (Lin et al., 2011). Thus, the current study
cannot rule out the possibility that stress affects dopaminergic reward regions such as the
striatum and ventromedial PFC. What can be stated, however, is that even if social
feedback-related signals were weak in the current study, stress and age effects were still
be observed in brain regions involved in motivated learning and decision making, albeit
at the level of attention, perception, and cognition-sensation-motivation integration (e.g.,
DLPFC).
In summary, the current study found that acute stress can alter social
reinforcement learning behavior and the motivations for this behavior, but effects of
stress may depend on age. Furthermore, measures of motivation, behavior, and brain
activation converged on the idea that older people prioritize processing of positive
feedback over negative feedback during social reinforcement learning. The study also
highlighted the influence of stress and age on involvement of attentional control and
vision perception regions during reinforcement learning. A more comprehensive
understanding of the brain mechanisms of stress effects on reinforcement learning across
the life span may be gained from investigations with different of reinforcers, longitudinal
studies, and analysis of functional interactions between brain regions (e.g., striatum and
DLPFC) during learning from positive and negative experiences.

95

CHAPTER 4: CONCLUSION
Summary
This dissertation makes several important contributions to our understanding of
the impact of acute stress on motivated learning and decision making in younger and
older adults.
Chapter 1 presented findings from a functional imaging study on stress and risky
decision making in younger adults – following up on behavioral findings indicating
finding that risk taking diverges under stress in young men and women (Lighthall et al.,
2009). Results in Chapter 1 underscore the importance of gender in determining how
stress may affect risk-related decision processing, revealing effects of stress on both
behavior and related patterns of brain activation during a risk-taking task involving
monetary reward. In particular, the study found no gender differences in risk-taking
behavior and brain activation under control conditions, with stress however, young men
had greater reward collection and faster decision speed relative to controls but effects of
stress were opposite in young females. The study in Chapter 1 also found a gender-by-
stress interaction for decision-related brain activation in the dorsal striatum and anterior
insula, brain regions involved in incentive processing (Clark, 2010; Ernst and Paulus,
2005; Haber and Knutson, 2010; Taylor et al., 2007) and the integration of cognitive,
affective, and sensory information (Balleine et al., 2007; Doya, 2008; Miyachi et al.,
2002; O’Doherty et al., 2004). With acute stress, activation in these regions was
increased in young males but decreased in young females. In terms of contribution to the

96

literature, this study is the first to demonstrate gender differences in response to decision
making among individuals exposed to acute stress.
The study in Chapter 2 tested the hypothesis that stress increases reward salience,
leading to more effective learning about positive than negative outcomes (“correct” and
“incorrect” feedback, respectively) in a probabilistic selection task, and aimed to
determine whether effects of stress were similar for younger and older adults. The study
found that stress enhanced positive-feedback learning, but not negative-feedback
learning, in both younger and older adults. This study is first to demonstrate feedback-
valence dependent stress effects for reinforcement learning in older adults – these
findings in older adults being generally consistent with those observed in previous studies
with only younger adults (Cavanagh et al., 2010; Petzold et al., 2010). One hypothesis
developed from this study, was that dopaminergic reward network regions including the
striatum and medial prefrontal cortex mediated stress effects – the hypothesis being
supported by Study 2 findings and previous research indicating that stress alters reward-
based learning.
This hypothesis was tested by the study presented in Chapter 3, which is the first
fMRI study to identifying neural and motivational mechanisms of stress effects on
reinforcement-based learning. In addition, the study included younger and older adults to
determine whether there were age differences these neural mechanisms. The study
included a reinforcement-learning task with social feedback (emotional faces and words)
and found that stress altered brain activation corresponding with choice option valuation,
prediction error, and feedback presentation. Results did not support the hypothesis that

97

stress alters reinforcement learning via modulation of dopaminergic reward network
regions. Instead, the study revealed effects of stress on task-relevant cognition and
perception regions; specifically, fronto-parietal regions involved in attentional control
and occipito-temporal regions involved in early visual processing. These patterns of
stress effects were similar in younger and older adults, however, older adults showed a
general positivity bias in their learning behavior and brain activation across stress
conditions. This latter age-related finding is consistent with the idea that aging is related
to a shift towards prioritizing positive over negative information (Mather & Carstensen,
2005).
Limitations and Future Directions
This dissertation makes several important contributions to our understanding of
how stress affects motivated learning and decision making in early and late adulthood,
but has some limitations that should be addressed in future research. Chief among them,
the reason that age differences in stress effects on reinforcement-learning behavior were
observed in one study (Study 3) but not another (Study 2) are presently unclear. These
mixed findings may be related to differences in feedback type and/or feedback
contingencies for the study tasks. The role of feedback type may be examined in future
studies with direct comparisons of stress effects on reinforcement learning involving
social (e.g., emotional faces/words as in Study 3) and non-social (e.g.,
“correct”/“incorrect” as in Study 2) feedback. Differences in stress effects on learning for
older adults could have also been related to how distinct choice options were from each
other. For example, it is possible that, for older adults, the enhancement of positive-

98

reinforcement learning with stress is more restricted to learning about choice option pairs
with very different types of emotional salience (e.g., highly positive/highly negative
option pairs, as in Study 2) versus choice option pairs that are less distinct (e.g., highly
positive/neutral option pairs, as in Study 3).
Another possibility is that, in Study 3, exposure to the scanner environment
significantly increased stress in older adults across stress conditions. For instance, older
adults in the control group may have experienced some stress, while older adults in the
cold-pressor group experienced even greater stress – pushing the latter group to a
detrimental level of stress. This possibility is supported by the fact that, like the younger
stress group, positive-feedback learning was significantly better than negative-feedback
learning in the older control group. Study 3 did not have a measure of stress response
related to the protocol generally, but future research may include subjective ratings and
stress hormone measures that can address possible age differences in stress response
related to the scanner environment. In addition, it is possible that limitations related to
power and/or design prohibited the observation of expected stress effects on
dopaminergic reward regions in Study 3. Future studies may be advised to include larger
samples and use reinforcers that are more likely to elicit reinforcement-learning signals
(e.g., food, monetary reward).
Finally, the current dissertation did not examine stress effects on risk-related
processing in older adults. This was because gender differences were expected, making
the required sample size to observe stress effects in younger and older adults
prohibitively large (i.e., six groups with ≥ 12 participants per cell). Thus, it is unclear if

99

gender modulates stress effects on risk-related brain activation in older adults, but well-
powered studies or investigations with only older adults may address this question.
Broader Implications
Across three studies, this dissertation demonstrated that stress affects motivated
learning and decision making in healthy younger and older adults – highlighting the
important role that transient emotional states play in guiding our choices across
adulthood. In particular, the findings from Study 1 provide insight into at least one source
of real-world gender differences in risk taking. For example, stress may contribute to
observed gender differences – specifically greater risk taking in men – for investment
decisions (Clark, D’ambrosio, McDermed, & Sawant, 2003; Jianakoplos & Bernasek,
1998; Sunden & Surette, 1998), alcohol use (Hill & Chow, 2002), and driving behavior
(Maxim & Keane, 1992). The findings from Study 1 suggest that experiences of acute
stress during risky decision making may increase the likelihood of hasty decisions in
men, and perhaps also enhance the allure of potential rewards via modulation of
dopaminergic brain regions (e.g, the striatum).
Studies 2 and 3 hold implications for understanding how stress may bias our
learning. The results of these studies indicate that we are more likely to remember
positive associations of past actions under stress. The clinical literature supports this
conclusion, as stress is one of the strongest predictors of drug relapse and increases drug
cravings in addicts (Sinha, 2009). This dissertation research finds consistent patterns in
healthy younger and older adults, such that the salience of rewards may be enhanced by
acute stress during learning. Further, Study 3 indicated that stress modulates motivation

100

to receive positive feedback and activation of attentional control and visual regions
during reinforcement-based learning. These findings fit well with existing models of
motivational effects on attention (Pessoa & Engelmann, 2010), which may suggest that
stress enhances motivation for reward, and increased motivation biases attentional control
and perception regions towards potentially rewarding stimuli. Thus, stressful experiences
may result in a “pop-out” effect, such that positive or rewarding choice options become
more perceptually salient.
Finally, this dissertation provides new insights into how older adults learn from
positive and negative experiences both with and without stress. Taken together, the
findings from Studies 2 and 3 indicated that older adults may sometimes experience
enhanced learning from positive feedback when under stress, but these stress effects may
be more affected by individual and situational factors like feedback type, level of task
difficultly, and whether increases to stress are small or large. These factors may also play
a role in determining when older adults exhibit enhanced learning from positive feedback
relative to younger adults. For example, the results of Study 3 are consistent with the idea
that positive social experiences are particularly important for older people
(Socioemotional Selectivity Theory; Carstensen et al., 1999). This finding extends the
scope of Socioemotional Selectivity Theory, indicating that when potential
reinforcements are social and positive, older adults can perform at the level of younger
adults on a challenging probabilistic-learning task. Thus, learning strategies and aids that
include positive social information may be particularly helpful for older adults.

101

In closing, this dissertation includes some of the first work in two emerging areas
of study, namely stress and decision making and decision neuroscience and aging. These
studies highlight several factors that influence learning and decision making, and also
draw attention to important issues that require further researcher to develop a full
understanding of the neural mechanisms of these cognitive functions. As the number of
older adults living independently and facing difficult financial, social, and health
decisions rapidly increases, this research is likely to fill a critical gap and may be used to
help older adults make better decisions in the coming years.

102

REFERENCES
Abercrombie, E. D., Keefe, K. A., DiFrischia, D. S., & Zigmond, M. J. (1989).
Differential effect of stress on in vivo dopamine release in striatum, nucleus
accumbens, and medial frontal cortex. Journal of Neurochemistry, 52, 1655-1658.
Almeida, D. M., Wethington, E., & Kessler, R. C. (2002) The daily inventory of stressful
events: an interview-based approach for measuring daily stressors. Assessment, 9,
41-55.
American Psychological Association. 2008. Stress in America Washington DC report.
Washington, DC: Harris Interactive Inc., Public Affairs and Policy.
Anstrom, K. K., & Woodward, D. J. (2005). Restraint increases dopaminergic burst firing
in awake rats. Neuropsychopharmacology, 30, 1832-1840.
Aslaksen, P. M., Myrbakk, I. N., Høifødt, R. S., & Flaten, M. A. (2007) The effect of
experimenter gender on autonomic and subjective responses to pain stimuli. Pain,
129, 260-8.
Bäckman, L., Lindenberger, U., Li, S. C., & Nyberg, L. (2010). Linking cognitive aging
to alterations in dopamine neurotransmitter functioning: recent data and future
avenues. Neuroscience and Biobehavioral Reviews, 34 670-677.
Bäckman, L., Nyberg, L., Lindenberger, U., Li, S. C., & Farde, L. (2006). The correlative
triad among aging, dopamine, and cognition: current status and future prospects.
Neuroscience and Biobehavioral Reviews, 30, 791-807.
Balleine, B. W., Delgado, M. R., & Hikosaka, O. (2007) The role of the dorsal striatum in
reward and decision-making. The Journal of Neuroscience, 27, 8161-5.

103

Bechara, A., Damasio, H., Tranel, D., & Damasio, A. R. (2005). The Iowa Gambling
Task and the somatic marker hypothesis: some questions and answers. Trends in
Cognitive Science, 9, 159-62.
Beckmann, C. F., & Smith, S. M. (2004) Probabilistic independent component analysis
for functional magnetic resonance imaging. IEEE Transactions on Medical
Imaging, 23, 137-52.
Beckmann, C. F., Jenkinson, M., & Smith, S. M. (2003) General multilevel linear
modeling for group analysis in FMRI. Neuroimage, 20,1052-63.
Blalock, H. (1972) Social Statistics. New York, NY: McGraw-Hill.
Bogdan, R., & Pizzagalli, D. A. (2006) Acute stress reduces reward responsiveness:
implications for depression. Biological Psychiatry, 60, 1147-54.
Bolla, K. I., Eldreth, D. A., Matochik, J. A., & Cadet, J. L. (2004) Sex-related differences
in a gambling task and its neurological correlates. Cerebral Cortex, 14, 1226-32.
Brandt, J., Spencer, M., & Folstein, M. F. (1988). The Telephone Interview for Cognitive
Status. Neuropsychiatry, Neuropsychology, & Behavioral Neurology, 1, 111–7.
Brewer DD, Catalano RF, Haggerty K, Gainey RR, Fleming CB. 1998. A meta-analysis
of predictors of continued drug use during and after treatment for opiate
addiction. Addiction. 93(1):73-92.
Calhoun, V. D., Adali, T., Pearlson, G. D., & Peker, J. J. (2001) A method for making
group inferences from functional MRI data using independent component
analysis. Human Brain Mapping, 14, 140-51.

104

Carstensen, L. L., Isaacowitz, D. M., & Charles, S. T. (1999). Taking time seriously: A
theory of socioemotional selectivity. American Psychologist, 54, 165-81.
Cavanagh, J. F., Frank, M. J., & Allen, J. J. (2010). Social stress reactivity alters reward
and punishment learning. Social Cognitive and Affective Neuroscience, 6(3), 311-
320.
Clark, L. (2010) Decision-making during gambling: an integration of cognitive and
psychobiological approaches. Philosophical Transactions of the Royal Society of
London. Series B, Biological Sciences, 365, 319-30.
Clark, R. L., D’ambrosio, M. B., McDermed. A. A., & Sawant, K. (2003). Sex
differences, financial education, and retirement goals. In Mitchell OS, Utkus SP
(Eds.), Pension Design and Structure. Oxford: Oxford University Press.
Coates, J. M., & Herbert, J. (2008) Endogenous steroids and financial risk taking on a
London, trading floor. Proceedings of the National Academy of Sciences of the
United States of America, 105, 6167-72.
Cohen, M. X., & Frank, M. J. (2009). Neurocomputational models of basal ganglia
function in learning, memory and choice. Behavioural Brain Research, 199, 141-
156.
Cox, K. M., Aizenstein, H. J., & Fiez, J. A. (2008). Striatal outcome processing in
healthy aging. Cognitive, Affective, & Behavioral Neuroscience, 8, 304-317.
Davis SW, Dennis NA, Daselaar SM, Fleck MS, Cabeza R. (2008). Que PASA? The
posterior-anterior shift in aging. Cerebral Cortex, 18, 1201-9.

105

Daw, N. D. (2011).Trial-by-trial data analysis using computational models. In: Delgado
MR, Phelps EA, Robbins TW, (Eds.) Decision Making, Affect, and Learning.
Oxford University Press.
Daw, N. D., O’Doherty JP, Dayan P, Seymour B, & Dolan RJ. (2006). Cortical substrates
for exploratory decisions in humans. Nature, 441(7095), 876-9.
Dedovic, K., D'Aguiar, C., & Pruessner, J. C. (2009) What stress does to your brain: a
review of neuroimaging studies. Canadian Journal of Psychiatry, 54, 6-15.
Deichmann, R., Gottfried, J. A, Hutton, C., & Turner, R. (2003). Optimized EPI for fMRI
studies of the orbitofrontal cortex. NeuroImage, 19, 430-41.
Del Arco, A., Segovia, G., de Blas, M., Garrido, P., Acuña-Castroviejo, D., Pamplona,
R., & Mora, F. (2011). Prefrontal cortex, caloric restriction and stress during
aging: studies on dopamine and acetylcholine release, BDNF and working
memory. Behavioural Brain Research, 216, 136-145.
Denburg NL, Cole CA, Hernandez M, Yamada TH, Tranel D, Bechara A, Wallace RB.
2007. The orbitofrontal cortex, real-world decision making, and normal aging.
Ann N Y Acad Sci. 1121:480-98.
Deuschle, M., Gotthardt, U., Schweiger, U., Weber, B., Körner, A., Schmider, J.,
Standhardt, H., . . . & Heuser, I. (1997). With aging in humans the activity of the
hypothalamus–pituitary–adrenal system increases and its diurnal
amplitudeflattens. Life Sciences, 61, 2239-2246.
Diamond, A. (2007). Consequences of variations in genes that affect dopamine in
prefrontal cortex. Cerebral Cortex, 17, i161-i170.

106

Dickerson, S. S., & Kemeny, M. E. (2004) Acute stressors and cortisol responses: A
theoretical integration and synthesis of laboratory research. Psychological
Bulletin, 130, 355-91.
Doll, B.B. & Frank, M.J. (2009). The basal ganglia in reward and decision making:
computational models and empirical studies. J. Dreher & L. Tremblay (Eds)
Handbook of Reward and Decision Making (pp. 399-425), Oxford: Academic
Press.
Dowling GJ, Weiss SR, Condon TP. 2008. Drugs of abuse and the aging brain.
Neuropsychopharmacology. 33(2):209-18.
Doya, K. (2008) Modulators of decision making. Nature Neuroscience, 11, 410-6.
Ebner, N. C., Riediger, M., Lindenberger, U. (2010). FACES--a database of facial
expressions in young, middle-aged, and older women and men: development and
validation. Behavior Research Methods. 42, 351-62.
Eppinger, B., & Kray, J. (2011). To choose or to avoid: age differences in learning from
positive and negative feedback. Jounal of Cognitive Neuroscience, 23, 41-52.
Eppinger, B., Hämmerer, D., & Li, S. C. (2011). Neuromodulation of reward-based
learning and decision making in human aging. Annals of the New York Academy
of Sciences, 1235, 1-17.
Eppinger, B., Herbert, M., & Kray, J. (2010). We remember the good things: Age
differences in learning and memory. Neurobiology of Learning and Memory, 93,
515-521.

107

Eppinger, B., Kray, J., Mock, B., & Mecklinger, A. (2008). Better or worse than
expected? Aging, learning, and the ERN. Neuropsychologia, 46, 521-539.
Ernst, M., & Paulus, M. P. (2005) Neurobiology of decision making: a selective review
from a neurocognitive and clinical perspective. Biological Psychiatry, 58, 597-
604.
Esposito, F., Scarabino, T., Hyvarinen, A., Himberg, J., Formisano, E., Comani, S., et al.
(2005) Independent component analysis of fMRI group studies by self-organizing
clustering. NeuroImage, 25, 193-205.
Feenstra, M. G., Botterblom, M. H., & Mastenbroek, S. (2000). Dopamine and
noradrenaline efflux in the prefrontal cortex in the light and dark period: Effects
of novelty and handling and comparison to the nucleus accumbens. Neuroscience,
100, 741-748.
Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). “Mini-Mental State” a Practical
Method for Grading the Cognitive State of Patients for the Clinician. Journal of
Psychiatric Research, 12, 189-198.
Frank, M. J., & Kong, L. (2008). Learning to avoid in older age. Psychology and Aging,
23, 392-398.
Frank, M. J., & O’Reilly, R. C. (2006). A mechanistic account of striatal dopamine
function in human cognition: Psychopharmacological studies with cabergoline
and haloperidol. Behavioral Neuroscience, 120, 497-517.

108

Frank, M. J., Santamaria, A., O’Reilly, R. C., & Willcutt, E. (2007). Testing
computational models of dopamine and noradrenaline dysfunction in attention
deficit/hyperactivity disorder. Neuropsychopharmacology, 32, 1583-1599.
Frank, M. J., Seeberger, L. C., & O’Reilly, R. C. (2004). By carrot or by stick: Cognitive
reinforcement learning in Parkinsonism. Science, 306, 1940-1943.
Gazzaley, A., Cooney, J. W., Rissman, J., & D’Esposito, M. (2005). Top-down
suppression deficit underlies working memory impairment in normal aging.
Nature Neuroscience, 8, 1298-1300.
Goldstein, J. M., Jerram, M., Abbs, B., Whitfield-Gabrieli, S., & Makris, N. (2010) Sex
differences in stress response circuitry activation dependent on female hormonal
cycle. Journal of Neuroscience, 30, 431-8.
Good, C. D., Johnsrude, I. S., Ashburner, J., Henson, R. N. A., Friston, K. J., &
Frackowiak, R. S. J. (2001). A voxel-based morphometric study of ageing in 465
normal adult human brains. NeuroImage, 14, 21-36.
Haber, S. N., & Knutson, B. (2010) The reward circuit: linking primate anatomy and
human imaging. Neuropsychopharmacology, 35, 4-26.
Hämmerer, D., Li, S. C., Müller, V., & Lindenberger, U. (2011). Life span differences in
electrophysiological correlates of monitoring gains and losses during probabilistic
reinforcement learning. Journal of Cognitive Neuroscience, 23, 579-592.
Heaney, J. L., Phillips, A. C., & Carroll, D. (2010). Ageing, depression, anxiety, social
support and the diurnal rhythm and awakening response of salivary cortisol.
International Journal of Psychophysiology, 78, 201-208.

109

Henckens, M. J., Hermans, E. J., Pu, Z., Joëls, M., & Fernández, G. (2009). Stressed
memories: how acute stress affects memory formation in humans. Journal of
Neuroscience, 29, 10111-9.
Hill, E. M., & Chow, K. (2002). Life-history theory and risky drinking. Addiction, 97,
401-3.
Hyvarinen, A. (1999) Fast and robust fixed-point algorithms for independent component
analysis. IEEE Transactions on Neural Networks, 10, 626-34.
Imperato, A., Puglisi-Allegra, S., Casolini, P., & Angelucci, L. (1991). Changes in brain
dopamine and acetylcholine release during and following stress are independent
of the pituitary-adrenocortical axis. Brain Research, 538, 111-117.
Jacobsen LK, Southwick SM, Kosten TR. 2001. Substance use disorders in patients with
posttraumatic stress disorder: a review of the literature. Am J Psychiatry.
158(8):1184-90.
Jianakoplos, N. A., & Bernasek, A. (1998). Are women more risk averse? Economic
Inquiry, 36, 620–30.
Jocham, G., Klein, T. A., & Ullsperger, M. (2011). Dopamine-mediated reinforcement
learning signals in the striatum and ventromedial prefrontal cortex underlie value-
based choices. Journal of Neuroscience, 31, 1606-1613.
Kaasinen, V., & Rinne, J. O. (2002). Functional imaging studies of dopamine system and
cognition in normal aging and Parkinson's disease. Neuroscience Biobehavioral
Review, 26, 785-93.

110

Kalivas, P. W., & Duffy, P. (1995). Selective activation of dopamine transmission in the
shell of the nucleus accumbens by stress. Brain Research, 675, 325-328.
Kirschbaum, C., & Hellhammer, D. H. (1994). Salivary cortisol in psychoneuroendocrine
research: recent developments and applications. Psychoneuroendocrinology, 19,
313-333.
Kirschbaum, C., Pirke, K. M., & Hellhammer, D. H. (1993). The 'Trier Social Stress
Test'—a tool for investigating psychobiological stress responses in a laboratory
setting. Neuropsychobiology, 28, 76-81.
Kudielka, B. M., Schommer, N. C., Hellhammer, D. H., & Kirschbaum, C. (2004). Acute
HPA axis responses, heart rate, and mood changes to psychosocial stress (TSST)
in humans at different times of day. Psychoneuroendocrinology, 29, 983-992.
Kudielka, B.M., Schmidt-Reinwald, A.K., Hellhammer, D.H., & Kirschbaum, C. (1999).
Psychological and endocrine responses to psychosocial stress and
dexamethasone/corticotropin-releasing hormone in healthy postmenopausal
women and young controls: the impact of age and a two-week estradiol treatment.
Neuroendocrinology, 70, 422-430.
Kudielka, B.M., Schmidt-Reinwald, A.K., Hellhammer, D.H., Schürmeyer, T., &
Kirschbaum, C. (2000). Psychosocial stress and HPA functioning: no evidence for
a reduced resilience in healthy elderly men. Stress, 3, 229-240.
Kuhnen, C. M., & Knutson, B. (2005) The neural basis of financial risk taking. Neuron,
47, 763-70.

111

Lejuez, C. W., Read, J. P., Kahler, C. W., Richards, J. B., Ramsey, S. E., Stuart, G. L, et
al. (2002) Evaluation of a behavioral measure of risk taking: The Balloon
Analogue Risk Task (BART). Journal of Experimental Psychology. Applied, 8,
75-84.
Levine, F. M., & De Simone, L. L. (1991) The effects of experimenter gender on pain
report in male and female subjects. Pain, 44, 69-72.
Lighthall, N. R., Gorlick, M. A., Schoeke, A., Frank, M. J., & Mather, M. (in revision).
Stress modulates reinforcement learning in younger and older adults.
Lighthall, N. R., Mather, M., & Gorlick, M. A. (2009). Acute stress increases sex
differences in risk seeking in the Balloon Analogue Risk Task. PLoS ONE, 4,
e6002. doi: 10.1371/journal.pone.0006002
Lighthall, N. R., Sakaki, M., Vasunilashorn, F., Nga, L., Somayajula, S., Chen, E. Y, …
Mather, M. (2011). Gender differences in reward-related decision processing
under stress. Social Cognitive and Affective Neuroscience, doi:
10.1093/scan/nsr026
Lin A, Adolphs R, Rangel A. (2012). Social and monetary reward learning engage
overlapping neural substrates. Social Cognitive and Affective Neuroscience, 7,
274-81.
Lovallo, W. (1975.) The cold pressor test and autonomic function: a review and
integration. Psychophysiology, 12, 268-282.

112

Lupien, S. J., McEwen, B. S., Gunnar, M. R., & Heim, C. (2009). Effects of stress
throughout the lifespan on the brain, behaviour and cognition. Nature Reviews
Neuroscience, 10, 434-445.
Mather, M. & Schoeke, A. (2011). Positive outcomes enhance incidental learning for
both younger and older adults. Frontiers in Neuroscience, 5, 129. doi:
10.3389/fnins.2011.00129
Mather, M., & Carstensen, L. L. (2005). Aging and motivated cognition: The positivity
effect in attention and memory. Trends in Cognitive Sciences, 9, 496-502.
Mather, M., & Lighthall, N. R. (2012). Risk and reward are processed differently in
decisions made under stress. Current Directions in Psychological Science, 21, 36-
41.
Mather, M., Gorlick, M. A., & Lighthall, N. R. (2009). To brake or accelerate when the
light turns yellow? Stress reduces older adults’ risk taking in a driving game.
Psychological Science, 20, 174-176.
Maxim, P. S., & Keane, C. (1992). Gender, age, and the risk of violent death in Canada,
1950-1986. Canadian Review of Sociology and Anthropology-Revue Canadienne
De Sociologie Et D Anthropologie, 29, 329–45.
Mell, T., Wartenburger, I., Marschner, A., Villringer, A., Reischies, F. M., & Heekeren,
H. R. (2009). Altered function of ventral striatum during reward-based decision
making in old age. Frontiers in Human Neuroscience, 3, 34.

113

Miyachi, S., Hikosaka, O., & Lu, X. (2002) Differential activation of monkey striatal
neurons in the early and late stages of procedural learning. Experimental Brain
Research, 146, 122-6.
Morcom, A. M., Bullmore, E. T., Huppert, F. A., Lennox, B., Praseedom, A., Linnington,
H., & Fletcher, P. C. (2010). Memory encoding and dopamine in the aging brain:
a psychopharmacological neuroimaging study. Cerebral Cortex, 20, 743-757.
Naliboff, B.D., Berman, S., Chang, L., Derbyshire, S. W., Suyenobu, B., Vogt, B. A., et
al. (2003) Sex-related differences in IBS patients: Central processing of visceral
stimuli. Gastroenterology, 124, 1738-47.
Nashiro, K., Mather, M., Gorlick, M. A., & Nga, L. (2011). Negative emotional outcomes
impair older adults’ reversal learning. Cognition and Emotion, 25(6), 1014-1028.
Nicolson, N., Storms, C., Ponds, R., & Sulon, J. (1997). Salivary cortisol levels and stress
reactivity in human aging. Journals of Gerontology Series A: Biological Sciences
and Medical Sciences, 52, M68-M75.
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004)
Dissociable roles of ventral and dorsal striatum in instrumental conditioning.
Science, 304, 452-4.
Ossewaarde, L., Qin, S., Van Marle, H. J., van Wingen, G. A., Fernández, G., &
Hermans, E. J. (2011) Stress-induced reduction in reward-related prefrontal cortex
function. Neuroimage, 55, 345-52.

114

Paulus, M. P., Hozack, N., Frank, L., & Brown, G. G. (2002). Error rate and outcome
predictability affect neural activation in prefrontal cortex and anterior cingulate
during decision-making. Neuroimage, 15, 836-846.
Pessoa, L., & Engelmann, J. B. (2010). Embedding reward signals into perception and
cognition. Frontiers in Neuroscience, 4, 17.
Petzold, A., Plessow, F., Goschke, T., & Kirschbaum, C. (2010). Stress reduces use of
negative feedback in a feedback-based learning task. Behavioral Neuroscience,
124, 248-255.
Piazza, P. V., Rougé-Pont, F., Deroche, V., Maccari, S., Simon, H., & Le Moal, M.
(1996). Glucocorticoids have state-dependent stimulant effects on the
mesencephalic dopaminergic transmission. Proceedings of the National Academy
of Sciences of the United States of America, 93, 8716-8720.
Pietschmann, M., Endrass, T., Czerwon, B., & Kathmann, N. (2011). Aging, probabilistic
learning and performance monitoring. Biological Psychology, 86, 74-82.
Porcelli, A. J., & Delgado, M. R. (2009) Acute stress modulates risk taking in financial
decision making. Psychological Science, 20, 278-83.
Preston, S. D., Buchanan, T. W., Stansfield, R. B., & Bechara, A. (2007) Effects of
anticipatory stress on decision making in a gambling task. Behavioral
Neuroscience,121, 257-63.
Preston, S. D., Buchanan, T. W., Stansfield, R. B., & Bechara, A. (2007). Effects of
anticipatory stress on decision making in a gambling task. Behavioral
Neuroscience, 121, 257-263.

115

Pruessner, J. C., Champagne, F., Meaney, M. J., & Dagher, A. (2004). Dopamine release
in response to a psychological stress in humans and its relationship to early life
maternal care: A positron emission tomography study using [11C]raclopride.
Journal of Neuroscience, 24, 2825-2831.
Pruessner, J. C., Dedovic, K., Khalili-Mahani, N., Engert, V., Pruessner, M., Buss, C., …
& Lupien, S. (2008). Deactivation of the limbic system during acute psychosocial
stress: evidence from positron emission tomography and functional magnetic
resonance imaging studies. Biological Psychiatry, 63, 234-240.
Putman, P., Antypa, N., Crysovergi, P., & van der Does, W. A. (2010) Exogenous
cortisol acutely influences motivated decision making in healthy young men.
Psychopharmacology, 208, 257-63.
Qin, S., Hermans, E. J., van Marle, H. J., Luo, J., & Fernández, G. (2009). Acute
psychological stress reduces working memory-related activity in the dorsolateral
prefrontal cortex. Biological Psychiatry, 66, 25-32.
Radloff, L. S. (1977) The CES-D scale: A self-report depression scale for research in the
general population. Applied Psychological Measurement,1, 385-401.
Rao, H., Korczykowski, M., Pluta, J., Hoang, A., & Detre, J. A. (2008) Neural correlates
of voluntary and involuntary risk taking in the human brain: An fMRI Study of
the Balloon Analog Risk Task (BART). NeuroImage, 42, 902-10.
Raz, N. (2004). The aging brain: Structural changes and their implications for cognitive
aging. In RA Dixon, L Backman, & L G Nilsson (Eds.), New frontiers in
cognitive aging (pp. 115-133), New York, Oxford University Press.

116

Resnick, S. M., Pham, D. L., Kraut, M. A., Zonderman, A. B., & Davatzikos, C. (2003).
Longitudinal magnetic resonance imaging studies of older adults: A shrinking
brain. Journal of Neuroscience, 23, 3295-301.
Rohleder, N., Kudielka, B.M., Hellhammer, D.H., Wolf, J.M., & Kirschbaum, C. (2002).
Age and sex steroid-related changes in glucocorticoid sensitivity of pro-
inflammatory cytokine production after psychosocial stress. Journal of
Neuroimmunology, 126, 69-77.
Rollo, C. D. (2009). Dopamine and aging: intersecting facets. Neurochemical Research,
34, 601-29.
Rutledge, R. B., Lazzaro, S. C., Lau, B., Myers, C. E., Gluck, M. A, & Glimcher, P. W.
(2009). Dopaminergic drugs modulate learning rates and perseveration in
Parkinson’s patients in a dynamic foraging task. Journal of Neuroscience, 29(48),
15104-4.
Saal, D., Dong, Y., Bonci, A., & Malenka, R. C. (2003). Drugs of abuse and stress trigger
a common synaptic adaptation in dopamine neurons. Neuron, 37, 577-582.
Samanez-Larkin, G. R., D'Esposito, M. (2008). Group comparisons: imaging the aging
brain. Social Cognitive and Affective Neuroscience, 3, 290-7.
Samanez-Larkin, G. R., Gibbs, S. E., Khanna, K., Nielsen, L., Carstensen, L. L., &
Knutson, B. (2007). Anticipation of monetary gain but not loss in healthy older
adults. Nature Neuroscience, 10(6), 787-791.

117

Samanez-Larkin, G. R., Kuhnen, C. M., Yoo, D. J., & Knutson, B. (2010). Variability in
nucleus accumbens activity mediates age-related suboptimal financial risk taking.
Journal of Neuroscience, 30, 1426-1434.
Schonberg, T, O’Doherty, J. P., Joel, D., Inzelberg, R., Segev, Y., & Daw, N. D. (2010).
Selective impairment of prediction error signaling in human dorsolateral but not
ventral striatum in Parkinson’s disease patients: evidence from a model-based
fMRI study. Neuroimage, 49(1), 772-81.
Schönberg, T., Daw, N.D ., Joel, D., & O’Doherty, J. P. (2007). Reinforcement learning
signals in the human striatum distinguish learners from nonlearners during
reward-based decision making. Journal of Neuroscience, 27(47), 12860-7.
Schoofs, D., Preuss, D., & Wolf, O. T. (2008). Psychosocial stress induces working
memory impairments in an n-back paradigm. Psychoneuroendocrinology, 33(5),
643-653.
Schoofs, D., Wolf, O. T., & Smeets, T. (2009). Cold pressor stress impairs performance
on working memory tasks requiring executive functions in healthy young men.
Behavioral Neuroscience, 123(5), 1066-1075.
Schott, B. H., Niehaus, L., Wittmann, B. C., Schütze, H., Seidenbecher, C. I., Heinze, H.
J., Düzel. E. (2007). Ageing and early-stage Parkinson's disease affect separable
neural mechanisms of mesolimbic reward processing. Brain, 130, 2412-2424.

118

Schwabe, L., Bohringer, A., Chatterjee, M., & Schachinger, H. (2008) Effects of pre-
learning stress on memory for neutral, positive and negative words: Different
roles of cortisol and autonomic arousal. Neurobiology of Learning and Memory,
90, 44-53.
Scott, D. J., Heitzeg, M. M., Koeppe, R. A., Stohler, C. S., & Zubieta, J. K. (2006).
Variations in the human pain stress experience mediated by ventral and dorsal
basal ganglia dopamine activity. Journal of Neuroscience, 26, 10789-10795.
Seamans, J. K., & Yang, C. R. (2004). The principal features and mechanisms of
dopamine modulation in the prefrontal cortex. Progress in Neurobiology, 74, 1-
58.
Serences, J. T. (2008). Value-based modulations in human visual cortex. Neuron, 60,
1169-81.
Sinha, R. (2008). Chronic stress, drug use and vulnerability to addiction. Annals of the
New York Academy of Sciences: Addiction Reviews, 1141, 105-130.
Sinha, R. (2009). Stress and addiction: A dynamic interplay of genes, environment, and
drug intake. Biological Psychiatry, 66, 100-1.
Smith, S. M. (2002). Fast robust automated brain extraction. Human Brain Mapping, 17,
143-55.
Soliman, A., O’Driscoll, G. A., Pruessner, J., Holahan, A. L., Boileau, I., Gagnon, D., &
Dagher, A. (2008). Stress-induced dopamine release in humans at risk of
psychosis: A [11C]raclopride PET study. Neuropsychopharmacology, 33, 2033-
2041.

119

Spreckelmeyer, K. N., Krach, S., Kohls, G., Rademacher, L., Irmak, A., Konrad, K., et al.
(2009). Anticipation of monetary and social reward differently activates
mesolimbic brain structures in men and women. Social Cognitive and Affective
Neuroscience, 4, 158-65.
Starcke, K., & Brand, M. (2012). Decision making under stress: a selective review.
Neuroscience and Biobehavioral Reviews, 36, 1228-48.
Starcke, K., Wolf, O. T., Markowitsch, H. J., & Brand, M. (2008). Anticipatory stress
influences decision making under explicit risk conditions. Behavioral
Neuroscience, 122, 1352-1360.
Sunden, A. E., & Surette, B. J. (1998). Gender differences in the allocation of assets in
retirement savings plans. American Economic Review, 88, 207–11.
Sutton, R. S., & Barto, A. G. (1988). Reinforcement Learning: An Introduction. MIT
Press.
Talairach, J., & Tournoux, P. (1988) Co-planar stereotaxic atlas of the human brain: 3-
dimensional proportional system: An approach to cerebral imaging. New York:
Thieme.
Taylor, S. F., Stern, E. R., & Gehring, W. J. (2007) Neural systems for error monitoring:
recent findings and theoretical perspectives. The Neuroscientist, 13, 160-72.
van den Bos, R., Harteveld, M., & Stoop, H. (2009). Stress and decision-making in
humans: performance is related to cortisol reactivity, albeit differently in men and
women. Psychoneuroendocrinology, 34, 1449-1458.

120

van Honk, J., Schutter, D. J., Hermans, E. J., & Putman, P. (2003). Low cortisol levels
and the balance between punishment sensitivity and reward dependency.
NeuroReport, 14, 1993-1996.
van Marle, H. J., Hermans, E. J., Qin, S., & Fernández, G. (2009). From specificity to
sensitivity: how acute stress affects amygdala processing of biologically salient
stimuli. Biological Psychiatry, 66, 649-55.
VanCauter, E., Leproult, R., & Kupfer, D.J. (1996). Effects of gender and age on the
levels and circadian rhythmicity of plasma cortisol. The Journal of Clinical
Endocrinology & Metabolism, 81, 2468-2473.
Waltz, J. A., Frank, M. J., Robinson, B. M., & Gold, J. M. (2007). Selective
reinforcement learning deficits in schizophrenia support predictions from
computational models of striatal-cortical dysfunction. Biological Psychiatry, 62,
756-764.
Wang, J., Korczykowski, M., Rao, H., Fan, Y., Pluta, J., Gur, R. C. et al. (2007) Gender
difference in neural response to psychological stress. Social Cognitive and
Affective Neuroscience, 2, 227-39.
Wang, J., Rao, H., Wetmore, G. S., Furlan, P. M., Korczykowski, M., Dinges, D. F., &
Detre, J. A. (2005). Perfusion functional MRI reveals cerebral blood flow pattern
under psychological stress. Proceedings of the National Academy of Sciences,
102, 17804-17809.

121

Watson, D., Clark, L. A., & Tellegen, A. (1988) Development and validation of brief
measures of positive and negative affect: the PANAS scales. Journal of
Personality and Social Psychology, 54, 1063-70.
Wechsler, D. (2001). Wechsler Test of Adult Reading. San Antonio, TX: Psychological
Corporation.
Welsh, K. A., Breitner, J. C., & Magruder-Habib, K. (1993). Detection of dementia in the
elderly using telephone screening of cognitive status. Neuropsychiatry,
Neuropsychology, & Behavioral Neurology, 6, 103-10.
Welte J, Barnes G, Wieczorek W, Tidwell MC, Parker J. 2001. Alcohol and gambling
pathology among U.S. adults: prevalence, demographic patterns and comorbidity.
J Stud Alcohol. 62(5):706-12.
Wise, R. A., & Rompre, P. P. (1989). Brain dopamine and reward. Annual Review of
Psychology, 40, 191-225.
Wolf, O. T., Convit, A., McHugh, P. F., Kandil, E., Thorn, E. L., De Santi, S., … de
Leon, M. J. (2001). Cortisol differentially affects memory in young and elderly
men. Behavioral Neuroscience, 115(5), 1002-1011.
Wood, S., Busemeyer, J., Koling, A., Cox, C. R., & Davis, H. (2005). Older adults as
adaptive decision makers: evidence from the Iowa Gambling Task. Psychology
and Aging, 20, 220-225.
Woolrich, M. W., Ripley, B. D., Brady, M. & Smith, S. M. (2001) Temporal
autocorrelation in univariate linear modeling of FMRI data. NeuroImage, 14,
1370-86.

122

Worsley, K. J. (2001) Statistical analysis of activation images. In: P. Jezzard, P. M.
Matthews, S. M. Smith (eds) Functional MRI: an introduction to methods, 1st
edition, Oxford: Oxford University Press. 251-70.
Xue G, Lu Z, Levin IP, Bechara A. (2010) The impact of prior risk experiences on
subsequent risky decision-making: the role of the insula. Neuroimage, 50, 709-16.
Yin, H. H., & Knowlton, B. J. (2006). The role of the basal ganglia in habit formation.
Nature Reviews Neuroscience, 7, 464-476.
Yoon, C., May, C. P., & Hasher, L. (2000). Aging, circadian arousal patterns, and
cognition. In D. C. Park & D. Schwarz (Eds.), Cognitive aging: A primer (pp.
151-171). Philidelphia, PA: Psychology Press.

123

APPENDIX A: CHAPTER 1. SUPPLEMENTARY TABLES

124

125

APPENDIX B: CHAPTER 1. SUPPLEMENTARY INDEPENDENT COMPONENT
ANALYSIS METHODS AND RESULTS
Methods
Independent Component Analysis
Independent component analyses (ICA; Calhoun et al., 2001) were conducted
using Brain Voyager QX (Brain Innovation BV) to identify task-related functional
networks in the brain and determine which brain regions are differentially involved in the
“task-related network” depending on one’s gender and stress status. Each participant’s
data was individually preprocessed in Brain Voyager (including slice-timing correction,
motion correction, realignment with T1-volumes, and transformation into the Talairach
template (Talairach and Tournoux, 1988).
Individual and self-organizing group-level ICA were applied to the functional
time-series. The single-subject ICA was conducted with a C++ implementation of the
fastICA algorithm (Hyvarinen, 1999). Using principal component analysis, first, the
initial dimensions of the functional dataset were reduced from number of time-points to
30. Then, 30 spatially independent components were estimated for each individual. The
resulting ICA decompositions from each subject were submitted to the self-organizing
group ICA (sogICA) for each of the four groups (i.e., by gender/stress group
individually). SogICA was implemented according to the procedure and the clustering
algorithm (Esposito et al., 2005). This permits clustering of components from each
individual based on the components’ mutual similarity measures. The similarity measures
were based on linear spatial correlations in a common whole-brain mask. Group-level

126

component maps were generated as random effects maps. The random effects statistic for
each voxel was calculated as the mean ICA Z-value of that voxel across the individual
maps divided by its standard error, resulting in a T-statistic. The resulting maps of T-
values were visualized using a threshold of P < .005 (uncorrected; T = 3.58 for stress
female and T = 3.50 for other three groups). Clusters that involved less than 20
contiguous voxels were discarded.
To define the task-related network in each group, temporal correlations between
each component activity and the active BART were obtained for each individual. Then,
for each group-level component obtained from the sogICA clustering, a group average
correlation between component activity and BART was calculated. The group-level
component that had the highest correlation with BART was defined as the “task-related
network” for each group. To determine whether the “task-related networks” showed
similar group differences as those observed in the whole brain GLM analysis, functional
masks were created from regions where stress effects were gender-specific in the whole
brain analysis (i.e., where gender by stress effects were present). Mean component
activity signals for these regions were extracted from each participant’s task-related
network and tested for group differences.

127

Results

128

129

APPENDIX C: CHAPTER 3. FEEDBACK WORD LIST FOR AUDIO CLIPS
Positive
1. Bravo
2. Brilliant
3. Excellent
4. Fantastic
5. Great
6. Yes
7. Outstanding
8. Superb
9. Terrific
10. Marvelous
11. Wonderful
12. Amazing

Negative
1. Idiot
2. Moron
3. No
4. Stupid
5. Wrong
6. Fool
7. Dim wit
8. Dummy
9. Loser
10. Failure
11. Imbecile
12. Clueless

Neutral
1. Chair
2. Desk
3. Paper
4. Stapler
5. Table
6. Telephone
7. Cup
8. Sink
9. Refrigerator
10. Printer
11. Notebook
12. Envelope

130

APPENDIX D: CHAPTER 3. COMPUTATIONAL MODEL FITTING
Psuedo-r
2
for each group:
Pseudo-r
2
Younger Control 0.36
Older Control 0.33
Younger Stress 0.43
Older Stress 0.29
Notes. The average of pseudo-r
2
as a measure of better fitting than purely random choices
(Daw, O’Doherty, Dayan, Seymour, & Dolan, 2006) in each group. Higher values are
associated with better fits, where zero is associated with a model with purely random
choices.

Parameters used for generating regressors in each group

Notes. The meta-parameters of , , and were hand tuned from individual
participant data, while a constant was used for .

! ! !
d

Younger Control 0.09 62.8 -5.26 0.50
Older Control 0.26 35.5 -4.37 0.50
Younger Stress 0.18 32.9 -4.86 0.50
Older Stress 0.22 29.7 -0.56 0.50
! ! ! d

131

APPENDIX E: CHAPTER 3. CRITERIA FOR REMOVAL OF
NOISE COMPONENTS WITH MELODIC ICA
Components were removed from the data if: 1) the peak spike in the timecourse was
≥ six units (on the y axis) or ≥ three units more extreme than the third largest spike in the
timecourse, if ≥ one slice had a full ring of activation surrounding the brain, 2) one slice
had ≥ 90% or activation but an adjacent slice had < 30% activation, 3) there was
activation coverage (or nearly full coverage) in ventricles, and 4) there was a visual
“swirl” or “mosaic” pattern of activation covering ≥ 70% with nearly no activation in
adjacent slices.

132

APPENDIX F: CHAPTER 3. RESPONSE TO LEARNING PARAMETERS
ACROSS GROUPS
Details for the observed neural response to EV, PE, and feedback by valence type
are displayed in Table 3.2. Results in this table represent signals observed across stress
and age groups.
Functional Activation Associated with Expected Value (EV)
Parametric correlations with positive EVs for choice trials were observed in the
medial frontal gyrus and midcingulate, superior temporal gyrus and supplementary motor
area, sensory cortex, and extrastriate visual cortical areas. Positive EVs were inversely
correlated with activation clusters in the superior frontal gyrus, DLPFC, angular gyrus,
and lateral occipital cortex. Correlations with negative EVs were observed in clusters
including the superior frontal gyrus and anterior cingulate, middle frontal gyrus and
dorsolateral PFC, and the inferior frontal gyrus and anterior insula.
Functional Activation Associated with Prediction Error (PE)
Correlations between positive PEs for choice trials and brain activation were
found in the superior parietal lobule and middle frontal gyrus. We did not find any
significant positive correlations for negative PEs across groups but we did observe a
cluster of negatively correlated activation in the lateral occipital cortex. Weak PE signals
across groups may have been due to variability in the location or nature of these signals
by group combined with small cell sizes (e.g., n = 12 in each older group); however,
similar to the study by Lin et al. (2011), also failed to find PE signals for social feedback
(N = 24) and presented uncorrected striatal response to PEs across valence types.

133

Functional Activation Associated with Positive and Negative Social Feedback
Across groups, receipt of positive feedback (positive > neutral) was associated
with enhanced activation of the superior frontal gyrus, medial frontal gyrus and anterior
cingulate, as well as activation in temporal, sensory and motor cortices. Response to
negative feedback (negative > neutral) activated similar prefrontal regions, and also
regions of the parietal cortex and the inferior frontal gyrus.

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

How emotional arousal influences memory and learning in younger and older adults

PDF

The impact of emotion, valence, and arousal on differential memory processes in younger and older people

PDF

Structure and function of the locus coeruleus across the lifespan

PDF

Is stress exposure enough? Race/ethnic differences in the exposure and appraisal of chronic stressors among older adults

PDF

Internet communication use, psychological functioning and social connectedness at older ages

PDF

Self-perceptions of Aging in the Context of Neighborhood and Their Interplay in Late-life Cognitive Health

PDF

Age integration in late life: sociodemographic & psychosocial correlates of intergenerational-only, peer-only, and age-integrated social networks

PDF

Reacting to the negative consequences of your decisions: an ATSS comparison of cognitive and affective reactions in older versus younger adults

PDF

Emotion, attention and cognitive aging: the effects of emotional arousal on subsequent visual processing

PDF

Value-based decision-making in complex choice: brain regions involved and implications of age

PDF

Three essays on modifiable determinants of shingles: risk factors for shingles incidence and factors affecting timing of vaccine uptake

PDF

Examination of the long-term psychosocial and functioning effects of a healthy living, life management behavior intervention for older adults

PDF

Dedifferentiation of emotion regulation strategies in the aging brain: an MVPA investigation

PDF

Investigating the effectiveness of a social work intervention on reducing hospital readmissions among older adults

PDF

Emotional reactivity and memory biases in older and younger adults

PDF

Estimating survival in the face of pain: evidence from the health and retirement study

PDF

Lifting up ourselves by lifting up others: examining cognitive-affective states linking generativity to well-being

PDF

The role of age, mood, and time- perspective on the interpretation of emotionally ambiguous information

PDF

International sex and age differences in physical function and disability

PDF

Age differences in diffusivity in the locus coeruleus and its ascending noradrenergic tract

Asset Metadata

Creator Lighthall, Nichole Renee (author)

Core Title Mechanisms of stress effects on learning and decision making in younger and older adults

School Leonard Davis School of Gerontology

Degree Doctor of Philosophy

Degree Program Gerontology

Publication Date 08/03/2012

Defense Date 06/19/2012

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag aging,Decision making,Learning,OAI-PMH Harvest,reward,Stress

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Mather, Mara (committee chair), Bechara, Antoine (committee member), Tjan, Bosco S. (committee member), Zelinski, Elizabeth M. (committee member)

Creator Email nichole.lighthall@gmail.com,nichole.lighthall@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-86554

Unique identifier UC11290605

Identifier usctheses-c3-86554 (legacy record id)

Legacy Identifier etd-LighthallN-1132.pdf

Dmrecord 86554

Document Type Dissertation

Rights Lighthall, Nichole Renee

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA