University of Southern California Dissertations and Theses

Decoding the neural basis of valence and arousal across affective states

(USC Thesis Other)

Decoding the neural basis of valence and arousal across affective states

PDF

Download

Open document

Flip pages

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content DECODING THE NEURAL BASIS OF V ALENCE AND AROUSAL ACROSS AFFECTIVE STATES by Roshni Lulla A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree MASTER OF ARTS (PSYCHOLOGY) December 2023 Copyright 2023 Roshni Lulla ii TABLE OF CONTENTS List of Figures.................................................................................................................................iii Abstract ..........................................................................................................................................iv Chapter One: Introduction...............................................................................................................1 Chapter Two: Methods….................................................................................................................6 Participants….......................................................................................................................6 Experimental Procedure.......................................................................................................6 Stimulus Ratings..................................................................................................................7 fMRI Data Acquisition.........................................................................................................8 fMRI PreProcessing & General Linear Model....................................................................8 Representational Similarity Analysis……….......................................................................9 Empathy Correlation Analysis………………...................................................................10 Chapter Three: Results…...............................................................................................................11 Representational Similarity Analysis.................................................................................11 Interpersonal Reactivity Index Measures...........................................................................11 Chapter Four: Discussion…...........................................................................................................14 Representational Similarity of Valence and Arousal.........................................................14 Empathy and Affective Simulation....................................................................................15 Chapter Five: Conclusion…..........................................................................................................18 Bibliography…..............................................................................................................................19 iii LIST OF FIGURES Figure 1: Range of arousal and valence ratings across all four emotions and neutral, for each of the stimuli, including the IAPS image and ANET text caption………………………8 Figure 2: Valence and arousal RSA maps, displaying regions that significantly represent either valence ratings (blue) or arousal ratings (yellow) across the IAPS stimuli presented in the scanner……………………………………………………….11 Figure 3: Arousal RSA map correlated with IRI Perspective-Taking scale……………………...12 Figure 4: Scatter plot of arousal similarity values averaged across the vmPFC and individual IRI Perspective-Taking scores, displayed with the linear correlation between the two………………………………………………………………………………….12 Figure 5: Arousal RSA map correlated with IRI Fantasizing scale……………………………...13 Figure 6: Scatter plot of arousal similarity values averaged across the lateral occipital cortex (LOC) and individual IRI Fantasizing scores, displayed with the linear correlation between the two……………………………………………………...13 Figure 7: vmPFC mask in blue from Vaccaro et al. (2022) MVPA results, portraying predictive voxels across emotion categories correlated with individual IRI Perspective-Taking scores. The arousal RSA map was masked with this vmPFC region and correlated with IRI Perspective-Taking scores, displayed here as inverse p-values in red………………………………………………...16 iv ABSTRACT Affective states have been shown to share core properties of valence and arousal, which can be mapped across the brain. Emerging multivariate analysis methods on neuroimaging data have identified areas in which the distinctiveness of neural patterns meaningfully relate to properties of affect. However, the link between neural patterns and complex aspects of cognition such as empathy has yet to be explored. We investigated this relationship using an affective simulation paradigm in which participants immersed themselves into a variety of affective scenarios within a functional magnetic resonance imaging scanner. We used a whole-brain searchlight Representational Similarity Analysis (RSA) to identify regions in which the pattern of neural activity related to a pattern of valence or arousal ratings of the affective stimuli and correlated these regions with individual trait empathy scores. We identified overlapping regions which encoded both valence and arousal, although arousal was more widely represented across the brain. Individual perspective taking abilities correlated with arousal representation in the default mode network including the ventromedial prefrontal cortex, aligning with previous findings from our group. These findings suggest that complex empathic processes such as taking the perspective of others may rely on the multidimensional encoding of affective intensity. 1 CHAPTER ONE: INTRODUCTION The field of social and affective neuroscience has debated about how our brains encode affective states, defined here as including both emotions and feelings. Emotions refer to action patterns, both motoric and physiological, that occur in response to internal or external stimuli that shift our homeostatic state. These emotions may lead to feelings, which are the conscious experiences of emotions, our subjective sense of the homeostatic states of the body (Carvalho & Damasio, 2021; Damasio & Carvalho, 2013). We use the term “affective states” to refer to emotions and feelings collectively. Originating from interoceptive signals, affective states allow our brains to translate homeostatic indicators that typically have a certain level of intensity as well as pleasant or unpleasant qualities. In other words, these states and their underlying interoceptive signals inherently vary in terms of valence (positivity or negativity) and arousal (intensity). Therefore, we can say the representation of valence and arousal in the brain is thus inextricably linked to our systems for regulating homeostasis (Vaccaro et al., 2020). Recent work has proposed a link between the internal representation of affect and our ability to understand and experience the states of others. This places an emphasis on the importance of individual affective processes in effectively utilizing empathy, which is typically conceptualized as a collection of cognitive and affective processes. Neural processes related to affect have been associated with affective empathy, or the ability to share the states of others (Christov-Moore & Iacoboni, 2016; Meltzoff & Decety, 2003). One popular way of measuring empathy is the Interpersonal Reactivity Index (Davis, 1980) which divides the construct into four subscales: our tendency to adopt the point of view of others (Perspective Taking), the tendency to transpose ourselves imaginatively into fictitious characters (Fantasizing), other-oriented feelings of concern for others (Empathic Concern), and self-oriented feelings of unease in interpersonal settings (Personal Distress). The affective aspects of empathy related to simulating the states of others have been associated with our ability to understand and robustly represent our own affective states (Ochsner et al., 2004). This is often referred to as the ‘simulationist’ theory of empathy, which highlights the importance of effective simulation of affective states in empathic function. It has been found to be particularly important for affective empathy, with brain regions involved in action simulation such as the inferior frontal gyrus related to affective but not cognitive aspects of empathy (Oliver et al., 2018). Given that aspects of empathy involve the mapping of others’ affective states onto one’s own, it is reasonable to expect that individual 2 differences in the neural implementation of affective states relates to trait differences in empathy. Indeed, recent work from our group has found that the distinctiveness of affect-related neural patterns in medial prefrontal cortex (mPFC) and orbitofrontal cortex (OFC) are related to individual differences in empathy (Vaccaro et al., 2022). Yet the current neuroimaging literature on valence and arousal has not reached consensus as to the neural processes related to these dimensions. The dimension of valence has been researched extensively in particular, possibly driven by a number of conflicting hypotheses regarding its representation in the brain. A 2015 meta-analysis tested competing theories regarding the neural basis of valence, pulling from a range of neuroimaging studies across a span of 18 years. These theories included the bipolarity hypothesis, bivalent hypothesis, and valence general hypothesis. The bipolarity hypothesis proposes that positive and negative affect lie on a one-dimensional scale, with certain brain regions linearly encoding valence. For example, regions in which activation increases for positive stimuli and decreases for negative stimuli. The bivalent hypothesis argues that positive and negative affect lie on separate scales, with independent brain systems responsible for processing positive and negative states. The valence general hypothesis proposes that valence is encoded more broadly throughout the brain and may not be strictly defined by positivity or negativity. This meta-analysis found limited evidence for the bipolar or bivalent hypotheses and concluded that valence may be more flexibly implemented throughout the brain. Evidence supporting the valence-general hypothesis identified regions of the anterior insula, OFC, amygdala, anterior cingulate cortex, and various areas of the prefrontal cortex. Collectively, these regions have been referred to as the ‘salience network’ and are expected to be involved in responding to affective stimuli (Lindquist et al., 2016). Other studies have shown how the method of modeling valence can significantly affect neural findings. Mattek et al. (2020) identified conflicting regions of interest when modeling valence and arousal as two separate dimensions and using a bipolar definition of valence, versus modeling valence as bivalent and deriving arousal from the dimensions of positivity and negativity (Mattek et al., 2020). Regardless of the model used to define these dimensions, there is a clear need to investigate more complex representations of affective states. Previous studies have generally used univariate approaches aimed at identifying brain regions that are engaged as participants view stimuli varying on the dimensions of valence and arousal. Wilson-Mendenhall et al. (2013) tested valence and arousal across discrete emotion 3 categories (fear, sadness, and happiness) using an emotion induction technique where participants were immersed into a range of auditory scenarios designed to elicit the given emotion. This study found evidence of valence and arousal mapping onto distinct brain regions both within and across emotion categories, indicating the prevalence of some core properties across affective states. Valence related to activity in the medial OFC, where activation was driven by increasing positive affect. This region has been implicated in encoding value during decision-making, which may relate to the encoding of increasing positive affect. Given the implications of the OFC in encoding value during decision-making, we can see the encoding of positive affect as a way of placing higher value on pleasant rather than unpleasant stimuli. Arousal related to activation in clusters of the left amygdala and correlated with increasing subjective arousal ratings (Wilson-Mendenhall et al., 2013). Others have investigated these dimensions by looking not only at valence and arousal as separate constructs, but also considering the interaction of these dimensions with one another. By splitting stimuli into four categories of high arousal positive images, low arousal positive images, high arousal negative images, and low arousal negative images, Nielen et al. (2009) were able to look at differences across categories as well as responses compared to neutral images. Emotional images across all four categories elicited stronger activation in the bilateral amygdala when compared to neutral images, likely because of the overall salience of these images. In terms of the interaction of the dimensions, arousal modulated responses to negative and positive stimuli differently. Highly arousing negative stimuli seemed to be processed in the anterior insula, while highly arousing positive stimuli were processed in various regions including the visual cortex, fusiform gyrus, and posterior cingulate cortex (Nielen et al., 2009). These studies focused on identifying specific regions related to the representation of valence and arousal, rather than relating these dimensions to functional patterns of the brain. Multivariate analyses provide an advantage by allowing us to investigate a more distributed encoding of these attributes across the brain. Univariate approaches to neuroimaging have inherent limitations. Emerging methods such as multivoxel pattern analysis (MVPA) allow us to uncover deeper insights by taking into account information from multiple voxels simultaneously. This proves useful for conditions which have a distributed multidimensional coding, in which voxels within a certain region may carry non-identical information about the condition. Univariate methods fail to detect these multidimensional effects, which may account for the lack of consensus in current literature 4 regarding the neural basis of affective states. MVPA has the ability to infer the dimensionality of a given neural code by calculating voxel-level variability, providing an avenue to understand the multidimensional encoding of affective states (Davis et al., 2014). These methods can also be performed in a searchlight fashion, providing a spatial map of information content across the entire brain rather than within specific regions of interest (Kriegeskorte et al., 2006). Recent literature has used MVPA to decode the multidimensional nature of affective states. Early MVPA studies focused on how various modalities may impact the experience of affective states, testing stimuli ranging from facial expressions to voice recordings, as well as both explicitly or implicitly induced states (explicit being where participants were told to think about the emotional consequences to an affective cue). Researchers were able to use multivariate analyses to successfully classify distinct emotion categories in the superior temporal gyrus and Heschl’s gyrus for vocally expressed emotions (Ethofer et al., 2009), and in areas of the superior temporal sulcus for dynamic facial expressions (Said et al., 2010). MVPA has been leveraged in a variety of cases to decode affective states, investigating both emotion categories as well as properties of valence and arousal, with the conclusion that further research on a more multidimensional encoding is necessary (Kragel & LaBar, 2016). Mirroring the design of Nielen et al.’s univariate analysis, a 2012 study tested stimuli looking at the interaction of valence and arousal dimensions, creating four categories of images (high arousal positive, low arousal positive, high arousal negative, and low arousal negative). Although the sample size was low, researchers were able to successfully predict high or low valence and arousal, as well as the four stimuli categories across various regions in the brain using MVPA both within and across participants. They identified informative voxels spanning the superior temporal gyrus and sulcus, mPFC, insula, anterior cingulate, and regions of the occipital cortex (Baucom et al., 2012). A more recent study explored a dynamic experience of affect using naturalistic stimuli, using MVPA to predict valence using either a bipolar or unipolar model of valence. The bipolar valence model was successful in predicting valence across video segments, with informative voxels across the superior temporal sulcus, medial prefrontal cortex, and middle frontal gyrus (Kim et al., 2020). While these studies aim to predict certain properties of stimuli, Representational Similarity Analysis (RSA) is an alternative form of MVPA focused on patterns of similarity across attributes of stimuli in order to quantify the correspondence between neural activity patterns and psychological states (Kriegeskorte et al., 2008). 5 RSA allows us to identify regions of the brain where the pattern of neural activity matches the pattern of ratings across properties such as valence and arousal. This would allow us to identify areas that may carry information about these dimensions across affective states, giving insight into not only which areas may be encoding this information, but whether these areas relate to individual differences in empathy. Research has proven RSA useful in decoding multidimensional information related to valence during simulated affective states using the International Affective Picture System (IAPS) (Lang et al., 1997), identifying regions of the mPFC, OFC, and insula in encoding valence. Chavez et al. investigated the representation of sociality and valence across IAPS images within four ROIs selected with a large-scale meta- analysis (Chavez & Heatherton, 2015). Chikazoe et al. investigated valence more broadly using a gustatory task paired with an IAPS task, specifically focusing on the OFC (Chikazoe et al., 2014). Our study implemented a similar affect simulation paradigm using IAPS images, however rather than selecting specific ROIs within the brain, we implemented a searchlight procedure to look across the whole brain. Each IAPS image was presented in the fMRI scanner accompanied with a descriptive caption, across four affective states: fear, disgust, happy, sad, and neutral. These images were independently rated on levels of valence and arousal in order to understand the interaction between the neural representation of these ratings and trait-level empathy measures collected from the scanned participants. Valence was modeled using the bipolarity hypothesis, with stimuli rated as either positive or negative. We conducted two whole-brain searchlight RSA analyses, one investigating the representation of valence and one for arousal. This allowed us to identify regions in which the pattern of neural activity strongly correlated with the pattern of valence or arousal stimulus ratings, for which we hypothesized be in areas related to affective cognition including the mPFC, OFC, insula, and cingulate cortex. Our secondary analysis incorporated self-reported trait empathy measures, in which we regressed empathy subscale scores against the RSA maps to identify regions that predicted individual differences in empathic ability. In line with the simulationist theory of empathy, we anticipated that individual differences in empathic ability would relate to the similarity amongst neural activation patterns, although the representation of valence and arousal may differentially relate to empathy. We predicted that areas of the prefrontal cortex, specifically the mPFC, could be related to perspective taking abilities given the implications of this region in affective simulation. 6 CHAPTER TWO: METHODS Participants 110 participants (56% female, mean age 26.5, range 20 – 31) were recruited from Los Angeles, California through recruitment flyers from the University of Southern California. All participants were healthy, right-handed native English speakers with normal or corrected-to- normal vision. Participants were screened to ensure they did not have any history of or current psychiatric conditions. All participants provided written informed consent prior to the experiment. All study protocols were approved in accordance with the Institutional Review Board approval guidelines provided by the University of Southern California. Experimental Procedure Participants first completed tasks in a magnetic resonance imaging scanner, followed by behavioral measures outside of the scanner. During the scan, participants were shown images presented as a photo centered on the screen along with descriptive text underneath. The study stimuli consisted of 45 images, gathered from a subset of images in the International Affective Pictures Set (IAPS) (Lang et al., 1997). Images were counterbalanced and spanned four affective categories: happy, sad, disgust and fear, as well as a neutral set, with 9 images in each set. The descriptive text underneath was chosen across corresponding affective categories, primarily selected from the Affective Norms for English Text (ANET) (Lang et al., 1997). Text was presented in the second person, presenting a situation that the subject may find themselves in. For example, ‘You race across the finish line; you’ve won first place’ (happy) or ‘You just found out your grandfather passed away’ (sad). Images and text were matched so that the text described a situation associated with the image. Participants were told to view the image, read the text, and imagine as if the situation was happening to them. For images without corresponding ANET text or text without corresponding IAPS images, study coordinators found text or images from the web to fit the given stimuli. Stimuli were presented using an event related design through PsychToolBox in MATLAB. The 45 stimuli (images paired with text) were randomized and counterbalanced across three functional runs, with 15 images presented in each run. Each stimulus was presented for 12 seconds with a 12 second fixation cross between each presentation as a rest period, resulting in 6-minute functional runs. Participants were instructed to lay still while observing the 7 stimuli, attempting to simulate the situation presented as strongly as possible. Prior to task presentation, participants were given the following instructions: “When the experiment begins, you will see a series of pictures and words. Each picture will appear on the screen in front of you with a short sentence below it. Please read the statement, look at the picture, and try to feel the emotions and feelings described by the scenario as strongly as possible. The picture will stay on the screen for several seconds. When the picture disappears, do your best to put it out of your mind. After a short period, the next picture will appear.” On a separate day after completion of the scanning protocol, participants returned to complete behavioral tasks including the Interpersonal Reactivity Index (IRI) (Davis, 1980). This self-report measure is designed to evaluate individual differences in empathy across four seven- item subscales. These subscales are: Empathic Concern (assessing ‘other-oriented’ feelings of sympathy and concern for unfortunate others), Fantasy (tapping respondents’ tendencies to transpose themselves imaginatively into the feelings and actions of fictitious characters in books, movies, and plays), Personal Distress (measuring ‘self-oriented’ feelings of personal anxiety and unease in tense interpersonal settings), and Perspective Taking (assessing the tendency to spontaneously adopt the psychological point of view of others). Empathy subscales were evaluated separately for each participant, resulting in four scores per subject. Stimuli Ratings Stimuli were rated for valence and arousal using an independent set of participants via Qualtrics. Images were independently rated due to the pairing of IAPS images with ANET text, since these specific pairings did not have standardized ratings from previous literature. 51 participants viewed each of the IAPS images as well as the paired ANET text and were told to rank both how positive or negative the image made them feel (valence) as well as how intense the feeling was (arousal). Valence and arousal ratings were collected on one-dimensional scales, so that images could be rated as either positive or negative, and either high intensity or low intensity. Ratings across for all images across emotions are shown below (Figure 1). 8 Figure 1: Range of arousal and valence ratings across all four emotions and neutral, for each of the stimuli, including the IAPS image and ANET text caption. fMRI Data Acquisition All neuroimaging data was collected at the Dornsife Neuroimaging Institute at the University of Southern California. Data was collected on a 3-Tesla Siemens Prisma System scanner using a 32-channel head coil. T1-weighted anatomical images were acquired across the whole brain with a 3D magnetization-prepared rapid acquisition gradient-echo (MPRAGE) sequence (voxel resolution = 1mm, TE = 2.26ms, TR = 2.30ms, flip angle = 9°, field of view = 87.5 x 87.5mm, matrix size = 224 x 224). A weighted T2 volume was acquired for review by an independent neuroradiologist in compliance with the scanning center’s policy, as well as guidelines set forth by USC’s Institutional Review Board. T2 images were not analyzed in this study. Functional images were acquired with a T2-weighted gradient-echo sequence (repetition time [TR] / echo time [TE] = 10,000/88 ms, 3.5-mm slices, flip angle 120°). fMRI Preprocessing and GLM Standard data preprocessing and general linear model analysis were done using fMRIPrep, a Neuroimaging PreProcessing Tool (NiPreps) application (Esteban et al., 2019). fMRIPrep was used to implement basic processing steps on BIDS-curated data, including coregistration, normalization, unwarping, noise component extraction, segmentation, and skull 9 stripping. Functional data were registered to each participant’s own anatomical image and the anatomical data were registered to the standard MNI Brain (Montreal Neurological Institute) using FreeSurfer’s boundary-based registration. The data were modeled with a regressor for each trial, or image viewed in the scanner, the temporal derivative all task regressors, and six motion parameters to account for residual motion effects. This was used to create voxel-wise z-statistic maps across every trial (image), resulting in 45 z-statistic maps per participant (3 runs x 5 images per run) displaying the voxels that responded significantly to that given image. These z-statistic maps were rearranged per participant so that each trial was in the same order from 1 through 45 (i.e. z-statistic 5 for Participant 1 represented the same trial as z-statistic 5 for Participant 2). The resulting maps were merged across time into a 4-dimensional file per subject, concatenating the 45 trials in the same order across participants. These 4-dimensional files were the input for the following representational similarity analysis. Representational Similarity Analysis Whole-brain searchlight representational similarity analysis (RSA) was completed using PyMVPA. Data were subjected to RSA to test the similarity between the behavioral ratings of stimuli across valence and arousal with their corresponding patterns of brain activity. We performed a whole-brain searchlight procedure, allowing us to identify regions of the brain that encode multidimensional information about valence or arousal. For each participant, the searchlight analysis was performed on their 4-dimensional z-statistic file, creating a spherical region with a radius of 3 voxels centered at every voxel of the brain. Neural activity patterns from each z-statistic map were extracted for each of these searchlight regions for each of the 45 trials, or images. The local voxel values were vectorized to create a neural representational dissimilarity matrix (RDM), calculating the Euclidean distance between vectorized voxel values for each of the 45 regions. This RDM contained measures of the difference between brain activity patterns across trials, or images, for the given searchlight region. The neural RDM was compared to two behavioral RDMs, one for valence and one for arousal. Using ratings from our independent set of participants, we created two behavioral RDMs, one displaying the Euclidean distance of rating values for valence and one displaying the Euclidean distance of rating values for arousal. The Spearman correlation between the neural RDM and each of the behavioral RDMs was calculated, resulting in two Spearman p values for each searchlight sphere. The p 10 value was mapped back onto the central voxel of the searchlight sphere. This yielded two whole- brain maps per participant, one showing the representational similarity of neural activity with the pattern of valence stimulus ratings and one showing the representational similarity of neural activity with the pattern of arousal ratings. Results were corrected for multiple comparisons, first using maximal statistical permutation testing with threshold free cluster enhancement (TFCE), and then with voxel-based thresholding using the null distribution of the max voxel-wise test statistic. V oxel-based thresholding results are reported below. Empathy Correlation Analysis In order to understand the relationship between the representation of valence or arousal with self-reported empathy, we used Interpersonal Reactivity Index (IRI) scores as regressors. Using FSL’s Randomise tool, individual scores of the four IRI subscales were demeaned and used as regressors across the whole-brain RSA searchlight maps for both valence and arousal. Regressors included Perspective-Taking, Personal Distress, Fantasizing, and Empathic Concern scores. Regressors were contrasted both positively and negatively with each voxel’s similarity value using a parametric permutation testing approach. Using a bootstrapped approach, the similarity values at every voxel were randomly associated with the regressor value (one of the four empathy sub-scales) to compute a test statistic, repeated 10,000 times to create a null distribution. This analysis resulted in significance maps, one of which displayed the 1 minus p- value for the relation between the regressor and similarity value at every voxel. Results were corrected for multiple comparisons using threshold-free cluster enhancement (TFCE). 11 CHAPTER THREE: RESULTS Representational Similarity Analysis Whole-brain searchlight similarity maps highly overlapped for valence and arousal. Multiple regions significantly represented both valence and arousal ratings of affective stimuli, even using the more conservative voxel-based thresholding method. These included multiple regions of the salience network, including much of the prefrontal cortex as well as the frontal pole, precuneus, inferior temporal gyrus, lateral occipital cortex, and precentral gyrus (Figure 2). Arousal representation was more widespread throughout the brain when compared to valence representation. Regions that specifically represented arousal included the majority of the right temporal lobe, bilateral frontal gyri, right occipital pole, as well as the basal ganglia and cerebellum. Valence representation was less widespread, with regions that specifically encoded valence ratings including the frontal pole, parahippocampal gyrus, along with smaller clusters of the cerebellum. Figure 2: Valence and arousal RSA maps, displaying regions that significantly represent either valence ratings (blue) or arousal ratings (yellow) across the IAPS stimuli presented in the scanner. Interpersonal Reactivity Index Measures Valence representation across the brain did not significantly correlate with any of the IRI subscales. Arousal representation positively correlated with Perspective Taking, or the ability to spontaneously adopt the point of view of others, in the prefrontal cortex, including the mPFC and OFC, postcentral gyrus, bilateral fusiform gyrus and lateral occipital cortex, and cerebellum (Figure 3). This correlation was particularly pronounced in the ventromedial prefrontal cortex 12 (vmPFC). Arousal similarity values in the vmPFC correlated with Perspective Taking scores are displayed below (Figure 4). Arousal representation positively correlated with Fantasizing, or the tendency to transpose one’s self imaginatively into the feelings and actions of fictious characters, in clusters of the right lateral occipital cortex and fusiform gyrus (Figure 5). Arousal similarity values in the lateral occipital cortex correlated with Fantasizing scores are displayed below as well (Figure 6). Figure 3: Arousal RSA map correlated with IRI Perspective-Taking scale. Figure 4: Scatter plot of arousal similarity values averaged across the vmPFC and individual IRI Perspective-Taking scores, displayed with the linear correlation between the two. -15 -10 -5 0 5 10 -0.1 -0.05 0 0.05 0.1 0.15 0.2 IRI Perspective-Taking Scores Arousal Similarity Values in vmPFC Arousal Similarity in vmPFC Correlated With Perspective Taking 13 Figure 5: Arousal RSA map correlated with IRI Fantasizing scale. Figure 6: Scatter plot of arousal similarity values averaged across the lateral occipital cortex (LOC) and individual IRI Fantasizing scores, displayed with the linear correlation between the two. -15 -10 -5 0 5 10 15 -0.1 -0.05 0 0.05 0.1 0.15 0.2 IRI Fantasizing Scores Arousal Similarity Values in LOC Arousal Similarity in LOC Correlated With Fantasizing 14 CHAPTER FOUR: DISCUSSION Multivariate methods have been gaining popularity due to their ability to decode multidimensional information, providing insight into complex cognitive processes such as affect. Our study used a whole-brain searchlight similarity analysis to further elucidate the brain regions involved in affective simulation, identifying multiple areas that may carry information about the valence or arousal of these simulated states. We use this analysis to demonstrate how the neural representation of affect may relate to individual trait differences. We found that the neural encoding of arousal significantly relates to individual perspective taking and fantasizing empathic abilities, providing implications for the use of advanced multivariate methods such as RSA, as well as a deeper understanding of how the neural patterns underlying affect may relate to empathy. Representational Similarity of Valence and Arousal We identified multiple regions in which neural patterns were significantly correlated with the pattern of both valence and arousal stimulus ratings. Regions that encoded both valence and arousal indicated involvement of the default mode network (DMN), with clusters in the mPFC, posterior cingulate cortex, and precuneus. Despite the anatomical proximity to sensorimotor cortices, the DMN is thought to be involved in more complex processes such as perception that may be unrelated to the external environment (Smallwood et al., 2021). The DMN has also been associated with social cognition, mind-wandering, and imagination. Using a collection of meta- analyses, Mars et al. displayed the strong overlap between the DMN and social cognition networks implicated in other-related processes and theory of mind (Mars et al., 2012). This suggests DMN involvement during other-related simulation such as the affective simulation task done in our study. Our results display DMN foci related to a robust encoding of both valence and arousal across simulated affective states, proposing a stronger relationship between the DMN and complex social cognition. Higher-order visual areas such as the bilateral lateral occipital cortex (LOC) also significantly represented both valence and arousal. This is supported by previous findings regarding emotion perception and visual cortical structures such as the LOC, related to the motivational saliency of the stimulus (Sabatinelli et al., 2007). This suggests an interaction between the neural encoding pattern of a stimulus during sensory perception and affective simulation happening in medial frontal and cingulate regions. 15 The neural encoding of arousal was distributed more widely compared to valence. Much of this spanned the bilateral fusiform gyri, with larger clusters in the right hemisphere. The difference in valence and arousal similarity maps could be due to a number of reasons. Our study measured valence using a bipolar model, with positivity and negativity measured on a one- dimensional scale. This model may not have been able to capture the true variance in valence across stimuli, considering many images had an ambiguous valence. Therefore, our method of measuring valence would not have been suitable for stimuli that elicited both positive and negative affect at the same time (Vaccaro et al., 2020). Arousal, on the other hand, had a more distinct pattern of variation across stimuli. This may account for the lack of significance for valence pattern similarity. Given the fact that arousal is dependent on stimulus intensity or salience, regions in which the pattern of neural activity related to arousal ratings would be expected to be distributed widely throughout the brain. The extent to which an external stimulus or affective simulation is salient provides us with information relevant to survival and would be present within neural encodings across multiple regions of the brain, incorporating regions responsible for basic sensory perception and recognition as well as complex cognitive processes. Empathy and Affective Simulation We identified several brain regions in which the similarity of neural patterns to arousal ratings was significantly correlated with individual perspective taking traits, particularly areas related to processing affect. Given the multivariate nature of our analyses, this places an emphasis on the importance of a robust encoding of these affective states, particularly on the salience or intensity these states induce. Frontal regions such as the mPFC and OFC had a positive relationship with the ability to adopt others’ viewpoints, consistent with our hypotheses regarding affective simulation. Notably, clusters of the PCC also exhibited a correlation with perspective taking, indicating potential involvement of the DMN. The extent to which neural patterns within these regions meaningfully relate to the intensity of each simulated state may explain why certain people are better at taking the perspective of others. Recent work from our lab identified overlapping regions related to perspective taking using the same affective simulation task. Rather than exploring the similarity between neural patterns and stimulus ratings, Vaccaro et al. (2022) used MVPA to identify regions in which patterns of neural activity were informative in classifying across the four distinct emotion 16 categories of happy, sad, fear, and disgust. Higher levels of individual perspective taking were related to predictive clusters in the mPFC and OFC, regions which significantly overlap with our findings related to arousal similarity and perspective taking. Specifically, regions of the ventromedial prefrontal cortex (vmPFC) overlap across these two analyses (Figure 7), a region that has been associated with incorporating interoceptive signals related to emotions and conscious feelings (Vaccaro et al., 2022). This is often referred to as the ‘somatic marker hypothesis’, which implicates the ventromedial areas of the prefrontal cortex in encoding our body-state structure in response to bioregulatory signals. Patients with vmPFC lesions show deficits in affective function, likely reliant on changes in our brain’s representation of somatic states (Damasio, 1996). This emphasizes the vmPFC’s role in affective simulation and therefore individual differences in the ability to take the perspective of others. Given the overlap of significant regions with the Vaccaro et al. (2022) analysis, we display how the distinctiveness of neural patterns in the vmPFC during our affective simulation task relates to perspective taking. Our results further these findings by exploring the degree to which ratings of arousal are encoded across the brain, suggesting that the pattern of neural activity within the vmPFC is not only informative in classifying across emotion categories, but may be driven by the way this region specifically encodes arousal. In other words, the intensity of a simulated state may account for the variability of neural encodings responsible for distinguishing across affective states. Figure 7: vmPFC mask in blue from Vaccaro et al. (2022) MVPA results, portraying predictive voxels across emotion categories correlated with individual IRI Perspective-Taking scores. The 17 arousal RSA map was masked with this vmPFC region and correlated with IRI Perspective- Taking scores, displayed here as inverse p-values in red. The ability to transpose ourselves imaginatively into fictitious characters related to higher-order visual regions, particularly clusters of the right LOC. This suggests a multidimensional sensory input that may explain individual differences in immersion during plays or films. These narratives rely on successful immersion into fictitious worlds and characters, which may be driven in part by robust patterns of activity in integrative visual areas. Importantly, the LOC is implicated in emotional perception, with activity in this region specifically related to stimulus intensity or arousal (Sabatinelli et al., 2007). The LOC has also been found to be functionally connected with the DMN during more complex object perception, such as bistable perception, indicating these dynamic systems may work together during cognitive processes involving simulation (Karten et al., 2013). Our results suggest that the extent to which neural patterns in the LOC meaningfully relate to intensity of a given stimulus may explain individual levels of successful narrative immersion, imagination, and feelings of compassion towards fictitious characters. 18 CHAPTER FIVE: CONCLUSION In conclusion, these findings contributed to our understanding of the neural encoding of affective states, specifically in relation to the dimensions of valence and arousal. Using Representational Similarity Analysis, we identified regions across the brain in which the pattern of neural activity significantly related to the pattern of stimulus ratings for either valence or arousal. The similarity maps strongly overlapped, suggesting multiple regions of the brain were responsible for encoding both valence and arousal. However, these representations were not identical, with arousal encoded across wider areas of the brain. While this could be attributed to the importance of stimulus saliency, it could also be due to the way we measured valence using a bipolar model. Future studies could extend these findings by measuring valence using a bivalent model, or a more complex method of modeling affect using more than two dimensions. In terms of how individual levels of trait empathy relate to the neural encoding of valence and arousal, we found regions related to arousal correlated with Perspective Taking and Fantasizing scores from the Interpersonal Reactivity Index. Importantly, this displayed a relationship between a participant’s empathic traits and their own neural patterns, emphasizing the value of searchlight RSA in understanding complex processes related to taking the perspective of others. Further analyses could probe complementary self-reported traits such as emotional intelligence or personality, potentially linking the encoding of affect to more nuanced individual differences. These findings reinforce the importance of investigating multidimensional patterns of neural activity, specifically how these patterns can allow us to learn about individual differences in cognition. 19 BIBLIOGRAPHY Baucom, L. B., Wedell, D. H., Wang, J., Blitzer, D. N., & Shinkareva, S. V . (2012). Decoding the neural representation of affective states. Neuroimage, 59(1), 718-727. https://doi.org/10.1016/j.neuroimage.2011.07.037 Carvalho, G. B., & Damasio, A. (2021). Interoception and the origin of feelings: A new synthesis. Bioessays, 43(6), e2000261. https://doi.org/10.1002/bies.202000261 Chavez, R. S., & Heatherton, T. F. (2015). Representational similarity of social and valence information in the medial pFC. J Cogn Neurosci, 27(1), 73-82. https://doi.org/10.1162/jocn_a_00697 Chikazoe, J., Lee, D. H., Kriegeskorte, N., & Anderson, A. K. (2014). Population coding of affect across stimuli, modalities and individuals. Nat Neurosci, 17(8), 1114-1122. https://doi.org/10.1038/nn.3749 Christov-Moore, L., & Iacoboni, M. (2016). Self-other resonance, its control and prosocial inclinations: Brain-behavior relationships. Hum Brain Mapp, 37(4), 1544-1558. https://doi.org/10.1002/hbm.23119 Damasio, A., & Carvalho, G. B. (2013). The nature of feelings: evolutionary and neurobiological origins. Nat Rev Neurosci, 14(2), 143-152. https://doi.org/10.1038/nrn3403 Damasio, A. R. (1996). The somatic marker hypothesis and the possible functions of the prefrontal cortex. Philos Trans R Soc Lond B Biol Sci, 351(1346), 1413-1420. https://doi.org/10.1098/rstb.1996.0125 Davis, M. H. (1980). <Davis1980.pdf>. JSAS Catalog of Selected Documents in Psychology. https://doi.org/https://doi.org/10.1037/t01093-000 Davis, T., LaRocque, K. F., Mumford, J. A., Norman, K. A., Wagner, A. D., & Poldrack, R. A. (2014). What do differences between multi-voxel and univariate analysis mean? How subject-, voxel-, and trial-level variance impact fMRI analysis. Neuroimage, 97, 271-283. https://doi.org/10.1016/j.neuroimage.2014.04.037 Esteban, O., Markiewicz, C. J., Blair, R. W., Moodie, C. A., Isik, A. I., Erramuzpe, A., Kent, J. D., Goncalves, M., DuPre, E., Snyder, M., Oya, H., Ghosh, S. S., Wright, J., Durnez, J., Poldrack, R. A., & Gorgolewski, K. J. (2019). fMRIPrep: a robust preprocessing pipeline for functional MRI. Nat Methods, 16(1), 111-116. https://doi.org/10.1038/s41592-018- 0235-4 Ethofer, T., Van De Ville, D., Scherer, K., & Vuilleumier, P. (2009). Decoding of emotional information in voice-sensitive cortices. Curr Biol, 19(12), 1028-1033. https://doi.org/10.1016/j.cub.2009.04.054 20 Karten, A., Pantazatos, S. P., Khalil, D., Zhang, X., & Hirsch, J. (2013). Dynamic coupling between the lateral occipital-cortex, default-mode, and frontoparietal networks during bistable perception. Brain Connect, 3(3), 286-293. https://doi.org/10.1089/brain.2012.0119 Kim, J., Weber, C. E., Gao, C., Schulteis, S., Wedell, D. H., & Shinkareva, S. V . (2020). A study in affect: Predicting valence from fMRI data. Neuropsychologia, 143, 107473. https://doi.org/10.1016/j.neuropsychologia.2020.107473 Kragel, P. A., & LaBar, K. S. (2016). Decoding the Nature of Emotion in the Brain. Trends Cogn Sci, 20(6), 444-455. https://doi.org/10.1016/j.tics.2016.03.011 Kriegeskorte, N., Goebel, R., & Bandettini, P. (2006). Information-based functional brain mapping. Proc Natl Acad Sci U S A, 103(10), 3863-3868. https://doi.org/10.1073/pnas.0600244103 Kriegeskorte, N., Mur, M., & Bandettini, P. (2008). Representational similarity analysis - connecting the branches of systems neuroscience. Front Syst Neurosci, 2, 4. https://doi.org/10.3389/neuro.06.004.2008 Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (1997). <IAPS.TechManual.1-20.2008 (1).pdf>. Lindquist, K. A., Satpute, A. B., Wager, T. D., Weber, J., & Barrett, L. F. (2016). The Brain Basis of Positive and Negative Affect: Evidence from a Meta-Analysis of the Human Neuroimaging Literature. Cereb Cortex, 26(5), 1910-1922. https://doi.org/10.1093/cercor/bhv001 Mars, R. B., Neubert, F. X., Noonan, M. P., Sallet, J., Toni, I., & Rushworth, M. F. (2012). On the relationship between the "default mode network" and the "social brain". Front Hum Neurosci, 6, 189. https://doi.org/10.3389/fnhum.2012.00189 Mattek, A. M., Burr, D. A., Shin, J., Whicker, C. L., & Kim, M. J. (2020). Identifying the representational structure of affect using fMRI. Affect Sci, 1(1), 42-56. https://doi.org/10.1007/s42761-020-00007-9 Meltzoff, A. N., & Decety, J. (2003). What imitation tells us about social cognition: a rapprochement between developmental psychology and cognitive neuroscience. Philos Trans R Soc Lond B Biol Sci, 358(1431), 491-500. https://doi.org/10.1098/rstb.2002.1261 Nielen, M. M., Heslenfeld, D. J., Heinen, K., Van Strien, J. W., Witter, M. P., Jonker, C., & Veltman, D. J. (2009). Distinct brain systems underlie the processing of valence and arousal of affective pictures. Brain Cogn, 71(3), 387-396. https://doi.org/10.1016/j.bandc.2009.05.007 Ochsner, K. N., Knierim, K., Ludlow, D. H., Hanelin, J., Ramachandran, T., Glover, G., & Mackey, S. C. (2004). Reflecting upon feelings: an fMRI study of neural systems 21 supporting the attribution of emotion to self and other. J Cogn Neurosci, 16(10), 1746- 1772. https://doi.org/10.1162/0898929042947829 Oliver, L. D., Vieira, J. B., Neufeld, R. W. J., Dziobek, I., & Mitchell, D. G. V . (2018). Greater involvement of action simulation mechanisms in emotional vs cognitive empathy. Soc Cogn Affect Neurosci, 13(4), 367-380. https://doi.org/10.1093/scan/nsy013 Sabatinelli, D., Lang, P. J., Keil, A., & Bradley, M. M. (2007). Emotional perception: correlation of functional MRI and event-related potentials. Cereb Cortex, 17(5), 1085-1091. https://doi.org/10.1093/cercor/bhl017 Said, C. P., Moore, C. D., Engell, A. D., Todorov, A., & Haxby, J. V . (2010). Distributed representations of dynamic facial expressions in the superior temporal sulcus. J Vis, 10(5), 11. https://doi.org/10.1167/10.5.11 Smallwood, J., Bernhardt, B. C., Leech, R., Bzdok, D., Jefferies, E., & Margulies, D. S. (2021). The default mode network in cognition: a topographical perspective. Nat Rev Neurosci, 22(8), 503-513. https://doi.org/10.1038/s41583-021-00474-4 Vaccaro, A. G., Heydari, P., Christov-Moore, L., Damasio, A., & Kaplan, J. T. (2022). Perspective-taking is associated with increased discriminability of affective states in the ventromedial prefrontal cortex. Soc Cogn Affect Neurosci, 17(12), 1082-1090. https://doi.org/10.1093/scan/nsac035 Vaccaro, A. G., Kaplan, J. T., & Damasio, A. (2020). Bittersweet: The Neuroscience of Ambivalent Affect. Perspect Psychol Sci, 15(5), 1187-1199. https://doi.org/10.1177/1745691620927708 Wilson-Mendenhall, C. D., Barrett, L. F., & Barsalou, L. W. (2013). Neural evidence that human emotions share core affective properties. Psychol Sci, 24(6), 947-956. https://doi.org/10.1177/0956797612464242

Abstract (if available)

Abstract Affective states have been shown to share core properties of valence and arousal, which can be mapped across the brain. Emerging multivariate analysis methods on neuroimaging data have identified areas in which the distinctiveness of neural patterns meaningfully relate to properties of affect. However, the link between neural patterns and complex aspects of cognition such as empathy has yet to be explored. We investigated this relationship using an affective simulation paradigm in which participants immersed themselves into a variety of affective scenarios within a functional magnetic resonance imaging scanner. We used a whole-brain searchlight Representational Similarity Analysis (RSA) to identify regions in which the pattern of neural activity related to a pattern of valence or arousal ratings of the affective stimuli and correlated these regions with individual trait empathy scores. We identified overlapping regions which encoded both valence and arousal, although arousal was more widely represented across the brain. Individual perspective taking abilities correlated with arousal representation in the default mode network including the ventromedial prefrontal cortex, aligning with previous findings from our group. These findings suggest that complex empathic processes such as taking the perspective of others may rely on the multidimensional encoding of affective intensity.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

The behavioral and neural correlates of music-evoked nostalgia across the lifespan

PDF

Neural processing of temporal distance in film

PDF

Neural and behavioral correlates of fear processing in first-time fathers

PDF

Noradrenergic mechanisms of arousal-enhanced memory selectivity

PDF

The neuroscience of ambivalent and ambiguous feelings

PDF

Neurobiological correlates of fathers’ transition to parenthood

PDF

Spatial and temporal patterns of brain activity associated with emotions in music

PDF

The impact of emotion, valence, and arousal on differential memory processes in younger and older people

PDF

Age-related differences in amygdala subregion functional connectivity with Ventromedial Prefrontal Cortex and Dorsolateral Prefrontal Cortex

PDF

Effects of online choir and mindfulness interventions on auditory perception and well-being in older adults during the COVID-19 pandemic

PDF

Fathers’ social-cognitive network connectivity: associations with prenatal father-infant attachment and postpartum parenting behavior

PDF

Mapping the neural architecture of concepts

PDF

Inhibitory spillover in adolescence: a fMRI study

PDF

The role of music training on behavioral and neurophysiological indices of speech-in-noise perception: a meta-analysis and randomized-control trial

PDF

Fathers' cross-sectional and longitudinal white matter microstructure organization during the transition to parenthood

PDF

Negative valence neurons in the larval zebrafish pallium

PDF

Structure and function of the locus coeruleus across the lifespan

PDF

Behavioral and neural influences of interoception and alexithymia on emotional empathy in autism spectrum disorder

PDF

Neural correlates of learning and reversal of approach versus withdraw responses to in- and out-group individuals

PDF

The neural correlates of creativity and perceptual pleasure: from simple shapes to humor

Asset Metadata

Creator Lulla, Roshni (author)

Core Title Decoding the neural basis of valence and arousal across affective states

School College of Letters, Arts and Sciences

Degree Master of Arts

Degree Program Psychology

Degree Conferral Date 2023-12

Publication Date 09/01/2023

Defense Date 06/08/2023

Publisher Los Angeles, California (original), University of Southern California (original), University of Southern California. Libraries (digital)

Tag affect,arousal,multivariate,neuroimaging,OAI-PMH Harvest,social neuroscience,valence

Format theses (aat)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Kaplan, Jonas (committee chair), Habibi, Assal (committee member), Martins-Klein, Bruna (committee member)

Creator Email lulla@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC113302899

Unique identifier UC113302899

Identifier etd-LullaRoshn-12304.pdf (filename)

Legacy Identifier etd-LullaRoshn-12304

Document Type Thesis

Format theses (aat)

Rights Lulla, Roshni

Internet Media Type application/pdf

Type texts

Source 20230901-usctheses-batch-1089 (batch), University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu