University of Southern California Dissertations and Theses

Behavioral and neural investigations of perceptual affect

(USC Thesis Other)

Behavioral and neural investigations of perceptual affect

PDF

Download

Open document

Flip pages

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content BEHAVIORAL AND NEURAL IN V E S T IG A T IO N S OF PERCEPTUAL AFFECT by Edward Allen Vessel A D issertation Presented to th e FACULTY OF THE GRADUATE SCHO OL U N IV E R S IT Y OF SOUTHERN C A LIFO R N IA In Partial Fulfillm ent o f the R equirem ents fo r th e D eg ree DO CTOR OF PHILOSOPHY (N EU R O SC IE N C E) M ay 2 0 0 4 C o pyrig ht © 2 0 0 4 Edw ard Allen Vessel Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. UMI Number: 3140568 INFORMATION TO USERS The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleed-through, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion. ® UMI UMI Microform 3140568 Copyright 2004 by ProQuest Information and Learning Company. All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. ProQuest Information and Learning Company 300 North Zeeb Road P.O. Box 1346 Ann Arbor, Ml 48106-1346 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. ACKNOW LEDGMENTS ii First and foremost, I would like to thank my advisor, Irving Biederman, for his professional, financial, and personal support. He gave me this wonderful opportunity to work on this incredibly interesting idea, and the confidence to believe that I could do great things with it. He provided extensive guidance during the design, execution, and analysis of the experiments, and very efficiently gave me feedback on drafts of this document. Most important of all, he put up with my many idiosyncrasies and enthusiastically encouraged my successes. Special thanks are also due to Mark Cohen of the UCLA Brain Mapping Center for his extensive guidance and friendship through all the phases of the brain imaging study, including design, running subjects, data analysis, and interpretation. I would also like to thank the members of my dissertation committee. Zhonglin Lu gave me assistance with brain imaging analysis and group statistics in addition to general guidance & support. My discussions with Richard Thompson about learning and habituation proved very helpful, as was his general guidance and support. Bartlett Mel had many helpful thoughts on the modeling of preference. Mitchell Earleywine provided helpful Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. iii comments on the factor analysis of preference, as well as personal support in becoming a better writer and researcher. In addition, I am indebted to a number of other USC faculty have helped me with aspects of this work. Bosco Tjan provided an algorithm for creating the spectrally equivalent images, assisted with the computation of RMS contrast, provided scripts for the LO localizer scan, and gave extensive assistance in brain imaging analysis and Linux system administration. Alan Watts and Larry Swanson provided valuable insights into peptide systems, cortical output, anatomy, and emotion. The simulated annealing algorithm used to design imaging sequences was designed with the assistance of Laurent Itti, and Laura Baker provided assistance with the multiple regression analyses used in both the behavioral and imaging experiments. This work would also have been quite difficult without the extensive input and support from my fellow graduate students, and I would like to thank them. In particular, Michael Mangini engaged me in many helpful discussions on this work, assisted with the multiple regression analysis, and provided some much needed distractions. My officemate Marissa Nederhouser provided assistance with statistical analyses, good music, and interesting conversation over the years. Several undergraduates also assisted in running subjects Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. iv and creating stimuli, including Michelle Greene, Henry Nguyen, Tiger Nguyen, Viet Nguyen and Ali Narayan. This work would also not have been possible without extensive help from a number of researchers around the world, and they deserve my heartfelt thanks. Moshe Bar (currently at Massachusetts General Hospital) provided many helpful discussions on preference, priming, and scene perception, and assisted in my selection of imaging parameters and fMRI sequence design. David Glahn (currently at UTHSCSA) assisted with the imaging study design and analysis, particularly in the use of the AFNI software. At UCLA, graduate student Richard Albistegui-Dubois provided extensive instruction on and assistance with operating the magnet and data analysis, and both Zrinka Bilusic and Anne Firestine provided analysis help and assistance with the magnet. Craig Stark (JHU) provided assistance with the AFNI software and the region of interest (ROI) analysis. Geoff Boynton (UCSD) aided extensively in the design and analysis of the imaging experiment. Russ Poldrack (UCLA) provided assistance with impulse response functions and general analysis. Bob Cox (NIMH) provided help with the AFNI software, especially as it pertains to the general linear model, deconvolutions, and motion correction. Douglas Ward (Medical College of Wisconsin) assisted with the deconvolution of my imaging data using the program 3dDeconvolve. Stephen Smith and the rest of FMRIB group (Oxford University Center for Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Functional MRI of the Brain) were a tremendous help with the analysis of my imaging data using the FSL software. Randy Buckner (W USTL), also provided helpful analysis information. I would also like to thank Jean-Marc Fellous (UCSD Salk Institute) for bringing the opioid gradient to the attention of Dr. Biederman. I would like to my heartfelt gratitude to my many friends who put up with me while working on these experiments, and for keeping me sane. In particular, my housemates Jason, Trystan, Ian, and Adam deserve thanks for allowing me to bring my work home with me, as do Roxanne, Ann, and Walid for interesting discussions and being great friends. Lastly, Shari Cha deserves special thanks for providing a peaceful place to write, helping me relax, keeping me motivated, and cheering my spirits! The Dartmouth Summer Institute deserves acknowledgment for giving me a first chance to get some practical experience with imaging and analysis and for introducing me to a number of people that have been very helpful. While this work was being completed, I was supported by a variety of sources, including an NIH Predoctoral Cognitive and Computational Training Grant to USC (T32 M H20003-05), and grants awarded to Irving Biederman from the Human Frontiers Science Program Organization (RG 0035/2000B), Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. The Army Research Office (MURI ARO DAAG55-98-1-0293), the National Science Foundation (IMSC NSF EEC-9529152), and the James S. McDonnell Foundation (99-53). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. TABLE OF CONTENTS AC KN O W LEDG M ENTS................................................................................................... ii LIST OF TABLES............................................................................................................... x LIST OF FIG U R ES............................................................................................................xi ABBREVIATIO NS........................................................................................................... xiv ABSTRACT................................................................................................................... xvi I. IN TR O D U C TIO N .............................................................................................................1 II. A TH EO R Y OF CO G NITIVE AND PERCEPTUAL A FFEC T........................... 9 Positive Affect in the Ventral Visual Pathway............................................... 9 A Neurochemical Index of Interpretability.......................................12 Behavioral Effects of Opioid Antagonists.......................... 16 Opioid Systems in the Brain.................................................. 17 Novelty, Priming, and Competitive Learning..................................26 Perceptual Affect in a Systems Perspective..................................32 Predictions and Extensions..............................................................................37 III. VISUAL PR EFERENCE...........................................................................................43 Background.......................................................................................................... 43 Complexity vs. Interpretability............................................................ 43 Effects of Previous Exposure.............................................................46 Arousal and Emotional Valence........................................................ 54 A Stimulus Set for Testing Positive Perceptual Affect............................. 57 Calibration Experiment.........................................................................57 Stimuli...........................................................................................58 Subjects.......................................................................................58 Method.........................................................................................59 Results......................................................................................... 61 IV. R E PETITIO N .............................................................................................................. 65 Experiment 1: Effects of Repetition, Between-Subjects Design 66 Methods....................................................................................................66 Subjects....................................................................................... 66 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. viii Stimuli...........................................................................................67 Procedure....................................................................................67 Results...................................................................................................... 68 Conclusions from Experiment 1........................................................ 73 Experiment 2: Selection of Image Sets for FM R I......................................75 Methods....................................................................................................76 Subjects.......................................................................................76 Stimuli...........................................................................................76 Procedure....................................................................................79 Results...................................................................................................... 80 Conclusions from Experiment 2 ........................................................ 88 V. ENVIRONM ENTAL UTILITY...................................................................................90 An Evolutionary Account of Scene Preference..........................................90 Experiment 3: Predicting Scene Preference from Environmental Utility........................................................................................99 Methods....................................................................................................99 Stimuli...........................................................................................99 Subjects.....................................................................................100 Procedure................................................................................. 101 Results....................................................................................................106 Individual Factor Results...................................................... 106 Consistency Results.............................................................. 113 Multiple Regression Results................................................114 Conclusions & Discussion.................................................................117 Biophilia and the Preference for Natural Environments.........................................................................120 VI. FMRI INVESTIGATION OF SCENE PR EFER EN C E...................................125 Background........................................................................................................ 125 Experiment 4: Neural Correlates of Perceptual Affect.......................... 131 Methods..................................................................................................132 Subjects.....................................................................................132 Apparatus..................................................................................132 Imaging Protocol.....................................................................133 Preference Experiment......................................................... 135 Procedures.................................................................. 135 Trial Sequence........................................................... 137 LOC Localizer..........................................................................139 Procedures.................................................................. 139 Analysis and Results.......................................................................... 140 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Single Subject Analysis Procedure................................... 140 Group Analysis: Whole Brain Voxelwise A N O VA 143 Group Analysis: Cluster-Level Significance................... 150 Selection of a Subset of Subjects......................... 158 Group Analysis: Regions of Interest ANO VA.................163 Individual Differences in Activation Patterns.................. 169 Repetition “Rebound”........................................................ 172 Image-Based Correlation Analysis....................................173 Conclusions .......................................................................................181 VII. CONTRIBUTIO NS AND FUTURE DIRECTIO NS......................................184 Contributions......................................................................................................184 Future Directions.............................................................................................. 186 Behavioral Studies.............................................................................. 187 Eyetracking............................................................................................ 188 Individual Differences in the Neural Correlates of Preference...........................................................................................189 Pharmacology....................................................................................... 190 Natural vs. Urban: Distribution of Image Factors in the Ventral Visual Pathway....................................................................190 Learning Effects................................................................................... 192 Cross-Modal and Conceptual Priming.......................................... 193 Direct Measures of Preference........................................................194 BIBLIOGRAPHY............................................................................................................ 195 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. LIST OF TABLES Table 4.1. Correlation of Average Preference Ratings for Experiments 2a and 2b ........................................................................................................................... 86 Table 5.1. Factor Summary Statistics.................................................................... 107 Table 5.2. Consistency of Factor Ratings............................................................. 113 Table 5.3. Correlates of Preference: Multiple Regression Results............... 114 Table 5.4. Correlates of Preference: Part Correlations for Each Model 116 Table 6.1. Summary of BOLD Effects: Preference and Repetition............... 162 Table 6.2. Correlation with Preference by R O I....................................................179 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. xi LIST OF FIGURES Figure 2.1. The Ventral Visual Pathway and Object Recognition Speed 11 Figure 2.2. A Gradient of p Opioid Receptors in the Ventral Visual Pathway................................................................................................................. 13 Figure 2.3. Cortical Areas Comprising the Ventral Visual Pathway............... 14 Figure 2.4. Pathways from Perceptual Affect to Behavior.................................35 Figure 2.5. Predictions of a Theory of Cognitive and Perceptual Affect 38 Figure 3.1. Droodles: An Illustration of Rich Interpretability..............................46 Figure 3.2. Effects of Mere Exposure on Preference Judgments...................48 Figure 3.3. Berlyne’s Evidence for Habituation of Preference......................... 51 Figure 3.4. Distribution of Average Preference Scores......................................62 Figure 3.5. Normal P-P Plot of Average Preference Ratings........................... 62 Figure 3.6. Most and Least Preferred Scenes.......................................................63 Figure 4.1. Exp. 1: Trial Sequence.......................................................................... 68 Figure 4.2. Exp. 1: Preference vs. Trial Number..................................................69 Figure 4.3. Exp. 1: Average Preference vs. Exposure.......................................70 Figure 4.4. Exp. 1, Block 1: Average Preference Score vs. Exposure, Grouped by Initial Preference...................................................................................... 70 Figure 4.5. Exp. 1: Results for Practice Block...................................................... 71 Figure 4.6. Exp. 1: Results for Block 2 ....................................................................72 Figure 4.7. Exp. 2a: Distribution of Preference Ratings by Image Category............................................................................................................................. 77 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Figure 4.8. Exp. 2b: Distribution of Preference Ratings by Image Category............................................................................................................................. 78 Figure 4.9. Exp. 2: Change in Average Preference Over Trial Blocks 81 Figure 4.10 Exp. 2a: Preference vs. Exposure for Images Categorized by A Priori Preference.........................................................................................................82 Figure 4.11. Exp. 2a: Data from Eight Individual Subjects.................................84 Figure 4.12. Exp. 2b: Preference vs. Exposure for Images Categorized by A Priori Preference.......................................................................................................... 85 Figure 4.13. Exp. 2a: Sex Differences in Preference vs. Exposure.............86 Figure 4.14. Exp. 2b: Sex Differences in Preference vs. Exposure.............87 Figure 5.1. Komar & Melamid’s Most and Least Wanted....Paintings............. 98 Figure 5.2. Examples of Spectrally Equivalent, Phase Scrambled Images.............................................................................................................................. 105 Figure 5.3. Scatterplots of Preference vs. Factors.............................................107 Figure 5.4. Examples of Scenes rated High and Low on Each of the Six Image Factors......................................................................................................... 110 Figure 6.1. A General Linear Model Used to Analyze Single Subject Data....................................................................................................................................142 Figure 6.2. Average Brain Activation Across All Conditions (Compared to No Im age)............................................................................................ 146 Figure 6.3. Effects of Preference............................................................................. 148 Figure 6.4. Effects of Repetition...............................................................................149 Figure 6.5. Low Preference Minus High Preference for All Sixteen Subjects............................................................................................................................152 Figure 6.6. First Presentation Minus Fifth Presentation for All Sixteen Subjects............................................................................................................................153 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. xiii Figure 6.7. Rebound of Activity in Early Visual Areas for Later Repetitions.......................................................................................................................155 Figure 6.8. The Effects of Repetition and Preference in Parietal Cortex................................................................................................................................ 156 Figure 6.9. Effect of Repetition in the Right Superior Frontal Gyrus...........158 Figure 6.10. Voxelwise Overlap of Subject’s Preference Effects.................. 160 Figure 6.11. High Preference Minus Low Preference for Nine Subjects....161 Figure 6.12. Region of Interest (ROI) Analysis for Nine Subjects................. 166 Figure 6.13. Left Parahippocampus ROI: Canonical HRF Model Fit............169 Figure 6.14. Preference Effects for Two Individual Subjects.......................... 170 Figure 6.15. Preference Effects for a Third Individual Subject.......................171 Figure 6.16. Rebound Effects are Not Due to Stimulus Timing..................... 173 Figure 6.17. Image-Based Correlation Analysis for Four RO I’s.....................180 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. ABBREVIATIONS xiv AC Anterior Commissure AFNI Analysis of Functional Neuroimages software package ANOVA Analysis of Variance BA Brodmann’s Area BOLD Blood Oxygen Level Dependent DAGO [D-Alanine-N-Methyl-Phenylalanine, Glycinol] Enkephalin EM Endomorphin EMG Electromyography EPI Echo-Planar Imaging ER-FMRI Event-Related FMRI FILM FMRIB’s Improved Linear Model FLIRT FM RIB’s Linear Image Registration Tool FMR-A FMRI Adaptation FMRI Functional Magnetic Resonance Imaging FMRIB Oxford Center for Functional Magnetic Resonance Imaging of the Brain FSL FMRIB Software Library FW HM Full Width at Half-Maximum GABA Gamma-Amino-n-Butyric Acid GLM General Linear Model HRF Hemodynamic Response Function Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. XV IAPS International Affective Picture System IRF Impulse Response Function IT Inferotemporal Cortex LCD Liquid Crystal Display LO Lateral Occipital — The posterior section of the LOC LOC Lateral Occipital Complex LSF Least-Squares Fitting MNI Montreal Neurological Institute ms millisecond(s) mRNA messenger Ribonucleic Acid PC Posterior Commissure PET Positron Emission Tomography PF Posterior Fusiform - The anterior section of the LOC PHG Parahippocampal Gyrus RMS Root Mean Squared (contrast) ROI Region of Interest RSVP Rapid Serial Visual Presentation SPM 99 Statistical Parametric Mapping 99 software package TE The anterior section of inferotemporal cortex TEO The posterior section of inferotemporal cortex TR Repetition Time VTA Ventral Tegmental Area Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. ABSTRACT xvi People prefer some perceptual inputs to others, an effect readily manifested in visual fixations during free viewing. This preference may be based on the activity of a gradient of p opioid receptors (ligand = endomorphin) that, surprisingly, is found in the ventral cortical pathway for visual recognition. This gradient, discovered in the macaque, is sparse in V1 and increases in density through V2, V4, TEO, IT and the parahippocampal cortex. The magnitude of endomorphin activity would determine perceptual and cognitive preference, resulting in a preference for patterns that are both richly interpretable (because they activate many associations in the opioid rich anterior regions of the ventral pathway) and novel. Repetition of a scene would result in less activity because of competitive interactions. Using a set of full color scenes calibrated for their a priori preference and presented for one second, repetition led to decreases in preference for all levels of a priori preference. In addition, the environmental utility afforded by a scene (such as its naturalness, or the presence of a good view) was a good predictor of a priori preference, suggesting that preferences may, in part, reflect an evolutionarily conserved mechanism for the rapid assessment of one’s environment. Using ER-fMRI, we recorded brain activity as subjects attended to one second presentations of scenes that varied in their rated preference and were repeated five times. Highly preferred and novel scenes produced Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. greater BOLD activity than less preferred and repeated scenes in more anterior areas of the ventral pathway, such as the parahippocampal cortex, consistent with the endomorphin hypothesis. In contrast, activity in the lateral occipital complex (LOC) was inversely related to preference, and early visual areas generally showed an inverted-U shaped function with repetition, likely reflecting an active reexamination of the visual scenes. Surprisingly, fronto parietal networks associated with voluntary attention were more active for later repetitions and low preference images, suggesting that endogenous attentional effort is required to attend to stimuli which do not, on their own, provide a novel, richly interpretable experience. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. CHAPTER 1: INTRODUCTION 1 A casual survey of what a person spends their day doing reveals a very large role for what are commonly referred to as “play” or “ exploratory” behaviors that have little or no immediate survival value. While most evident in human behavior, exploratory behaviors which are not in the service of other drive states have also been documented in nonhuman primates and, to a lesser extent, rodents. For example, Harlow describes the development of what he calls “curiosity motivation” in four stages starting immediately after birth, beginning with reflexive orienting and leading to aggressive object manipulation and interaction with other animals (Harlow, 1986,182). Clearly we engage in behaviors that can be seen as motivated by drives for thirst, hunger, and the avoidance of harm. Yet such commonplace pastimes as going to a movie, reading a book, engaging in a conversation, visiting a museum, listening to music, surfing the internet, or driving down a scenic coast highway hardly seem, at first glance, to be understandable within the context of a drive state. Even when one considers activities which do fit within the standard drive reduction paradigm, which restaurant we eat at, who we choose to court, and where we go to enjoy a beverage with that chosen someone are decisions we make tens of times a day in the absence of any discriminable survival value for one choice over another. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 2 On a shorter timescale, the moment to moment shifts in where we choose to look about our visual world are also not predictable from standard drive states, yet are not random either. Clearly, certain parts of a visual scene, and more broadly speaking, certain types of information, are preferentially selected over others. For example, a person walking into a room in which he or she has never been does not stare at blank sections of walls, but instead fixates regions of “interest.” Previous attempts to understand how regions of a visual scene are spontaneously selected have focused either on the poorly defined concept of complexity (e.g. Berlyne, 1958) or on saliency (e.g. Itti, Koch & Niebur, 1998). Yet neither complexity, which lacks generalizability across multiple stimulus types, nor feature based saliency, which fails to incorporate higher level relationships among the elements of a scene, can account for human spontaneous visual selection (Kaplan, 1992; Oliva et al., 2003; Heidenreich & Turano, 2003). W e propose that spontaneous visual selection is an example of another, very basic drive separate from those commonly defined in the study of motivated behaviors: information foraging. More specifically, humans engage in many behaviors in order to maximize the rate of acquisition of novel, yet richly interpretable information: we are “infovores.” This drive, which likely operates continuously in the absence of more pressing biological needs Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 3 (such as harm avoidance or hunger), is unlike the traditional drives in that information foraging behavior has no clear end point: one does not simply become “satiated” with information and stop foraging. Instead, it is likely that an information seeking system would be wired in such a way that the acquisition of information itself acts as a reward, thus leading to the “ win- shift” behavior typical of information seeking behaviors. The importance of an information foraging system stems from the fact that it is often impossible to know what aspects of our environment contain potentially useful information for future decisions. Humans have a vast capacity for remembering specific episodes of experience, as well as the names and appearance of objects, people, and places we encounter. The decisions we make at any moment are informed by a combination of sensory cues collected at that moment and a vast repository of experience. By selectively attending to those aspects of an environment that strike a balance between novelty and interpretability (e.g. relatability to previous memories), an information foraging system allows for the preferential encoding of information which is maximally beneficial for making life saving decisions at a later time. Such a system likely evolved alongside the dramatic expansion of the mammalian brain as survival strategies shifted focus away from largely instinctual reactions towards solutions which depended on general problem solving abilities and the observation of information patterns over long periods Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. of time. As an example, consider a situation in which an early hominid might casually observe a friendly interaction between two other individuals. While that bit of information may have no bearing on his current situation, it would become relevant if he were to consider a fight with one of the individuals later. Not only would he have to contend with one person, but he now knows that he might potentially have to contend with the second person as well. In the words of Stephen Kaplan, “ when drives are satisfied, when nothing else is going on would be precisely the time when the individual could be obtaining information about the environment for use at a later time.” (Kaplan, 1992, pp. 584) In the next chapter, we present a theory for how an information foraging system could be built into the architecture of the brain’s perceptual systems. A key element to understanding information foraging as a drive state is that attaining new information is experienced as pleasurable: every act of perception, by virtue of its information content, gives rise to an affective response. W e suggest that this affective response may be mediated by a gradient of p opioid receptors in perceptual pathways. In this way, the activity in perceptual systems mediating the recognition of interpretable information can influence a wide range of behaviors and learning processes, such as moment to moment eye movements, spontaneous preferences for certain scenes over others, the formation of long term aesthetic dispositions, Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 5 and the identification of likely sources of information in one’s environment (such as TV, urban centers, movies, and other people). Stephen Pinker, in his 1997 book “How the Mind Works,” provides another poignant example for how built-in information foraging biases and links to preference would have been critical for the survival of early hominids. Quoting from Tooby & Cosmides (1992): Imagine that you are on a camping trip that lasts a lifetime. Having to carry water from a stream and firewood from the trees, one quickly learns to appreciate the advantages of some campsites over others. Dealing with exposure on a daily basis quickly gives one an appreciation for sheltered sites, out of the wind, snow, or rain. For hunter-gatherers, there is no escape from this way of life: no opportunities to pick up food at the grocery store, no telephones, no emergency services, no artificial water supplies, no fuel deliveries, no cages, guards, or animal control officers to protect one from the predatory animals. In these circumstances, one’s life depends on the operation of mechanisms that cause one to prefer habitats that provide sufficient food, water, shelter, information, and safety to support human life, and that cause one to avoid those that do not. Pinker makes explicit the direct analogy between human preferences, as studied within the field of environmental aesthetics, and the biological study of habitat selection: people enjoy being in some places more than others (Pinker, 1997, Ch. 6). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 6 As a means of assessing this system, we have chosen to investigate visual preference behavior and its neural correlates. While there exists only a minimal literature on direct investigations of visual preference, a variety of different disciplines have touched on related issues of information foraging or emotional responses to stimuli. In Chapter Three, we discuss some of the wider influences on visual preference that can be gleaned from some of these fields. Early studies of preference and orienting behavior by Berlyne and his colleagues and a large number of studies documenting the effects of previous exposure on affective responses to stimuli provide the most relevant background within cognitive psychology. Preferential looking has been used as a method to assess the development of visual abilities in infants for decades with a very limited understanding of why infants show such preferences in the first place. More recently, studies of visual attention and high level scene perception have investigated the contributions of low level features, scene gist and informativeness to spontaneous visual selection, yet without any reference to the affective consequences of these processes. A majority of the relevant findings on visual preference are to be found in the field of environmental psychology and landscape assessment. Stephen Kaplan’s influential work has provided a bridge between the practical need to develop an understanding of landscape management and the deeper theoretical implications of observed preferences for certain types of landscapes over others. Finally, the effects of stimulus valence and arousal Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 7 on positive and negative affect have been studied in the context of scenes and faces. Yet the main focus of this body of research has been on the induction or perception of negative emotions and the role of the amygdala in these responses. Very few of these studies even address positive affect, or have done so using inadequate stimulus sets or with a focus on highly arousing stimuli (such as erotica). Unlike these approaches, we have chosen to focus on low valence, mildly positive affect as a method for studying the properties of information foraging separate from other drive states responsible for avoidance of harm or sexual excitement. Within these various domains, affective responses to visual stimuli have been assessed using eye movements, physiological measures, and rated preferences, yet none of these fields have provided an adequate exploration of the links between perception and affective preference, nor an explanation for the brain mechanisms underlying the affective response to potentially interesting aspects of the environment. In Chapters Four and Five, we present a series of behavioral experiments which explore human visual preference using full color scenes. W e demonstrate that preferences for visual scenes are highly predictable across subjects and that increased exposure produces a reliable decrease in scene preference. In addition, we show that the interpretability of a scene, and hence its preference, may be largely understood within the context of the evolutionary value of certain Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 8 scene characteristics such as the presence of natural elements, the degree of mystery, and the presence of an expansive view. Such “built-in” preferences are those which would have been critical to our early hominid ancestors for successful habitat selection. In Chapter Six, we present imaging data supporting the claim that differential activity distributions in the ventral visual pathway give rise to differences in stimulus preference, a result which is in agreement with the hypothesis that p opioid receptors may mediate the link between perception of highly interpretable information and positive affective responses. Finally, in Chapter Seven, we outline several directions for future research and summarize our contributions within the field of perceptual affect. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 9 CHAPTER 2: A THEO RY OF COG NITIVE AND PERCEPTUAL AFFECT When you walk down a busy street, hike along a mountain path, or walk into a room in which you have never been before, where do you look and what mechanisms determine this? W e propose a neurocomputational theory of cognitive and perceptual affect based upon a gradient of p opioid receptors in the primate visual pathway mediating recognition. This theory provides an explanation for how the perception of novel, richly interpretable information can provide a signal used for both the spontaneous visual selection of “regions of interest” and the accompanying positive affective reaction elicited by information foraging. Positive Affect in the Ventral Visual Pathway What scenes we choose to look at and where we look within a scene are decidedly nonrandom activities. On the order of three times a second, we make saccades around our visual world which are clearly not random. W e do not stare at blank walls nor uninterpretable masses, are fascinated by art and computer animation, and gaze longingly at scenery. That is, we don’t look at regions where nothing is happening nor do we look at regions of uninterpretable variation. Instead we look at “regions of interest.” But how can we understand “interest?” In addition, we generally show a preference Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 10 for novelty, preferring a new book, movie, or conversation to one experienced previously. Once a particular source of information has been adequately explored, there is very little benefit from revisiting that same source in the near future. How is this “ win-shift” behavior implemented in the brain? The challenge posed by these questions is to understand how a motivational system can be built into a real time perceptual-cognitive system that allows for the exploration of new aspects of one’s environment that are not so foreign as to be uninterpretable. A major constraint imposed on the possible instantiations of such a system is the ability to control real-time eye movements based upon an affective evaluation of the regions of one’s visual environment. Feedforward activity in the macaque ventral visual pathway is estimated to reach ventral temporal cortex on the order of 80 to 100 milliseconds, with eye movement planning and execution occurring within 250 milliseconds (Thorpe & Fabre-Thorpe, 2001; see Figure 2.1). These estimates are slightly longer for humans, where differential EEG activity for categorization tasks has been observed at 150 milliseconds, with behavioral responses typically occurring on the order of 400 milliseconds (and as fast as 250 ms) after stimulus onset (Thorpe, Fize & Marlot, 1996; Thorpe & Fabre-Thorpe, 2001). Therefore, this component of the affective response to a perceptual stimulus cannot rely on lengthy cognitive operations nor feedback from limbic structures, but instead Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 11 involves signals arising directly from perceptual processing in a feedforward manner. It is not an “afterthought” or attributional process, but an integral aspect of perception, which by virtue of its immediate nature and links to the brain’s effector systems, guides spontaneous visual selection about one’s environment and produces other measurable indices of positive affect (Winkielman & Caccioppo, 2001; Winkielman et al., 1997; Zajonc, 1980). mand Categorical judgments, decision making 160-1 Simple visual forme, , corners »- Tc spinal cord -160-220 ms muscle Figure 2.1. The ventral visual pathway is able to extract the form of visual objects and scenes in under 150 ms. Reprinted with permission from Thorpe & Fabre-Thorpe, Science, 291:260-63 (2001). Illustration: Carin Cain. Copyright 2001 American Association of the Advancement of Science. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. A Neurochemical Index of Interpretability 12 W e posit a specific function for the gradient of p opioid receptors discovered by Lewis and his colleagues in the ventral visual pathway of the macaque monkey as mediators of this positive perceptual affect (Lewis et al., 1981; Wise & Herkenham, 1982). Using radioactively labeled naloxone ([3H]naloxone), a potent opiate antagonist, they quantified the relative amounts of the p opioid receptor in different cortical and subcortical regions of a rhesus (Macaca mulatta) monkey. As can be seen in Figure 2.2, the amount of naloxone binding was found to increase monotonically from cortical areas V 1 , V2, V4, TEO, IT, fusiform cortex (TF), parahippocampal cortex (TH), and periamygdalar cortex (A) on both the medial and lateral surfaces. Note that the labels for the cortical fields used by the authors differ from the currently used monkey designations, with OC corresponding to V 1 , OB roughly to V2, and OA roughly to V4. In contrast, the distribution of the 5 opioid receptor, as assessed by binding of tritiated Leu-enkephalin was found to be uniformly distributed across cortical regions, indicating that the p receptor might be involved in a specific aspect of perceptual processing. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 13 Visual eoftteat projactton and related area* 4.0 • Lateral surface Medial surface 2.0 ’ 1 J 0 • Figure 2.2. The density of p-like opioid receptors as labeled by tritiated naloxone is relatively low in early cortical visual areas (OC = striate cortex, OB = peristriate, OA = preoccipital) and gradually increases on both the lateral and medial surfaces of the ventral visual pathway. Reprinted with permission from Lewis et al., Science 211:1166-1169 (1981). Illustration: ??. Copyright 1981 American Association for the Advancement of Science. In humans, the ventral visual pathway is greatly expanded over that of the macaque, and includes a large swath of tissue extending from the occipital pole along the ventral, lateral, and medial aspects of the brain into the temporal pole. This includes the lingual and parahippocampal gyri on the medial aspect, the fusiform gyrus, and the inferotemporal gyrus on the lateral aspect. Figure 2.3 shows a ventral view of the human brain with the cerebellum removed and several of these structures labeled. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 14 Figure 2.3. The ventral aspect of a human brain with the cerebellum removed. The ventral visual pathway includes the lingual (22) and parahippocampal gyri (17) medially, the fusiform gyrus (18) along the center, and the inferior occipital (23) and inferior temporal gyri (19) on the lateral aspect. Adapted from Bergman et al., 2001. Why would opioid activity be associated with a perceptual pathway? The exact role of these receptors is not clear, but the demonstration that endogenous opioids often play an important role in motivation, reinforcement, and pleasure raises an interesting possibility. Noting a possible parallel with the gating function of opioids in the spinal cord, Lewis et al. speculate that “opiatergic neurons may be involved in the filtering of sensory stimuli at the cortical level, and thus play a role in selective attention.” (Lewis et al., 1981, 1168). The crucial aspect of this discovery is the direct match between the density of p opioid receptors and the sequence of processing stages of the Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 15 ventral visual stream underlying the perception of visual form, often referred to as the “ what” pathway (Ungerleider & Mishkin, 1982; Macko et al., 1982; Goodale & Milner, 1992). The assumption is that as the activity from earlier visual areas is passed on to later cortical fields, this activity leads to the release of endogenous opioids and subsequent binding to p opioid receptors. The significance of finding a gradient that is most dense in association cortex is profound: this pattern suggests that maximal p opioid activity (and hence, reinforcement) would result from visual experiences that lead to a high degree of interpretation. Stimuli which are merely “busy” but not richly interpretable would provide little pleasure. Our proposal is that richly interpretable perceptual inputs would lead initially to the most neural activity — and hence endogenous opioid activity — in the anterior stages of the ventral visual pathway (IT and parahippocampal cortices). High activity would then be associated with high preference (or interest). As will be described below, once this activity is unleashed in the endomorphin-rich association areas, competitive learning sets in, resulting in a diminution of the activity when the input is repeated resulting in reduced preference. The net effect is that the strongest preference will be manifested for novel, richly interpretable stimuli. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 16 A similar gradient was discovered from primary to secondary auditory cortex, and some evidence of such a gradient was seen in somatosensory areas as well. Such gradients may provide a neurochemical index of interpretability by which activity in perceptual pathways can influence selection mechanisms and lead to affective responses in an immediate and bottom-up manner. Behavioral Effects of Opioid Antagonists Investigations of p opioids in humans using the antagonist naloxone are consistent with a possible role for this system in perceptual affect. Reports from subjects that have been administered naloxone both in research and clinical settings indicate no pervasive effects on mood, though there are occasional reports of anhedonia (flattened affect) in which the world seems devoid of much of its normal excitement, and dysphoria (unhappiness) particularly in situations in which the subject is required to remain passive (Arnsten et al., 1984). Naltrexone, a related, milder opiate antagonist, is currently in use as a treatment for severe alcoholism because of its tendency to reduce the rewarding aspects of alcohol that are responsible for sending an alcoholic on a binge (Volpicelli et al., 1992). It appears as if the opiate antagonist slightly reduces the subject’s initial desire to have a drink, but also strips the alcoholic experience of its pleasing aspects which, in turn, reduces the desire. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 17 Naloxone has also been investigated with regards to its effects on auditory stimuli. Goldstein found that, in some subjects which often experience “chills” in response to music, the administration of naloxone attenuated those chills (Goldstein, 1980). Opioid Systems in the Brain There likely exist many different functional brain systems which are influenced by opioid mediated signaling, and while a majority of the work on these systems to date has been on the subcortical pathways involved in nociception, reward, and stress, the presence of opioid receptors and endogenous opioid ligands in the cortex is well established (Lewis et al., 1981; Lewis et al., 1983). Three main opioid receptor types, p, 5, and k , have been identified across different mammalian species and their cortical and laminar distribution studied using radioreceptor assays, receptor autoradiography, and more recently, in situ hybridization studies of receptor mRNA (Peckys & Landwehrmeyer, 1999; Hiller & Fan, 1996; Mansour et al, 1995; Lewis et al, 1983; Wise & Herkenham, 1982; Lewis et al, 1981). The subcortical and cortical distribution of the three receptor types, which all belong to the superfamily of G-protein-coupled receptors (Peckys & Landwehrmeyer, 1999), are quite different, implicating them in a variety of Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 18 systems with widely ranging function. Most opioid systems investigated to date have found the receptors to be coupled to inhibitory G proteins that ultimately produce a decrease in neuronal excitability through some combination of activation of inwardly rectifying potassium channels and decreases in calcium channel conductance (Peckys & Landwehrmeyer, 1999; Hille, 1992, 186-7). The initial discovery of the p opioid receptor in nervous tissue in 1973 marked a revolution in the understanding of how drugs have their effect on the nervous system, and began the race for the discovery of the natural, endogenous ligands for the receptors (Pert & Snyder, 1973; Zadina et al., 1999). A number of nonspecific endogenous ligands, including (3-endorphin (which binds to p and 5), and ligands with higher affinity and selectivity for the 5 and k receptors (such as Met-enkephalin, Leu-enkephalin and dynorphin, which show weak p activity but primarily bind to 5, 5, and k respectively) were discovered in the seventies (Khachaturian et al., 1985; for a review, see Corbett et al., 1993). But it was not until 1997 that two potent, selective endogenous agonists for the p receptor were discovered in the human brain (Zadina et al., 1997; Hackler et al., 1997; Zadina et al., 1999; Zadina, 2002). Termed endomorphins (short for “endogenous morphine”), these tetrapeptides show distributions in the central nervous system consistent with the localization of p opioid receptors and a role in a diversity Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 19 of functions including pain and stress gating, reward, and other autonomic and endocrine functions (Martin-Schild et al., 1999; Zadina et al., 1999). Below, I will briefly review the known cortical distributions of p opioid receptors and their possible ligands in rats, monkeys, and humans. Much of the early work on opioid receptors sought to understand the systems underlying morphine’s analgesic ability, and therefore focused on elucidating the spinal and brainstem opiatergic systems. Yet clinical observations of patients on morphine clearly indicate that it has effects on “higher” processes such as attention, mood, and motivation (Buchsbaum et al., 1982). These observations led Michael Lewis and his colleagues at NIMH to reexamine the cortical distribution of opioid receptors in light of known functional cortical systems. As discussed above, the most striking aspect of their findings was the graded increase in naloxone binding from primary sensory areas to higher sensory and association areas. These gradients peaked in periamygdaloid cortex on the medial surface of the temporal lobe (Lewis et al., 1981). Differences in naloxone binding were also observed in the frontal cortex. The lowest amount of binding was seen in the motor strip, while orbitofrontal cortex showed a high degree of binding. The olfactory tubercle, was also associated with strong naloxone binding. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 20 The following year, another group at NIMH reexamined naloxone binding in the macaque monkey with an interest in characterizing the laminar patterns of opioid receptor distributions, a feat not possible in the earlier study, which used brain homogenates (Wise & Herkenham, 1982). They reported that the ventral temporal and orbital frontal fields had among highest levels of binding in the cortex, with “progressively higher levels of opiate binding were observed with increasing proximity to the most primitive type of cortex.” (Ibid., 218). The majority of cortical binding was strongest in layer V; in the striate and peristriate cortex, they saw enriched naloxone binding in layers V and VI, as well as some binding in layer IV as well. Conversely, the frontal eye fields had the strongest binding in layer III. Since layer V is the predominant source of corticofugal projections out of cortex, the authors suggested that a main effect of cortical p opioid receptor activity is to influence the outflow of cortical fields, with an emphasis on polymodal information procession and limbic function. Studies in rat show rough agreement with the distribution of p opioid receptors across cortical fields, though there are species differences in the lamination patterns (Lewis et al., 1983). The distribution of opioid receptors in humans have since been investigated using both receptor radiography and mRNA immunohistochemistry, with somewhat conflicting results. A radiographic study using tritiated DAGO, a potent p agonist with greater selectivity than naloxone, found p receptors to Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 21 predominate in layer III (especially Ilia), with a second peak of density in either layer V (frontal) or IV (temporal and occipital; Hiller & Fan, 1996). On the other hand, mRNA for the p receptor was found to predominately label pyramidal cells in layer V (Peckys & Landwehrmeyer, 1999), though this study did not look at as many different cortical divisions as the radiographic study. The authors of the latter study suggest that while the overall amount of opioid mRNA appeared to be lower in human than rat, the relative amounts in cortex and hippocampus appear much higher in human. While an earlier study of DAGO binding in human cortex reported that p receptors are most concentrated in superficial layers (I and II), both of the more detailed DAGO studies are in good agreement on the issue of the p receptor distributions in the ventral visual pathway. Referring to the gradients reported by Lewis et al. (1981 and 1983), Quirion & Palapil state that “similar organizations may exist in human brain cortex.” (Quiron & Palapil, 1991,104) These studies found virtually no evidence for p receptors in striate cortex, with little or no binding seen in area 18 (referred to as lateral occipital cortex by Hiller & Fan) either. The lateral occipitotemporal gyrus (BA 37) showed an intermediate degree of binding and the inferior temporal gyrus (BA 20) showed one of the highest degrees of p opioid receptor labeling. The parahippocampal gyrus also showed a high degree of DAGO binding (highest in layer IV followed by layer III), though not as high as the Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 22 inferior temporal or medial temporal (BA 21) gyri (Hiller & Fan, 1996; Quiron & Palapil, 1991). The hippocampus proper and dentate gyrus contain very few p opioid receptors, a finding that is quite contrary to the animal literature, while high levels are seen in all divisions of the amygdala, cerebellum, striatum, globus pallidus, nucleus accumbens, and other nuclei of the basal forebrain (Quiron & Palapil, 1991). In comparison to the level of detail available on the distribution of opioid receptors, the current evidence available on the distribution of the endogenous ligands for these receptors is quite sparse. This is especially true for the p receptor given the recency of the discovery of potent, selective endogenous ligands. Reports on the cortical distribution of EM1 and EM2 (two different forms of endomorphin) in the human brain are lacking, and studies in rat have focused primarily on subcortical structures. W hat is known is that there are some clear differences in the distribution of EM1 and EM2; EM2 is more prevalent in the spinal cord, while EM1 is more widely distributed in areas associated with reward, arousal, limbic and cognitive functions (Martin-Schild et al., 1999). In the rat, fibers containing EM1 have been visualized in frontal and limbic cortices, all divisions of the amygdala, nucleus accumbens, septal nuclei, the diagonal band of Broca, bed nucleus of the stria terminalis, midline thalamic Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 23 nuclei, locus coeruleus, and the striatum (Zadina et al., 1999; Zadina, 2002). The highest densities of EM1 immunoreactive fibers are in the posterior hypothalamus (dorsomedial, ventrolateral, and arcuate nuclei) and the nucleus of the solitary tract, which are also the only observed locus of cell bodies containing EM1 immunoreactivity (Martin-Schild et al., 1999). However, the authors stress that visualization of immunoreactive cell bodies depends on the administration of colchicine to prevent axonal transport. Their administration of colchicine into the lateral ventricles may not have been sufficient to allow for the visualization of EM containing cell bodies in parts of the brain not immediately surrounding the ventricles, making it likely that additional cells containing endomorphin will be identified in future studies. In addition, there are likely to be species differences in the distribution of EM containing cells. As an example of a clear inadequacy in our understanding of p opioid systems, the authors point to the mismatch between the presence of p opioid receptors in the striasomes of the rat striatum, yet an absence of any endomorphin-like immunoreactivity (Ibid.). It is therefore plausible that some p opioid systems, including those in the cortex, may interact with some of the mixed affinity enkephalins1 or as of yet unidentified endomorphins. 1 These enkephalins have been shown to be synthesized in rat cortex (Khachaturian et al., 1985; Mansour et al., 1995) and to exist in human occipital and ventral temporal cortex (Hurd, 1996). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 24 Given the presence of both p and 5 opioid receptors in cortex in differing distributions, it remains an interesting possibility that the release of enkephalin compounds from cortical and/or subcortical sources with varying p/5 affinity ratios could provide a rich diversity of neurochemical modulation of sensory signals. Significant levels of proenkephalin mRNA (the precursor gene for Met- and Leu- enkephalin) have been observed in all regions of cortex, and their laminar distribution matches relatively well the distribution of human p binding sites (Hurd, 1996; Quiron & Palapil, 1991). Moderate to high levels were observed in temporal, occipitotemporal, occipital, frontal and motor cortices and the olfactory gyrus. In general, the ventral visual pathway contained a quite high amount of proenkephalin mRNA, with an especially high concentration in the lingual gyrus. In contrast, the limbic cortices showed very little proenkephalin mRNA expression (Hurd, 1996). The proposed role of p opioids as mediators of increased perceptual pleasure is somewhat counterintuitive given the fact that most opioid signaling mechanisms are linked to inhibitory G proteins and therefore act to inhibit their targets. The available evidence on the distributions of opioid receptors within neurons indicates that opioids may act as neuromodulators in some cases and as direct neurotransmitters in other systems (Simon & Hiller, 1994), and it is likely that in many cases the cortical influence of opioid systems is through the presynaptic inhibition of neurotransmitter release Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 25 (Hiller & Fan, 1996). One possibility is that the co-release of endomorphin by a glutamatergic cortical neuron binds to p opioid receptors on the presynaptic terminals of nearby GABAergic neurons. Activation of these neurons would then serve to reduce the tonic level of inhibition impinging back on the glutamatergic neurons in that part of the cortex. This arrangement is analogous to the disinhibitory role of opioid receptors in other brain systems that have been studied. For example, opioid receptors are found on GABAergic neurons in the rat hippocampus (Morris & Johnston, 1995; Cohen, Doze & Madison, 1992), and in the brain’s reward circuits, where the euphoric effects of morphine are thought to result from an inhibition of striatal GABAergic neurons and a subsequent disinhibition of their dopamine releasing target cells in the substantia nigra (Peckys & Landwehrmeyer, 1999; Mansour et al., 1995). Finally, dopamine releasing neurons in the VTA projecting to the nucleus accumbens are also thought to be released from GABAergic inhibition as a result of the activation of p receptors on the GABAergic neurons, implicating a role for endogenous opioids in sexual behavior as well (L. M. Coolen, colloquium. USC, January 15, 2003). Consequently, endomorphin activity in perceptual pathways likely results in a selective reduction of local inhibition, allowing greater activation of areas associated with reward such as the amygdala, orbitofrontal cortex, and subcortical dopaminergic pathways. This local increase might also potentiate Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 26 synaptic plasticity and play a key role in the network dynamics underlying competitive interactions. Novelty, Priming, and Competitive Learning As will be discussed more extensively in the next chapter, stimuli are often less preferred the second time they are experienced. While there may be some comfort associated with familiarity in the face of uncertainty, interest is much more often associated with novel environments, ideas, sounds, and pictures. Even in familiar settings, we still seek to assimilate novel information in picking up a book or turning on the TV or going to a movie or desiring a good conversation. The familiar setting may offer fewer distractions (i.e. competing sources of novel information) to these information rich sources. In addition to providing a basis for a preference of highly interpretable stimuli, our theory can also account for the general preference for novelty. Given the assumption that increased activity in the more anterior stages of the ventral visual pathway leads to increased preference and visual interest, there exists a wide base of theoretical and experimental support for the proposition that stimulus repetition decreases this activity. Behavioral studies typically show faster recognition times and increased recognition performance with repetition of the stimuli (see Schacter, Chiu & Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Ochsner, 1993 for a review). This repetition priming occurs for stimuli which are both explicitly recognized as well as stimuli which are presented too briefly to be recognized on a single exposure (Bar & Biederman, 1998). At longer presentation times, the magnitude of the priming is robust to changes in retinal position, mirror reflection, size, and as long as an object’s parts remain readily discernible, rotation in depth (Biederman & Cooper, 1991; Fiser & Biederman, 2001; Biederman & Gerhardstein, 1993; Biederman & Bar, 1999). Priming is generally believed to be a product of the reactivation of the same cellular networks by a repeated stimulus, which occurs with the strengthening of the connections between a neuron responding to some stimulus aspect and the inputs that cell receives from earlier stages of processing. Importantly, the increase in local connection weights is accompanied by an increase in inhibition from the most active unit(s) to units less strongly activated by the initial presentation. Priming involving stimuli which change in their low level properties (such as object position or orientation in depth) presumably occurs as a result of overlapping cellular activation at a higher level of processing in a manner which reflects the level of representation at which the two stimuli are equivalent. For example, the degree of translation invariance observed in supraliminal priming implicate the human homologue Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 28 of monkey TE as the locus for the neural changes underlying the facilitation of recognition (Bar & Biederman, 1999). Neurophysiological studies of stimulus repetition in the rhesus monkey indicate that priming may be the result of the formation of a sparser code for representing that stimulus. Using complex, multicolored pictures (such as faces, objects, patterns, and textures) which were familiar to the animal, Miller, Li, & Desimone (1993) found that 48% of the cells they recorded from in the anterior, ventral portion of IT (TE) discriminated between a matching (repeated) and a non-matching stimulus, even when a number of stimuli intervened between the sample and the match. Of these, 92% showed decreased firing upon repetition of a stimulus. In an earlier study, the same group found that many IT neurons developed stimulus-specific suppression lasting long periods when they were tested with novel stimuli repeatedly until they became familiar (Miller, Li, & Desimone, 1991b). They propose the existence of a passive, suppressive mechanism as well as an active mechanism for tagging a particular stimulus as a target. Another group, also demonstrated what they term stimulus specific adaptation (SSA) in macaque IT cells, and report that such adaptation only occurred for those neurons with an excitatory response to the stimulus (Sobotka & Ringo, 1994). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 29 It is important to note that in addition to the widespread suppression observed in these studies, a very few cells, likely those which come to represent the stimulus, increase their firing over repeated exposure. This evidence is consistent with what would be expected if the cells that respond best to the initially novel stimulus act to suppress the firing of the general population of less active cells. Cellular networks in higher order visual areas mediating recognition (TE, TEO, and perhaps V4 and other earlier areas to a lesser degree) undergo competitive interactions, whereby cells mutually inhibit each other and readjust their connection strength from earlier layers in a process that maximizes individual cell firing. This refinement of initially broad and noisy activity leads to the development of a robust, sparse representation for a novel stimulus as it becomes familiar. The global effect of this process would be an overall decrease in the amount of neuronal activity for a stimulus which has been seen previously. Brain imaging studies in humans confirm that stimulus repetition leads to both short term adaptation and longer term reorganization of cortical circuits, seen as reduced FMRI activity in the human extrastriate and inferotemporal cortex (Buckner et al., 1998; James et al., 2000, van Turrenout, Ellmore & Martin, 2000). This reduction in visual areas tends to occur to a greater extent in the later stages of processing, and is sometimes absent from early visual areas completely (Buckner et al., 1998). A study of repetition priming by Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 30 Vuilleumier et al. (2002) reported that the decreases observed in area LO and the posterior inferior temporal cortex (including the fusiform) were invariant to changes in the retinal size of the images, and some fusiform adaptation was observed for changes in viewpoint as well. Finally, the presentation of a different exemplar with the same name as a previously presented object did not produce changes in any visual areas, but did affect left inferior frontal cortex. There have been some reports of increases of activity to repetition of stimuli, particularly when the stimuli are unfamiliar (Henson, Shallice & Dolan, 2000) or shown for only very brief durations (James et al., 2000). Conflicting reports indicate that both familiar (nameable) and unfamiliar (nonsense) objects lead to decreases (and no observable increases) in left and right inferior occipital gyri and fusiform gyri (van Turrenout, Ellmore & Martin, 2000; Vuilleumier et al., 2002), though the authors do note that the locus of regions showing decreased activity was more variable across subjects for nonsense objects. A more recent study has reported that while lateral occipital and posterior inferotemporal cortex show repetition suppression for both nameable and novel objects, the fusiform gyrus only shows decreases for nameable objects (and overall lower activity for nameable objects). The likely resolution for these conflicting accounts lies in whether or not the repetition of a stimulus leads to an increase in interpretability, as would be Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 31 expected if its activity propagated forward (more anterior) on a second presentation, or merely activates the same representation a second time, leading to a decrease in activity. Stimuli which are presented too briefly to be fully perceived, or are so uninterpretable as to require longer viewing for understanding would therefore lead to increases on a second presentation. In agreement with this idea, an imaging study by Bar et al. in 2001 demonstrated that subjective ratings of increased interpretability for an initially unrecognized stimulus led to an anterior shift in the focus of activity in the temporal cortex for that stimulus. These ideas will be explored in greater detail in the next chapter when we consider the behavioral effects of subliminal exposures on preference judgments. For repetition of a stimulus which does not lead to an increase in interpretability, then, there is a decrease in the overall amount of neural activity in the ventral visual pathway, resulting in less pleasure from viewing that stimulus a second time. There are some apparent exceptions to preferences for novelty. Yet further investigation of these cases reveal that they do not run counter to our theory, but are instead cases where interpretability is lacking and additional processing is needed to achieve understanding. A commonly cited example is the mere exposure effect, in which the “mere” exposure of a stimulus leads Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 32 to a preference for that stimulus over novel items (Zajonc, 1968). W e will review the evidence for and against this effect in greater detail in Chapter Three. It is likely that this effect arises when a stimulus is presented in such a manner that a single exposure is well masked or very brief, preventing an understanding on a first exposure. A second example is the preference of a child to have the same bedtime story read to him or her on many occasions. Again, it is likely that the child does not fully understand the story, even though they may be able to recite the story verbatim. Each subsequent exposure provides an opportunity for the child to extract something new, such as an association between a word and a picture or a deeper understanding of the story being told. In an analogous manner, the effort required of an adult to achieve mastery of a new subject (such as the struggle to understand a new theorem) can be frustrating before the moment of comprehension, but the knowledge of that impending moment and its accompanying pleasure provides motivation to continue struggling. Perceptual Affect in a Systems Perspective While we have focused on understanding how activity in perceptual streams may be modulated by differences in the novelty and interpretability of a stimulus, it is important to understand how these differences affect other systems in the brain that ultimately produce the behavioral effects of Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 33 selection, pleasure, and recall. An extensive overview of these systems is beyond the scope of this theory, but it is helpful to think about how activity in the anterior stages of the ventral visual pathway may exert influence on other brain areas. The anterior portion of the inferotemporal cortex (area TE) and the parahippocampal cortices are heavily connected to the hippocampus (via the perirhinal and entorhinal cortices; Saleem & Tanaka, 1996; Suzuki & Amaral, 1994a), the lateral, basal, and basal accessory nuclei of the amygdalar complex (Turner, Mishkin & Knapp, 1980; Iwai & Yukie, 1987; Cheng, Saleem & Tanaka, 1997), and a variety of subcortical structures, including the striatum (including the ventral striatum, e.g., nucleus accumbens, both directly from TE and via the amygdala and hippocampus; Cheng, Saleem & Tanaka, 1997; Friedman et al., 2002), the superior colliculus (though this connection appears to be stronger in infants than adult monkeys; Webster, Bachevalier & Ungerleider, 1993; Webster, Bachevalier & Ungerleider, 1995), the pulvinar (Webster, Bachevalier & Ungerleider, 1993) and the claustrum (Cheng, Saleem & Tanaka, 1997). Cortical targets include the prefrontal cortex (including the dorsolateral prefrontal cortex and frontal eye fields, BA 8 and 45, and the orbitofrontal cortex, BA 12,11,13; Webster, Bachevalier & Ungerleider, 1994; Bullier, Schall & Morel, 1996; Lavenex et al., 2002), the superior temporal sulcus (Saleem et al., 2000), the insular, Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 34 cingulate, and retrosplenial cortices (Lavenex et al., 2002), and regions in the parietal cortex (such as LIP and 7a; Lavenex et al., 2002; Webster, Bachevalier & Ungerleider, 1993). In addition, the multiple regions in the anterior portion of the temporal lobe are highly interconnected and give rise to large feedback projections to earlier visual areas. Interestingly, a large number of studies have recently shown that the anterior portion of area TE appears to be functionally and anatomically divided into a ventral aspect (TEav) and a dorsal aspect (TEad), with these divisions giving rise to different connectivity patterns in many target structures (Saleem & Tanaka, 1996; Cheng, Saleem & Tanaka, 1997; Saleem et al., 2000; Yoshida et al., 2003). Therefore, information about object form influences a large number of brain systems, such as those involved in decision making and movement planning, the formation of episodic and factual memories, the formation of stimulus- reward associations, and emotional responses such as fear, pleasure, disgust, etc. As it relates to information foraging, the systems which are of importance to us are those involved in spontaneous visual selection and the subjective experience of pleasure. Through its projections to the frontal eye fields (FEF) and the superior colliculus, activity distributions in the anterior inferotemporal cortex could directly guide eye movements during free viewing to areas of high interest. Conversely, the affective component of Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 35 information foraging behavior, i.e., the “ thrill” elicited from the acquisition of novel and highly interpretable information, is likely a result of activity in the nigrostriatal system (dopaminergic afferents from the ventral tegmental area, nucleus accumbens, the substantia nigra, and their targets in hypothalamic and brainstem motor nuclei; Robbins & Everitt, 1996). As is illustrated in Fig. 2.4, the ventral visual pathway can exert influence on this system either directly (Cheng, Saleem & Tanaka, 1997), or via activity in the hippocampus, amygdalar nuclei and prefrontal areas (Friedman, Aggleton & Saunders, 2002). It is likely that affective preference is mediated by the direct projections, which provide a mechanism by which information acquisition itself can act as a rewarding stimulus in the absence of any external reinforcers. Cortical Motor and Planning areas Explicit — *- Behavioural Responses Language . r Cortex v Amygdala and Orbitofrontal Cortex Association Cortex i Striatum Secondary Cortex , Primary Reinforcers e.g. taste, touch, pain Implicit — *- Behavioural Responses Premotor Cortex etc — » ■ Thalamus Primary Cortex t V en tral Striatum brainstem spinal cord Reflexes INPUT Figure 2.4. The affective component of information foraging behavior is likely mediated by direct projections from sensory cortices to the striatum (from Rolls, 1999, pp. 257). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 36 An important question raised by the divergent projections arising from the anterior aspect of the ventral visual stream is how activity distributions in these areas can differentially affect mnemonic and affective systems. Stimuli which are recognized as being familiar do not necessarily produce strong positive affective responses, and vice versa. An interesting possibility is suggested by the previously mentioned process of “ sparsification” implicated in the neural studies of stimulus repetition. Stimuli which are both novel and richly interpretable have not yet come to be represented in a sparse manner, and will lead to widespread activity distributions in area TE and the parahippocampal cortex (though individual cells may not fire at their maximal rate). As a person becomes familiar with a stimulus, competitive interactions lead to a change in the representation of that stimulus, as a very few cells which are initially most activated adjust their connection strength in a way which leads to even greater activation by the same stimulus in the future, while the majority of weakly activated cells adjust their connection strength in a manner which decreases their firing to the stimulus. It is possible that the projections from area TE and the parahippocampal cortex to brain pathways mediating perceptual affect and spontaneous visual selection integrate over large populations of cells in order to be most sensitive to wide, shallow activity distributions. On the other hand, connections to mnemonic systems are likely to be more specific in nature. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Predictions and Extensions 37 This theory has broad implications for many aspects of human behavior. Below, we will outline some predictions as they pertain to visual preference, learning likely sources of new information, the moment to moment control of eye movements, and the possible basis for individual differences in information foraging. In the next chapter, we will introduce visual preference as a method for testing this theory and present a review of known factors influencing visual preference. Figure 2.5 presents a series of predictions based upon hypothetical activity distributions in a p opioid gradient. Blank walls are not preferred because they result in little activity in this pathway. Uninterpretable inputs (such as random-appearing masses) are not preferred because they result in little activity in the endomorphin-rich anterior areas such as TE. Novel inputs that result in extensive interpretation and/or association maximize endomorphin release, and repetition of stimuli will result in less activity in areas high in endomorphin receptors. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. HYPOTHETICAL ACTIVITY DISTRIBUTION IMAGE Figure 2.5. Hypothetical activity distributions along the gradient of endomorphin receptors in the ventral visual pathway for visual scenes of increasing interpretability. If information acquisition itself can act as a reward, it seems reasonable that the dopamine brain systems underlying stimulus-reward associations(See Tremblay & Schultz, 1999; Schultz, Tremblay & Hollerman, 2000) should be able to learn that certain uninteresting behaviors or objects themselves can lead to large payoffs in information. For example, the act of waiting in line at a movie theater may not be pleasurable in itself, but is done in the service of Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. the likely payoff produced by seeing the movie. Similarly, people know that television, radio, the internet, cultural functions, other people, and cities are all likely sources of rich information, and may develop associations and habits around these things similar to those seen with items associated with food or drug reward. While these activities represent a deviation from the maximization of novel and richly interpretable information in the short term, they are performed in the service of this drive. A full characterization of the effect of the information foraging drive on moment to moment behavior will have to integrate the probability of reward over multiple time scales, though it should be noted that in many cases both long term payoffs and short term interest can be served simultaneously. A major aspect of information foraging is the real time control of eye movements about one’s visual environment. Unlike a camera, only the central region of an image on the retina is processed in detail. By saccading to regions of interest, the foveal region of the retina can maximize the amount of information gathered on each fixation (Barlow, 1961). The demonstration that p opioids often act via synaptic transmission (Simon & Hiller, 1994) allows that activation of an opioid gradient could have the temporal resolution necessary to influence selection mechanisms in real time. Perceptually driven affective responses would allow eye movements to track novelty and interpretability in a fast, feedforward manner that is not Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 40 dependent on extensive feedback from limbic areas nor lengthy cognitive deliberations. Several recent studies demonstrate that spontaneous eye movements are sensitive to this type of information. Using a gaze-contingent display in which a section of a scene could be moderately blurred as a fixation was released, Loschky, McConkie, Yang, and Miller (2001) showed that saccades in free viewing would avoid the blurred regions, even when they were in the periphery. This result could reflect the “ calculation” that it would be difficult to obtain good information from such regions. Other recent studies of eye movements while looking at scenes and artwork confirm that image factors in addition to salience influence the position of fixations (Oliva et al., 2003; Heidenreich & Turano, 2003). A key element for a successful information foraging strategy is the interpretability of a stimulus. A stimulus’ interpretability is heavily dependent on past experience and can differ from person to person. By providing a link between ventral visual activity and visual preference, our theory provides a basis for studying individual differences in preference. If a particular picture were to lead to more activity in endomorphin-rich areas in one subject than in the general population (for example when an art connoisseur views modern Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 41 art), we would expect that subject to show higher than average preference for that image A second source of individual variability in information foraging may be related to the rate at which an individual habituates to novel information. Some individuals are natural “sensation seekers” who get bored easily and are always seeking out new experiences.(Zuckerman, 1995). Others, according to Zuckerman, take a much more conservative approach to novelty, and are often quite content with that which they are familiar with. Zuckerman has proposed that sensation seeking is related to the activity of monoamine systems (particularly dopamine and norepinephrine; Zuckerman, 1995), suggesting that individual differences in sensation seeking could be the result of differing levels of monoamines, receptors, inputs to these systems, or enzymes responsible for their breakdown (such as monoamine oxidase, MAO). An individual who gets “bored” easily may have a hypoactive dopamine system or alternatively, a reduced input to the nigrostriatal dopaminergic system as a result of a decrease in the strength of cortical p opioid gradients. Tests of the link between perceptual affect and sensation seeking may have important consequences for the clinical treatment of anhedonia and depression. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 42 The neurocomputational theory outlined above has the potential of accounting for how interest can be expressed in real time in a perceptual pathway and why habituation occurs so that, as stimuli become familiar, once they are fully understood, there is less pleasure in perceiving them again. This theory assumes only that greater neural activity in perceptual pathways results in greater endomorphin release, and that the greater the endomorphin release, the more pleasurable the experience. W e predict that high levels of neural activity in association areas should be experienced as pleasurable. In addition to providing an input to the brain systems mediating the qualitative experience of pleasure, this activity would serve to inform frontal and subcortical mechanisms guiding spontaneous visual selection. The magnitude of this endomorphin activity would subserve perceptual and cognitive affect, resulting in a preference for patterns which are both novel (because they have yet to undergo competitive interactions), and richly interpretable (as such patterns would initially activate a rich set of associations). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. CHAPTER 3: VISUAL PREFERENCE 43 Our theory predicts that novel, highly interpretable stimuli produce high activity in temporal lobe structures and thus high endomorphin release, resulting in high visual preference. W e have developed a paradigm to measure the functioning of this system in which we measure the rated preferences of individuals as they view pictures of various degrees of initial interest. Background The literature on visual preference, while relatively sparse, indicates that visual preference may possibly be influenced by a number of factors, including stimulus complexity, previous exposure, arousal, emotional valence, and the evolutionary significance of a scene. Complexity vs. Interpretability A large number of studies from approximately 1950 to the present by Daniel Berlyne and others have investigated the role that “ collative” stimulus properties such as novelty, “complexity,” “surprisingness,” and “incongruity” have on a variety of different measures, such as subjective novelty ratings, Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. pleasingness, interestingness, and galvanic skin response (Berlyne, 1958, 1963; Aitken, 1974). This extensive series of studies used a large number of different stimulus types, including irregular polygons (such as those studied by Attneave), groupings of lines and dots, simple line drawings, and occasionally black and white versions of paintings. While Berlyne was quite successful in showing that these properties exhibited robust effects on a number of different measures, efforts to pin down exactly what determined stimulus “complexity” were unsuccessful. As a result, effects of complexity on preference were generally not achieved. For a relatively simple class of objects, such as irregular polygons, it appears that on average subjects find figures of low and high complexity, defined by number of sides, least preferable, while those somewhere in the middle are most preferred (Aitken, 1974). Typically, ratings of “interestingness” show a similar pattern, with a peak at slightly higher levels (Ibid). However, this inverted-U shaped function does not generalize to all types of stimuli, especially since Berlyne’s theory does not adequately explain what comprises “complexity” across different stimulus sets. From the perspective of opioid-mediated perceptual preference, we can see that it is not complexity per se that underlies preference but the extent of opioid activity. Very simple polygons might not allow many novel interpretations; highly complex ones may merely resemble texture masses Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 45 that also do not allow much interpretation. One can imagine that polygons with an intermediate number of sides might suggest some object or interpretation, thereby resulting in increased associative activity and, consequently, increased opioid activity. But as examples with droodles presented below will illustrate, it is not the complexity of the pattern that matters but the interpretability of that pattern. In general, the approach we have taken differs from that of Berlyne in that we do not attempt to define the “complexity” of our stimuli but seek instead to relate a stimulus’ preference to its interpretability. A stimulus which is highly interpretable is one which makes many contacts with memory. The sheer number of elements in a stimulus is not a direct measure of its interpretability. As an example of what is meant by interpretability, consider the cartoons, or “droodles,” illustrated in Figure 3.1. As a test for yourself, look at the drawings alone before consulting the caption for the interpretations. By themselves, these simple drawings are easily understood (and most definitely not visually complex) but not very interesting either. However, when a novel and meaningful interpretation is paired with the droodle, the experience is pleasurable. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 46 A D E t? j Figure 3.1. These “droodles” illustrate the principle that rich interpretability (in the absence of visual “complexity”) is pleasurable, a) A soldier and his dog walking behind a wall, b) A close-up of a pig reading book titles in a library. In 1968 Zajonc first reported that moderately complex visual stimuli which had been seen for very brief durations were preferred over equally complex stimuli that had not been seen at all. Many studies since that time (e.g. Seamon et al., 1984; Bornstein, 1992) have expanded on his findings to include a wide range of stimuli, and have demonstrated that the effect is robust (and perhaps even stronger) when subjects have no memory of having seen the stimuli. However, an equally large number of researchers have failed to find a robust effect of previous exposure under a large range of Effects of Previous Exposure Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 47 conditions, including ourselves (Vessel & Biederman, unpublished data), Bornstein (1989), and Berlyne (1970). W e attempted to replicate the effect of mere exposure on several classes of stimuli, including line drawings of objects, two-dimensional Fourier descriptors (smooth, closed outlines) created by Shepard & Cermak (1973), and, like Zajonc, irregular octagons generated using an algorithm described by Attneave & Arnoult (1956). In only one experiment out of 5 did our results show any evidence of better than chance performance on an affective choice task (i.e. which stimulus do you prefer?). Irregular octagons which were pre exposed ten times for 13.3 ms and followed by a 40 ms mask produced 57% liking and 55% recognition, collapsed over judgment order (95% confidence interval for liking judgment: 50.3 — 63.8). W e did not observe a benefit for affect over recognition, nor was affect significantly better than chance, for images pre-exposed one or four times. In addition, the results for affect were heavily dependent on both the order in which the subjects made their judgments (affect first or recognition first) and on the sex of the subject. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 48 c n Recognition Affect 012 34 56 7 89 10 01 Number of exposures 2 3 4 5 6 7 8 9 10 Block 1 Block 2 Figure 3.2. Results from a mere exposure experiment in which subjects indicated in a 2 alternative-forced choice paradigm which of two stimuli they recognized or preferred. As indicated by the blue lines, when both the recognition and the affect judgments occurred on the first block, ten exposures produced a preference for the previously seen image 58% of the time and recognition only 53% of the time. Notice, however, that the order in which subjects made the judgment interacted significantly with the effect: Block 2 recognition was slightly better than Block 2 affect at all three exposure levels. Importantly, the effects of exposure in these types of experiments are miniscule in comparison to the effects of a priori image preference. In a previous experiment, we observed a correlation of 0.87 between the average rank preference ordering given to the irregular octagons by a group of five subjects and the number of times each stimulus was chosen by a different group of twenty subjects in a mere exposure paradigm using forced choice preference judgments (regardless of the number of previous exposures). In the experimental results reported above, these rank orderings were used to create stimulus pairs which were matched on a priori preference. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 49 In an interesting 1970 paper entitled “Novelty, complexity, and hedonic value,” Berlyne showed subjects a large number of sequences of repeating stimuli of differing complexity, and found that all the sequences showed a highly significant general trend towards a preference for novel stimuli, counter to Zajonc’s mere exposure effect. The data from the first two of these experiments is shown in Figure 3.3. Condition four, which used differently shaped and colored stimuli on every trial, did not show this decrease. Only in a third experiment, in which a wide variety of very complex, different stimuli were used on each trial did Berlyne observe any increase in preference throughout the sequence. Finally, in a fourth and fifth experiment, Berlyne repeatedly assessed subjects preference for two “simple” stimuli (one nonrepresentational, one a portrait) and two “ complex” stimuli (one nonrepresentational, one a reproduction of a crowded painting with many elements), with intervening presentations between each judgment. He found that while preferences for the simple stimuli declined for each judgment, preferences for more complex stimuli initially rose and then fell in an inverted U shaped function (Berlyne, 1970). Berlyne interpreted these results as indicating that very complex stimuli, when they do not have to be rated each trial, are less susceptible to what he referred to as a “ tedium” factor. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. In 1990, Bornstein et al. published a similar observation in a paper entitled “Boredom as a limiting condition on the mere exposure effect,” highlighting the fact that mere exposure effects only seem to work with short experiments, using very few stimuli, shown for very brief exposures, with happy, well adjusted subjects. The overwhelming amount of evidence, including decades of preference habituation studies in infants, indicates that under most circumstances people show a preference for novelty. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 51 cim * i p m m t x m m m g r n * * !!} P M A5 £ ( *<*#** C U f5 * 9 t * % I ***SS 3r 4r 5TWUU EXP.S ( MTCMTSnmNESS) * * * * * * A * » * Figure 3.3. Judgments of A) preference and B) interestingness decline with repeated presentation with a variety of stimulus types and presentation conditions. The only exception are experiments in which novel patterns are used on each trial (bottom row of A and B). Individual data points showing a discontinuous jump in preference are from trials where a different pattern was presented. From Berlyne, 1970. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 52 Interestingly, recent EEG and FMRI work in visual priming may indicate that on a very short timescale, in the period before an object or scene has been recognized, priming might actually result in an increase in neural activity (James et al., 2000). If this is the case, then it is likely that mere exposure effects are the result of this transitory, early increase in visual activity during the first hundreds of milliseconds before a person recognizes an object. During this period, brief exposure increases the interpretability of stimuli, akin to Seamon’s concept of perceptual fluency (Seamon, Marsh & Brody, 1984), as neural activity propagates further through the ventral visual stream, but is not yet subject to competitive interactions. Within the FMRI literature, Bar et al. have shown that the focus of visual activity moves progressively anterior in the ventral pathway as subjects report increased knowledge regarding stimulus identity (Bar et al., 2001). More recently, Zago & Bar have shown that priming a 500 ms presentation of an object with a presentation of that same object leads to a change in the BOLD signal across multiple areas of the ventral visual stream that is dependent on the duration of the prime (Zago & Bar, 2003). Both behavioral priming and the observed suppression of the BOLD signal peaked between 150 to 250ms, with both shorter (40 ms) and longer (350, 500, or 1900 ms) primes resulting in less of a reaction time savings and less BOLD signal reduction than these peak times. If preference is coupled directly to activity in this pathway through the action of Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 53 endomorphin receptors, these studies would provide an explanation for why, with exposures which are not long enough to allow a subject to completely recognize a stimulus, preference for subsequent exposures increases as activity propagates further through the ventral visual pathway allowing for increased interpretability. Bar & Biederman (1999) proposed that the human homologue to monkey V4 or TEO is the locus of subliminal priming; the presence of moderate levels of endomorphin receptors in these areas could mediate the mild increase in affect observed for subliminal primes. Longer pre-exposure durations that allow for a stimulus to be recognized in a single presentation reliably produce decreases in preference on subsequent presentation. Using EMG as a physiological marker of positive affect, Winkielman & Cacioppo (2001) showed that increases in the degree of processing as a result of increased presentation time or the presentation of an outline prime elicit automatic, positive affective reactions. Once again, these effects are found to be rather weak. Due to the fragile nature of these biases, such priming based effects would be largely washed away by the later effects of preference habituation as a stimulus becomes familiar. Returning to Zajonc’s work, it appears as if the basic finding of increased affect for previously seen stimuli only operates within a relatively narrow domain; the major lesson to be learned from this body of work appears to be that preferences are Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 54 automatic, requiring no inferences or attributional operations (Zajonc, 1980, Winkielman et al., 1997). In the context of the opioid gradient account, it could be that the positive effects of brief exposures may be a consequence of their being processed to a greater extent in the ventral pathway than the control stimuli, but not to the extent that competitive learning would reduce their advantage. Arousal and Emotional Valence The next two factors shown to influence visual preference, arousal and emotional valence, have largely been studied in the context of fear conditioning and reactions to aversive stimuli. Cahill et al. (1996; O ’Carroll et al., 1999) has explored in depth the likely involvement of the amygdala and the influence of its beta-adrenergic projections on a subject’s ability to recall highly arousing, aversive scenes. Patients with damage to the amygdala do not show enhanced memory for highly arousing, aversive stimuli, and PET activity in the amygdala correlates with long-term, free recall (Cahill et al., 1996). In addition, blockade of beta-adrenergic activity at encoding also led to a reduction in recall for highly arousing, aversive stimuli (O ’Carroll et al., 1999). More recently, it has been claimed that amygdala damage may lead to a deficit in remembering the “ gist” of highly arousing negative scenes, but not for remembering visual details (Adolphs et al., 2001). In addition to Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. certain amygdala nuclei, the orbitofrontal cortex, the temporal pole, and the superior frontal gyrus are also likely to play a role in the reaction to emotionally valent stimuli (Royet et al., 2000). It is likely that this system, which has a very high evolutionary value, is separate from a putative information foraging system. While the amygdala has been shown to be involved in positively reinforcing behaviors such as food reward (Rolls, 1999), it is unclear what role it may play in information foraging. For these reasons, we have chosen to use stimuli which do not depict highly arousing negative scenes nor erotica, such as those in the International Affective Picture System (IAPS) developed by P. Lang at the Center for the Study of Emotion and Attention (e.g. Lang et al., 1997a). Our focus also differs significantly from that of the IAPS stimulus set in that we aim to investigate visual preference within the larger context of information foraging across many scene types, and not just as it pertains to particular objects with high emotional content such as happy babies or mutilated bodies. While it may be the case that visual orienting responses and certain physiological responses to these stimuli still obey the general framework of our theory of perceptual affect (Lang et al., 1993), as noted previously, other motivational systems would be expected to override that of information Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 56 foraging. Support for a dissociation of information foraging and the fear response comes from a recent study of the rate of memory formation for scenes with positive and negative affective content, which reported separable contributions of valence and arousal (Maljkovic & Martini, 2003). Using rapid serial visual presentation (RSVP) of scenes covering a range of emotional valence, they found that the rate of memory acquisition (as measured by the performance on an old/new recognition test for sequences of varying exposure duration) was related to valence. Highly arousing scenes of either valence were better remembered overall, but negative scenes were poorly remembered at the shortest presentation rates and increased steeply, while positive and neutral scenes showed constant increase in memory performance with increasing exposure. The authors concluded that highly negative scenes may lead to an initial freezing response with later accelerated processing. These conclusions agree with our proposal that highly informative scenes, regardless of their emotional valence, will initially lead to a greater positive affective response, though this initial response may then be superceded if the stimulus engages another drive state such as avoidance or sexual interest. Of greater interest to our work, however, is the likelihood that evolution has shaped our preferences for certain visual scenes within the more neutral Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 57 domains of navigation, monitoring, and habitat selection. Chapter Five will deal extensively with this topic. A Stimulus Set for Testing Positive Perceptual Affect As a method to test the functioning of an information foraging system and its links to perceptual affect, we have developed a stimulus set of full color scenes for use in visual preference experiments. Given the wide range of interpretability of different classes of stimuli, we chose to focus our efforts on understanding how preference varied within a set of stimuli which were all of relatively high interpretability. W e used full color, natural scenes likely to activate the entire ventral visual pathway in an attempt to maximize the range of any preference effect. Calibration Experiment Based upon previous reports of the reliability and intuitive nature of preference judgments (Berlyne, 1970; Zajonc, 1980), we selected a simple preference rating task as our basic behavioral measure. Our first step was to gather baseline preference data on all of the images in our stimulus set. In Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 58 addition, we chose an exposure duration of one second, which was clearly long enough to allow an inspection of each scene sufficient for a rich interpretation, if one was to be had.1 Stimuli Using a combination of images collected using a digital camera and the MasterClips Premium Image Collection from IMSI2 we selected a set of 200 full color scenes which varied widely on a number of dimensions, such as natural vs. urban/man-made, close-up vs. scenic, animate vs. inanimate, and cluttered vs. simple. All images were cropped to 640x480 pixels using Adobe Photoshop and saved as high quality rgb jpeg images.3 Subjects Twenty subjects participated in this experiment either for credit in an undergraduate psychology course or as a paid volunteer. Informed consent was obtained in accordance with the guidelines established by the University 1 At 100 ms, subjects can readily determine the setting of a scene and the interactions and roles of its major entities (Potter, 1976; Biederman, 1981; Biederman, Mezzanotte & Rabinowitz, 1982). 2 IMSI MasterClips© and MasterPhotos™ Premium Image Collection, 1895Fransisco Blvd., East, San Rafael, CA 94901-5506, USA. 3 The full stimulus set is available online at http://geon.usc.edu. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 59 of Southern California Institutional Review Board (IRB). The subjects consisted of ten males and ten females, all of which were right handed. The mean age was 20.3 years, and all subjects had normal or corrected to normal vision. Method Subjects were instructed that they were taking part in an experiment designed to investigate how people make judgments about scenes. They were shown each scene in our full image set for one second each and asked to indicate how much they liked the scene on a seven point scale. They were to give the scene a six or seven if they found it very visually interesting and enjoyed looking at it and to give the scene a one or two if they found the scene boring, uninteresting, or if they didn’t like looking at the scene. If they had no strong feeling either way, they were to give the scene a middle rating. The instructions stressed that a good way to think about this judgment was to ask themselves “how much would I like to see this picture again?” The images were shown to the subject on a Sony Trinitron display controlled by a Macintosh G3 computer running Matlab and the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997). The scenes were shown at a resolution of 82 pixels per inch, creating images which were 7.75 x 5.875 inches on the Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 60 screen. At a distance of 32 inches, these scenes subtended approximately 14 degrees of visual angle horizontally. Following an adaptation period during which the subject’s eyes were allowed to adjust to the dark room, each trial began with a fixation point followed by a scene for one second, displayed on a mid gray background. The order of presentation of the scenes was randomized across subjects, with the even numbered subjects seeing the reverse sequence of the odd numbered subjects, resulting in the same average serial position across subjects for all scenes. After the scene disappeared, a prompt appeared on the screen which read “Enter Preference: 1 (don’t like it) ^------> 4 <------ > 7 (really like it).” The subject responded using the numeric keypad on the computer keyboard, and was given unlimited time to respond. Subjects were instructed to rate each image based on their preference for that image at that moment, regardless of the ratings they had given on other trials, and to try and use the entire seven point scale. When the subject responded, the prompt disappeared, the screen remained blank for 500 milliseconds, and the next trial began. Subjects were given the opportunity to take a break after every 40 trials. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 61 Results Any errors or omissions on the part of the subject were recorded by the experimenter and corrected before analysis. Average preference ratings were computed for all 200 scenes. The average preference rating across subjects was 3.97 + 0.62, and the average variance in ratings across subjects was 2.62. Figure 3.4 shows the distribution of preference scores for the whole image set, and the normal probability plot in Figure 3.5 indicates that this distribution is roughly normal across the rating scale. The six most and six least preferred scenes are shown in Figure 3.6 along with their average rating. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 62 Distribution of Average Preference Scores N = 200 Images Average Preference Score Figure 3.4. The distribution of average preference ratings across twenty subjects for the full 200 scene image set. Normal P-P Plot of Average Preference Ratings For 200 Im ages 1.00 - P -75 L L I 0.00 0.00 .25 .50 .75 1.00 Observed Cumulative Probability Figure 3.5. A plot of the expected cumulative probability versus observed cumulative probability for the average preference ratings of all 200 images, showing only slight deviations from normality. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 63 6.10 6.10 6.00 A 5.85 5.85 5.85 2.20 B 2.20 2.40 2.40 Figure 3.6. The a) six most and b) six least preferred scenes in the calibration experiment. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 64 It is clear that the preferred and non-preferred scenes differ in a number of ways. Perhaps the most obvious from the examples shown in Figure 3.6 is the prevalence of natural vistas in the set of preferred scenes and the number of more man-made, urban elements in the low preferred scenes. In addition, the low preferred scenes commonly contained a large amount of clutter (e.g. the pile of garbage) or texture, often as the result of many repeated elements (e.g. the pile of bricks or rows of chairs). A measure of cross-subject consistency was calculated by correlating a single subject’s preference ratings on all 200 images with the mean of the other 19 subjects. The mean of the square of these correlations gives an indication of the overall amount of variance in the ratings that can be attributed to the stimulus set: on average, 31% of the variance in an individual subject’s preference scores can be predicted by the group mean for the stimulus being shown. In a set of subsequent experiments, we used these ratings to explore 1) what aspects of the images might account for these differences in initial preference, 2) how these preferences change with repeated presentation, and 3) how the ventral visual stream differentially responds to images of varying preference. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. CHAPTER 4: REPETITION 65 The theory we presented in Chapter Two predicts that novel, highly interpretable stimuli should be preferred as they lead to the most activity in the anterior regions of the ventral visual pathway. Here we present two experiments which tested the effects of stimulus repetition on rated preferences. Our results provide support for the hypothesis that repetition decreases preference, and does so in a consistent manner for scenes of varying initial interest. In addition, these experiments allowed us to develop a stimulus set and experimental parameters for use in brain imaging experiments. A major hurdle to overcome in the experimental design was to control for the local novelty environment such that scene novelty was not confounded with time (or trial number) in the experiment. That is, it should not be the case that novel scenes occur early in the session and repeated scenes later. In each of the experiments reported below and our imaging experiment, measures were taken to deconfound repetition and trial number. Given the demonstration by Berlyne that the preference for a novel image presented in the context of repeated images is increased above what would Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 66 be expected amongst a sequence of other novel images (Berlyne, 1970), we also sought to control the “preference context” of our images. Experiment 1: Effects of Repetition, Between-Subjects Design In this first experiment, our goal was to assess the effects of multiple repetitions on stimulus preference over the entire set of 200 stimuli. In order to assess the limits of repetition effects, we chose a design which employed a large number of repetitions but only allowed for each subject to see a subset of the total set of images. Methods Subjects Forty volunteers participated in this experiment either for payment or credit in psychology courses. The procedures for informed consent were followed in this and all subsequent experiments unless noted otherwise. The sample consisted of 11 men, 29 women, with a mean age of 19.8. Information on handedness was not collected. None of the subjects had previously seen any of the images^ and all subjects had normal or corrected-to normal-vision. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 67 Stimuli Stimuli consisted of 200 full color scenes selected to vary on a wide number of dimensions and independently rated for a priori preference (see the Calibration Experiment in Chapter Three for details on the stimuli). Procedure Each subject saw a subset of thirty images with a full range of a priori preference values (based on the pretest ratings), grouped into three sets of ten images. Each of these lists covered the range of preference ratings from high to low. The first set was used as “practice” stimuli at the beginning of the experiment and were subsequently used as “spacers” throughout the test trials. The second set was used for “Block 1 during which a new image was introduced within each group of ten trials and then repeated in each subsequent group of trials up to ten repetitions. The third set was used for “Block 2,” which continued to introduce new images at a controlled rate, but did not result in a full ten repetitions for each image (see Fig. 4.1). This trial sequence controlled for the average serial position of the scenes, the local novelty context (novelty was not confounded with trial number for the Block 1 images), and the preference context (based on the pretest results). Over the set of forty subjects, the design allowed each image to be evaluated as a Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 68 Block 1 image by two subjects. The instructions, trial timing, and the subject’s task were identical to that of the calibration experiment. “Practice” rv r\ a Test A random seauence 20 Blocks of 10 trials, randomized: sequence of: Bl-1 Bl-2 Bl-3 Bl-10 B2-1 P-l shown 1 time P-l Bl-1 Bl-2 Bl-9 Bl-10 P-2 shown 2 times P-2 P-l Bl-1 Bl-8 Bl-9 P-3 shown 3 times w P-3 P-2 m m P-l ■ ' * Bl-7 m Bl-8 ■ » F-10 shown 10 times • ■ P-9 P-8 • P-7 * Bl-1 • Bl-2 times Figure 4.1 . The trial sequence was designed in such a way as to maintain the local novelty context while achieving ten full repetitions of the Block One images. Images labeled as P were part of the practice list, B 1 the Block One list, and B2 the Block Two list. Results The mean preference rating was 4.22 + 0.12. No significant change over the course of the test block occurred, indicating that the design of the trial sequences adequately controlled for the novelty context (see Fig. 4.2). As can be seen in Figure 4.2, the control for novelty context was very much needed insofar as there was a decline in preference over the “Practice” Block (first three trial blocks), where novelty was confounded with trial block. For Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 69 the Test Blocks (Fig. 4.3), the average rating given for the first exposure of Block 1 images was 5.21, which declined steadily over the ten exposures of those same images to 3.6. It is evident from Figure 4.4 that there is a consistent decline with repetition for all preference levels, except for a floor effect with the least preferred images. On the other hand, highly rated scenes may be asymptoting well above floor. Preference vs. Trial Number ij— CL Practice" Test Block (Block 1 images evenly distributed) Block (Error bars are standard errors of subject means) * 1 " 1 1 I "I 1 1 I I I 'T ' ' I ' I < N =40 0 1 2 3 4 5 6 7 8 9 1 0 11 1 2 1 3 Trial Block (20 trials each) Figure 4.2. The average preference ratings over the course of the test block remained constant. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 70 a > 6 o u U ) a) 5- u c a > £ a) Q . ^ n J cn 3- ro !_ a) > < 2 Average Preference vs. Exposure . N = 40 (Error bars are averaqe standard errors of the subject means') H 1 ------------ 1 ------------ 1 ------------ 1 ------------ 1 -------------1 ------------ 1 ------------ 1 -------------1 -------------1 — 0 1 2 3 4 5 6 7 8 9 Exposure # Figure 4.3. Average preference declined over repeated presentations. 6 r 7 10 Block 1: Average Preference Score vs. Exposure, Grouped by Initial Preference Number of images given this initial rating J4 ® 5 Q _ C D 3 O) > 2 -1 ► 40 subjects (Error bars are standard errors) 1 2 3 4 5 6 7 8 9 1 0 Exposure # Figure 4.4. Notwithstanding floor effects, average preference declined over repeated presentation for all images regardless of initial preference. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 71 The pattern of results for the Practice images and Block 2 images was similar, though with a bit more variability due to the unequal number of presentations of each image in Block 2 and less than optimal balancing for the Practice images (Fig. 4.5 & 4.6). Practice: Average Preference Score vs. Exposure, Grouped by Initial Preference Number of images given this initial rating 60 57 CL 45 cn (error bars are standard errors) N = 40 9 7 8 10 5 6 4 2 3 1 Exposure # Figure 4.5. Preferences for the Practice and Block Two images showed a similar pattern to that of the more controlled Block One data. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 72 Block 2: Average Preference Score vs. Exposure, Grouped by Initial Preference Number of images given this initial rating C l cn 21 24 N =40 2 3 4 5 6 7 8 9 10 1 Exposure # Figure 4.6. Preferences for the Block Two images showed a similar pattern to that of the more controlled Block One data and the Practice block. The Block Two data does not contain error bars since the images were used to fill in the end of the experiment, resulting in fewer data points as one progresses to later exposures. In order to evaluate the consistency of this decline over individual images, we computed the linear and quadratic trends for all 200 images. The two sets of ratings obtained for each image as a Block 1 stimulus were first averaged before being submitted to a hierarchical regression analysis and tested for significance using a two-tailed F test. Out of the 200 images, sixty-two did not show a significant trend at the 0.05 level. 128 images showed a significant negative linear trend, while only a single image showed a Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 73 significant positive trend. Thirty three images showed a significant positive quadratic trend (falling then rising) and ten had a significant negative quadratic trend over and above any linear trend. These data indicate that the overwhelming effect is for images to be less preferred as they are repeated. W e also performed a trend analysis on individual subjects to evaluate the universality of the observed decreases in preference. Out of the forty subjects, eight did not show a significant change in preference with repetition. The remaining thirty-two subjects all showed a significant negative linear trend. The data from twelve of these subjects also showed an additional quadratic component, all but one of which was positive. Conclusions from Experiment 1 From this experiment we can conclude that under these experimental conditions, preference declines with repeated presentation. This habituation occurs for images with a wide range of a priori preference. While the data seem to indicate that preferences decline at a relatively consistent rate for all levels of initial rated preference, we cannot draw any quantitative conclusions regarding the rate of habituation as a function of initial preference because we cannot assume that the preference judgments are made on an interval or ratio scale. A physiological measure of preference such as eye fixations or Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 74 the monitoring of facial muscles using EMG would allow for one to quantitatively assess the effects of repetition and a priori preference, though the problem remains that these measures may not be linearly related to the underlying psychological variable of preference. In Chapter Seven, we outline how this “ win-shift” behavior might be explored within a preferential looking paradigm to avoid the use of a subjective rating scale. From the practice blocks of the trial sequence data (Fig. 4.2), it is clear that preference is influenced by the local novelty context, which we term a “contrast” effect. Additionally, the subsequent rebound and leveling of preference during the trial blocks indicates that an intermixed design such as the one used here is an adequate control for such effects. There are also several aspects of these results that are important when considering possible designs for an FMRI experiment. First, a significant habituation effect was seen in as few as five exposures, allowing us to drastically reduce the number of individual image presentations. Second, a priori preferences were relatively stable across subjects, but not perfectly predictable. In Experiment 2 we overcame this by selecting a set of images with low variance in their cross-subject preference scores. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 75 Furthermore, reducing the number of preference levels from seven to three, a low, medium and high preference group, allows us to have good control over a subject’s preference for an image without having to collect behavioral responses during an imaging session. Such a design allows us both to scan our subjects in a “naive” state (one in which they are not making explicit preference judgments) and to use a fully factorial experimental design. Experiment 2: Selection of image sets for FMRI In two preliminary experiments, we created three groups of images with consistent a priori preferences across subjects to test the predictability of preference ratings from a priori image preference and repetition. In addition, the use of a fully factorial design allowed us to test for the presence of an interaction in the ratings between a priori preference and repetition related changes. As noted previously, because we could not assume an interval or ratio scale, no inference could be made as to any processes responsible for the interaction or lack thereof. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Methods 76 Subjects Two groups of twenty volunteers participated in this experiment either for payment or credit in psychology courses. The first sample consisted of 6 men and 14 women, 4 of whom were left handed, and 16 right handed. The mean age of the sample was 21.1 + 2.9. The second sample consisted of 4 men and 16 women, all of whom were right handed, with a mean age of 19.3 + 1.5. None of the subjects had previously seen any of the images, and all subjects had normal or corrected to normal vision. Stimuli Using the preference ratings from the calibration experiment, the full set of two hundred images was ranked according to preference and variance. Importantly, these rankings were based upon the data from only the first twelve of the twenty subjects in the calibration experiment, as the final eight were collected at a later time. Therefore, there were slight differences between the preference ordering used for this first pass image selection and the final values used as a priori preferences. A subset of twenty images was selected from each of three ranges: 5 to 7, 3 to 5, and 1 to 3. In addition to Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 77 the mean preference score and low variance constraint, images were selected to minimize the semantic and visual similarity of the entire image set so, for example, there was only one parking lot scene and one bedroom. An additional twenty four images were selected to serve as buffers at the beginning and end of the experiment. Figure 4.7 shows the distribution of these three image groups across the range of a priori preferences computed from the full set of twenty subjects in the calibration experiment. Distributions of Preference Ratings by Image Category - - High Medium Low Buffers High m Medium Low LA.lm UHnlI Preference Figure 4.7. Distribution of a priori image preferences (based on all twenty subjects from the calibration experiment in Chapter Three) for the stimuli used in Experiment 2a. The bars show the number of images at each preference level for the three image categories, while the lines show fitted normal curves for each of the three distributions and the buffer images. Note that the images were selected based on the average ratings from only twelve of the subjects in the calibration experiment. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 78 Following Experiment 2a, 13 of the stimuli were replaced following an analysis of their preference ratings in order to achieve an image set with more consistent ratings. This new set of 60 stimuli were used in Experiment 2b. Figure 4.8 shows the distribution of a priori preference ratings for this set of images based upon the calibration experiment. The decision to keep certain images that overlapped in a priori preference with images of another category was based upon the image ratings from Experiments 1 and 2a. Distributions of Preference Ratings by Image Category / \l - - High ----- Medium ----- Low 1 I II 1 Buffers ■ 1 High 1 1 Medium Low a > £ D o 75 E o z T 3 0 Preference Figure 4.8. Distribution of a priori image preferences (based on all twenty subjects from the calibration experiment in Chapter Three) for the stimuli used in Experiment 2b. The bars show the number of images at each preference level for the three image categories, while the lines show fitted normal curves for each of the three distributions and the buffer images. Note that the images were selected based on the average ratings from only twelve of the subjects in the calibration experiment. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 79 Procedure The instructions and trial timing were identical to that used in the calibration experiment and Experiment 1. Unlike Experiment 1, however, all subjects saw all images in a within subjects, full factorial design. The experiment began with the presentation of a sequence of buffer images designed to pre expose the subject to three images once, a different three images twice, and other sets three and four times. This sequence of thirty images (3 + 6 + 9 + 12) was followed by twenty four blocks of fifteen images. Each block of fifteen images contained one image in each of the fifteen conditions: three preference levels by five exposures. For the first four of these blocks, the images comprising the latter repetitions were taken from the set of pre exposed buffers. During each block of fifteen trials, a new high, medium, and low image were introduced and subsequently repeated on successive blocks until all twenty stimuli in each preference category had been shown five times. The last four blocks of trials also required the interjection of buffer images to allow for the experimental images introduced last to be shown the second, third, fourth, and fifth times. This design was necessary to maintain a constant novelty and preference context throughout the entire experiment and deconfound trial number with repetition number. Within each block of fifteen trials, the condition order was counterbalanced and verified to prevent the repetition of an image at the borders between blocks. The order of image Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 80 introduction was different for each subject, with one subject receiving the images in the opposite direction as another to control for the overall position of each image over the entire set of subjects. Results To evaluate the effectiveness of our experimental design on controlling the local novelty environment, we evaluated the change in preference over the course of the experiment averaged over blocks of fifteen trials (Fig. 4.9). An omnibus F-test for any differences among the trial blocks revealed a significant difference for Experiment 2a, F(25,475) = 2.65, p < 10"4. This test may be too sensitive to random fluctuations within a the sequence due to the extremely high degrees of freedom in the error term. A more appropriate test of any significant linear trend over the course of the experiment found a significant slope of -0.18 (t = -4.22, p < 10'4. Therefore, there did appear to be some overall decrease in preference over the course of this experiment. However, it is not likely that this decrease was responsible for the decrease in preference over repetition, as the magnitude is much less. Over five blocks in the experiment, the overall preference went down by 0.13 units on average, while the change as a result of image repetition was 1.1 units over five repetitions (collapsed across image preference). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 81 The F-test for block differences in Experiment 2b (which contained the actual stimuli used in the imaging experiment) was not significant, F(25,475) = 1.02, p = 0.44. A Change in Average Preference over Trial Blocks <2 Standard error of the subject means N = 20 Block (15 trials each) B Change in Average Preference Over Trial Blocks 7 ii _ 2o Standard error of the subject means 5 10 15 20 Block (15 trials each) Figure 4.9. No overall change in average preference scores was observed in either a) Experiment 2a, nor b) Experiment 2b. The dashed lines above and below the datapoints indicate the standard error of the subjects’ mean preference rating for each block of fifteen trials. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 82 Figure 4.10 shows the results from Experiment 2a. Preference declined with repeated presentation for images at all levels of a priori preference. In addition, the average preference of the high, medium, and low image groups showed good separation in the group data at all repetition levels. A two-way analysis of variance (ANOVA) revealed main effects of preference (F = 127.7, p « 1x10'G ) and repetition (F = 25.4, p < 1x10‘6) with no significant interaction (F = 1.8, p = 0.085). P referen ce vs. Exposure fo r Im a g e s C a te g o rize d by A P rio ri P referen ce “ High ™ Medium " A * ■ Low 3- Error bars are pooled standard errors of the subject condition means______ N = 20 3 5 2 1 4 E xposure # Figure 4.10. Results from Experiment 2a. Preference declines with repeated exposure for all a priori preference groups. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 83 Figure 4.11 shows data from eight of the individual subjects from Experiment 2a. Clearly, the separation of image categories and decline with repetition are present in individual subjects, though to varying degrees. Using the preference ratings from Experiment 2a, we found it necessary to make several changes in the stimuli used in each category. These changes were aimed at maximizing the separation between each of the three preference groups. A total of thirteen images were replaced from the original image set of sixty images. Figure 4.12 shows a different set of twenty subjects run on this new set of images. There were highly significant effects of both preference (F = 231.5, p « 1x1 O'6) and repetition (F = 39.5, p < 1x1 O'6), and an interaction approaching significance (F = 1.9, p = 0.058). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 84 Preference vs. Repetition for Eight Individual Subjects High Medium - A - Low 6n. 1 2 3 4 5 1 2 3 4 5 1 2 3 4 4 5 Exposure # Figure 4.11. Data from eight individual subjects in Experiment 2a. All subjects showed consistent effects of repetition and a clear separation of different image groups. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 85 Preference vs. Exposure for Im ages Categorized by A Priori Preference ■ “ • I “ High *■*•■** Medium Low cn C l Error bars are pooled standard errors of the subject condition means l\l = 20 2 3 5 1 4 Exposure # Figure 4.12. Results from Experiment 2b. This experiment replicates the finding with a different set of twenty subjects and a slightly different image set that preference declines with repetition for all preference groups. Using the forty-eight images which were common to both Experiments 2a and 2b, we calculated the correlation of the average preference score for each exposure across the two experiments. These values are shown in Table 4.1. The overall correlation was 0.939, with the average rating on the first exposure reaching a correlation of 0.954. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 86 Correlation Between Average Preference Ratings for Experiments 2a and 2b Exposure 1 2 3 4 5 Correlation (Pearson’s r) 0.954 0.934 0.942 0.931 0.931 Table 4.1. Average preference ratings for the forty-eight images common to Experiments 2a and 2b are highly correlated across all exposures. Both experiments were then analyzed for the effects of gender on preference judgments. Figure 4.13 shows the data from Experiment 2a by gender, and Figure 4.14 shows the effects of gender in Experiment 2b. A significant interaction of gender with repetition was found for Experiment 2a (F = 5.2, p < 0.001) and this trend approached significance in Experiment 2b (F = 1.6, p = 0.11). While the initial image ratings did not differ, men showed a greater decline in preference with repetition. Sex Differences in Preference vs. Exposure C D C 4 -J 2 5 < D U C £ 4 £ < u u C L c 3 r u < D —• - High - M Medium - M Low - M High - F — — Medium - F - » > Low - F . IN = 20 (Error bars are standard errors of the subject means) H ------------1 ------------1 ----------------------------------------------- i i i " 1 2 3 4 5 Exposure # Figure 4.13. In Experiment 2a, initial preferences were not significantly different for men and women, but men showed a larger effect of repetition. This interaction was highly significant (F = 5.2, p < 0.001). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 87 Sex Differences in Preference vs. Exposure “ High - M ~ Medium - M n ■ Low - M O l - High-F ■ ■ Medium - F • L O W - F c 3- (Error bars are standard errors of the subject means) N = 20 2 3 5 1 4 Exposure # Figure 4.14. In Experiment 2b, initial preferences were not significantly different for men and women, but men showed a trend towards a larger effect of repetition. This interaction was not significant (F = 1.9, p = 0.1). Finally, we also performed a regression analysis on the group data from Experiment 2b using the ratings obtained in the calibration experiment. In this analysis exposure number and a priori preference accounted for 88% of the variance in the group average preference ratings. This analysis also found no evidence for any interaction of a priori preference and the effects of repeated exposure. A similar analysis performed on the data from individual subjects found was able to account for 46% (+ 11%) of the variance in individual preference ratings on average. Since we will not be collecting preference ratings in the magnet, this value provides an upper bounds estimate of the expected reliability of our FMRI results. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 88 Conclusions from Experiment 2 This experiment confirms that preference habituation is reliable and occurs for images of differing initial preferences. The selection of images into high, medium, and low preference categories produces stable differences in preference which persist over multiple repetitions. In addition, large effects of repetition can be seen in five or fewer repetitions. Average preference ratings from one group of subjects allow for excellent prediction of a second sample, as well as decent prediction at the single subject level. This design, therefore, should be powerful enough to produce observable changes in the FMRI BOLD signal without having the subjects make explicit preference judgments. While the group averages are quite well behaved, there are clearly some differences in individual subjects. Overall, the pattern of results is the same for all subjects, differing only in the rate of habituation and the exact preferences for specific images. While initial ratings do not differ by gender, men appear to show a faster rate of habituation than women. However, a more rigorous test of this effect (with equal numbers of men and women) is needed. An interesting possibility is Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. that the faster rate of habituation for men is related to sensation seeking,; men typically score higher on tests for this trait (Zuckerman, 1995). Overall, these two experiments provide strong evidence for the repetition expectations of the theory presented in Chapter Two. Novel images are consistently preferred, with repetition leading to a predictable decrease in preference for images of a wide range of a priori preference. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. CHAPTER 5: ENVIRONM ENTAL UTILITY 90 As is evident from the calibration of our image set presented in Chapter Three and the subsequent tests of preference changes over repetition in Chapter Four, different samples of subjects show a high degree of consistency in their mean preference ratings. From this consistency, one can surmise that there are certain aspects of visual stimuli which activate common brain mechanisms subserving visual preference. In this chapter, we present experiments and analyses focusing on attributes that determine initial scene preference. Importantly, this analysis is driven by an evolutionary account of scene preference, which leads to a better understanding of why particular features, patterns, and arrangements present in a scene make it preferred, and gives us clues as to how a “dumb” mechanism such as natural selection could lead to the development of a general purpose system subserving perceptual affect and information foraging. An Evolutionary Account of Scene Preference While it is obviously true that there is large variation in the particular paintings, pictures, clothing, and movies people find most enjoyable, certain Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 91 factors have been found to be highly reliable predictors of the visual patterns that a person finds appealing. Research in the fields of landscape assessment, environmental aesthetics, environmental psychology, geology, and architecture have examined a wide range of both low level and content- related factors using stimuli as diverse as classic paintings, geometric shapes, photographs of real and artificial scenes, and drawings of trees, buildings, and landscapes. Within the field of landscape assessment, one can find on the order of fifty separate experiments which have correlated subjects’ preference ratings with judges' ratings of how well certain factors describe a set of scenes. A good majority of these studies find that such factors can account for greater than fifty percent of the variance (and as much as eighty-five percent in some cases) across a wide range of image categories and experimental paradigms (Kaplan & Kaplan, 1995). In his 1984 book, Biophilia, Edward A. Wilson put forward the hypothesis that human beings have an innate emotional affiliation towards other living organisms. According to this idea, the positive (biophilic) and negative (biophobic) biases that humans have for orienting towards, remembering, and responding to biological stimuli are largely genetically programmed. The range of phenomena that might potentially be explained by this hypothesis includes the tendency for people to decorate their offices with plants and nature paintings, the mass exodus from cities that occurs every year during Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 92 Labor Day and Memorial Day, the high cost and desirability of property near the ocean or on a hilltop, the widespread phobia of snakes and spiders, and the supposed healing powers of Japanese zen gardens! The underlying notion of biophilia is that these genetically programmed biases are expressed behaviorally as preferences and phobias for certain types of environmental stimuli (Kaplan, 1992; Heerwagen & Orians, 1993, Wilson, 1993). More specifically, Ulrich proposes that positive and negative associations with certain stimuli are the result of biologically prepared learning: while it may not be the case that there is a completely innate tendency to respond a particular way, a few instances of positively or negatively reinforced encounters with such stimuli are enough to form lasting associations (Ulrich, 1993). These associations differ from those with stimuli that were not significant during evolution. Such stimuli typically require much more extensive exposure for their conditioning and the conditioned responses are more readily extinguished. A frequently cited example is the difference between the long lasting negative associations often formed with snakes, an evolutionary threat that has little relevance to today's city dwellers, and the relative weakness of associations made with handguns, which are a far more real threat in modern times (Ulrich, 1993). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 93 While the underlying premise of biophilia probably holds true for all the senses, the close link between vision and navigation places a special emphasis on the likely effect that natural selection has had on shaping our visual preferences for particular places, objects, and patterns. Stephen Kaplan, in a 1992 article entitled “Environmental preference in a knowledge- seeking, knowledge-using organism,” explores the notion that the evolutionary significance of landscape arises from the need to unconsciously assess one’s environment quickly while not straying too far from the known: In this way preference would help keep the individual in an environment where orientation and access to new information can be maintained easily - quite apart from the particular purposes that individual was pursuing at that moment. (Kaplan, 1992, 595) Human beings have been living in industrialized societies for less than three centuries, and as recently as ten thousand years ago, were entirely nomadic, roaming the savanna in search of food, water, and shelter from predators and the elements. Under such conditions, any variation in the genome which might convey an advantage for finding habitats with access to these resources (and protection from threats) would become highly conserved in successive generations. Kaplan provides evidence that certain factors predicting environmental utility, such as how well an individual can view the environment without being seen, might be good predictors of rated preferences for natural settings. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 94 This idea was first formalized in Jay Appleton’s “Prospect and Refuge” theory, proposed in his 1975 book The Experience of Landscape. Appleton’s studies of British landscape paintings led him to conclude that the main components of landscape preference are Prospect (having a grand view allowing one to see the layout of the land and approaching threats), Refuge (having a safe hiding place from which one can see without being seen), and Hazard (containing an aspect which might present a danger or threat, such as a precipice; Appleton, 1988). Interestingly, this third aspect was largely ignored, possibly since it afforded a negative bias for a particular environment. Appleton favored an evolutionary interpretation of this theory, stating that a preference for such environments would confer an adaptive advantage, and saw his theory as providing a much needed functional interpretation for data on environmental aesthetics. Heerwagen & Orians have interpreted Appleton’s results and theory as support for the existence of a general mechanism subserving habitat selection, or more specifically, habitability assessment (Heerwagen & Orians, 1993). They argue that humans have innate preferences for those environments which are similar to those which were most habitable during human evolution, namely, the savanna. The wide open spaces allowing for easy assessment of nearby animals, shelters, water sources, and predators, combined with the relative ease of harvesting vegetation from low lying trees provided an ideal Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 95 environment for a bipedal animal with free swinging arms and upright posture. Active mechanisms for seeking habitable locales were necessary. Rarely did habitats occupied by humans during most of our evolutionary history provide reliable resources long enough to enable permanent occupancy of sites. Frequent moves through the landscape were the rule even though traditional sites might be revisited on an annual basis. (Heerwagen & Orians, 1993,140) Evidence for such preferences agrees with the argument made by several researchers that evolution has not endowed us with hard wired preferences, but instead acts through subtle biases which interact with a person’s experience. Kaplan cites a 1982 study by Balling and Falk which reported that young children (eight and eleven years old) show a substantial preference for savanna over all other biomes, but that slightly older children (age fifteen) show an equal preference for savanna and hardwood forest, the dominant biome in the area in which the study was conducted (Kaplan, 1992). All animals must balance the opposing drives for exploration of an unfamiliar environment with the protection afforded by staying put, and these genetic biases likely reflect natural selection’s optimal solution for suitable prior assumptions as to what cues are worth exploring and when it is time to leave an overused habitat. Not only does exploration of the environment come at the cost of possible safety, but in extreme cases, it is also at odds with an exhaustive understanding of one’s immediate environment. Stephen & Rachael Kaplan Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 96 provide a preference model which incorporates both of these aspects of information gathering (understanding and exploration) within a framework that also makes a distinction between information which is immediately available in the sensory input and inferred sources of information. This gives rise to four factors which may act as predictors for preference (Kaplan, 1992). The first of these factors, coherence, is the ability to immediately understand the sensory input; it is the ease with which one can grasp the organization of a scene. The second factor, legibility, refers to the degree to which a scene lends itself to navigation, and draws on an understanding of the three dimensional structure of the scene (and is hence an inferred understanding). The third category, complexity, refers to the number of elements in the scene, and is intended to capture the immediate comprehension of the scene as related to exploration. The final factor, mystery, captures the inferred aspect of scene exploration: a scene high in mystery is one which holds the promise for more information if one were able to explore the scene in depth. While it is likely that the usage of the terms coherence, complexity, mystery, and legibility by other researchers is not identical to the Kaplans’ definition, their preference model has served as a theoretical standard for a large number of studies in environmental aesthetics and landscape assessment. According to their view, results which highlight the predictive value of Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 97 content-based categories (such as natural vs. urban, or scenes with a good view) actually support their model insofar as these scenes are typically those which lend themselves to high functioning within the scene (Kaplan, 1992). A rather unexpected source of confirmation for these ideas can be seen in the “Most W anted” and “Least W anted” paintings created by Vitaly Komar and Alex Melamid in response to polls conducted in the United States and more than a dozen other countries. The 1993 survey of 1001 adult Americans contained forced-choice questions about the respondents’ preferences for certain content-based categories, such as favorite colors, modern vs. traditional art styles, indoor vs. outdoor scenes, animals, geometric vs. random patterns, etc. From these results, Komar and Melamid created paintings summarizing the preferred and non-preferred responses. The “Most W anted” paintings for the vast majority of the countries polled were outdoor pastoral scenes with a view into the distance, containing elements of mystery, people, animals, and other obvious signs of a fertile environment (Fig. 5.1a). The “Least W anted” paintings were typically geometric patterns or abstract shapes (Fig. 5.1b). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 98 Figure 5.1. Komar and Melamid ‘s A) “Most Wanted” and B) “Least Wanted” paintings, based on the results of a survey of what Americans prefer to see in art. Although these works will likely never qualify as great art, they may function as a form of visual “common denominator” reflecting the elements of aesthetic judgment common to most individuals. Komar and Melamid’s findings provide evidence that the environmental utility of a natural scene, including its navigability and availability of resources, is a basis for initial preference. W e have begun to explore this idea in our work by looking at how well factors which might correlate with utility can explain people’s preferences for visual scenes. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 99 Experiment 3: Predicting Scene Preference from Environmental Utility Based on the work of Kaplan, Appleton, and others, we selected a set of factors reflecting the utility of an environment for the purposes of information gathering, navigation, and habitability. W e collected ratings for each scene in our stimulus set on these factors and used a multiple regression approach to parcel out the variance in preference ratings attributable to these factors. In addition, a number of other measures were also included in this analysis to control for the contributions of low level stimulus properties (e.g. contrast energy) and the presence of particular stimulus types (e.g. faces, places and objects). Methods Stimuli In this experiment we used the full set of 200 stimuli from the calibration experiment (see Chapter Three for details). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 100 Subjects Twelve participants rated all 200 scenes on a seven-point scale for five factors selected from the work of Kaplan and Appleton. This sample consisted of five men and seven women, mean age of 20.3 + 3.5 years. The participants were a mix of novice paid volunteers and experienced behavioral subjects. A different set of four subjects (two male, two female, mean age 21.3 + 3.3 years) rated the set of scenes for the extent to which they depicted natural or urban/manmade settings. Three of the subjects were experienced observers, while one was a paid novice volunteer. Additionally, the authors (IB & EV) rated the scenes for the degree to which they depicted a place, and whether the scene contained human faces or objects. Finally, a different set of twenty subjects (five male, fifteen female, three left handed, seventeen right handed, mean age 19.6 + 1.1 years) rated their preferences for a set of images which were spectrally equivalent to the scene stimuli. None of these subjects had previously seen any of the stimuli used Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 101 in any of our experiments. Subjects participated in this experiment either for credit in an undergraduate psychology course or as a paid volunteer. Procedure Subjects were instructed that they were participating in an experiment designed to investigate how people make decisions about visual scenes. Their task was to indicate the degree to which each scene “agreed” with each trait. They were to give the scene a six or seven if the scene rated highly in that quality, a one or two if that quality did not describe the scene very well, and a middle rating if they had no strong feeling either way. The subject was then shown the following descriptions of each factor on the computer screen, as well as a paper copy for them to consult during the experiment. Mystery: If you were able to continue viewing the scene, how likely is it that something new might occur or appear? If you could venture deeper into the scene, how likely is it that you would obtain different information from changes in your vantage point? Vista: How good is the view? Can you see a wide expanse of area, or for a long distance? Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 102 Refuge: Is there a position in the scene where you can have a good vantage point without being seen? Is there a good “hiding place” that is well protected? Coherence: How rapidly and easily did you understand what the scene was about? Imagine a quick blink at the scene. Would you be able to get it in a single glance or would a number of fixations be required? Legibility: Let’s say you were dropped randomly in the scene. If it is a close up, imagine that you are very small. How easily would you be able to determine where you were in the scene and navigate through it? If you had to return to where you started, would you be able to? Presumably this would be difficult for a scene that was homogeneous, and easy if there were a number of distinct landmarks. It should be noted that unlike many of the studies reviewed by Kaplan, we did not include the factor of complexity for reasons previously mentioned. Also, our definition of refuge is what some researchers would term secondary refuge (Kaplan, 1992), as it is not assessing whether the scene is shown from a protected vantage point, but whether such a vantage point exists in the scene. However, it is possible that subjects might allow for either of these to influence their judgments of refuge. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 103 The subjects were asked to read over the descriptions carefully and ask the experimenter if they required any clarifications. In addition, the subjects were instructed to try and use the entire seven point scale when making their judgments. Each subject then viewed all 200 scenes and rated the scenes on each of the five factors. The subject was given a break every forty trials. In addition, they were shown a block of forty scenes a second time (in a different order) to derive a measure of reliability. On each trial, the scene was shown in the center of the screen and remained visible for an unlimited amount of time while the subject used the mouse to rate each scene on each of the factors. The factor name appeared on the screen below the image and remained there until the subject clicked on one of seven boxes indicating their rating, at which point the next factor name appeared. The order of the scenes was counterbalanced across subjects, with pairs of subjects receiving the scenes in the reverse order. The order in which the subjects rated the factors remained constant for a block of forty trials (after which the order was changed) and the question order for any one image was balanced across subjects. In a separate experiment, four subjects were shown all 200 scenes and asked to rate each scene for the degree to which it depicted a natural or an Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 104 urban/manmade setting. The subjects responded on a seven point scale using the keyboard. The order of presentation was reversed for pairs of subjects, resulting in the same average serial position for all scenes. The resulting ratings were then averaged across subjects and used in a multiple regression procedure assess how much of the variance in the calibration experiment preference ratings could be explained by these factors. As a method for parceling out any contribution to preference from low-level stimulus factors, we included two additional regressors. The first was a simple measure of stimulus contrast energy derived from computing the standard deviation of the image pixels in each color channel and then averaging over the three channels (root mean squared, or RMS contrast). The second measure was an average preference rating obtained in a separate experiment for a set of 200 phase scrambled images which each contained the same spectral energy as one of the experimental scenes. These images were produced by transforming each of the three color channels of the experimental images into Fourier space, scrambling the phase spectrum (but not the magnitude spectrum), and transforming the resulting image back into Euclidian space (Fig. 5.2). These images therefore Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 105 contained the same amount of oriented, colored information at different spatial scales as the original scenes, but with no definable structure. Subjects rated their preference for these images in a design identical to the calibration experiment. The average preference rating over subjects was computed for each spectrally equivalent image and included in the multiple regression. Figure 5.2. Examples of spectrally equivalent, phase-scrambled images used to assess the contribution of low level factors to scene preference. These images were created by randomizing the phase of the different spatial frequency components (but not the magnitude) of the image set used in the calibration experiment. Finally, ratings of the degree to which each image depicted a place, contained human faces, or contained objects were obtained from two subjects and included in the analysis. For the object ratings, the subjects adhered to several important criteria. First, the presence of a partially occluded or cut off object reduced the rating of an image’s objectness, as it is likely that such an incomplete view reduces its ability to be perceived as an Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 106 object. Second, human beings were counted as objects, but the ratings for humans were also reduced in comparison to other objects (animals and inanimate objects) as it is likely that the presence of human faces and body parts activates specific processing mechanisms separate from a general purpose object processing mechanism. Finally, scenes which contained a large number of repeated objects were given medium to low ratings of objectness, as it is likely that such images are encoded as texture rather than as discrete objects. Images which produced large disagreement between the two judges were reevaluated to arrive at a more consistent set of criteria. Results Individual factor results The mean ratings and standard deviations of each factor are shown in Table 5.1, and each of the factors are plotted against preference in Fig. 5.3. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 107 = actor Summary Statistics Factor Mean (N=200) Std. Dev. (N=200) Mean (N=60) Std. Dev. (N=60) Coherence 5.22 0.98 5.45 0.76 Legibility 4.16 0.83 4.34 0.76 Mystery 3.40 0.73 3.27 0.72 Vista 4.13 1.33 4.30 1.40 Refuge 3.16 1.16 3.40 1.08 Natural/Urban 3.71 2.22 3.22 2.22 Place 3.91 2.23 4.19 2.24 Face 1.73 1.52 1.99 1.85 Object 3.84 1.76 4.08 1.73 RMS Contrast 60.8 15.3 59.25 16.14 Preference for Spectral Equivalent 3.55 0.87 3.49 0.85 Preference (from Calibration Expt.) 3.97 0.98 3.93 1.24 Table 5.1. Each of the factors were rated on a one to seven scale, except for RMS contrast, which can range from 0 (no contrast) to 128 (maximum pixelwise contrast). Std. Dev. = Standard Deviation. A 7 B 6 5 s c i 4 « g £ a 3 8 c £ ■ 8 £ a. 2 Rsq = 0.0843 Rsq = 0.0035 1 100 6 2 3 4 5 7 Contrast Figure 5.3. Scatterplots of preference versus the low level image factors and environmental utility factors for all 200 stimuli. A) RMS contrast B) Preference ratings for spectrally equivalent images C) Coherence D) Legibility E) Vista F) Refuge G) Mystery and H) Naturalness. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 108 ^4' Rsq = 0.0075 Rsq = 0.0289 Coherence Legibility E 8 c £ • 8 £ a Rsq = 0.2021 1 3 2 4 6 5 7 F 8 c £ ■ 8 £ a. Rsq = 0.0325 1 2 3 4 5 6 Vista Refuge G H 8 C £ I a 8 c £ £ C L Rsq = 0.0893 Rsq = 0.3286 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 0 1 2 3 4 5 6 7 8 Mystery Natural vs. Urban (Naturalness) Figure 5.3 (continued). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 109 It was possible that legibility may have benefited from transformation to a quadratic variable. W e tested this by performing an additional regression analysis in which we included a quadratic transformation of legibility. This led to no change in any of our results, so we have not included this analysis in the regression reported below. In addition, coherence and the preferences for spectrally equivalent images show a compression to a narrow range, possibly reducing our ability to measure their effects on preference accurately. Figure 5.4 shows examples of scenes rated high and low on each of the six factors of interest. W e report these summary statistics for both the full set of 200 images as well as the subset of sixty scenes selected in Experiment 2 for use in imaging experiments. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Figure 5.4. Examples of scenes rated high and low on a) mystery, b) vista, c) refuge, d) coherence, e) legibility, and f) naturalness (natural versus urban/manmade). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Figure 5.4 (continued). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 1 1 2 E Legibility High Low F Naturalness High Low Figure 5.4 (continued). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Consistency Results 113 In order to assess the reliability and consistency of our subject’s ability to rate the stimulus set on these factors, each subject was shown a subset of the images a second time. Table 5.2 lists the mean and standard deviations across the twelve subjects for the correlation of the first and second image presentations. Consistency data was not collected for the other factors. Consistency of Factor Ratings Factor Avg. Correlation (N = 12) Coherence 0.69 + 0.17 Legibility 0.61 + 0 .1 5 Mystery 0.54 + 0.19 Vista 0.77 + 0.13 Refuge 0.63 + 0.23 Table 5.2. Correlations between factor ratings on the first and second presentation of a subset of images to each subject. It is clear that while there is a range of consistency, several of the factors show very good consistency within a subject, particularly vista and coherence. It should be noted that the “mean minus one” correlations for the naturalness ratings across the four subjects was 0.91, indicating a high degree of agreement on this factor. In addition, some feeling for the reliability of the spectrally equivalent images can be obtained by noting that the standard deviation of preference judgments for these images across subjects was 0.73, which is less than the standard deviation across subjects Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 114 of preference judgments for the original images (1.08). Overall, these results indicate that individual subjects show a range of consistency, but on the whole are very reliable when rating such image factors, particularly for vista and naturalness. Multiple regression results Using a three step multiple regression, we calculated the amount of variance in preference ratings accounted for by factors which assess an environment’s evolutionary utility above that accounted for by low level stimulus factors, and whether the presence of faces or places accounted for any additional variance over that of the other factors. Table 5.3 shows the overall R2, R2 change, and change in F for each of these models. Table 5.4 shows the part correlations for each factor in each of the three models. Model 1, which tests the role of low-level image factors, did account for a small, but significant amount of the variance in preference ratings for both stimulus sets (0.08 in the full set and 0.14 in the sixty image subset). Increased preference for the spectrally equivalent images predicts increased preference for the normal image, while RMS contrast had no effect (Table 5.4). Adding in the environmental utility factors (Model 2) led to an increase Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 115 Correlates of Preference: Multip e Regression Results Model R2 Adj. R2 R2 A F A Sig. F A All 200 Stimuli Model 1: Low Level Stimulus Factors 0.08 0.08 0.08 9.1 0.002 Model 2: Model 1 + Env. Utility Factors 0.51 0.49 0.42 27.6 < 1a 4 Model 3: Model 2 + Face, Place & Object Factors 0.53 0.50 0.02 3.0 0.033 Sixty Stimulus Subset Model 1: Low Level Stimulus Factors 0.14 0.11 0.14 4.8 0.01 Model 2: Model 1 + Env. Utility Factors 0.66 0.61 0.51 12.8 < 1a 4 Model 3: Model 2 + Face, Place & Object Factors 0.67 0.59 0.01 0.4 0.78 Table 5.3. Regression results summary for the three hierarchical models tested. The top portion of the table shows the results for each model regressed against preference scores for the full set of 200 stimuli from the calibration experiment, while the bottom portion shows the results for each model regressed against preference scores for only those sixty stimuli used in Experiment 2 and the imaging experiment. R2, proportion of variance in preference scores accounted for by the model; Adj. R2, R2 adjusted for multiple comparisons; R2 A, change in R2 over previous model; F A, change in F over previous model; Sig. F A, significance of the change from previous model. Bold italics show those effects significant at p < 0.05. in the overall R2 values of 0.42 and 0.51, while the factors added in Model 3 (faces and places) contributed only an additional 0.02 to the R2 value for the full image set and produced no change for the sixty image subset. The best model for the full image set is Model 3, which accounts for 53% of the variance in preference ratings, while Model 2 is the preferred model for the sixty image subset, accounting for 66% of the variance in the group preference ratings for those images. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 116 Correlates of Preference: Part Correlations for Each Model Factor All 200 Images Sixty Image Subsel Part Corr. T Sig. t Part Corr. t Sig. t Model 1: Low Level Stimulus Factors Contrast 0.007 0.11 0.91 0.07 0.56 0.58 Spect. 0.28 4.2 < I f f 4 0.37 3.0 0.004 Model 2: Model 1 + Environmental Utility Factors Contrast 0.03 0.63 0.53 0.001 0.014 0.99 Spect. 0.15 3.0 0.003 0.001 0.016 0.99 Coherence 0.10 1.9 0.06 -0 .0 2 0.29 0.77 Legibility -0 .0 2 -0.42 0.67 0.06 0.70 0.48 Mystery 0.18 3.5 < I f f 3 0.17 2.0 0.046 Natural/Urban 0.38 7.5 < I f f 4 0.43 5.2 < I f f 4 Vista 0.20 3.9 I f f 4 0.18 2.2 0.03 Refuge -0.01 -0.21 0.83 0.03 0.35 0.73 Model 3: Model 2 + Face & Place Factors Contrast 0.05 0.99 0.32 0.02 0.20 0.84 Spect. 0.15 3.0 0.003 -0.006 -0.07 0.94 Coherence 0.13 2.6 0.011 -0.004 -0.05 0.96 Legibility 0.001 0.02 0.98 0.07 0.84 0.40 Mystery 0.20 4.1 I f f 4 0.19 2.2 0.03 Natural/Urban 0.36 7.2 < I f f 4 0.40 4.8 < I f f 4 Vista 0.10 1.9 0.055 0.11 1.3 0.19 Refuge -0.03 -0.64 0.53 0.01 0.16 0.88 Face -0.13 -2.65 0.009 -0.07 -0.89 0.39 Place 0.02 -0.35 0.72 -0.01 -0.15 0.88 Object -0.04 -0.82 0.41 -0.02 -0.30 0.77 Table 5.4. Part correlations for each of the factors in the hierarchical regression analysis. The left hand side shows the part correlations for the regression on the full set of 200 images, while the right hand side shows the part correlations for the regression on the subset of sixty images used in Experiment 2 and the imaging experiment. Part Corr., part correlation for each factor of the model; t, t statistic for that part correlation, Sig. t, significance of the factor’s part correlation; Spect., preference for spectrally equivalent images. Bold italics show those effects significant at p < 0.05. For the full 200 image set, Naturalness, Mystery, Coherence and low level spectral properties are significant positive predictors of preference, while the presence of faces is a significant negative predictor (Model 3). In addition, Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 117 Vista approached significance in this model and was highly significant in the reduced model. For the sixty images used in Experiment 2, Naturalness, Vista, and Mystery are significant predictors of preference (Model 2). Low level image properties and stimulus attributes such as faces, places and objects do not account for any additional variance in preference beyond that of environmental utility. Conclusions & Discussion In support of Kaplan’s theory, we found that factors which reflect the environmental utility of a particular scene are good predictors of preference. The large role of Naturalness in subject’s preference ratings will be discussed in depth later in this chapter. The predictive value of Mystery and Vista for rated preferences provides strong support for a link between information foraging and preference. Not only does it benefit people to monitor parts of the visual environment which may be potential sources of change, it is pleasurable to do so. The lack of any predictive power for Refuge may reflect the fact that this factor may only play a significant role in assessing a landscape’s utility or habitability during certain times, such as in the evening when it is getting dark or when one is seeking protection from the elements. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 118 The failure of Legibility as a significant predictor is not surprising given the relatively low level of variation on this factor. The same may be true for Coherence, which was less variable in the sixty image subset than in the whole set of 200 images, and quite high overall (Table 5.1). In addition, it is likely that a scene which is comprehended quickly and easily (and as such, carries no surprises) may not provide much in the way of rich and novel interpretation. The low level factors, particularly the preference ratings for the spectrally equivalent images, do account for a small, but significant amount of variance in preference ratings when considered alone. However, as can be seen from the part correlations for Model 2, this effect is not significant for the sixty image subset. This may indicate that some of the variance accounted for by the descriptive (environmental utility) factors is carried in rather low level information, particularly for the subset of images we selected as having low variance across subjects. For these images, the preference for spectrally equivalent images correlates positively with naturalness and vista (0.42 and 0.43 respectively). RMS contrast did not account for any significant variance in any of the models. Therefore, the preferences we observed for our larger stimulus set of 200 images were slightly dependent on low level stimulus factors, but within the set of images selected in Experiment 2 for use in Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 119 FMRI, there was no preference information carried by low level stimulus differences that was not correlated with a scene’s environmental utility. Given the likelihood that natural scenes and vistas are more textural and have a more open spatial envelope, one would expect them to be correlated with spectral energy to a degree. The fact that no additional variance in preference is accounted for by low level factors once environmental utility is taken into account is a rather striking demonstration of the usefulness of environmental descriptors. The stimulus factors which were added in Model 3--the presence of faces, the degree to which a stimulus describes a place, and the presence of clear, central objects -- accounted for some additional variance in the preference ratings above that of the environmental utility model in our full stimulus set, but not for the sixty image subset. Our interest in including such factors stems from the demonstration that faces, places and objects activate specific cortical fields in the anterior ventral visual stream (Kanwisher, McDermott & Chun, 1997; Epstein et al., 1999; Tong et al., 2000; Ishai et al, 1999; Grill- Spector et al., 1998; Kourtzi & Kanwisher, 2000). From the part correlations, it is clear that the presence of a face is correlated with a decrease in preference for our full stimulus set. However, it appears that this finding does not hold true for images which are most stable across subjects, as the part correlation does not even approach significance for the sixty image Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 120 subset. Interestingly, whether a stimulus codes for a place does not correlate with increased preference, though it is likely that the high correlation of the place factor and vista (0.81) is a major reason why vista failed to reach significance in Model 3 for the sixty image subset, and only approached significance for the full image set. The presence of objects also does not act as a significant predictor of preference in either image set, though the high correlations of these ratings with coherence and legibility (0.56 and 0.61 respectively) do indicate that the presence of clear, central objects make an image easier to comprehend. Biophilia and the Preference for Natural Environments For both image sets, the Natural vs. Urban factor was by far the strongest predictor of preference. This holds true even when the correlations of this stimulus dimension with vista, refuge, mystery, low level image properties, faces, and places are factored out. This finding agrees with a large body of evidence in landscape assessment showing that the amount of natural versus urban/manmade content is consistently one of the best predictors of preference (e.g. Kaplan, Kaplan, & Wendt, 1972; Ulrich, 1974,1993). Even for studies which have focused on specific urban or natural environments, the relative prominence of natural and urban elements played a significant role in preferences. For industrialized nations, the concept of Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 121 "natural" extends to anything that is predominantly vegetation, including parks, pastures, and golf courses. Kaplan cites examples from a number of studies, including one in which scenes from an Urban Nature category far outranked other urban scenes (such as Alley/Factory), and another in which waterscapes with mountain peaks in the background were much preferred to waterscapes with human intrusions (Kaplan & Kaplan, 1995, 44). The presence of human elements, such as power lines, concrete, signs, and even air pollution drastically reduces preference for natural scenes. As summarized by Roger Ulrich, In very general terms, European, North American, and Japanese adult groups tend to respond to scenes as natural if the landscape is predominantly vegetation, water, and mountains, if artificial features such as buildings, automobiles, and advertising signs are absent or inconspicuous, and if the dominant visual contours or edges are curvilinear or irregular rather than starkly rectilinear or regular. (See Rodenas, Sancho-Royo, and Bernaldez 1975, Ulrich 1983, 1986a, Wohlwil! 1983). (Ulrich, 1993, 95) Ulrich also points out that the similarities in preference ratings for individuals and groups from European, North American, and Asian cultures far outweigh the differences. This is not to say that there is no aspect of learning involved in landscape preference: One study by Jack Nasar comparing American and Japanese subjects rating urban scenes from both countries found that both sets of subjects preferred those scenes taken from their non-native country (Nasar, 1988). This agrees with the preference for novelty we have seen in our own work. However, the two groups of subjects in Nasar’s study showed Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 122 a high degree of agreement both for their overall preferences (0.54), and in the degree to which factors such as nature (0.64 and 0.44 for the U.S. and Japanese subjects, respectively), upkeep (0.53 and 0.55), and the presence of vehicles (-0.40 and -0.61) correlated with preference. Why does the natural environment have such a strong effect on how enjoyable a scene is to look at? It is clear that the natural vs. urban distinction is a very salient one; even when not asked to explicitly make preference judgments, the natural vs. urban dimension has a prominent effect on subjects’ categorizations of scenes. As part of a study on the spatial representations underlying scene recognition, Oliva and Torralba asked subjects to successively split a set of eighty-one scenes into smaller and smaller groups, with explicit instructions to not use criteria related to objects or scene semantic groups (e.g. forest, street, beach). Sixty-five percent of the subjects spontaneously used "naturalness" as the first criterion for separation, with another twelve percent using this as their second criterion (Oliva & Torralba, 2001). However, this does not explain why preference judgments should be influenced so heavily by naturalistic content, and only begs the question of why this aspect of our visual environment is so readily detected. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 123 Returning to the preference matrix proposed by Stephen and Rachael Kaplan, naturalness would be preferred not so much on its own, but only by virtue of the fact that natural scenes may afford better opportunities for exploration and understanding of one’s environment. The data, however, tend to tell a different story. Both in our studies and in those of landscape assessment, several of their factors do not appear to be good predictors of preference across their entire range, and the natural vs. urban dimension has consistently been a good predictor of preference independent of its covariation with the factors of the Kaplan preference matrix. In a summary of the studies on landscape assessment, Kaplan admits that “it is not clear that high degrees of either [coherence or complexity] increases preference linearly.” (Kaplan, 1992, 589) The results for complexity are not surprising in light of evidence from earlier studies indicating that within a particular stimulus domain, figures of medium complexity tend to be preferred (Vitz, 1966). Therefore, complexity might be a decent predictor, but not in a linear fashion, and perhaps not in the most predictable way across different stimulus categories. Legibility has also met with mixed success, although as Kaplan points out, it has been studied less extensively. Mystery has been the only consistently significant predictor in the literature. Correlations between preference and mystery often match or exceed those for the natural vs. urban dimension, with partial correlations between 0.31 and 0.56 (Kaplan, 1992). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 124 Within Kaplan’s framework, these results suggest that understanding a scene is not sufficient to induce preference, while the capacity for information gains upon exploration is a very strong predictor of preference. However, Kaplan’s framework does not adequately address the fact that natural scenes are not always those which have the highest capacity for exploration, and are preferred over and above what the preference matrix would predict. In addition, our results and those of several other researchers have found prospect (or vista) and refuge to be significant predictors independent of the factors in Kaplan’s preference matrix (Heerwagen & Orians, 1993). But when considered in an evolutionary framework, it seems more parsimonious to postulate a causative role for the adaptive and nourishing aspects of nature in the predictive power of these perceptual attributes. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 125 CHAPTER 6: FMRI INVESTIGATION OF SCENE PREFERENCE W e sought to understand how neural activity in the ventral visual pathway is related to perceptual affect. The neurocomputational hypothesis outlined in Chapter One predicts that stimuli which are preferred will lead to greater activity in the anterior regions of this visual recognition pathway. To test this hypothesis, we used functional magnetic resonance imaging (fMRI) to observe an indirect measure of brain activity while subjects viewed scenes of differing preference. Background To date, the only neural investigations of preference have examined brain activity either within the context of a mere exposure paradigm or emotionally arousing stimuli. As was discussed in Chapter Three, both of these paradigms may be related to information foraging, but likely do not provide a direct test of the theory we have proposed. Elliot and Dolan (1998) used Positron Emission Tomography (PET) to observe the brain activity of nine subjects while they made either a preference or a recognition choice between pairs of Chinese ideograms. In Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. addition to judgment type, the experimental scans differed according to whether both stimuli of each pair were unfamiliar or whether one of the stimuli had been seen previously in a pre-exposure block before the scan began. Importantly, the pre-exposed stimuli were shown for only 50 ms and subsequently masked with a checkerboard. Each stimulus was shown ten times. Behaviorally, the subjects showed chance recognition performance but greater than chance liking (59.3% ) for the pre-exposed stimuli. The authors observed greater activity associated with making a preference judgment as compared to a memory judgment in the left anterior frontal cortex (BA 8), left premotor cortex (BA 6), left caudate nucleus and left pulvinar, right superior temporal gyrus (BA 22) and right fusiform gyrus (BA 20), and bilateral cerebellum and bilateral medial occipital gyrus (BA 18,19; Elliot & Dolan, 1998, 4699). Objective stimulus novelty as compared to familiarity was associated with increased activity in the left mediodorsal thalamus, left fusiform gyrus (BA 19), left superior temporal gyrus (BA 38), right cuneus (BA 19), and right hippocampal gyrus (BA 35). Unfortunately, the design of the study did not allow the authors to look at activity related to preference itself, as any one scan contained both the preferred and non preferred image from each pair. In addition, the scans which they classified as being “ familiar” as opposed to novel still contained one novel stimulus for each image pair (versus two novel Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 127 stimuli for the “unfamiliar” scans). The authors found no significant activations greater for familiar than unfamiliar stimuli. As was discussed extensively in Chapter Three, the mere exposure effect is not very robust and likely relies on the small increase in interpretability afforded by subliminal pre-exposure to a stimulus. Given the fact that the stimuli used in this experiment do not differ in their a priori interpretability (two novel Chinese ideographs versus one novel and one pre-exposed ideograph) it is not surprising that that the only observed effect of stimulus familiarity was the standard reduction in activity for a primed stimulus. While our theory would predict that increased preference for the repeated stimulus would be the product of increased activity, it is likely that the difference occurs so early and is of such a small magnitude in this study as to be undetectable. In contrast, our interest is not in the activity underlying the judgment of preference, but in the neural activity that forms the basis for that judgment. Therefore, we chose not to ask subjects to make any type of judgment at all during our scans, and we used a stimulus set which attempted to maximize the differences in a priori stimulus preference. A second FMRI study of preference was conducted by Lang et al. (1998) using the International Affective Picture System (IAPS; Lang, Bradley & Cuthbert, 1997). They found significantly greater blood flow for high valence Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 128 unpleasant and pleasant pictures over neutral pictures in sections of the occipital cortex, including early visual areas, extrastriate cortex, and the posterior fusiform gyrus in addition to loci in the inferior and superior parietal lobules. Pleasant stimuli produced greater activity than neutral stimuli in the right lingual gyrus, right peristriate cortex (BA 19), and the left fusiform gyrus. Several major limitations limit the relevance of this study to our hypothesis. Only four slices in the extreme posterior portion of the brain were imaged so the effects of their image set on later stages of processing could not be assessed. Second, the images were presented in a block design, which our behavioral data indicate could result in extensive context effects as the overall preference was not held constant across the duration of the scan. In addition, it is possible that the behavioral state induced in the subject (such as disgust during the highly arousing negative stimuli) could lead to additional differences between conditions other than just the nature of the stimuli. Finally, as was discussed extensively in Chapter Three, highly emotionally valent images may activate specialized brain mechanisms dedicated to the processing of such stimuli that would supercede the activity of an information foraging system. In all of our behavioral experiments, repeated exposure of a stimulus has had a strong and consistent effect on preference. While some individual subjects Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. may show a slower rate of change, all subjects show some degree of decreased preference for stimuli as they are repeated. As was discussed in Chapter Three, both the behavioral and neural work on repetition priming is highly consistent with the hypothesis that visual preference may be computed directly from the distribution of activity in the ventral visual pathway. Therefore, we predicted that repetition of a stimulus would lead to decreases in the BOLD signal along the ventral visual pathway, with those decreases being most pronounced for the most anterior stages of processing. Given the involvement of the lateral occipital complex (LOC) in object recognition (Grill-Spector et al., 1998; Grill-Spector et al., 1999; Kourtzi & Kanwisher, 2000) and its possible homology with macaque IT (Tanaka, 1997), we sought to characterize the effects of preference in this area. Given its position as a middle-to-late stage in object processing, we expected that the magnitude of the preference effect in this area would be intermediate between earlier stages and later stages thought to encode more specific features such as faces and places. The demonstration that the LOC may actually be comprised of two distinct but similar areas one which is more dorsal (superior), posterior, and lateral, and one which is more anterior and ventral (Grill-Spector et al., 1999), raises the possibility that we might see differential activity in these two object processing subareas. These two Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 130 areas have been referred to as LO (the more posterior, lateral, and dorsal locus) and PF (for posterior fusiform - the more ventral and anterior locus). This latter division was reported to be more invariant to object size and position compared to LO. W e included an independent localizer scan for object responsive areas, which, while not allowing us to specifically distinguish between these two subareas, provided a way for us to independently assess the effects of preference in object selective voxels (a volume element). Proceeding in an anterior direction, we also sought to assess the effects of a priori preference and repetition in the fusiform and parahippocampal gyri. These areas have been shown to respond to a variety of specific stimulus classes, such as faces and pictures of places in a manner which shows a high degree of specificity (Epstein et al., 1999; Tong et al., 2000) yet maintains some degree of distributed representation (Ishai et al., 1999; Haxby et al., 2001). In addition, recent work demonstrating the role of the parahippocampal gyrus in the processing of context makes it a likely candidate for showing greater activity for richly interpretable, and hence pleasurable stimuli (Bar & Aminoff, 2003). W e were also interested in exploring the role of the ventral striatum (nucleus accumbens), amygdala and the orbitofrontal cortex in visual preference, as Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 131 these areas represent possible output pathways from the ventral visual pathway which may mediate some of the rewarding aspects and behavioral selection of visual preference (Rolls, 1999, 257; Robbins & Everitt, 1996). In order to prevent factors such as preference context (see Chapter Four), subject fatigue, and any awareness on the part of the subject of what type of image they might expect to see on any given trial, we used a jittered, event- related (ER) FMRI design (Burock et al., 1998; Buckner & Braver, 1999) allowing us to present stimuli in a manner virtually identical to that used in our behavioral experiments (Experiment 2). Experiment 4: Neural Correlates of Perceptual Affect W e investigated the effects of a priori image preference and stimulus repetition on brain activity using an event-related FMRI paradigm. Subjects were asked to pay attention to a sequence of full color scenes consisting of images in High, Medium, and Low a priori preference categories, each shown for a total of five exposures. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Methods 132 Subjects Sixteen healthy adult volunteer subjects participated in this experiment. The mean age of our subjects was 26.6 (+ 3.9). Nine were male, seven female, fifteen right handed, one left handed. All subjects consented to participate in accordance with the Institutional Review Boards of both the University of Southern California and the University of California, Los Angeles. The subjects were selected according to their willingness to take part in an MRI experiment and their ability to remain attentive and still during the session. In addition, all subjects were required to pass a rigorous safety and neurological screening to ensure that none of them had any condition which might compromise their safety in the magnet (such as a metal implant or heart condition) or the neurological profile of the subject pool (such as a diagnosed mental illness or pathological condition). Apparatus All scanning took place under the direction of Mark Cohen at the UCLA Ahmanson-Lovelace Brain Mapping Center on a 3 Tesla, General Electric Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 133 magnetic imager, fitted with an ANMR (Woburn, MA) head coil for echo- planer imaging. Imaging Protocol Following the consenting process and safety screening, the subject was taken into the magnet chamber, given an explanation of the general process, and moved into the bore of the magnet. Each subject was equipped with earplugs to dampen the noise from the magnet, headphones to allow for two way communication, video goggles, and a panic button. The imaging sequence began with a shimming sequence designed to detect and correct inhomogeneities introduced into the main magnetic field. This process was reiterated until the inhomogeneity in precession frequency across the field was no more than 20 Hz. A series of very short, low resolution anatomical scans were then acquired to allow for the accurate slice prescription of a T1 weighted high resolution anatomical scan and the echoplanar (EPI) scans. Using these initial scans as a guide, we acquired a twenty-six slice, high resolution (1.562 mm in the plane of the slice) EPI image covering the whole brain for the purposes of registration, ROI selection, normalization and viewing. These axial slices Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 134 were 5 mm thick (signal averaged over a 4 mm distance with a 1 mm space between slices) and were aligned to the anterior and posterior commissures. If necessary, a correction procedure was run to correct for any Nyquist ghost observed in the EPI data. From these twenty-six slices, we selected a subset of sixteen slices from which to collect our “ functional” data using the BOLD (blood oxygenation level dependent) technique. W e selected these slices in such a manner as to completely cover the ventral surface of the occipital and temporal poles, extending upwards as far as the slices would allow. As described in detail below, we then performed our main preference experiment split into two ten minute scans, separated by a short break (typically about thirty seconds). W e collected a full brain image every 2.5 seconds from sixteen 5mm slices (in plane resolution 3.125mm x 3.125 mm). For this and all subsequent EPI scans, we used a flip angle of 90°. Following the preference experiment, we collected a second high resolution EPI anatomical image of the ventral portion of the cortex, aligned obliquely with the ventral surface to get complete coverage of the ventral occipital and temporal lobes. This image was used to select a subset of six slices from which to collect data for our LOC localizer scan. As described in more detail below, we then ran a 5.5 minute scan designed to allow for the functional Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 135 identification of object selective voxels for use in a region-of-interest (ROI) analysis of the LOC. Finally, we collected a T1-weighted, high resolution three dimensional volume (resolution: 1.01 x 1.01 x 1.2 mm) for use in cross-subject normalization and visualization. Preference Experiment The stimuli used in this experiment were identical to those used in Experiment 2b: twenty images in each of a high, medium, and low preference group along with twenty four images used as buffers at the beginning and end of the experiment. Importantly, these three preference groups did not differ in their average RMS contrast (F[2,38j = 0.34, p = 0.71). Procedures. Subjects passively viewed the sequence of full color scenes over the course of two ten minute scans, and were instructed to pay attention to the scenes for the duration of the scans. Our primary interest in this study was to isolate activity related to bottom up, stimulus driven affective responses, separate from any activity related to making an active aesthetic judgment. Therefore, at no point were the subjects asked to make Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 136 preference judgments, nor was there any mention of our interest in visual preference. The image sequence was generated using the Psychophysics Toolbox for Matlab (Brainard, 1997; Pelli, 1997) and displayed at a resolution of 72 dots per inch on Resonance Technology LCD goggles worn by the subject. W e employed a jittered, event-related design similar to that described by Buckner and his colleagues (Buckner et al., 1998; Burock et al., 1998; Dale & Buckner, 1997), which allowed us to use trial sequences which were virtually identical to those used in Experiment 2. Critically, this again allowed us to deconfound trial number and repetition as well as maintain a consistent local novelty context throughout the scans. A full brain volume was acquired every 2.5 seconds; this time is commonly called one TR (repetition period). There was a .75 probability of an image actually being presented per TR and a .25 probability of nothing being presented (termed a “blank”) per TR. W hen an image was shown, it was shown for 1 sec. This sequence produced a rough approximation to a Poisson distribution with an average presentation rate of one stimulus every 3.33 seconds. The sequence consisted of a buffer block followed by twenty-four blocks of twenty TRs, each composed of five fixations and fifteen (three preference levels by five exposure levels) intermixed experimental trials. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 137 Trial sequence. In addition to the constraints mentioned above, rapid pace event related imaging designs impose an additional trial ordering constraint as a result of the relatively slow hemodynamic response and their reliance on the deconvolution of these temporally overlapping signals. Studies of the typical impulse response function for the FMRI BOLD signal indicate a peak activation around six seconds following stimulus onset, a slight undershoot in some areas, and a return to baseline on the order of twenty to thirty seconds (Menon & Goodyear, 2001). Therefore, if one wishes to minimize the possible effects one condition might have on other conditions when averaging multiple trials in a mixed trial sequence, each trial type should, on average, be preceded by every other trial type. That is, the average “history” of every trial type should ideally be identical to prevent any misattribution of late activation in one condition to a different condition (Burock et al, 1998). In practice, the effects from more than one or two previous TRs have a vanishingly small effect on the estimated signal for a condition, and attempting to control for such “lookback” effects becomes increasingly difficult with more complex designs (Donaldson & Buckner, 2001). In our study, we controlled for the one-back condition history while simultaneously controlling for novelty context and satisfying our repetition design using a “simulated-annealing” algorithm (Kirkpatrick, Gelatt, & Vecchi, 1983). Initial condition orders were generated to satisfy the repetition Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 138 ordering constraints (i.e. 24 blocks of 20 trials, each containing five fixations and one trial of each condition). An error term was then calculated by looking at the one-back history for each condition over the entire sequence and subtracting it from an ideal matrix (which would have each condition preceded by every other condition and an equal number of fixations). Pairwise swaps were then made within blocks of 20 trials by randomly selecting a block and then randomly swapping two trials within that block; the swap was accepted with a probability of p if it resulted in a decrease in the error term. The value of p was calculated as p= 1 1 + ede/T where de is the decrease in error resulting from a successful swap and T is a “ temperature” which was decreased every 3000 iterations. This probabilistic acceptance rule helped prevent the algorithm from falling into local minima, thereby improving the overall fitness of the final solution. Given the constraints imposed on the condition ordering, there did not exist any “perfectly” ordered sequences; we instead chose the top two sequences and generated different image orders by shuffling the order in which images were first introduced into the sequence across subjects. For the most part, pairs of subjects were introduced to the different images in reversed order. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 139 LOC Localizer W e utilized a technique developed by Kourtzi & Kanwisher (2000) to localize voxels which responded better to objects than to scrambled versions of those same objects. This technique uses line drawings and grayscale versions of both familiar and novel objects, which allows us to identify voxels that respond to object form independent of familiarity or low level cues. Area LOC is, in fact, defined by: a) its greater response to objects than to scrambled versions of these objects, and b) its independence of object familiarity so that an unfamiliar object, just as an abstract sculpture unlikely to have been seen previously by the subjects, gives a BOLD response as strong as a familiar object. Procedures. The subjects were instructed that they were going to see a series of pictures flashed on the screen at a very fast rate, and that they were to maintain fixation on the black dot presented at the center of the screen for the duration of the scan, which lasted five minutes and thirty-six seconds. The scan consisted of twenty-one blocks, each sixteen seconds in length. During these sixteen seconds, twenty images were presented in a blocked design from one of eight image categories: line drawing of a novel object, line drawing of a familiar object, grayscale rendered image of a novel object, Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 140 grayscale photograph of a familiar object, and the scrambled version of each of these object classes. The remaining five blocks were blank periods used for estimation of a resting baseline. Importantly, the scan controls for the low level image features (edges) introduced by scrambling an image by placing a grid of lines over all images used in the experiment. During this scan, we measured the BOLD signal in six slices, obtaining an image once every second. Analysis and Results Single Subject Analysis Procedure Analyses were performed using FSL, the FMRIB Software Library developed at the Oxford Centre for Functional Magnetic Resonance Imaging of the Brain. Each subject’s functional timeseries from the two scans (16 slices oriented along the AC-PC axis, with 64x64 voxels per slice) were concatenated, preprocessed to correct for slice timing and motion correction (Jenkinson et al., 2002a). The 4D timeseries was then subjected to non brain removal (Smith, 2002), spatial smoothing using a Gaussian kernel with a filter width at half maximum (FW HM) of 6mm, mean-based intensity normalization of all volumes by the same factor, and high pass temporal Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 141 filtering (Gaussian-weighted LSF straight line fitting, with sigma=2.5 sec.) to filter out any frequencies with a wavelength larger than 50 seconds. The parameters of the blurring and high pass filtering were determined by comparing the peak voxel activation of our statistical analysis over several values, including 5, 6, and 8mm blurring, and 30 and 50 second high pass filtering. Whole brain time-series statistical analysis was carried out using FM RIB’s Improved Linear Model (FILM) with local autocorrelation correction (Woolrich et al., 2001 b). Each of the subject’s scans were fit to a sixteen parameter general linear model1, with one parameter specifying each of the fifteen conditions plus one additional parameter for the buffers (Fig. 6.1). Using these parameter estimates, contrasts of interest and accompanying t-tests were computed. Data from individual subjects were first used in a group analysis across all subjects to explore the general effects of repetition and a priori preference. W e will return to an analysis of particular data patterns in individual subjects after presenting these group results. 1 The temporal derivatives of each parameter were also included to account for any slight temporal mismatch of the model and the data caused by differences in slice timing or imperfect alignment (Woolrich et al, 2001 b). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 142 0 1 .1.0*1 1 0 0 c 0 0 8 0 ( I 0 t 'X 'X - 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 9 Lo*2 0 a 1 0 0 D 0 0 0 8 0 •B 0. 0 0 « u 1 8 0 0 0 0 0 0 0 0 0 3 lo*3 6 D '0 0 . 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 a 0 0 0 4 L(»4 0 0 '0 0 0 0 1 0 0 0 * 0 0 0 0 0 0 fl_ 1 0 0 0 0 0 0 0 0 0 0 W m m 0 0 o 0 8 a 0 0 c 0 0 0 0 1 0 0 0 o’ 0 0 0 0 0 0 0 0 0 0 0 a O f *»<u 0 a 0 0 0 0 0 0 0 t 0 0 0 0 0 G 0 0 0 0 O 0 0 0 0 0 0 tn X e d S 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 m iM3 3 0 0 c 0 0 8 8 0 0 0 0 0 0 0 0 0 9 S e d 4 0 O ' 0 G 0 0 0 0 0 0 0 0 0 0 1 0 0 0 8. 0 0 0 0 0 0 0 0 a 0 0 0 1 0 J t a d S a 0 0 0 0 0 0 0 0 a 0 0 0 0 0 0 1 0 0 0 0 0 . : o 0 0 0 0 < 0 Oil Eighl 0 0 0 0 0 0 0 0 0 ■ B p ,0-0 0 0 0 ■ 0 0 0 0 0 0 0 0 0 0 0 012 M tfa H 0 0 0 0 o 0 0 0 0 o u 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 3 Mgh3 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 H I 0 O ' ,0 0 0 0 0 0 0 0 0 0 0 0 0 1 4 a 0 0 0 0 0 0" 0 0 0 t 0 0 , 0 0 0 0 0 0 0 0 1 0 0 0 0 0 B IS HifhS 0 0 0 0 0 0 0 0 0 a 0 0 0 0 0 0 0 0 8 0 0 8 0 0 0 0 1 0 0 0 Figure 6.1. An example of the general linear model used to analyze a single subject’s data. Each row of the figure is one parameter (a condition and its temporal derivative), created by convolving the stimulus presentation sequence for that condition with a canonical, double gamma hemodynamic response function. The lower part of the figure shows how individual linear contrasts for each of the fifteen conditions are derived from these parameters. The small bar on the left indicates period of the lowest temporal frequency used by the model. The entire length of the columns corresponds to 488 2.5 s TRs (twenty minutes and twenty seconds). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Group Analysis: Whole Brain Voxelwise ANOVA 143 Using a single timepoint from the functional timeseries, a six parameter (3 degrees of translation and three degrees of rotation) registration was computed between each subject’s functional timeseries and the coplanar, anatomical echo planar image acquired just prior to the functional scans (in plane resolution: 1.562 x 1.562 mm). For this, and all subsequent registrations, we used FSL’s Linear Registration Tool (FLIRT; Beckman et al., 2001; Jenkinson et al., 2002a). For most subjects, this transformation consisted only of a translation in the z direction to account for the slices on the top and bottom of the coplanar image which were not present in the functional data. For a few subjects, however, small additional translations or rotations were required to correct for any head movement that had occurred between the collection of the coplanar anatomical EPI and the midpoint of the functional timeseries used as a motion correction target. Each subject’s coplanar EPI anatomical image was registered with their own high resolution, T1-weighted anatomical image, which was subsequently warped to an “average” brain created by the Montreal Neurological Institute (MNI) from T1 structural scans of 152 different subjects. This was accomplished using a twelve parameter affine transformation (translation, rotation, scaling, and shearing along each of the three axes) between each Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 144 subject’s T 1 anatomical image and the MNI standard brain. The computed transformation matrices from this series of registrations were then concatenated into a single affine transformation matrix and applied to each subject’s individual parameter estimate and variance images for all fifteen conditions. This anatomical normalization procedure was compared to a different procedure which relied on the creation of a custom anatomical template from all sixteen subjects (Woods et al., 1998). While neither procedure produced a perfect alignment, the alignment to the MNI standard brain described above appeared to be slightly better. All group analyses were performed within a volume defined by the overlap of the data space of all sixteen individual subjects. Due to the variability in the placement of slices and individual anatomy, this led to a shrinkage of the total area of the brain in which we were able to test for group effects. The group statistical analysis was performed using FILM in a voxelwise manner using a thirty parameter model, with sixteen parameters used to specify the mean of each subject and the additional fourteen parameters implementing an effect coded, 3 x 5 analysis of variance. Two parameters were used to code the effects of preference, four were used to code the effects of repetition, and the remaining eight parameters encoded the interaction terms. F tests for each main effect and the interaction were computed from these Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 145 terms, and linear contrasts of interest and their accompanying t-statistics were calculated as well. Averaged over all subjects and conditions, image presentation led to highly significant changes in activation compared to periods of no image in widespread regions of occipital, temporal, parietal, and frontal cortex (Fig. 6.2). It is interesting to note that the major path of significant activation follows a route which is quite medial, down through the lingual and fusiform gyri and into the parahippocampal gyri on both sides. It is possible that full field, real world color scenes lead to activity distributions with significantly more lingual activity than simple object stimuli typically used in most imaging experiments studying visual processing. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 146 (V o x e lw is e Z ) Figure 6.2. A) Average activation across all conditions (compared to no image) for all sixteen subjects. These nine axial slices progress from the most inferior (upper left) to most superior (lower right). The color indicates the voxel level z value at each position. Warm colors indicate increases above rest, while cool colors indicate decreases from rest. This image was thresholded at p < 1x1 O '5 . B) This midsaggital section shows the locations of the nine slices in A. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 147 W e then performed a series of linear contrasts to investigate the main effects of preference and repetition across all voxels in our group data. As this is an exploratory study, we first looked at the general pattern of results using voxelwise statistics, which are very liberal. As will be reported below, we followed this with a more stringent test of significance based up Gaussian Random Field theory (Worsley, 2001). W e observed effects of a priori image preference in regions of the occipital, temporal, and parietal cortices (Fig. 6.3), including greater activity for High over Low preferred images in the parahippocampal cortex, and the reverse pattern in the inferior and medial occipital gyri, and areas of inferior parietal cortex. No differences were observed at the group level in the amygdala, ventral striatum, the orbitofrontal cortex, or striate visual areas. W eak effects were observed in the frontal and parietal cortices, though none of the frontal effects survived the more stringent statistical tests described below. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 148 -5 .1 8 (V o x e lw is e Z ) Figure 6.3. Effects of Preference. A) The contrast for High preference images minus Low preference images. These nine axial slices progress from the most inferior (upper left) to most superior (lower right). The color indicates the voxel level z value at each position. Warm colors indicate greater activity for High preference compared to Low preference images, while cool colors indicate greater activity for Low preference as compared to High preference images. This image was thresholded at p < 0.01. B) This midsaggital section shows the locations of the nine slices in A. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 149 -7 .7 9 (V o x e lw is e Z ) Figure 6.4. Effects of Repetition. A) The contrast for the First presentation of each image minus the Fifth presentation. These nine axial slices progress from the most inferior (upper left) to most superior (lower right). The color indicates the voxel level z value at each position. Warm colors indicate greater activity for the First presentation over the Fifth presentation, while cool colors indicate greater activity for the Fifth presentation over the First presentation. This image was thresholded at p < 0.01. B) This midsaggital section shows the locations of the nine slices in A. Stimulus repetition (Fig. 6.4) resulted in extensive decreases in activity in the lingual, fusiform, and parahippocampal gyri, peaking in a region at the Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 150 junction of these three areas bilaterally. Decreases were also observed for a dorsal aspect of the cuneus (extrastriate cortex, BA 19), and to a lesser degree, some regions of the earlier (extrastriate) visual areas in the ventral stream. Increases in activity over repetition were observed in inferior parietal areas and the prefrontal cortex. Group Analysis: Cluster-Level Significance To evaluate the significance and summarize the above results, we used a clustering procedure and correction for multiple comparisons derived from Gaussian Random Field Theory (Worsley, 2001). Contrasts of interest were first masked by any a priori hypotheses we had about the locus of an activation (such as all those voxels which overlapped in all subjects, or all those voxels which showed significantly greater average activity for scenes over blank), and then thresholded at a Z level which produced well defined clusters. These clusters were then tested at a false positive rate of p < 0.05 based upon the calculated smoothness of the data, which gives an estimate of the true number of independent resolution elements (resels) in a dataset. This method has the advantage of being much more forgiving than an overly stringent Bonferroni correction for multiple comparisons. However, its potential drawback is a bias against activations with a small spatial extent, as the size of the activation in relation to the calculated reset size is weighed Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 151 heavily. Therefore, small activations which are highly significant at the voxel level may be more likely to be rejected than a wider cluster containing voxels of lower Z values. W e first sought to characterize the presence or absence of significant effects within those regions which showed greater activity for visual presentation as compared to a blank screen. Using the contrast shown in Figure 6.2, we first thresholded this image at a voxelwise Z value of 3.5 and tested for significance of the resulting clusters. The largest cluster contained a total of 28,858 voxels and included the entire swath of activity from early visual areas through the parahippocampal cortex. A mask was created from this cluster to test for the effects of preference and repetition within areas activated by visual stimulation. Each contrast tested was first masked by this dataset, then thresholded at a Z of 3.8. The resulting clusters were tested for statistical significance. No clusters survived this procedure for the High preference greater than Low preference contrast. In the reverse direction, however, three clusters were found (Fig. 6.5). Regions at the anterior/ inferior border of the inferior occipital gyrus and the fusiform gyrus were significantly greater for Low than High preferred images on both sides (Fig. 6.5a). These loci are possibly coincident with what Grill-Spector et al. (1999) refer to as PF, the anterior Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 152 subsection of the LOC. A more lateral, posterior, and superior activation was also seen on the right side (Fig. 6.5b), in an area likely corresponding to LO. Both of these areas show greater activity for objects as compared to their scrambled controls, suggesting that the presence of more objects in our Low preference images may be driving this difference in activity. Using the ratings of the presence of objects in our stimuli from Chapter Five, we found that there were significantly more objects in the Low preference group compared to the High preference group (t = 2.65, p = 0.01). Interestingly, the Medium preference group had the highest ratings of “objectness.” -5 0 (V o x e lw is e Z ) C lu s te r S ig n ifican ce: p < 0 .0 2 Figure 6.5. Low preference minus High preference for all sixteen subjects, within those voxels showing a visual response as defined by a contrast of all conditions minus blank. The Z map was thresholded at 3.8 and the resulting clusters tested for significance. A) An axial slice at z = -11 (11mm inferior of the anterior commissure) showing significant clusters bilaterally at the border of the inferior occipital gyrus and the posterior fusiform gyrus. B) A coronal slice at y = -69 (69mm posterior of the anterior commissure) showing a significant cluster in the right middle occipital gyrus. Note that the activation shown in B is not visible in A. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 153 The contrast of First presentation minus Fifth presentation (Fig. 6.6) produced large, bilateral clusters which start in the posterior fusiform and lingual gyri and grow stronger in the anterior direction, peaking in a posterior region of the parahippocampal gyrus. No clusters were found in visual areas which showed greater activity for the Fifth than the First presentation. (V o x e lw is e Z ) C lu s te r S ig n ifican ce: -lo g lO (p ) > 12 Figure 6.6. First presentation minus Fifth presentation for all sixteen subjects, within those voxels showing a visual response as defined by a contrast of all conditions minus blank. The Z map was thresholded at 3.8 and the resulting clusters tested for significance. A) An axial slice at z = -8 (8mm inferior of the anterior commissure) showing bilateral activation of the posterior parahippocampal gyri extending posteriorly into the fusiform gyri. B) A coronal slice at y = -40 (40mm posterior of the anterior commissure) showing the same activations as in A. The change in activity from the first to fifth presentation was not monotonic in all parts of the brain. Early visual areas in particular showed a significant rebound on the fifth presentation (Fig. 6.7). This result, which was quite unexpected, is perhaps one which would not have been seen with fewer repetitions, as is usually the case in studies of fMRI adaptation (FMR-A; Grill- Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 154 Spector, Kourtzi & Kanwisher, 2001). When asked what their experience was like during the experiment, several of our subjects spontaneously reported that on the latter repetitions of an image they often found themselves searching the image for some type of new detail or change in the stimulus. This rebound in early activity on the fifth presentation (and possibly the third and fourth in some conditions) likely reflects an active reexamination of the scene as a consequence of the much reduced activity produced by the stimulus aspects which were attended on the first few presentations. In order to maximize firing (and preference), the subjects begin to attend to different aspects of the scene. This explanation is further supported by the cluster results obtained when we examined the effects of repetition and preference outside of those areas defined by the Stimulus minus Rest mask. Both later repetitions (Fig. 6a), and, to a lesser extent, low preferred images (Fig. 6b), led to greater activity in regions of the parietal cortex implicated in the voluntary control of attention (Yantis & Serences, 2003; Corbetta & Shulman, 2002). Given the nature of the task - “attend as best you can for the entirety of the scan” - this result suggests that allocating attention to non-preferred scenes requires more attentional effort than paying attention to scenes which are pleasurable to look at. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced w ith permission o f th e copyright owner. Further reproduction prohibited without permission. A Exposure Num ber: 1 2 3 4 5 Figure 6.7. After an initial decrease, activity rebounds on the fifth exposure. A) Three axial slices shown from an inferior (top picture) to superior (bottom picture) position in the brain, showing the average activity (in percent signal change from baseline) over the five exposures. Note that these are not statistical images, but estimated signal magnitudes (greater than baseline). B) A coronal view showing the z position of the three slices in A: -12, -3, and 6 (Talairach z coordinates). 156 (V o x e lw is e Z ) C lu s te r S ig n ifican ce: p < 0 .0 5 Figure 6.8. The effects of Repetition and Preference in parietal cortex. A) Fifth presentation minus First presentation, for all sixteen subjects, showing a significantly greater activity in the precuneus and the right inferior parietal lobule for the last presentation over the first. The Z map was thresholded at 3.8 and the resulting clusters tested for significance. Note that an activation in the supramarginal gyrus was significant at a lower level Z threshold but was extensively connected to a large cluster. B) Low minus High preference, showing greater activity for Low preferred scenes in the parietal cortex. The Z map was thresholded at 1.9 and the resulting clusters tested for significance. Note that the resulting cluster was quite large and extended throughout the parietal cortex, but that the effects were in general quite weak. C) This midsaggital section shows the z level (34 mm superior to the anterior commissure) for the slices shown in A and B. Unlike traditional accounts of attention, which might predict greater activity in attentional networks when viewing stimuli which a person enjoys looking at for a long time, this result provides a novel interpretation of attention - endogenous attentional effort is required when one must pay attention (either due to an external pressure or in service of another drive state) to stimuli which do not, on their own, provide a novel, richly interpretable experience. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 157 The main reason why we were able to see this effect is that we fixed the amount of time that a picture was viewed, rather than letting spontaneous selection determine the duration of attention to a specific scene. If we had done the latter then, no doubt, we would have observed that preferred pictures had more attentional effort devoted to them relative to nonpreferred pictures. The issue is the amount of effort required per unit time that a picture is attended: Preferred pictures require less attentional effort per ms of attention than nonpreferred pictures. Recent accounts detailing the existence of two distinct cortical attentional networks (Corbetta & Shulman, 2002) subserving voluntary attention (inferior parietal sulcus/superior parietal lobule/superior frontal cortex) and stimulus driven attention (temporoparietal cortex and inferior frontal cortex) are consistent with this interpretation. Indeed, our results also show greater activity in the right superior frontal gyrus (and, to a lesser extent, the left superior frontal gyrus) for the fifth compared to the first presentation (Fig. 6.9). It is important to note that these results showing heightened activation in attentional areas for scenes that were of tow initial preference or viewed after Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 158 (V o x e lw is e Z ) C lu ster S ignificance: p < 0 .0 0 6 Figure 6.9. The right superior frontal gyrus shows greater activity on the Fifth presentation of a stimulus than the First (sixteen subjects). The Z map was thresholded at 3.8 and the resulting clusters tested for significance. extensive repetitions are opposite to what was found for later areas (the parahippocampal and anterior lingual/fusiform gyri) in which preferred and novel scenes produced the greatest activation. This pattern of results rules out the possibility that the later activity merely reflected the feeding forward of activity from earlier stages. Selection of a subset of subjects. Given the high spatial variability in the locus of individual preference effects and restricted size, we asked whether a subset of our subjects showed overlap in the locus of any preference effect2. The High > Low preference contrast image for each subject (normalized into 2 Given that, on average, only 46% of the preference ratings of individual subjects was predictable from the group averages, considerable variability would not be unexpected between subjects in the neural activity that was predictable from the group averages. A planned study would collect preference ratings in the magnet for each subject rather than relying on predictions from a group of subjects. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 159 standard space) was thresholded at a very low level (z > 1), and summed across subjects (Fig. 6.10). The highest degree of overlap was observed in the posterior part of the left parahippocampal gyrus, with nine out of sixteen subjects showing overlap at seven voxels, and eight of those showing an overlap at twenty-eight voxels. W e then performed an analysis for group effects within this subset of nine subjects. Again, we were interested in asking whether any voxels within that region showing activity to visual stimulation (i.e., stimulus greater than no stimulus) showed greater activity to the image set with High a priori preference over the image set with Low a priori preference. Within this subset of subjects, we found a single cluster of voxels in the left parahippocampal gyrus which was significant at the group level (Fig. 6.11). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 0 (Number of Subjects) Figure 6.10. Voxelwise overlap of preference effects. A) The number of subjects showing greater activity for High than for Low preference images at each voxel in standard space. This image was produced by thresholding each subject’s High minus Low contrast at z > 1 and adding these images together. These nine axial slices progress from the most inferior (upper left) to most superior (lower right). B) This coronal section shows the location of the 9 slices shown in A, which are spaced 4mm apart. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 161 C lu ster Threshold p < 0 .0 5 -4 .4 7 (V o x e lw is e Z ) T alairach : -14 (Left), -34 (Posterior), -14 (Inferior) Figure 6.11. High minus Low preference for nine subjects. Within those voxels which showed a significant effect of image presentation versus rest (with a cluster level significance of p < 0.015), we first thresholded the High minus Low contrast at a z of 3.8 to produce clusters, and then tested for the significance of those clusters at p < 0.05. One cluster in the left parahippocampal gyrus survives, which is shown in a A) coronal view, B) saggital view, and C) axial view. Table 6.1 presents a summary of the effects we found for both preference and repetition. Overall, the effects of repetition were several orders of magnitude greater than those of a priori image preference, though this is not very surprising given the established upper limit for the ability of a priori image preference for this stimulus set to predict preferences for individual subjects as noted earlier (and in Chapter Four). It is perhaps surprising that the largest effects of a priori image preference were negative: activity in the lateral occipital complex (both a lateral, more superior locus on the right side and more inferior bilateral loci on the borders of the fusiform gyri) was greatest for low and medium preferred images. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 162 Summary of BOLD Effects: Preference and Repetition Structure Size (Voxels) Max Z Max Location Signif. Iog10(p) X y z (16 Subjects) Repetition: 1> 5 L parahippocampal g. (36) 1291 7.79 18 -40 -10 16.2 R parahippocampal g. (36) 896 7.01 -25 -42 -8 12.9 R cuneus (19) 395 6.17 30 -84 25 7.69 L. Lingual g. (17) 137 4.73 -17 -87 4 4.09 L Cuneus (17) 82 4.54 -5 -79 10 3.07 L middle occipital 9- (18) 70 4.94 -43 -77 1 2.82 L Middle Occipital g. (19) 29 4.75 -35 -87 20 1.83 L parahippocampal g./subiculum(27/30) 21 4.31 -17 -32 -2 1.59 R parahippocampal g. /perirhinal (35) 14 4.19 20 -19 -13 1.36 Table 6.1. Summary of significant effects. Brodmann areas or functional designations are in parentheses. The location of the voxel with the maximum Z within the cluster is reported in Talairach coordinates, g., gyrus; post., posterior; LOC, lateral occipital complex; PF, posterior fusiform, signif., significance; max, maximum. All effects were computed by thresholding the contrast’s Z image at 3.8 and then testing for significance at p < 0.05, **except for the right supramarginal gyrus — this activation was extensively connected to other clusters and so was tested only at the voxelwise level. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 163 Structure Size (Voxels) Max Z Max Location Signif. Iogi0(p) X y z Repetition: 5 > 1 L precuneus (7) 476 5.76 -5 -66 35 7.68 R superior frontal g. (10) 90 5.09 30 57 15 2.28 R inferior parietal lobule (40) 46 4.79 66 -30 30 1.33 Preference: L > H R middle occipital g. (LOC) 175 5.18 46 -69 7 4.71 L post. Fusiform (PF) 36 4.3 -41 -56 -11 2.02 R post. Fusiform (PF) 26 4.52 34 -54 -11 1.74 R supramarginal g. ** 3.11 42 -44 35 ** (9 Subjects) Preference: H > L L parahippocampal g. (36) 15 4.47 -14 -34 -14 1.31 Table 6.1 (continued). Group Analysis: Regions of Interest ANOVA Given the problems with subject registration and anatomical variability in the whole brain group analysis, we sought to create a set of more constrained statistical hypotheses using a region of interest analysis. By selecting a set Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. of voxels in each subject’s individual data space and averaging the timecourses over those voxels, one can eliminate the drastic reduction in statistical power one suffers when performing a whole brain analysis. In our region of interest analysis, we focused on the effects of preference and repetition within a region of the parahippocampal gyrus, LO, early visual cortex, and the lingual gyrus. Regions of interest were drawn based on either anatomical scans (for the parahippocampal gyrus, lingual gyrus, and early visual cortex) or separate localizer scans (for LO) in each individual subject. For the LO localizer scan, we examined a contrast of Intact > Scrambled, collapsed across all image types (novel, familiar, line drawing, and grayscale object). This contrast was then superimposed on the subject’s high resolution EPI scan and used to select a contiguous region of voxels in the lateral occipital cortex, primarily in the middle and inferior occipital gyri. No attempt was made to differentially identify a more posterior (LO) versus anterior (PF) locus, though it is likely that the majority of ROI’s drawn reflect the more posterior and superior (LO) part of the lateral occipital complex. These ROIs were then used to extract the original waveform from the individual subject’s timeseries data, which was analyzed using a GLM-based averaging procedure implemented within AFNI (Analysis of Functional Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 165 Neurolmages; Cox, 1996). Importantly, instead of using a canonical impulse response function (IRF) to fit the data, this procedure estimates the impulse response function from the data itself. The data from the nine subjects selected in the previous analysis are shown in Fig. 6.12. However, the data for both left and right LO are from only seven of those subjects, as independent localizer scans were not available for two of the nine subjects. High preferred images led to a greater response than either Medium or Low preferred images in the left parahippocampal ROI (Fig. 6.12a), and a greater response than Low preferred images in the right parahippocampal ROI (Fig. 6.12b). LO showed less activity for High preferred images compared to Low preferred, while the Medium image group showed the greatest response in these ROI’s (Fig. 6.12 c & d). This result is in agreement with the finding that the Medium image group contained, on average, more objects than the Low image group, which in turn contained more objects than the high image group. Striate cortex showed no difference in the three preference groups (Fig. 6.12e), though it did show much higher average activity. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 166 A Left Parahippocam pal R O I: Preference U .3 — High — Medium — Low 0.2 cn (Dashed lines indicate standard errors of the subject means) o.i 5.0 7.5 10.0 12.5 Time (seconds from picture onset) 15.0 17.5 2.5 B Right Parahippocampal ROI: Preference 0.4 High Medium Low 0.35 0.3 0.25 0.2 cn 0.15 0.05 (Dashed lines indicate standard errors of the subject means) 0.05 17.5 7.5 10.0 12.5 Time (seconds from picture onset) 15.0 2.5 5.0 Figure 6.12. Region of Interest (ROI) analysis for the nine subjects subselected in the previous section. The average hemodynamic impulse response functions for High, Medium, and Low Preference are plotted for ROI’s in A) left parahippocampal cortex, B) right parahippocampal cortex, C) left lateral occipital cortex, D) right lateral occipital cortex, E) and striate cortex. The data for the LO ROI’s does not include two subjects for which an independent localizer scan was not available. Dashed lines indicate the standard errors of the subject means. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 167 C Left Lateral Occipital (L O ) R O I: Preference U.bi High Medium Low 0.4 ■ e □ 0.2 7 5 c g > In (Dashed lines indicate standard errors of the subject means) 0.2 15.0 17.5 7.5 10.0 12.5 Time (seconds from picture onset) 2.5 5.0 D Right Lateral Occipital (LO) ROI: Preference 0.7 High Medium Low 0.6 0.5 0.4 0.1 0.1 (Dashed lines indicate standard errors of the subject means) i i i 0.2 15.0 17.5 7.5 10.0 12.5 Time (seconds from picture onset) 2.5 5.0 Figure 6.12 (continued). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 168 S triate C ortex R O I: Preference — High — ~ Medium — Low 0.8 0.6 o > 0.4 0.2 _ (Dashed lines indicate standard errors of the subject means) 0.2 o 2.5 5.0 7.5 10.0 12.5 Time (seconds from picture onset) 15.0 17.5 Figure 6.12 (continued). Using the canonical hemodynamic response function provided with the SPM 99 software (Friston et al., 1995), a fit was calculated by multiplying this model function with the hemodynamic response functions estimated for each condition in each ROI (Fig. 6.13). Within the left parahippocampal ROI, High preferred images produced greater activity than Low preferred images for the first three exposure levels, with the difference disappearing on the fourth and fifth presentation. The Medium image category, however, showed a somewhat different profile. On average, activation in this condition fell between that for the High and Low image categories (Fig. 6.12a), though its activity on the first presentation in the left parahippocampal ROI was less than that of the Low preference images (Fig. 6.13). In addition, it appears as Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 169 if the slight rebound seen on later repetitions in this area was largely driven by the Medium image category. Left Parahippocampus ROI: Canonical HRF Model Fit High Medium Low 0.3 c 0.15- cn .X o . i 0.05- (Dashed lines indicate standard errors of the subject means) Exposure Num ber Figure 6.13. Using the canonical hemodynamic impulse response function (HRF or IRF) from SPM99 as a model, a fit was derived for each condition and plotted. Dashed lines indicate the standard errors of the subject means. Individual Differences in Activation Patterns While the effects of repetition appear to be quite robust across individual subjects, the effects of preference are much more variable. Overall, it appears as if a majority of subjects show a positive effect of a priori image Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 170 preference in a posterior region of the parahippocampal gyrus, though to varying degrees. Given the variability in the statistical reliability and spatial locus of effects in individual subjects, it is difficult to define a single quantitative measure capturing the presence or absence of this effect in all subjects. However, it appears as if approximately twelve of the sixteen subjects show at least some degree of preference related activity in an area in or near the parahippocampal cortical locus we have identified. As shown previously, nine of these show overlap of these effects on a small subset of voxels. At the individual subject level, however, these effects can be quite robust (Fig. 6.14). (Voxelwise Z) C luster Threshold: p < 0 .0 5 Figure 6.14. Two individual subjects showing strong effects of preference in the parahippocampal gyrus. The High > Low contrast was thresholded at 3.1 for the subject in A and 2.5 for the subject in B to produce well defined clusters. These clusters were then tested for significance at the 0.05 level. These statistical images have been overlaid on the subject’s own high resolution EPI image. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 171 In addition to this characteristic activity pattern, several subjects showed effects of preference scattered throughout the ventral visual pathway. Figure 6.15 shows an example of a subject with preference effects at several loci in the frontal cortex and early visual areas, in addition to an activation in the right lingual gyrus (just posterior to the locus observed in the group average, but in the opposite hemisphere). 3 .8 0 Cluster Significance: p < 0.05 (Voxelwise Z) Figure 6.15. An individual subject showing preference effects (High preference greater than Low preference images) at a number of locations within the ventral visual pathway and frontal cortex. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 172 Future studies which collect preference ratings in the magnet for each individual subject may reduce the amount of variability in the individual preference results, and allow for a more meaningful investigation of the differences between subjects. Repetition “ Rebound” The group data show a rebound in large sections of the ventral visual pathway for the later repetitions (most evident on the fifth repetition but present on earlier repetitions as well). This effect is also evident at the single subject level. To rule out the possibility that this effect may have been due to slight differences in the mean position of an image’s first repetition and its last repetition (Fig. 6.16a), we examined the effects of repetition for a different general linear model which did not differ in the mean position of the repetition conditions. The same effects are present in this image-based correlation analysis, which included all of the “buffer” images in the parameter estimates for the five repetitions, eliminating any difference in the mean trial position (Fig. 6.16b). In addition, as can be seen from the parahippocampal ROI data (Fig. 6.13), this rebound was not identical for all three preference levels, while there were no differences in the average serial position of the images in the different preference categories. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 173 Figure 6.16. The rebound seen on the fifth presentation of an image is not due to differences in stimulus timing. A) Input vectors for First (top, red) and Fifth (bottom, green) repetitions for the High Preference images in the fifteen parameter factorial model. The fifth presentation of each image came, on average, 80 TR later than the first presentation. The vectors for Medium and Low conditions are similarly staggered. B) Input vectors for the First (top, red) and Fifth (bottom, green) repetitions in the six parameter image-based correlation analysis (this analysis was analyzed separately for the two sessions). Note that for this model there is no overall temporal shift in the input vectors for First and Fifth presentations, as the buffer images were included. However, a rebound in signal on the fifth presentation was still observed. C) Representative data from one voxel analyzed using the parameters in A or B. Image-Based Correlation Analysis The grouping of our image set into high, medium, and low preference categories allowed us to design a fully factorial, mixed design experiment. However, these groupings were based on the mean ratings of a separate set Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 174 of subjects, allowing us to perform an image-based correlation analysis which capitalized on the variation within each preference level. W e used this analysis technique to create a summary of how preference correlated with activity at various stages within the ventral visual stream. W e sought to assess the plausibility of a graded change in preference-related activity as would be predicted by the involvement of p opioid receptors as mediators of visual preference. A major problem with this analysis is the lack of easily defined anatomical “stages” in the ventral visual pathway: the subject variability in both the physical placement of a particular anatomically defined cortical field and the mapping between anatomy and functionality make it very difficult to define a set of processing stages in all subjects. W e did not perform independent localizers for faces, places and retinotopic cortex in all of our subjects, which would have allowed us to select such cortical fields based on the functional response of each voxel. Instead, we overcame this variability by choosing a set of four cortical regions defined by a combination of individual anatomy, independent functional localizers, and stimulus factor deconvolutions of our scene preference experiment. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 175 The first ROI chosen was striate cortex, defined anatomically in the subject’s data space as described above. The second ROI chosen was defined as object responsive cortex. For this ROI, we used both the LO ROI defined above using an independent localizer scan as well as an ROI defined from the scene preference experiment using the “objectness” factor described in Chapter Five. The extraction of this and the next two RO I’s will be described in detail below. Our third ROI was defined as “ face” responsive cortex, and our fourth ROI as those voxels responding to our “place” factor. While these four RO I’s did not cover the entirety of the ventral visual pathway, they allowed us to look at the correlation of a priori image preference and the BOLD signal at four points along the ventral visual pathway defined in individual subjects. The latter three RO I’s were defined using a stimulus based deconvolution of the data from our scene preference experiment. One subject was excluded from this analysis completely due to a lack of data for some of the sixty images used. For the remaining fifteen subjects, four regressors were created by taking the image sequence used for that subject and specifying each image’s rating on a subset of the factors reported in Chapter Five. For this analysis, we used RMS contrast, objectness, the presence of faces, and the degree to which each image depicted a place. Each of these factors was scaled to vary between zero and one, and we also included a fifth factor Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 176 specifying the exposure number of each image, also scaled to vary between zero and one. These five factors were then used in a general linear model similar to that described above for the fully factorial analysis (Fig. 6.1), including each factor plus its temporal derivative as parameters in FILM. RMS contrast and exposure number were included to help make the model a more complete description of the signal components we expected to see and remove any contribution of contrast or repetition from the estimates of the other factors. The computed parameter estimates were then used to derive contrasts and t tests for each factor. Unlike other work which has used pure stimuli to nominate stimulus selective cortical fields, we did not create contrasts which, for example, nominated voxels which responded greater to places than to objects and faces. Since our basic method of analyzing each factor’s contribution to a voxel’s activity is through correlation, any voxel which responds strongly to more than one factor will already have a lower weight than one which responds more selectively. W e therefore used contrasts which weighed the factor of interest as one and all other factors as zero. In order to produce masks with well defined regions, we then thresholded these contrasts (t-statistic images) at a voxelwise t of 2.58 (corresponding to p < 0.01 uncorrected), and subsequently eliminated all clusters which were not significant at the cluster level threshold of 0.3 (using Gaussian Random Field Theory as described Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 177 previously). This very relaxed threshold produced factor related clusters for most subjects and most factors. Each of the masks was then edited to remove any clusters which were not in the ventral visual pathway, loosely defined as the occipital lobe plus the inferior temporal, fusiform, and parahippocampal gyri. While this method allows us to draw “stimulus specific” regions of interest, there is a potential drawback. These regions were not selected using “pure” stimuli, but only a specification of when each image contained the specified stimulus type. Therefore, any spurious correlations between different stimuli present in the images will hinder our ability to define category specific voxels. While the correlations between most of the factors were low enough to not make this a concern, the correlation of 0.43 between the presence of objects and faces may have led to some difficulty in defining separate clusters of voxels. In the present analysis, we allowed voxels which were nominated strongly by more than one factor to be part of both ROI’s. The surviving voxels for each covariate were then used as a mask to select voxels from each subject’s scene preference experiment. The timecourses of each set of voxels were averaged together to form single timecourses for object responsive voxels, face responsive voxels and place responsive voxels. A similar timecourse was constructed by averaging the voxelwise Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 178 timecourses of the anatomically defined striate ROI and the LO ROI on each side defined by the independent localizer scan. Note that unlike this ROI, the ROI’s defined by the factor covariates did not distinguish between the left and right hemisphere, and in some cases included sets of non-contiguous voxels. For example, it should be noted that the “ face responsive voxels” include two non-contiguous areas on each side of the brain, one in a more ventral location corresponding to the fusiform face area and a second on the lateral surface of the occipital lobe. These timecourses were then used to derive IRF’s for each of the sixty images used in the experiment for each ROI in each subject. This IRF was then averaged over the set of subjects with data for that ROI, and each timepoint of this average IRF was then correlated with a priori image preference for each ROI individually. As can be seen in the ROI plots in figure 6.12, the second timepoint is the maximum of the IRF. W e therefore report the correlation between preference and average activity of the second timepoint for each factor. These correlations are reported in Table 6.2. Figure 6.17 shows the average location of each of these 4 RO I’s superimposed on an average anatomical brain with the corresponding correlations. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Correlation with Preference by ROI Region of Interest Correlation of Activity and Preference Number of Subjects Striate Cortex -0.053 15 Object Responsive Cortex (from preference scan) -0.295 6 Left LOC (from LOC localizer) -0.101 11 Right LOC (from LOC localizer) -0.350 11 Face Responsive Cortex -0.091 11 Place Responsive Cortex 0.213 9 Table 6.2. Correlations of activity and preference for four regions of interest. Activity was defined as the maximum height of the average hemodynamic impulse response function over subjects for which each ROI could be established. While the correlations do not progress in a monotonic fashion from early to late visual areas, these results do indicate an increase in the importance of more anterior cortex for stimulus preference. It is likely that the increased complexity of the human ventral visual pathway over that of the macaque may have led to some departure from a strictly linear increase in the importance of anterior cortical fields for preference. It is also possible that the inclusion of non-ventral voxels in the “ face selective” ROI may have weakened the correlation between activity in this ROI and preference. However, as was shown in Chapter Five, the presence of a face in a stimulus is not a good predictor of preference. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 180 A -0.05 Striate Ctx. -0.30 -0.09 Faces 0.21 Placet Figure 6.17. Correlation of image preference and maximal activity in four ROI’s within the ventral visual pathway. The correlation is near zero in striate cortex, is negative in object responsive voxels, approaches zero again in face responsive voxels, and is positive for place responsive voxels. These correlations were computed by taking the average waveform in the preference experiment over all voxels of the ROI (disregarding laterality), computing the IRF for each image, and correlating this with that image’s a priori preference. A) The regions of interest are displayed on a series of axial slices of the MNI standard atlas spaced 8mm apart, starting with the most inferior slice in the upper left (Talairach z = -19) to the most superior in the lower right (Talairach z = 16). B) This rendering shows the position of the four ROI’s superimposed on the MNI standard atlas with the inferior and posterior sections of brain remaining. The position of the brain is as if it were being viewed from above and slightly to the left. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Conclusions 181 Scenes that received high preference ratings produced greater activity in the parahippocampal gyrus than scenes that received low preference ratings, an effect consistent with the endomorphin hypothesis. However, individual subjects appear to show large variation in the anatomical locus of preference related activity. The fact that several individual subjects show very strong effects of preference, despite their lack of anatomical overlap, may be an indication that: a) the spatial distribution of endomorphin receptors is variable across individuals, and/or b) the neural loci of the cells sensitive to variables that affect scene preference are variable across individuals. While it is likely that human opioid gradients would follow a general increase from early to late areas, either or both of these factors could be the reason for two people seeing the world in the same way but with preferences for differing aspects of the same stimulus. This positive effect of preference was generally not apparent in the earlier stages of the ventral pathway and negative in area LO. The negative relationship between preference and the LO response provides strong evidence that overall activity in the ventral pathway is not what leads to preference for a stimulus, but only activity in the more anterior regions that Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 182 are likely activated by rich interpretations and are associated with high levels of endomorphin receptors. Repetition produced a consistent decline in activity in the parahippocampal gyrus which tracked the preference ratings but produced a U-shaped function in more posterior areas. The increase in activity over repetitions 3 through 5 likely reflected an increase in active processing of the scenes. It is interesting to note that repetition, which accounted for less of the variance than a priori preference in Experiment 2 (9.4% and 78% respectively) led to much greater activity differences in the resultant BOLD signal. This suggests that the activation of a very few cells in the most anterior, endomorphin-rich areas of the temporal lobes can outweigh the activity from a larger population of cells in a more posterior area. In a sense, the neural activity subserving preference effects may retain the mapping observed in the ventral pathway in general whereby there is a many-to-one mapping in that receptive fields in anterior temporal lobe (which in TE can be larger than 70 degrees) are dramatically larger than those in V1 (0.5-2.0 degrees). Our finding of preference and repetition effects in a posterior portion of the parahippocampal gyrus have interesting implications for a growing body of research implicating a similar, if not overlapping region in the representation of places (Epstein et al., 1999; Talairach coordinates 20, -39, -5 and -28, -39, Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 183 -6) and the analysis of visual context (Bar & Aminoff, 2003; Talairach coordinates -24, -41, -4). Our preference related activation was just anterior and inferior to this activation (-14, -34, -14), while our repetition related decreases were virtually overlapping (18, -40, -10 and -25, -42, -8). From an adaptation standpoint, it seems reasonable that the repetition of an image set comprised primarily of full color scenes would lead to repetition related decreases in an area which is involved in the representation of place. In addition, highly contextual objects are more relatable to memory and lead to richer interpretations. What our work demonstrates is that objects and scenes rich in contextual, interpretable information are also pleasurable to look at - at least the first few times. Our results provide some confirmation of the theoretical account of information foraging behavior: high activity in the opioid receptor rich areas of the ventral temporal cortex provides a “ win” signal to the brain systems mediating response selection and preference. Neural adaptation of this signal through competitive learning leads to a decrease, triggering a “shift” response to other stimuli (or aspects of the scene) which is likely to result in additional novel associations. Under circumstances in which there is no alternative to a boring stimulus yet there is a motivation to maintain attention (what else are you going to do in the magnet?), voluntary attentional effort is required. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 184 CHAPTER 7: CONTRIBUTIO NS AND FUTURE DIRECTIO NS The goal of this thesis was to develop a theory of cognitive and perceptual pleasure by investigating the behavioral properties of visual preference and its relation to the known organization of the ventral visual pathway. Below I will outline the specific contributions this work has made to our understanding of positive perceptual affect. In addition, I will outline several future directions suggested by the work. Several of these questions will be addressed as part of my postdoctoral research efforts. Contributions A major contribution of this work is the recognition of a need for a theory of perceptual affect. Understanding how the brain analyzes the visual world does not answer the question of motivation: why are some things processed preferentially, and how does the brain choose what gets ignored and what gets processed? Although issues of stimulus selection have been studied under the guise of visual attention, the need for links to stimulus interpretability and the pleasurable aspects of obtaining information had not been recognized. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 185 The behavioral data presented here have contributed significantly to our understanding of perceptual affect. Using a well defined, naturalistic stimulus set and rigorous testing situations, we replicated previously shown effects of stimulus repetition on preference, and showed that these effects hold over a wide range of initial preferences. Second, we replicated previous reports in the landscape assessment literature on the predictability of preference from mystery, naturalness, and the presence of a good view (vista). W e showed that for full color scenes, these types of factors account for a larger proportion of stimulus preference than factors such as the presence of objects, faces, and places. In addition, we also provided a unifying framework for understanding the effects of novelty, familiarity, and stimulus interpretability on preference. Third, our brain imaging experiments have had a significant impact on our understanding of the physiological basis of perceptual affect. Using BOLD fMRI, we found that highly preferred images and novel images lead to greater activity in parts of the parahippocampal cortex, and that these same areas showed the highest correlation with image preference. This finding of preference-related activity in a perceptual pathway provides evidence for the possible involvement of opioid gradients present in the human and non human primate ventral visual pathway. A major aspect of this contribution was the development of a balanced, event-related fMRI paradigm for testing Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 186 the effects of repetition in a mixed design that also controlled for effects of preference context. Our imaging results also show that these effects observed in later visual areas are not merely feedforward effects from earlier areas, as preference was negatively correlated with activity in area LO. This data underlines the importance of a differential weighting of activity like that which would be produced by a neurochemical gradient. Finally, our results provide a novel interpretation of voluntary attention, in that we found evidence for the activation of brain networks thought to be involved in the deployment of voluntary attentional resources when stimuli are non-preferred either due to less intrinsic interest (interpretability) or extensive repetition. Future Directions The theory we have proposed raises a large number of questions which we have only begun to address in the series of experiments presented here. Below, I will outline several directions in which this work could be expanded in order to test specific claims of the theory and its generality to other stimulus domains. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Behavioral Studies 187 The most basic extension of the work we have done is to test whether the decrease in repetition with repeated exposure we observed in Experiments 1 and 2 are solely a function of the number of times an image has been seen, or whether the observed effect is modulated by presentation time. In all of our experiments, the images were shown for a full second. As was discussed in Chapter Three, the effects of preference may be linked with the phenomenon of priming. Zago & Bar (2003) showed that a prime on the order of 250 ms is most effective, while both shorter and longer durations are less effective as primes. Due to the hypothesized importance of visual preference for information foraging, it is possible that the peak of this function at 250 ms may be related to the time of an average fixation. A 250 ms exposure of an image may be sufficient to produce the repetition related decreases in preference we observed in our experiment. Similarly, it is possible that shorter exposure durations may not lead to repetition related decreases, and may even reflect an inverted U shaped function as a result of initial “mere exposure” effects and subsequent habituation. By testing several shorter exposure durations, we can test how tightly linked visual preference is to priming, and whether the mere exposure effect and repetition related decreases in preference are two ends of a continuum. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 188 In addition to timing effects, we also plan to investigate whether there are Differences in preferences when ratings are collected on every presentation versus passive viewing. This will allow us to address whether the act of making a preference judgment influences the degree of preference habituation. Eyetracking Although preference judgments are quite stable within an individual, it is unclear whether such explicit judgments provide an accurate measure of perceptual affect. The subject’s expectations, social norms and changing criterion may all adversely affect the link between a person’s perceptually driven affective reaction and their rated preference. Since a major functionality of an information foraging system is to guide spontaneous selection, we plan to measure how well eye movement patterns can be predicted from preference judgments. As a simple test, we plan on using a preferential looking paradigm like that used in infants, in which two pictures are shown to a subject side-by-side on the screen while eye movements are monitored. On any one trial, the subject will be shown two images side by side on a computer monitor which differ in their a priori preference (high, medium, or low), the number of times they have been seen before, or both. In a similar task, Berlyne (1958) and found that total fixation time (but not first Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. fixation) decreased monotonically to a repeated picture, though he did not assess effects of preference. W e predict that total fixation time and/or first fixations will be greater for images of high a priori preference and images which are novel. Individual Differences in the Neural Correlates of Preference Our ability to detect preference related changes in Experiment 4 was limited by the fact that group preference ratings are able to account for an average of only 46% of the variance in individual preferences. This tradeoff allowed us to observe preference related changes in the absence of explicit preference judgments. To complement our findings, we plan to perform an FMRI experiment in which preference ratings are collected for all 200 images as the subject views them in the magnet. This design, while it does not dissociate between activity related to affective preference and that related to the act of making preference judgments, will allow us to capitalize on the individual differences in preference and better localize preference related neural activity. In addition to the activity we observed in the ventral visual pathway and parietal regions in Experiment 4, we would predict that frontal networks shown to be involved in decision making and executive functions should also be active (Fuster, 2000). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Pharmacology 190 As a direct test of the involvement of the |J opioid receptor gradient in positive perceptual affect, normal subjects could be administered p opioid agonists (such as fentanyl) or antagonists (such as naloxone or naltrexone) while performing a visual preference task. In order to demonstrate a selective effect of a drug such as naloxone on visual preference, we would require the subject to perform several visual tasks in addition to the preference task, such as a visual recognition test and a fear cueing test. These same tasks could potentially be studied in a separate set of subjects on a GABAergic or cholinergic [antagonist] (such as benzodiazapene and scopolamine, respectively) to rule out any nonspecific effects of inhibition or memory impairment on visual preference (e.g. Robbins et al., 1997). Natural vs. Urban: Distribution of Image Factors in the Ventral Visual Pathway The high correlation of preference with the natural vs. urban dimension points to a potentially interesting prediction for how this factor is represented in higher order visual areas. Unlike objects, a landscape viewed from a distance does not require much rotational invariance and contains very few properties which would aid in the decomposition of the landscape into shape Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 191 primitives. Instead, classification of landscapes may be mediated by color, texture, and global form. Light blue above darker blue abutting beige may be a reliable cue combination for a beach, while a solid mass of green may signal forest or park. In a study comparing the naming speeds of “color- diagnostic” and “ color-nondiagnostic” scenes, Oliva & Schyns found that reversing the colors of a color-diagnostic scene (either a canyon, forest, coastline, or desert) increased subjects’ reaction times above that of pure luminance (black and white) versions of the scenes, which were in turn significantly worse than the originals (Oliva & Schyns, 2000). This was in contrast to color-nondiagnostic scenes (which included the categories city, shopping area, road, and room), which showed no cost for either the luminance only or color reversed versions. The authors claim that this difference comes from the fact that in color space, the color-diagnostic scenes are largely non-overlapping, while the nondiagnostic (urban) scenes significantly overlapped. An alternative, but complimentary explanation might be that the urban scenes show no cost due to the relative ease of extracting non-accidental properties and object independent geon structural descriptions. Variation in the natural vs. urban dimension is therefore likely to modulate the balance between structural and textural information and produce differences in the distribution of activity in the form perception pathway. It is possible that this dimension corresponds to a medial / lateral distinction in processing, with urban scenes leading to greater activity in the Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 192 lateral aspect of this pathway (leading out of LO) and natural scenes producing greater activity in the medial aspect (leading out of the lingual gyrus and into the parahippocampal gyrus). An fMRI study could be designed to map such a distribution. A set of such maps would then allow for a more complete picture of whether the activity distribution across the ventral visual pathway can be used to predict stimulus preference, as would be the case if a gradient of p opioid receptors mediates between neural activity and preference. Learning Effects W e plan to investigate how preference interacts with the learning of novel objects and object classes. Preference may affect how well an object is learned, and the context in which an object is learned may in turn affect its subsequent preference. Such relationships likely have important consequences for how particular stimuli are represented in the brain. W e plan to study these questions by varying the interpretability of the context in which an object is learned, and by studying the development of visual expertise for object classes of differing preference. One possible experiment along these lines would utilize a paradigm similar to that of Beauchamp et al. (2001) in which small tokens are shown moving in patterns which are either random, passive (like billiard balls), or mimicking human movement. By Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 193 varying the context in which a token is encountered, we can test whether its association with a rich interpretation leads to a preference for that token by itself. A second set of experiments would involve the development of a training program for a class of visual experts. Performance on an expert discrimination task would be measured within groups of subjects which are learning both a preferred visual discrimination and a nonpreferred discrimination. Such an experiment would allow us to asses whether a person’s preferences for certain types of expert discriminations affect their learning rate for the discriminations and the underlying neural representations of the new object class. Alternatively, a simpler experiment would involve the presentation of images and a preference rating during one session, followed several days (or weeks) later by a recognition test to see if preferred stimuli are better recognized than nonpreferred stimuli. Recognition memory performance and preference could also be related to the amount of neural activity in medial temporal lobe structures previously observed to predict recognition memory (Fernandez et al., 1999). Cross-Modal and Conceptual Priming The theory we presented in Chapter Two can be generalized to perceptual modalities other than vision, and even to the combination of cues from multiple modalities. In addition to testing whether novelty and interpretability Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 194 can be used to predict preference for auditory, haptic, or taste stimuli, it is important to understand how preference is influenced by the degree to which information from two modalities is integrated Direct Measures of Preference A major drawback of the research we have done so far is the lack of a more objective measure of positive affect. While measuring eye movement patterns moves us a bit closer towards a quantitative measure of preference, they are not a pure measure of the contribution of information foraging and may reflect other mechanisms (such as a response to sudden onsets or motivated searches) under certain circumstances. As a means of measuring positive affect more directly, it may be useful to explore the use of EMG. Winkielman & Cacioppo (2001) found that an electromyographic signal from the zygomatic muscle provides a reliable signal of positive perceptual affect even during subliminal exposures. While this technique has not been explored within the context of full color scenes nor under repetition of the same stimuli, it is possible that measuring the activity from the “smile muscles” during perceptual tasks would allow for a quantitative study of the effects of interpretability and habituation on positive perceptual affect. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. BIBLIOGRAPHY 195 Adolphs, R., Denburg, N. L., & Tranel, D. (2001). The amygdala's role in long-term declarative memory for gist and detail. Behavioral Neuroscience, 1/5(5), 983-992. Aitken, P. P. (1974). Judgments of pleasingness and interestingness as functions of visual complexity. Journal of Experimental Psychology, 103(2), 240-244. Appleton, J. (1975). The Experience of Landscape. London: Wiley. Appleton, J. (1988). Prospects and refuges revisited. In J. L. Nasar (Ed.), Environmental Aesthetics (pp. 27-44). New York: Cambridge University Press. Arnsten, A. F. T., Neville, H. J., Hillyard, S. A., Janowsky, D. S., & Segal, D. S. (1984). Naloxone Increases Electrophysiological Measures of Selective Information-Processing in Humans. Journal of Neuroscience, 4(12), 2912-2919. Attneave, F., & Arnoult, M. D. (1956). The quantitative study of shape and pattern perception. Psychological Bulletin, 53, 452-471. Bar, M., & Biederman, I. (1998). Subliminal visual priming. Psychological Science, 9(6), 464-469. Bar, M., & Biederman, I. (1999). Localizing the cortical region mediating visual awareness of object identity. Proc. Natl. Acad. Sci. USA, 96, 1790-1793. Bar, M., Tootell, R. B., Schacter, D. L., Greve, D. N., Fischl, B., Mendola, J. D., Rosen, B. R., & Dale, A. M. (2001). Cortical mechanisms specific to explicit visual object recognition. Neuron, 29(2), 529-35. Bar, M., & Aminoff, E. (2003). Cortical analysis of visual context. Neuron, 38, 347-358. Barlow, H. B. (1961). Possible principles underlying the transformation of sensory images. In W . A. Rosenblith (Ed.), Sensory Communication (pp. 217-234). Cambridge, MA: MIT Press. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 196 Beckmann, C. F., Smith, S. M., & Jenkinson, M. (2001). General multi-level linear modeling for group analysis in FM RI (Technical Report TR01CB1). Oxford, UK: Oxford Centre for Functional Magnetic Resonance Imaging of the Brain (FMRIB). Bergman, R., Williams, T. H., Gluhbegovic, N ., & Jew, J. (2001). Atlas of Human Anatomy in Cross Section, [Web document]. The University of Iowa: Virtual Hospital. Available: http://www.vh.org/Providers/Textbooks/HumanAnatomy/CrossSection Atlas.html. Berlyne, D. E. (1958). The influence of complexity and novelty in visual figures on orienting responses. Journal of Experimental Psychology, 55(3), 289-296. Berlyne, D. E. (1963). Novelty, complexity, incongruity, extrinsic motivation, and the GSR. Journal of Experimental Psychology, 56(6), 560-567. Berlyne, D. E. (1970). Novelty, complexity, and hedonic value. Perception & Psychophysics, 8{5A), 279-286. Biederman, I., & Cooper, E. E. (1991). Priming contour-deleted images: evidence for intermediate representations in visual object recognition. Cognitive Psychology, 23(3), 393-419. Biederman, I., & Gerhardstein, P. C. (1993). Recognizing depth-rotated objects: evidence and conditions for three-dimensional viewpoint invariance. Journal of Experimental Psychology: Human Perception and Performance, 19{6), 1162-1182. Bornstein, R. F. (1989). Exposure and Affect - Overview and Meta-Analysis of Research, 1968-1987. Psychological Bulletin, 106(2), 265-289. Bornstein, R. F., Kale, A. R., & Cornell, K. R. (1990). Boredom as a limiting condition on the mere exposure effect. Journal of Personality and Social Psychology, 58, 791 -800. Bornstein, R. F. (1992). Subliminal mere exposure effects. In R. F. Bornstein & T. S. Pittman (Eds.), Perception Without Awareness: Cognitive, Clinical, and Social Perspectives (pp. 191-211). New York: The Guilford Press. Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10, 443- 446. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 197 Buchsbaum, M. S., Reus, V. I., Davis, G. C., Holcomb, H. H., Cappelletti, J., & Silberman, E. (1982). Role of opioid peptides in disorders of attention in psychopathology. Annals of the New York Academ y of Sciences, 398, 352-365. Buckner, R. L., Goodman, J., Burock, M., Rotte, M., Koutstaal, W ., Schacter, D., Rosen, B., & Dale, A. M. (1998). Functional-anatomic correlates of object priming in humans revealed by rapid presentation event-related fMRI. Neuron, 20(2), 285-296. Buckner, R. L., & Braver, T. S. (1999). Event-related functional MRI. In C. T. W . M. P. A. Bandettini (Ed.), Functional M R I. Bullier, J., Schall, J. D., & Morel, A. (1996). Functional streams in occipito frontal connections in the monkey. Behavioural Brain Research, 76(1- 2), 89-97. Burock, M. A., Buckner, R. L., Woldorff, M. G., Rosen, B. R., & Dale, A. M. (1998). Randomized event-related experimental designs allow for extremely rapid presentation rates using functional MRI. Neuroreport, 9(16), 3735-3739. Cahill, L., Haier, R. J., Fallon, J., Alkire, M. T., Tang, C., Keator, D., W u, J., & McGaugh, J. L. (1996). Amygdala activity at encoding correlated with long-term, free recall of emotional information. Proceedings of the National Academy of Sciences of the United States of America, 93(15), 8016-8021. Cheng, K., Saleem, K. S., & Tanaka, K. (1997). Organization of corticostriatal and corticoamygdalar projections arising from the anterior inferotemporal area TE of the macaque monkey: A Phaseolus vulgaris leucoagglutinin study. Journal of Neuroscience, 17(20), 7902-7925. Cohen, G. A., Doze, V. A., & Madison, D. V. (1992). Opioid inhibition of GABA release from presynaptic terminals of rat hippocampal interneurons. Neuron, 9, 325-335. Corbett, A. D., Paterson, S. J., & Kosterlitz, H. W . (1993). Selectivity of ligands for opioid receptors. In A. Hertz (Ed.), Opioids I (1st ed., Vol. 104, pp. 645-673). Berlin: Springer. Corbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulus- driven attention in the brain. Nature Reviews Neuroscience, 3, 201- 215. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 198 Cox, R. W . (1996). Software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical Research, 2 9 , 162-173. Dale, A. M., & Buckner, R. L. (1997). Selective averaging of rapidly presented individual trials using fMRI. Human Brain Mapping, 5, 329- 340. Donaldson, D. L., & Buckner, R. L. (2001). Effective paradigm design. In P. Jezzard, Matthews, P.M., Smith, S.M. (Ed.), Functional M RI: An Introduction to Methods (pp. 177-195). Elliott, R., & Dolan, R. J. (1998). Neural response during preference and memory judgments for subliminally presented stimuli: a functional neuroimaging study. Journal of Neuroscience, 18(12 (June 15)), 4697- 4704. Epstein, R., Harris, A., Stanley, D., & Kanwisher, N. (1999). The parahippocampal place area: Recognition, navigation, or encoding? Neuron, 23(1), 115-125. Fernandez, G., Brewer, J. B., Zhao, Z., Glover, G. H., & Gabrieli, J. D. (1999). Level of sustained entorhinal activity at study correlates with subsequent cued-recall performance: A functional magnetic resonance imaging study with high acquisition rate. Hippocampus, 9, 35-44. Fiser, J., & Biederman, I. (2001). Invariance of long-term visual priming to scale, reflection, translation, and hemisphere. Vision Research, 41, 221-234. Friedman, D. P., Aggleton, J. P., & Saunders, R. C. (2002). Comparison of hippocampal, amygdala, and perirhinal projections to the nucleus accumbens: Combined anterograde and retrograde tracing study in the macaque brain. Journal of Comparative Neurology, 450(4), 345- 365. Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. B., Frith, C. D., & Frackowiak, R. S. J. (1995). Statistical parametric maps in functional imaging: a general linear approach. Human Brain Mapping, 2 , 189- 210. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 199 Fuster, Joaquin M. (2000). Executive frontal functions. Experimental Brain Research, 133(1), 66-70. Goldstein, A. (1980). Thrills in Response to Music and Other Stimuli. Physiological Psychology, 8(1), 126-129. Goodale, M. A., & Milner, A. D. (1992). Separate visual pathways for perception and action. Trends in Neurosciences, 15(1), 20-25. Grill-Spector, K., Kushnir, T., Hendler, T., Edelman, S., Itzchak, Y., & Malach, R. (1998). A sequence of object-processing stages revealed by fMRI in the human occipital lobe. Hum Brain Mapp, 6(4), 316-28. Grill-Spector, K., Kushnir, T., Edelman, S., Avidan, G., Itzchak, Y., & Malach, R. (1999). Differential processing of objects under various viewing conditions in the human lateral occipital complex. Neuron, 24(1), 187- 203. Grill-Spector, K., Kourtzi, Z., & Kanwisher, N. (2001). The lateral occipital complex and its role in object recognition. Vision Research, 4 1 ,1409- 1422. Hackler, L., Zadina, J. E., Ge, L. J., & Kastin, A. J. (1997). Isolation of relatively large amounts of endomorphin-1 and endomorphin-2 from human brain cortex. Peptides, 18(10), 1635-1639. Harlow, H. F. (1986). From Learning to Love: The Selected Papers o fH .F . Harlow. New York: Praeger Publishers. Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., & Pietrini, P. (2001). Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, 293(5539), 2425-2430. Heerwagen, J. H., & Orians, G. H. (1993). Humans, Habitats, and Aesthetics. In K. S.L. & E. O. Wilson (Eds.), The Biophilia Hypothesis (pp. 138- 172). Covelo, California: Island Press. Heindenreich, S. M., & Turano, K. A. (2003). What predicts where one will look when viewing artwork? (Poster Presented at the 3rd Annual Meeting of the Vision Sciences Society in Sarasota, Florida. MO80). Henson, R., Shallice, T., & Dolan, R. (2000). Neuroimaging evidence for dissociable forms of repetition priming. Science, 287(5456), 1269- 1272. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 200 Hille, B. (1992). Ionic channels of excitable membranes (2nd ed.). Sunderland, Mass.: Sinauer Associates, Inc. Hiller, J. M., & Fan, L.-Q. (1996). Laminar distribution of the multiple opioid receptors in the human cerebral cortex. Neurochemical Research, 21{11), 1333-1345. Hurd, Y. L. (1996). Differential messenger RNA expression of prodynorphin and proenkephalin in the human brain (vol 72, pg 767 ,1 9 9 6 ). Neuroscience, 74(1), 293-293. Ishai, A., Ungerleider, L. G., Martin, A., Schouten, J. L., & Haxby, J. V. (1999). Distributed representation of objects in the human ventral visual pathway. Proceedings of the National Academy of Science, USA, 96, 9379-9384. Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2 0(11), 1254-1259. Iwai, E., & Yukie, M. (1987). Amygdalofugal and amygdalopetal connections with modality-specific visual cortical areas in macaques (macaca fuscata, M. mulatta, and M. fascicularis). The Journal of Comparative Neurology, 261, 362-387. James, T. W ., Humphrey, G. K., Gati, J. S., Menon, R. S., & Goodale, M. A. (2000). The effects of visual object priming on brain activation before and after recognition. CurrBiol, 10(17), 1017-24. Jenkinson, M., Bannister, P., Brady, M., & Smith, S. (2002a). Improved optimisation for the robust and accurate linear registration and motion correction of brain images. Neuroimage, 17(2), 825-841. Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: a module in human extrastriate cortex specialized for face perception. Journal of Neuroscience, 17(11), 4302-4311. Kaplan, S., Kaplan, R., & Wendt, J. S. (1972). Rated preference and complexity for natural and urban visual material. Perception & Psychophysics, 12(4), 354-356. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 201 Kaplan, S. (1992). Environmental preference in a knowledge-seeking, knowledge-using organism. In J. H. Barkow, Cosmides, L. & Tooby, J. (Ed.), The Adapted Mind: Evolutionary Psychology and the Generation of Culture (pp. 581-598). New York: Oxford. Kaplan, R., & Kaplan, S. (1995). The Experience of Nature: A Psychological Perspective. Ann Arbor, Michigan: Ulrich's Bookstore. Khachaturian, H., Lewis, M. E„ Schafer, M. K. H„ & Watson, S. J. (1985). Anatomy of the CNS Opioid Systems. Trends in Neurosciences, 8(3), 111-119 Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220, 671-680. Komar, V., & Melamid, A. (1993). United States: Most Wanted Painting : Dia Art Foundation (http://www.diacenter.org/km). Kourtzi, Z., & Kanwisher, N. (2000). Cortical regions involved in perceiving object shape. J. Neuroscience, 20(9), 3310-3318. Lang, P. J., Greenwald, M. K., Bradley, M. M., & Hamm, A. O. (1993). Looking At Pictures - Affective, Facial, Visceral, and Behavioral Reactions. Psychophysiology, 30(3), 261-273. Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (1997a). International affective picture system (IAPS): Technical manual and affective ratings. (Technical R eport). Gainesville: The Center for Research in Psychophysiology, University of Florida. Lang, P. J., Bradley, M. M., Fitzsimmons, J. R., Cuthbert, B. N., Scott, J. D., Moulder, B., & Nangia, V. (1998). Emotional arousal and activation of the visual cortex: An fMRI analysis. Psychophysiology, 35(2), 199- 210. Lavenex, P., Suzuki, W. A., & Amaral, D. G. (2002). Perirhinal and parahippocampal cortices of the macaque monkey: Projections to the neocortex. Journal of Comparative Neurology, 447(4), 394-420. Lewis, M., Mishkin, M., Bragin, E., Brown, R. M., Pert, C. B., & Pert, A. (1981). Opiate receptor gradients in monkey cerebral cortex: correspondence with sensory processing hierarchies. Science, 2 1 7(13, March), 1166-1169. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 202 Lewis, M. E., Pert, A., Pert, C. B., & Herkenham, M. (1983). Opiate Receptor Localization in Rat Cerebral-Cortex. Journal of Comparative Neurology, 216(3), 339-358. Loschky, L. C., McConkie, G. W ., Yang, J., & Miller, M. E. (2001). Perceptual effects of a gaze-contingent multi-resolution display based on a model of visual sensitivity. Paper presented at the ARL Federated Laboratory 5th Annual Symposium - ADID Consortium Proceedings. Macko, K. A., Jarvis, C. D., Kennedy, C., Miyaoka, M., Shinohara, M., Sokoloff, L., & Mishkin, M. (1982). Mapping the primate visual system with [2-14C]Deoxyglucose. Science, 218(4570), 394-397. Maljkovic, V., & Martini, P. (2003). Different rates of memory formation for scenes with positive and negative affective content (Presentation given at the Vision Sciences Society 3rd Annual Meeting. TA5). Sarasota, FL. Mansour, A., Fox, C. A., Akil, H., & Watson, S. J. (1995). Opioid-receptor mRNA expression in the rat CNS: anatomical and functional implications. Trends in Neurosciences, 18(1), 22-29. Martin, A., & Weisberg, J. (2003). Neural foundations for understanding social and mechanical concepts. Cognitive Neuropsychology, 20, 575- 587. Martin-Schild, S., Gerall, A. A., Kastin, A. J., & Zadina, J. E. (1999). Differential distribution of endomorphin 1 - and endomorphin 2-like immunoreactivities in the CNS of the rodent. The Journal of Comparative Neurology, 405, 450-471. Menon, R. S., & Goodyear, B. G. (2001). Spatial and temporal resolution in fMRI. In P. Jezzard, Matthews, P.M., Smith, S.M. (Ed.), Functional M RI: An Introduction to Methods (pp. 145-158). Miller, E. K., Li, L., & Desimone, R. (1991b). A Neural Mechanism For Working and Recognition Memory in Inferior Temporal Cortex. Science, 254(5036), 1377-1379. Miller, E. K., Li, L., & Desimone, R. (1993). Activity of neurons in the anterior inferior temporal cortex during a short term memory task. Journal of Neuroscience, 1460-1478. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 203 Morris, B. J., & Johnston, H. M. (1995). A role for hippocampal opioids in long-term functional plasticity. Trends in Neuroscience, 18, 350-355. Nasar, J. L. (1988). Visual preferences in urban street scenes: a cross- cultural comparison between Japan and the United States. In J. Nasar (Ed.), Environmental Aesthetics: Theory, Research, and Applications (pp. 260-274). New York: Cambridge. O'Carroll, R. E., Drysdale, E., Cahill, L., Shajahan, P., & Ebmeier, K. P. (1999). Stimulation of the noradrenergic system enhances and blockade reduces memory for emotional material in man. Psychological Medicine, 29(5), 1083-1088. Oliva, A., & Schyns, P. G. (2000). Diagnostic colors mediate scene recognition. Cognitive Psychology, 41(2), 176-210. Oliva, A., & Torralba, A. (2001). Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision, 42(3), 145-175. Oliva, A., Torralba, A., Castelhano, M. S., & Henderson, J. M. (2003). Top- down control of visual attention in object detection. Paper presented at the IEEE International Conference on Image Processing, September 14-17, Barcelona, Spain. Peckys, D., & Landwehrmeyer, G. B. (1999). Expression of mu, kappa, and delta opioid receptor messenger RNA in the human CNS: A P-33 in situ hybridization study. Neuroscience, 88(4), 1093-1135. Pelli, D. G. (1997). The videotoolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision, 10, 437-442. Pert, C. B., & Snyder, S. (1973). Opiate receptor: demonstration in nervous tissue. Science, 1 7 9 ,1011-1014. Pinker, S. (1997). How the Mind Works. New York: W . W . Norton & Company. Pinker, S. (1997). How the Mind Works. New York: W .W . Norton & Company. Quirion, R., & Pilapil, C. (1991). Distribution of multiple opioid receptors in the human brain, Receptors in the Human Nervous System (pp. 103-121): Academic Press. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 204 Robbins, T. W ., & Everitt, B. J. (1996). Neurobehavioral mechanisms of reward and motivation. Current Opinion in Neurobiology, 6, 228-236. Robbins, T. W ., Semple, J., Kumar, R., Truman, M. I., Shorter, J., Ferraro, A., Fox, B., McKay, G., & Matthews, K. (1997). Effects of scopolamine on delayed-matching-to-sample and paired associates tests of visual memory and learning in human subjects: comparison with diazepam and implications for dementia. Psychopharmacology, 134(1), 95-106. Rolls, E. T. (1999). The Brain and Emotion. New York: Oxford. Royet, J. P., Zald, D., Versace, M., Costes, N., Lavenne, F., Koenig, O., & Gervais, R. (2000). Emotional responses to pleasant and unpleasant olfactory, visual, and auditory stimuli: a positron emission tomography study. Journal of Neuroscience, 20(20), 7752-7759. Saleem, K. S., & Tanaka, K. (1996). Divergent projections from the anterior inferotemporal area TE to the perirhinal and entorhinal cortices in the macaque monkey. Journal of Neuroscience, 76(15), 4757-4775. Saleem, K. S., Suzuki, W ., Tanaka, K., & Hashikawa, T. (2000). Connections between anterior inferotemporal cortex and superior temporal sulcus regions in the macaque monkey. Journal of Neuroscience, 20(13), 5083-5101. Schacter, D. S., Chiu, P., & Ochsner, K. N. (1993). Implicit memory: a selective review. Annual Review of Neuroscience, 1 6 ,159-182. Schultz, W ., Tremblay, L., & Hollerman, J. R. (2000). Reward processing in primate orbitofrontal cortex and basal ganglia. Cereb Cortex, 70(3), 272-84. Seamon, J. G., Marsh, R. L., & Brody, N. (1984). Critical importance of exposure duration for affective discrimination of stimuli that are not recognized. Journal of Experimental Psychology: Learning, Memory, and Cognition, 70(3), 465-469. Shepard, R. N., & Cermak, G. W . (1973). Perceptual-cognitive explorations of a toroidal set of free-form stimuli. Cognitive Psychology, 4(3), 351- 377. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 205 Simon, E. J., & Hiller, J. M. (1994). Opioid peptides and opioid receptors. In G. J. S. e. al. (Ed.), Basic Neurochemistry: Molecular, Cellular, and Medical Aspects (5th ed., pp. 321-339). New York: Raven Press, Ltd. Smith, S. (2002). Fast robust automated brain extraction. Human Brain Mapping, 77(3), 143-155. Sobotka, S., & Ringo, J. L. (1994). Stimulus specific adaptation in excited but not in inhibited cells in inferotemporal cortex of macaque. Brain Research, 646, 95-99. Suzuki, W . A., & Amaral, D. G. (1994). Perirhinal and Parahippocampal Cortices of the Macaque Monkey - Cortical Afferents. Journal of Comparative Neurology, 350{4), 497-533. Tanaka, K. (1997). Mechanisms of visual object recognition: monkey and human studies. Current Opinion in Neurobiology, 7(4), 523-529. Thorpe, S., Fize, D., & Marlot, C. (1996). Speed of processing in the human visual system. Nature, 387(June 6), 520-522. Thorpe, S. J., & Fabre-Thorpe, M. (2001). Seeking categories in the brain. Science, 297(5502), 260-263. Tong, F., Nakayama, K., Moscovitch, M., Weinrib, O., & Kanwisher, N. (2000). Response properties of the human fusiform face area. Cognitive Neuropsychology, 77(1-3), 257-279. Tooby, J., & Cosmides, L. (1992). The psychological foundations of culture. In J. H. Barkow, L. Cosmides, & J. Tooby (Eds.), The Adapted Mind: Evolutionary Psychology and the Generation of Culture (pp. 19-136). London: Oxford University Press. Tremblay, L., & Schultz, W . (1999). Relative reward preference in primate orbitofrontal cortex. Nature, 398(6729), 704-8. Turner, B., Mishkin, M., & Knapp, M. (1980). Organization of the amygdalopetal projections from modality-specific cortical association areas in the monkey. The Journal of Comparative Neurology, 191, 515-543. Ulrich, R. S. (1974). Scenery and the Shopping Trip: The Roadside Environment as a Factor in Route Choice. Unpublished Doctoral dissertation, University of Michigan, Ann Arbor. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 206 Ulrich, R. S. (1993). Biophilia, biophobia, and natural landscapes. In S. L. Kellert & E. O. Wilson (Eds.), The Biophilia Hypothesis (pp. 73-137). Covelo, California: Island Press. Ungerleider, L. G., & Mishkin, M. (1982). Two cortical visual systems. In M. A. G. D.J. Ingle, R.J.W. Mansfield (Ed.), Analysis of Visual Behavior (pp. 549-586). Cambridge, MA: MIT Press. van Turennout, M., Ellmore, T., & Martin, A. (2000). Long-lasting cortical plasticity in the object naming system. Nat Neurosci, 3(12), 1329-34. Vitz, P. C. (1966). Preferences for different amounts of visual complexity. Behavioral Science, 1 1 ,105-114. Volpicelli, J. R., Alterman, A. I., Hayashida, M., & O'Brien, C. P. (1992). Naltrexone in the treatment of alcohol dependence. Arch Gen Psychiatry, 49(11), 876-880. Vuilleumier, P., Henson, R. N., Driver, J., & Dolan, R. J. (2002). Multiple levels of visual object constancy revealed by event- related fMRI of repetition priming. Nature Neuroscience, 5(5), 491 -499. Webster, M. J., Bachevalier, J., & Ungerleider, L. G. (1993). Subcortical Connections of Inferior Temporal Areas Te and Teo in Macaque Monkeys. Journal of Comparative Neurology, 335(1), 73-91. Webster, M. J., Bachevalier, J., & Ungerleider, L. G. (1994). Connections of Inferior Temporal Areas Teo and Te With Parietal and Frontal-Cortex in Macaque Monkeys. Cerebral Cortex, 4(5), 470-483. Webster, M. J., Bachevalier, J., & Ungerleider, L. G. (1995). Transient Subcortical Connections of Inferior Temporal Areas Te and Teo in Infant Macaque Monkeys. Journal of Comparative Neurology, 352(2), 213-226. Wilson, E. O. (1984). Biophilia: The Human Bond with Other Species. Cambridge, MA: Harvard University Press. Wilson, E. O. (1993). Biophilia and the conservation ethic. In S. L. Kellert & E. O. Wilson (Eds.), The Biophilia Hypothesis (pp. 31-41). Covelo, California: Island Press. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 207 Winkielman, P., Zajonc, R. B., & Schwarz, N. (1997). Subliminal affective priming resists attributional interventions. Cognition and Emotion, 11(4), 433-465. Winkielman, P., & Cacioppo, J. T. (2001). Mind at ease puts a smile on the face: Psychophysiological evidence that processing facilitation elicits positive affect. Journal of Personality and Social Psychology, 81(G), 989-1000. Wise, S. P., & Herkenham, M. (1982). Opiate receptor distribution in the cerebral cortex of the rhesus monkey. Science, 218(22), 387-389. Woods, R. P., Grafton, S. T., Watson, J. D. G., Sicotte, N. L., & Mazziotta, J. C. (1998). Automated image registration II: Intersubject validation of linear and nonlinear models. Journal of Computer Assisted Tomography, 2 2 ,153-165. Woolrich, M., Ripley, B., Brady, J., & Smith, S. (2001b). Temporal autocorrelation in univariate linear modeling of FMRI data. Neuroimage, 14(G), 1370-1386. Worsley, K. J. (2001). Statistical analysis of activation images. In M. Jezzard, & Smith (Ed.), Functional M RI: An Introduction to Methods . Oxford, UK: Oxford University Press. Yantis, S., & Serences, J. T. (2003). Cortical mechanisms of space-based and object-based attentional control. Current Opinion in Neurobiology, 1 3 ,187-193. Yoshida, M., Naya, Y., & Miyashita, Y. (2003). Anatomical organization of forward fiber projections from area TE to perirhinal neurons representing visual long-term memory in monkeys. Proceedings of the National Academy of Sciences of the United States of America, 100(7), 4257-4262. Zadina, J. E., Hackler, L., Ge, L.-J., & Kastin, A. J. (1997). A potent and selective endogenous agonist for the mu-opiate receptor. Nature, 386, 499-502. Zadina, J. E., Martin-Schild, S., Gerall, A. A., Kastin, A. J., Hackler, L., Ge, L. J., & Zhang, X. (1999). Endomorphins: Novel endogenous mu-opiate receptor agonists in regions of high mu-opiate receptor density, Neuropeptides: Structure and Function in Biology and Behavior (Vol. 897, pp. 136-144). Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 208 Zadina, J. E. (2002). Isolation and distribution of endomorphins in the central nervous system. Japanese Journal of Pharmacology, 89, 203-208. Zago, L., & Bar, M. (2003). The rise and fall of visual priming (Presentation given at the Vision Sciences Society 3rd Annual Meeting. TA192). Sarasota, FL. Zajonc, R. B. (1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology, ^(Monograph supplement No. 2, Part 2). Zajonc, R. B. (1980). Feeling and thinking. American Psychologist, 35(2), 151-175. Zuckerman, M. (1995). Good and Bad Humors: Biochemical Bases of Personality and Its Disorders. Psychological Science, 6(November (6)), 325-332. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Face classification

PDF

Invariance to changes in contrast polarity in object and face recognition

PDF

The neural representation of faces

PDF

Characteristics and cortical localization of subliminal visual priming

PDF

Contribution of NMDA receptor-channels to the expression of long -term potentiation and metaplasticity: Electrophysiological and computational studies

PDF

Explicit encoding of spatial relations in the human visual system: evidence from functional neuroimaging

PDF

Cross-modal prepulse modification of ERP and startle

PDF

Dopamine related temporal processing errors in prenatally stressed rats

PDF

Associations and mechanisms among attention deficit hyperactivity disorder symptoms, cognitive functioning, and drinking habits

PDF

Glial activation and neurotoxicity: Age-related cytokine expression, ApoJ-induced microglial activation, and role of peroxynitrite in mediating neuron death

PDF

A context for timing, conditioning and modification of cerebellar function

PDF

Childhood videotaped neuromotor and social precursors of schizophrenia: A prospective investigation

PDF

Acquisition, consolidation and storage of an associative memory in the cerebellum

PDF

Analysis, synthesis and recognition of human faces with pose variations

PDF

Contributions of active dendrites and structural plasticity to the neural substrate for learning and memory

PDF

Apoptotic pathways involved in kainate excitotoxicity

PDF

Neuropsychological mechanisms predictive of adolescent addictive behaviors

PDF

Involvement of the secretory pathway for AMPA receptors in the expression of LTP

PDF

Estrogen effects on excitability and plasticity in hippocampus

PDF

Hyperactive symptoms, cognitive functioning, and drinking habits

Asset Metadata

Creator Vessel, Edward Allen (author)

Core Title Behavioral and neural investigations of perceptual affect

School Graduate School

Degree Doctor of Philosophy

Degree Program Neuroscience

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag biology, neuroscience,OAI-PMH Harvest,psychology, cognitive

Language English

Contributor Digitized by ProQuest (provenance)

Advisor Biederman, Irving (committee chair), Earleywine, Mitchell (committee member), Lu, Zhong-Lin (committee member), Mel, Bartlett W. (committee member), Thompson, Richard (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c16-530804

Unique identifier UC11335911

Identifier 3140568.pdf (filename),usctheses-c16-530804 (legacy record id)

Legacy Identifier 3140568.pdf

Dmrecord 530804

Document Type Dissertation

Rights Vessel, Edward Allen

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA