University of Southern California Dissertations and Theses

Functional models of fMRI BOLD signal in the visual cortex

(USC Thesis Other)

Functional models of fMRI BOLD signal in the visual cortex

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content FUNCTIONAL MODELS OF FMRI BOLD SIGNAL IN THE VISUAL CORTEX
Pinglei Bao
A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(Neuroscience Graduate Program)
December 2013
Copyright 2013 Pinglei Bao
i
Epigraph
Two roads diverged in a wood, and I,
I took the one less traveled by,
And that has made all the difference.
Robert Frost
ii
Acknowledgments
! First, I would like to express my gratitude to my thesis advisor, Dr. Bosco Tjan for
his continuing support, guidance and enthusiasm, making this thesis possible. Dr. Tjan
has been an excellent mentor and his patience, motivation, enthusiasm and immense
knowledge is invaluable to me. I could not have imaged having a better advisor and
mentor for my Ph.D study.
! I would also thank to Dr. Norberto Grzywacz and Dr. Judith Hirsch for being on
my thesis committee. Their encouragement, suggestion, insightful comments and
questions help me to have a deeper think of my project and research. I would also thank
Dr. Zhonglin Lu , Dr. James Weiland and Dr. Irving Biederman for giving good advices.
! I would also like to express my appreciation to the staffs and members of
Neuroscience Graduate Program and Dornsife imaging center for their help in various
stages of this dissertation, in particular Jiancheng Zhuang and Xiangrui Li.
! I would like thank to my friends in Neuroscience Graduate Program and Vision
science community in USC. Special thanks to Rishabh Jain for his endless
encouragement. Also special thanks to Wan-Qing Yu for organizing the amazing vision
journal club to let me have a broad view of the research areas that I am interested in.
iii
! T-lab does not only provide the amazing views for L.A.’s downtown, but also let
me to meet the best lab colleagues. Speical thanks to Dr. Anirvan Nandy, Dr. Miyoung
Kwon and Rachel Millin for the many stimulating discussions and the help from them .
I would also thank to Kilho Shin and Benjamin Files for all of your help with my project.
! Finally, I gratefully acknowledge my family for their support and love. Special
thank to my wife Jun Zheng for her understanding and support. She never asked me
such questions as “When are you going to get your Ph.D. Degree” or “When are you
going to earn 50,000$ per year?”
iv
Table of Contents
Abstract! viii
Chapter 1! 11
Introduction! 11
fMRI! 11
fMRI and visual system! 12
Relationship between BOLD response and the neural activities! 16
Functional connectivity! 18
Research Questions and summary! 21
Chapter 2! 22
Iso-eccentric organization of spontaneous activity in the human visual
cortex! 22
Abstract! 22
Introduction! 23
Results! 24
Methods! 34
Chapter 3! 38
fMRI additivity of attended and unattended image contents! 38
Abstract! 38
Introduction! 38
Results! 41
Discussion! 51
v
Methods! 55
Chapter 4! 63
Quantifying the relationship between the fMRI BOLD signal and
neural activity with an achiasmic human subject! 63
Abstract! 63
Introduction! 64
Results! 67
Discussion! 79
Conclusion! 83
Methods! 83
Chapter 5! 93
Conclusion and Future Direction! 93
Applying SLC model on human MT (hMT+)! 93
Differentiate neural suppression effect versus nonlinear BOLD effect with an
Achiasmatic subject! 94
Reference! 96
vi
List of Figures
Figure 2.1 The correlation map and the corresponding profile plot! ! ! 25
Figure 2.2 Individual normalized correlation maps! ! ! ! ! 26
Figure 2.3 The correlation map between low-level visual areas! ! ! 27
Figure 2.4 The spatial structure of evoked activities! ! ! ! ! 28
Figure 2.5 The correlation map simulated with feedback model! ! ! 33
Figure 3.1 The example stimuli for the experiments.! ! ! ! ! 44
Figure 3.2 The form-selective visual areas’ responses predicted by SLC model! 47
Figure 3.3 Ventral visual areas’s responses and fitted with SLC model! ! 48
Figure 3.4 All ventral visual areas’ coefficients, category specificity and attention
modulation! ! ! ! ! ! ! ! ! ! ! 50
Figure 3.5 R
2
and coefficients maps obtained from voxel-based SLC model fit.! 52
Figure 4.1 Retinotopy of the achiasmic subject S! ! ! ! ! ! 70
Figure 4.2 Behavioral and physiological evidences for non-interactivity of the co-
locating neural populations.! ! ! ! ! ! ! ! 73
Figure 4.3 Results of the pure BOLD summation experiment obtained from the
achiasmic subject with 6 s stimulus duration.! ! ! ! ! ! 75
Figure 4.4 fMRI BOLD response as a function of neural activity! ! ! 78
Figure 4.5 Cross studies comparison between spiking activity and neural activity
inferred from the BvZ function ! ! ! ! ! ! ! ! 80
Figure 4.6 Responses evoked with ROI-defining stimuli. ! ! ! ! 89
Figure 4.7 Results of the BOLD summation experiment (with 6 s stimulus duration)
with and without removing the global noise component.! ! ! ! 92
vii
Abstract
!
! Functional magnetic resonance imaging (fMRI) is a functional imaging procedure that
measures the neural activity by detecting the associate changes in the blood flow. It has become
one of the most widely used technologies for in-vivo human brain imaging.
! Since the first sets of fMRI images were acquired to compare the response in visual areas
with visual stimuli either on or off, fMRI has been prominent in the study of the visual system.
Not only fMRI extends our knowledge about the human visual system with different stimuli
and tasks, but also the visual system provides a venue for understanding the mechanism of the
fMRI signal and for testing new analysis methods. My three studies follow this tradition of
fruitful interactions between fMRI methods and findings about the visual system, and these
three projects are fully elaborated from chapter 2 to chapter 4.
! In chapter 2, we identify and explain the correlational structures of the spontaneous
BOLD fMRI in the low-level visual areas. We measured the BOLD response while subjects
rested with their eyes closed, without stimuli or any explicit tasks. We studied the measured
spontaneous activity in the visual cortex (V1-V3) at a fine scale. We found that the strongest
correlations in spontaneous activity are between points on the cortex with functional receptive
fields at the same distance from the point of gaze in the visual space, irrespective of whether the
receptive fields are in the same quadrant or hemi-field. No iso-eccentricity connections have
ever been reported in the visual area using anatomical methods. We used a simulation to
determine if temporally varying and spatially diffused feedback from at least two higher
viii
cortical areas with different amounts of coverage of the peripheral visual field can provide a
quantitative account of these findings.
! The purpose of the second study, described in chapter 3, is to build a signal strength
response model which accounts for category specificity and attention effect across different
visual areas. Using fMRI, we conducted three complementary experiments in which observers
had to compare either the face or scene component of a pair of briefly presented images, each
being an amalgamation of face, scene and random noise pattern. We found that the BOLD
response to each non-noise image component (face or scene) is linear in the “signal proportion”
of the component, defined as the ratio of the contrast energy of the component to the contrast
energy of the entire image. For a cortical area along the ventral visual pathway, the slope of this
linear function depends on whether the component is attended to and if it is preferred by the
cortical area. The net BOLD response of a cortical area is a simple sum of the responses to all
non-noise components. We validated this linear sum-of-components model by showing that a
model fitted to the data from any two of the three experiments can accurately predict the
empirical results of the third.
! The aim of my third study, described in chapter 4, is to understand the relationship
between the neural response and BOLD response with an achiasmatic subject. Achiasma is a
rare congenital condition that prevents the normal crossing of the optic nerves from developing.
Because the optic chiasm is absent, the left and right halves of the visual field project to the
same cerebral hemisphere, ipsilateral to the eye of origin. In such an achiasmatic individual,
subject S, we found that each fMRI voxel in the retinotopic areas V1-V3 responds equally to two
positions in the visual field. These two population receptive fields (pRFs) of a single voxel are
located mirror-symmetrically about the vertical meridian. Contrast masking and fMRI
ix
adaptation experiments revealed that these pRFs are served by two non-interacting groups of
neurons of comparable number and excitability. By presenting identical stimuli to both of these
receptive fields instead of just one, we can double the local neural activity, regardless of the
definition of “neural activity.” Using this in-vivo model, we found that BOLD response
amplitude is proportional to approximately the square root of the underlying neural activity.
x
Chapter 1
Introduction
! This chapter provides an overview of the scientific background related to this thesis. A
general background of how fMRI applied on the study of visual system, is described. Further
more, an introduction about the relationship between the Blood-Oxygen-Level Dependent
(BOLD) response and the underlying neural activity is given. Also, the correlation analysis
based on BOLD responses collected when subjects are in “resting state” is briefly discussed .
Lastly, a brief review of the thesis is provided.
fMRI
! It has been around two decades since the first fMRI image was acquired with using echo
planar imaging (EPI) sequence by comparing the response of visual stimuli on and stimuli off in
early of 90s (Ogawa et al., 1990b; Belliveau et al., 1991). fMRI has become one of the most widely
used technologies for in vivo human brain image. It is an MRI procedure that measures brain
activity by detecting associate changes in blood flow, which typically uses deoxygen
hemoglobin as the natural contrast agent to map the neural activity in the brain by measuring
so called as BOLD responses (Ogawa et al., 1990a; Ogawa et al., 1990b; Kwong et al., 1992;
Kwong, 1995).
! The advantage of fMRI could be attributed to the following reasons: 1) high spatial
resolution compared to electroencephalography (EEG) or magnetoencephalography (MEG); 2)
high temporal resolution, compared to other hemodynamic technique, such as positron
11
emission tomography (PET); 3) It is safe and totally non-invasive nature, allowing multiple
measurements; 4) the relatively easy accessibility of the MR equipment (most modern hospitals
have MRI machines capable of taking fMRI images).
! However, there are still some limitations about fMRI. The most fundamental one is that
fMRI did not measurement of the neural activity directly. As the consequence, the precise
relationship between neural activities and measured BOLD response are not completely
understood. The signal observed by fMRI might be predominately driven by the synaptic rather
than the neural firing (Bandettini and Ungerleider, 2001; Logothetis et al., 2001; Logothetis,
2008), which brings the confuse about the data’s interpretation. Also because of the lag and
point spread properties of hemodynamic response, the fMRI’s temporal and spatial resolution is
limited by the properties of the brain’s circulator system. The signal is also companied with the
physiological noise, such as heart rate or respiration rate.
fMRI and visual system
! fMRI has dramatically increased our detailed knowledge of the human visual cortex.
With fMRI, many visual areas have been precisely located, whose functional properties and
connectivity with other areas have been well studied, which have been considered as one of the
major accomplishments in the visual neuroscience (Grill-Spector and Malach, 2004). Also! , given
the big coverage of the fMRI measure, it is much easier to record the whole visual systems’
responses. Thus, it help us in understanding the organization of these visual areas as the visual
processing stages. In this part, we will review how fMRI help us to understand the visual
system, focusing on these two themes, mapping the visual areas, and understanding the
organization of the visual areas.
12
1) Mapping the human visual areas
! fMRI studies about the visual areas in human visual cortex used two criteria to define a
region(Felleman and Van Essen, 1991; Tootell et al., 2003), a) retinotopy, typically the visual area
should have a full representation of the visual field (combining hemispheres); b) global
functional properties.
! In the low-level visual areas, visual field topography is used to define and map the
visual areas (Engel et al., 1994; DeYoe et al., 1996; Engel et al., 1997). Mapping from the retinal
input to the visual cortex is topographic in that nearby regions on the retinal input project to
nearby cortical regions, which is typically described as a log-polar transformation, in which,
standard axes in the retina are transformed into polar axes in the cortex: eccentricity (distance
from fovea) and polar angle. The logarithmic component of the transformation account for
cortical magnification of the fovea representation (Schwartz, 1985; Duncan and Boynton, 2003).
In the other word, the transformation preserves the visual space relations but with distortion.
! Mapping the polar angle reveals multiple representation of the horizontal and vertical
meridian arranged in approximately parallel bands on the cortical surface. The vertical and
horizontal meridians representation defines the borders of the retinotopic areas, like V1, V2, V3,
and V3a (Engel et al., 1994; Engel et al., 1997).
! The definition of the higher-level visual cortex is more based on the global functional
properties, such as human MT (Tootell et al., 1995) and LOC (Grill-Spector et al., 1998). Human
MT (in some literature, it is also called as hMT+) is selectively activated by moving versus the
stationary stimuli. LOC might contain several visual areas, which shows greater fMRI response
to images of familiar and unfamiliar objects than texture, visual noise or scrambled objects.
13
Also, the fMRI also defined a set of regions highly selectively response to certain category of
images, such as fusiform face area (FFA) responds to face (Kanwisher et al., 1997), and
parahippocampal place area (PPA) to scene images (Epstein and Kanwisher, 1998).
! The human’s visual system contains numerous visual areas that collectively occupy a
large fraction of cerebral cortex. The total number of the visual areas, the identities of these
areas, and their locations has been much progress since the development of the fMRI. However,
the functional definition of visual areas beyond V3 is currently under vigorous debate. Thus,
even the aspect of mapping human visual areas remains a field in which we expect many new
findings in the near future.
2) The organization of the visual cortex
! The fMRI method offers a major advantage over other techniques applied in
neuroscience is from its ability to image the entire brain essentially at once. This advantage
provide a good opportunity to investigate the organization of the different areas.
! One distinctive organization of the visual areas along both the occipito-temporal and
occipitio-parietal pathways are organized hierarchically, which suggests that low-level inputs
are transformed into more abstract representation through successive stages of processing
(Riesenhuber and Poggio, 1999). The hierarchical processing along a sequence of ventral visual
areas as a function of two criteria: a) retinotopy and b) specificity. As the hierarchy increases
along the system, the degree of retinotopy will get lower and lower, which is due to the increase
of the receptive fields(Wandell et al., 2007). Also, the low level visual areas show a low degree of
specificity to either motion or form and the high level visual areas, like MT shows a strong
preference for moving versus stationary stimuli but does not exhibit object selectivity, and in
14
contrast LO responses more strongly to objects but has little response for moving versus
stationary.
! Also, the modulation of the abstract representations of the object will show bigger
modulation of the BOLD response in the higher cortical areas than in the lower-level visual
areas. Grill-Spector. et al.(1998) shows that the gradual scrambling experiment in which object
images are borken into an increasingly larger number of fragments reveals areas along the
hierachical sequence that are increasingly more sensitive to image scrambling. Also Tjan et al.
(2006) used scene images with the different amounts of the external noise and they found that
when the contrast of the stimulus was held constant, the BOLD response is more modulated by
signal-noise-ratio (SNR) when a cortical areas was further away from the retinal input.
! If the cortex is unfolded by introducing a cut along the calcarine sulcus, the hierarchical
processing can approximately be arranged along the back-to-front axis (which corresponds to
postero-medial to anterior-lateral axis on the real hemisphere). The axis that is situated
orthogonally to the hierarchy axis along the dorso-ventral direction in the lower visual areas can
be examined as the global framework of eccentricity. More interestingly, the Face- and scene-
selective areas are found in cortex with foveal and peripheral biases, respectively (Hasson et al.,
2002; Levy et al., 2004). Also, it has been found that the medial-to-lateral organization of big and
small object preferences in the ventral temporal cortex(Konkle and Oliva, 2012), mirrored along
the lateral surface. In conclusion, the orthogonal axes of hierarchy and center-periphery
specialization provide an overall framework to the entire organization of human visual areas.
15
Relationship between BOLD response and the neural
activities
! The success of fMRI as a tool for understanding brain functions in human or animals is
highly dependent on a deeper understanding of the relationship between the observed signal,
typically as the BOLD response, and the underlying neural activity that we think it presents.
The fMRI technique capitalize on the coupling of cerebral blood flow, energy demand and
neural activity(Buxton et al., 1998; Stephan et al., 2007; Griffeth and Buxton, 2011). The
interaction between these variables are overwhelmingly complex. Also the matter is further
complicated because “neural activity” is not a univariate quantity ( single- / multi-unit spiking
activities, local field potential of various frequencies, etc) (Logothetis et al., 2001; Logothetis and
Wandell, 2004; Logothetis, 2008). Also, attempts to infer this relationship from stimulus-
evoked BOLD response have been frustrated by the complex nonlinearity between stimulus and
neural activity (Boynton et al., 1996; Huettel and McCarthy, 2000).
! The relationship between the stimuli and BOLD responses are investigated in many
experiments. In general, the relationship is nonlinear. BOLD responses increases as a function of
signal energy of stimuli in a nonlinear, compressive manner (Boynton et al., 1996; Boynton et al.,
1999; Avidan et al., 2002). This nonlinear could be due to either one of two aspects or both of
them: the neural signal might be nonlinear to the stimulus’s energy and BOLD responses might
be nonlinear to the neural activity. The first possible has been observed in lots
electrophysiological experiments, which makes the second possible hard to know by just
manipulating the signal’s energy.
16
! This relationship between the BOLD responses and the duration of the stimuli is also
found to be nonlinear. For example, In Huettel et al (2000)’s paper, they measured double-pulse
response with different interpair intervals (IPI) and simple-pulse. They subtracted subjects’
single-pulse response from the double-pulse response to isolate the contribution of the second
stimulus. If the hemodynamic responses were fully additive, the residual should be a time-
shifted replica of the single stimulus. However, the amplitude of the second pulse’s response
was smaller and the amplitude decrements was dependent upon IPI. The smaller of the IPI, the
more decrements of the amplitude. Such refractory effects are also observed in other
experiments could also be due to two reasons: the nonlinearity of the temporal summation of
neural activity, which is typically described as adaptation effect, and the nonlinearity of the
temporal summation of the BOLD responses. To dissociate these two effects is also very
important, since lots studies assume the reduction responses for the second stimuli is due to the
neural adaptation effect and use this effect as a probe to reach the sub-voxel resolution.
! The approach by linking the stimuli and the BOLD to test the relationship has the
fundamental challenge: the relationship between stimulus and neural activity and between the
neural activity and the BOLD response are both nonlinear.
! Given the difficulty of the approach described above, the most straightforward way to
explore the relationship between the neural activity and the BOLD responses is to record these
two signals simultaneously, and using system identification method to determine the
relationship. However, the definition of the neural activity for the population neurons is vague.
It could be LFP at different frequencies, or MUA. In the experiment, people find that both LFPs
and MUAs are correlated with BOLD, and quantitative analysis reveals LFP can predict BOLD
17
better (Logothetis et al., 2001). However, it is still not deterministic to know which kind of
neural activity can drive the BOLD responses.
! The application of system-identification techniques on a trial-by-trial bias enable to build
a model of the BOLD mechanism and make an estimation of the impulse response function,
which can be tested with the conditions not used for estimating the impulse function. Since the
convolution is pure linear operator, if the predicted BOLD response based on the convolution
between impulse response functions and LFP match the recorded BOLD response, it will
suggests at least the relation between the BOLD and LFP is linear. However, not all the inferred
BOLD responses matched the recorded responses, especially when the stimuli are longer
(Logothetis, 2003). Magri (2013)‘s paper also suggests that the nonlinear compressive model to
describe the transformation from the neural activity (LFP and MUA) to the BOLD responses
give a better fit than the linear fit.
! In conclusion, the relationship between the BOLD responses and its underlying neural
activity is still not clear. As the result, the interpretation of nonlinearity observed in the fMRI
experiment is ambiguous about the origin of the nonlinearity. Moreover, this also prevents the
researcher to give a further interpretation of the BOLD activity in terms of the reflection of the
neural activity.
!
Functional connectivity
! Functional connectivity is defined as the temporal dependence of neural activity
patterns of anatomically separated brain regions (Aertsen et al., 1989; Friston et al., 1993)and
studies have been shown the feasibility of examining functional connectivity between brain
regions as the level of co-activation of functional MRI time-series measured during rest, which
18
is called as resting-state fMRI signal and believed to reflect functional communication between
brain regions (Biswal et al., 1995; Lowe et al., 2000a; Damoiseaux et al., 2006).
! The neural basis of the resting-state fMRI signal is not fully understood and in the past
years, there has been an ongoing debate about whether the resting-state signal result from
physiological noise, such as respiration and cardiac oscillations or these correlation origins from
co-activation in the underlying spontaneous neural activity (Birn et al., 2006; Shmueli et al.,
2007; Birn et al., 2008; Chang et al., 2009). The support for second idea comes from that most of
the resting-state patterns tends to occur between brain regions that overlap in the same function
domain, such as vision, motor and auditory network (Biswal et al., 1995; Lowe et al., 2000b;
Salvador et al., 2005; Damoiseaux et al., 2006; Nir et al., 2008). Further more, the spontaneous
BOLD fluctuation and simultaneous fluctuations in neural spiking show a strong association
support for a neural basis of resting-state BOLD (Shmuel et al., 2002; Shmuel and Leopold,
2008).
! The studies about the resting-state network have reported the formation of functional-
linked networks, which consist of anatomically separated but sharing a common function
(Beckmann et al., 2005; Damoiseaux et al., 2006; Fox and Raichle, 2007). There have identified
around eight functionally linked subnetworks, which include the visual network, the motor
network, default mode network, two lateralized networks consisting of superior parietal and
superior frontal regions, and network consisting of bilateral temporal/insular and anterior
cingulate cortex (Raichle et al., 2001; Fox et al., 2005; Fransson, 2005; Buckner and Vincent, 2007).
More interestingly, the functional networks are found not only in the wide-scale, but that
resting-state networks may show an internal topology that is strongly organized to their sub-
functions, such as the functional connectivity found in the primary motor resting-state network
19
have been reported to be ordered according to the somatotopic organization (van den Heuvel
and Pol, 2010).
! If resting-state fluctuations reflect ongoing neural activity and communication between
separated brain regions, we should expect to observe the existence of anatomical connections
between these functional linked brain regions to support the ongoing communication. There are
several studies (Koch et al., 2002; Honey et al., 2007; Hagmann et al., 2008) that have indeed
suggested a direct connection between functional and anatomical connectivity in the human
brain by combining resting-state fMRI with structural diffusion tensor imaging measurements
(DTI), which can reconstruct the white matter tracts by measuring the diffusion profile of free
water molecules in brain tissues. Most of studies suggest that those anatomical connected brain
regions tends to have high functional connectivity, however, the vice is not always true.
! There are also many studies which use resting state fMRI techniques in examining
possible functional disconnectivity effects in neurologic and psychiatric brain disorder,
including Alzheimer’s disease (Greicius et al., 2004; Rombouts et al., 2005), schizophrenia
(Bluhm et al., 2007; Liu et al., 2008), dementia (Rombouts et al., 2009). The change of the
functional connectivity, especially the change of the default mode network, have been reported
in these special condition, which suggests these neurodegenerative diseases are targeting
interconnected cortical network.
! In conclusion, by measuring the level of co-activation of resting-state fMRI time-series
between brain regions, the studies about the resting-state revealed interesting findings about the
“functional connections” between the brain regions and local networks, as well as the overall
organization of the functional communication in the brain network. The value of these new
functional connectivity tools can be used to examine the diseases about the connectivity.
20
Research Questions and summary
! My three studies follow this tradition of fruitful interactions between fMRI methods
and the findings about the visual system. In my first project, I explored whether the functional
connectivity corresponds to the anatomical connectivity in the lower visual areas, taking
advantage of the precise topological relationship between cortical space and visual space and
the known anatomical connectivity in these areas. I discovered that the strongest correlations of
fMRI spontaneous activity in low-level visual areas are between voxels represented the same
eccentricity. In my second project, I used fMRI to study how different external and internal
factors can affect the ventral visual areas’ responses to image contents. I did three
complementary experiments that led to the discovery of a simple quantitative model describing
the ventral visual regions’s BOLD response to compound images under different attention
statues. This model is linear in a quantity that we called "signal portion", which is simply the
ratio of contrast energy of the content to the contrast energy of the full image. In my third aim, I
explored a more basic question -- the relationship between neural activity and the BOLD
response. I quantified the relationship between the neural activity and fMRI BOLD responses in
an achiasmatic subject, taking advantage of the fact that in achiasmia, the left and right visual-
field representations are fully overlapped in V1-V3.!
21
Chapter 2
Iso-eccentric organization of spontaneous activity in the
human visual cortex
Abstract
! Neural activity measured in the absence of any externally driven behaviors or stimuli
exhibits systematic and bilateral correlations across the brain. Such correlations are often taken
to imply functional connectivity between different brain areas, but the underlying causes of
correlations are unclear. We used functional magnetic resonance imaging to study spontaneous
activity in the human visual cortex (V1-V3) at a fine scale, capitalizing on the detailed
retinotopic maps in these visual areas. We found that the strongest correlations in spontaneous
activity are between points on the cortex with functional receptive fields at the same distance
from the point of gaze in the visual space, irrespective of whether the receptive fields are in the
same quadrant or hemi-field. This long-distance iso-eccentric organization of the spontaneous
activity is robust. Moreover, when we visually stimulate at one eccentricity in one quadrant of
the visual field, we can detect a signal from cortical locations that correspond to the same
eccentricity in the non-stimulated quadrants. These findings can be accounted for quantitatively
by the temporally varying and diffuse feedbacks from two or more higher cortical areas that
exhibit different degrees of eccentricity biases.
22
Introduction
! The brain is active in the absence of any explicit task or stimulation. As much as 80% of
energy consumption at “rest” can be associated with neural signaling at the synapses (Sibson et
al., 1997; Sibson et al., 1998; Shulman et al., 2001; Shulman et al., 2004). Local energy
consumption due to task-related activity can be as low as 1% beyond this resting-state baseline
(Raichle and Mintun, 2006). When measured with fMRI, the spontaneous BOLD signals during
rest are found to have a low-frequency fluctuation (<0.1 Hz) that is spatially organized (Fox and
Raichle, 2007). Such fluctuations in BOLD signal appear to correlate with the amplitude
fluctuations in the γ band of local field potential (Leopold et al., 2003) and can account for inter-
trial variability in human behavior (Fox et al., 2007). These and other findings suggest that
spontaneous BOLD signals in resting-state are the neurovascular signature of the underlying
spontaneous neural activity.
! An increasingly important application of spontaneous BOLD fluctuations has been to
identify functional connectivity between brain regions in terms of the temporal correlation
between the spontaneous BOLD fluctuations (Biswal et al., 1997; Cordes et al., 2000; Hampson
et al., 2002; Fox et al., 2005). While it has been shown that for regions that are anatomically
connected, the correlation between BOLD fluctuations is high (Vincent et al., 2007), the converse
is not guaranteed. However, in order to use correlated BOLD fluctuations to map connectivity, it
is necessary for the converse to be true, at least in an indirect sense. We tested the idea that
correlated BOLD fluctuation implies functional connectivity by measuring spontaneous BOLD
signal fluctuations in low-level visual areas V1, V2 and V3, taking advantage of the precise
topological relationship between cortical space and visual space in these visual areas.
23
Results
Spatial structure of resting-state correlations in V1-V3
! Spontaneous BOLD fluctuations were measured from six healthy subjects at rest with
their eyes closed and video projector powered off. All voxels (3 mm isotropic) from areas V1,
V2, and V3 of both hemispheres between retinal eccentricity of 0.2º to 8º were included in the
analysis. Each voxel was used as a seed voxel to compute the resting-state temporal correlation
against all other voxels using non-smoothed, slice-timing and motion corrected BOLD signal
between 0.004–0.1 Hz. Standard method for retinotopic mapping with rotating wedge and
expanding ring was also conducted on these subjects during separate scanning sessions. (See
Methods and Materials.)
! When resting-state correlations are expressed as function of the retinotopic coordinates
of the voxels, a distinct pattern emerged: the highest correlations are between voxels at the same
retinal eccentricity (distance from the center of gaze) but not necessarily at the same retinal
position. To quantify the spatial structure of resting-state correlations while preserving the
vertical and horizontal meridians, we projected the correlation coefficients between a seed voxel
and all the target voxels on to a normalized visual space, where the retinal eccentricity of a
target voxel was replaced by the ratio of the eccentricity of the target voxel to that of the seed
voxel. We also flipped the correlation map about the meridians, if needed, such that the seed
voxels and thus any correlations between retinotopically nearby voxels are confined to the
lower right quadrant (Figure. 2.1 B). This was done to distinguish correlations between voxels
representing proximal retinal locations from those representing distal locations. When the
correlation maps from all seed voxels were averaged in the normalized visual space, ignoring
24
the proximal correlations, a distinct peak appeared at an eccentricity ratio of 1.0, independent of
the eccentricity of the seed voxel (Figure 2.1 C, D).
! The iso-eccentric correlations remained strong even when the seed and target voxels
were in different visual quadrants. The average inter-quadrant temporal correlation (Pearson r)
between seed and target voxels at the same eccentricity was 0.51±0.04. The iso-eccentric
organization of spontaneous correlations is evident in all six subjects we tested (Figure 2.2).
Figure 2.1: The correlation map and the corresponding profile plot
A) The thresholded correlation maps (highest r of top 10%) with two seed voxels in the low-level visual
cortex (green cross), which are represented in cortical surface and visual space(with the retinotopic
map).
B) Normalization of different correlation maps from different seed voxels. Based on the seed voxel’s
position in the visual space, the whole correlation map will be expanded, like figure b,1 or shrunk,
like b,2 in order that all the seed voxels will be at the normalized eccentricity 1. If needed, the
correlation map will be flipped around meridian once or twice, so that the center of the receptive field
of the seed voxel with be confined in the lower-right quadrant in the visual space.
C) The average normalized correlation map. The dark circle represents the normalized eccentricity.
D) The profiles of the average normalized correlation map of different eccentricities. The dark line
represents the profile which the seed voxels sit in across the whole eccentricity. And the red line
0.1 1 10
0
0.1
0.2
0.3
0.4
0.5
Visual Space
Left hemisphere Right hemisphere

r
0
0.1
0.2
0.3
0.4
0.5
(a)
(b)
(c)
0.7°
1.9°
5.2°
All
Relative Eccentricity
r
(d)
25
represents the profile which the seed voxels’ eccentricity around 0.7 degree; the blue line represents
the profile which the seed voxels’ eccentricity is around 1.9 degree; the purple line represents the
profile which the seed voxels’ eccentricity is around 5.2 degree.
! The iso-eccentric correlations remained strong even when the seed and target voxels
were in different visual quadrants. The average inter-quadrant temporal correlation (Pearson r)
between seed and target voxels at the same eccentricity was 0.51±0.04. The iso-eccentric
organization of spontaneous correlations is evident in all six subjects we tested (Figure 2.2).
Figure 2.2 Individual normalized correlation maps
The seed voxels were from V1, and target voxels were from V2 and V3. For each subject, target-seed
correlation peaks for target voxels that are at the same or similar eccentricity as the seed voxels.
Spatial invariance of the resting-state correlations
! The iso-eccentric organization of resting-state correlations is robust regardless of
whether the seed and target voxels are within or between visual areas (Figure 2.3A). To a first
order approximation, the correlation coefficient (r) decreases with a slope of -0.25 as a function
of log eccentricity ratio between target and seed voxels, independent of the eccentricity of the
26
seed voxel or its cortical origin (Figure 2.3B). This means that r decreases linearly in terms of the
cortical distance between a target voxel and the nearest voxel at the same eccentricity as the
seed voxel, since cortical distance along the radial direction is approximately proportional to log
eccentricity. We found that on average, r decreases 0.3 every 32 mm, 27 mm, and 26 mm in V1,
V2, and V3 respectively, consistent the cortical magnification functions of these areas
(Dougherty et al., 2003).
Figure 2.3: The correlation map between low-level visual areas
A) The average normalized correlation maps computed in one visual area with the seed voxel that
drawn from the same or a different visual area. The normalized coordinates of the seed voxels are
confined to the lower-right quadrants by the normalization process (see figure 2.1)
B) The profiles of the corresponding normalized correlation maps, showing that correlation bandwidth
is essentially a constant, regardless of the visual areas of the seed and target voxels.!
Spatial structure of evoked activities
27
! In a separate experiment with the same six subjects, we presented a flickering arc-
shaped checkerboard at one of two eccentricities in one of the four visual quadrants in a blocked
design (Figure. 2.4A). We observed weak modulation in the three non-stimulated quadrants,
which as expected did not rise above the conventional statistical threshold. However, when we
compared the modulation amplitude in an non-stimulated region of interest (ROI) for a
stimulus at the same eccentricity as that represented in the ROI and for a stimulus at a different
eccentricity, we found that the median of this difference (same-eccentricity minus different-
eccentricity) was greater than zero for each of our six subjects, which is statistically significant
(sign test, p<0.02). Thus, correlations in both spontaneous and evoked activities are organized
by eccentricity in the visual cortex.
Figure 2.4: The spatial structure of evoked activities
1 2 1 2 1 1 2 2
ï ï ћ n
VDPHHFF ћ n
GLI IHUHQWHFF 2
2
2
2
1
2
1
2
1
2
1
2
(a) (b)
(c)
S1 S2 S3 S4 S5 S6
28
A) Two arcs of flickering checkboard, alternating between one at a small eccentricity and on at a large
eccentricity, are used to stimulate one of the four quadrants in a block design.
B) The Regions of interest, shown on a flattened cortex, correspond to the 4 quadrants and 2 different
eccentricities.
C) The box plot shows for each of the six subjects the contrast modulation of non-stimulated ROIs
between when the flickering checkboard is at the same eccentricity and when the checkboard at a
different eccentricity from that represented in the ROI.
A feedback account of iso-eccentric organization of resting-state correlations
! To the best of our knowledge, there is no report of any anatomical structure within each
of the tested visual areas (V1-V3) as well as in any of the upstream stages (LGN, retina) that
preferentially connects neurons with receptive fields at the same retinal eccentricity. While
horizontal connections in V1 can depend on orientation (Gilbert and Wiesel, 1983; Schwarz and
Bolz, 1991), there is no evidence to suggest any anisotropy in orientation tuning as a function
retinal eccentricity or azimuth except perhaps at the V1/V2 border. Studies using retrograde
tracers (Kennedy and Bullier, 1985) as well as functional measurements (Engel et al., 1997),
showed strong retinotopy in these cortical areas, forming essentially a one-to-one mapping
between visual space and cortical space. Even the electrophysiological recording of the cat’s
visual cortex show that the section of corpus callosum will demolish the oscillatory response
that can synchronize across two hemispheres (Engel et al., 1991), it is hard to imagine that the
corpus callosum can induce the isoeccentric connection. Given the conspicuous absence of any
evidence for anatomical connections that have an iso-eccentric bias, we do not think our finding
is indicative of any such connections.
! The observed iso-eccentric organization of resting-state correlations can be a result of
feedbacks from multiple higher-level cortical areas, each receiving its feed-forward inputs from
neurons representing a different mix of central and peripheral vision fields. Most if not all of
29
the forward projections in the cortex are reciprocated with feedback projections (Mignard and
Malpeli, 1991; Hupe et al., 1998). We may assume that if a higher-level cortical area received
most of its inputs from central vision, for example, the feedbacks from this higher-level area
would also mostly target central vision. A second assumption is that there is a common
component in the intrinsic fluctuation of the feedbacks originating from the same cortical area.
These two simple assumptions, the reciprocity in terms of strength between feed-forward and
feedback connections, and the presence of a common mode in the intrinsic fluctuation of
feedbacks, together with the fact that different higher-level visual areas have different central-
peripheral emphases (Hasson et al., 2002; Levy et al., 2004) are sufficient to explain our finding.
! The intuition is as follows. Feedback signals from higher-level cortical areas are a part of
the observed spontaneous activity. Any common component (common mode) in these signals
form a basis for the correlation observed in the spontaneous activity across the cortex. The
resting-state correlation between a pair of voxels within a retinotopic visual area depends on the
composition of feedbacks they received from the higher-level areas they connect to and the
strength of any uncorrelated fluctuations.If the only sources of correlation are the common
modes of the feedbacks, the correlation reaches a maximum when the two voxels receive an
identical composition of feedbacks.. Higher-level visual areas are known to receive inputs from
the central and peripheral visual fields with different proportionalities. Regions in the
inferotemporal (IT) cortex that are selective to faces or words predominantly receive input from
the visual field at or near the point of gaze (Hasson et al., 2002), while other visual areas such
MT, MST, and V8 are less foveal-centric and have much broader peripheral coverage (Huk and
Heeger, 2002). With different higher level visual areas having a different central-peripheral
emphasis, and given our assumption on the reciprocity of strength between feedback and feed-
30
forward connections, two voxels in a retinotopic visual area can have identical composition of
feedbacks only if their receptive fields are at the same eccentricity – hence the iso-eccentric
organization of resting-state correlations.
! We can also demonstrate this claim with a simple simulation that quantitatively matched
our data (Figure. 2.5c). Here we assume two hypothetical higher-level cortical areas with
opposite central-peripheral preferences. The central-peripheral preferences take the form of
linear functions in terms of the logarithm of eccentricity, which is approximately proportional to
the cortical distance from the center of the foveal representation in a visual area. This simple
setup shows robust iso-eccentric resting-state correlation that matches the amplitude and shape
of the observed correlation profile.
! The simulation may be considered as a minimal setup required for quantitatively
accounting for the empirical data: the strength of feedback from a down-stream area must be a
function of eccentricity, and there must be at least two sources of feedback with independent
common modes. The reality most likely involves feedback from more than two down-stream
areas, and their central-peripheral preferences need to be diametrically opposite to one another.
Note that our theory does not require the correlation between the feedback signals from the
same cortical area to be iso-eccentrically organized. We only require the strength of the
feedbacks varies with eccentricity, and such variation can be monotonic.
31
Figure 2.5: The correlation map simulated with feedback model
A) The scheme of the model.
B) The profile of the connection strength of the feedbacks
C) The average normalized map for model data and subjects’ data(seed region is V1, and target region is
V2)
D) The profiles for the average normalized map based on model data and subjects’ data.
Discussion
! Excluding trivial local correlations and correlations associated with known retinotopic
projections (lower-right quadrants of the normalized correlation maps), resting-state
correlations in the retinotopic visual areas V1, V2, and V3 are organized by eccentricity – the
correlation between two voxels is at a maximum where the population receptive fields of the
voxels are equidistance from the center of gaze. The correlation decreases as the difference in
eccentricity increases and is largely invariant to the absolute eccentricity of the voxels. This
finding was robustly observed in all of our subjects. We further found that stimulus-evoked
32
activities leave detectably larger modulation in a non-stimulated ROI when the stimulus is at
the same eccentricity represented in the ROI.
! These findings would have suggested a robust functional connectivity in the form of
concentric rings centered at the point of gaze. However, no anatomical connections that match
this pattern have ever been reported. Given that anatomical connections within these lower-
level visual areas have been extensively studied, it is rather unlikely that such a prominent and
distinct structure is missed. In fact, what we have produced is a counter-example that disputes
any strong links between physical and functional connectivity.
! Our results can be explained quantitatively in terms of generic properties of feedback
signals received by these lower-level retinotopic visual areas and the assumption that there is a
common component among the feedback signals originating from the same higher-level cortical
area. The presence of such a common mode in feedback signals has not been directly observed.
A direct observation of the common modes can be challenging because each lower-level area
projects to and therefore receives feedbacks from multiple higher-level areas, each with its own
common mode signal. In fact, our explanation of the concentric ring-shape structure of resting-
state correlation relies on the intermixing of these common mode signals, since none of the
feedbacks is spatially organized into concentric rings.
! Our finding with evoked activities suggests that the common mode signal can be
modulated with visual stimuli. Unlike the flickering checkerboard stimulus we used, which is
not selective in terms of which higher-level areas it activates, stimuli of certain natural
categories (e.g. faces) can selectively evoke certain higher-level visual areas with different
central-peripheral biases (Hasson et al., 2002). It should be possible to isolate the feedbacks
from a higher-level cortical area by stimulating with a specific category of objects and observing
33
the activity from the higher-level region in a lower-level retinotopic visual area that corresponds
to a non-stimulated visual field. If the stimulus predominantly evokes higher-level areas with a
central-field bias, then the evoked common mode (feedback) signals will not reach maximum
amplitude at the same eccentricity as the stimulus, as was the case with our non-specific
checkerboard stimulus. Instead, the evoked common mode will be a stronger at lower
eccentricity, reaching a maximum at the cortical regions that represent the fovea. This
prediction is consistent with the finding of William et al. (2007) that they could decode the
identity of peripherally presented objects from activation patterns in foveal retinotopic cortex.
! In conclusion, while we found strong and robust iso-eccentric organization of resting-
state correlations among the retinotopic visual areas, the absence of any anatomical evidence in
the extensively studied cortical areas preclude us from taking these findings to imply any
concentrically organized connectivity, functional or otherwise. Instead, we show that the
concentric arrangement of resting-stating BOLD fluctuation can be entirely due to a set of
spatially diffused but eccentricity-biased feedbacks from higher cortical areas. Studying
intrinsic fluctuation in the retinotopic cortex thus provides a window for investigating cortical
feedbacks and their roles in visual processing.
Methods
Subjects.
! Six observers ages 22 – 40 years, participated the experiments. The observers had either
normal vision or corrected-to-normal vision via MRI-compatible glasses. The Institutional
Review Board of the University of Southern California approved the experimental protocol, and
each subject provided written informed consent.
34
MRI data acquisition.
! All (f)MRI scans were acquired at the Dana and David Dornsife Cognitive Neuroscience
Imaging Center at the University of Southern California. MRI recording used a standard
birdcage head-coil on a Siemens 3T MAGNETON Trio MRI system with TIM. For each subject,
anatomical sagittal images (256 × 256 × 192) of 1-mm
3
isotropic spatial resolution were obtained
with a T1-weighted 3D MPRAGE sequence (T1 = 900ms, TR = 2,300 ms, TE = 2.98ms, flip angle
= 9°). BOLD activity was measured with a T2*-weighted echo planar imaging (EPI) sequence
(TR = 1,000ms, TE = 25ms, flip angle = 60°, FOV = 192 × 192 mm).
Experiment Procedure
! All visual stimuli were generated by a Power Macintosh G9 running a Matlab program
that used the Psychtoolbox (Brainard, 1997; Pelli 1997). The stimuli were displayed on a 32 × 24-
cm rear-projection screen at a viewing distance of 91cm.
Retinotopic scans
! The retinotopic scans include a rotating wedge and expanding ring, which are composed
of a flickering radial color checkboard pattern. Retinotopic scans were used to identify
retinotopic visual areas of each observer, and to determine the mapping between the visual
cortex and the visual space.
Resting-state scan
! The duration of the resting state scan was 520 seconds. The projector and lights in the
scan room were switched off and subjects were instructed to close their eyes and relax in the
scanner.
Scans with visually evoked activities
35
! In each of four 260-second scans, a flickering checkerboard stimulus was shown in one
of the four quadrants of the display. Only one quadrant was stimulated in each scan. Each
checkerboard formed an arc with radial boundaries of 1° to 2° (inner) or 5° to 7° (outer) (Figure.
2.4 A). Each arc stayed within one visual quadrant and away from the meridian by 15 degree in
polar angle. The inner and outer checkerboard was presented in alternating 24-second blocks,
separated by 16 seconds of rest. Subjects performed the same color-identification task as in the
resting-state scan with fixation.
Data processing
Preprocessing of the data
! Preprocessing the EPI-time series was performed using Brainvoyager software. The first
twenty volumes of each function scan were discarded. Preprocessing steps included 3D motion
correction and high-pass temporal filtering to remove any frequency component below 2 cycles
per scan.
! For the resting-state scans, the data were further processed. They were low-pass filtered
with a cutoff frequency of 0.1 Hz using a Butterworth filter. Six head motion parameters and the
mean white-matter signal were regressed out of the data.
Normalized correlation map
! We begin with a correlation map. A correlation map for a resting state scan shows, in
visual space, the pattern of correlation between the time courses of the seed and target voxels.
Each point in the map corresponds to the visual-space coordinates of a target voxel as
determined by retinotopy. It displays the correlation coefficient (r) between the target voxel and
the seed voxel. A correlation map was generated from each seed voxel. The correlation map was
then scaled by shrinking or expanding also that all the seed voxels appear at the normalized
36
eccentricity 1. The scaled map was then reflected about the horizontal and/or vertical meridian
as needed, so that the coordinates of the seed voxels are confined to the lower right visual
quadrant. After normalization (scaling and reflecting), all correlation maps were averaged. The
resulting map preserved the vertical and horizontal meridians. The radial coordinate of the map
present the eccentricity of the target voxel relative to that of the seed voxel in log scale
(approximately linear in cortical distance.
! The profile of an averaged normalized correlation map was calculated by averaging the
correlation across the polar angles, excluding the lower right used to confine the seed voxels.
Spatial structure of the subthreshold responses to evoked activities
! Eight ROIs (two eccentricities × four quadrants) were defined by contrasting the
response to inner vs. outer checkerboard stimuli. Only one quadrant is stimulated during each
scan. For each stimulus location and for each non-stimulated quadrant, we estimated the sub-
threshold evoked response from the two ROIs in the non-stimulated quadrant using a standard
general linear model (the main regressor was a boxcar function presenting the stimulation,
convoluted with the canonical hemodynamic response function of BOLD signal). The parameter
estimates (beta) obtained from non-stimulated ROI that was at the same eccentricity as the
stimulus was contrasted against the beta from the non-stimulated ROI that was at at the
different eccentricity. For each subject, the distribution of the 24 beta contrasts (8 stimulus
locations × 3 non-stimulated quadrants) and median are shown with a boxplot.
37
Chapter 3
fMRI additivity of attended and unattended image
contents
Abstract
! Neural activities along the ventral visual pathway are selective to image content and
modulated by content-selective attention. We found an remarkably simple relationship that
underlies the fMRI BOLD response to a complex scene: the BOLD response to each component
of a compound image is linear in the “signal proportion” of the component, defined as the ratio
of the contrast energy of the component to the contrast energy of the entire image. The net
response a compound image is simply the sum of the responses to all of its non-noise
components. The slope of the linear BOLD response depends on the category preference of the
cortical area and attention. We cross-validated this sum-of-linear-components model by
showing that a model fitted to the data from two of the three complementary experiments can
accurately predict the data from the third.
Introduction
! Cortical areas along the ventral visual pathway are associated with representation and
recognition of shapes(Haxby et al., 1991; Goodale and Milner, 1992; Kobatake and Tanaka, 1994).
38
Their responses to visual input are jointly determined by the object or feature category! ,
attention, and visibility. For a typical natural scene with multiple objects, the response profiles
of these areas are expected to be rather complex. A succinct formulation that can sufficiently
describe responses in these areas in terms of the extrinsic visual input, ! intrinsic task and
attention states would represent a significant advance and enable systematic and quantitative
investigations into the response properties of these areas. Our goal here is to provide such a
formulation for population responses measured with BOLD fMRI.
! The visibility of objects or image components drives the ventral visual areas’ response.
Visibility can be manipulated by adjusting image contrast or by adding noise to an image
(Legge and Foley, 1980; Legge et al., 1987; Lu and Dosher, 1999). In the lower visual areas, such
as V1 and V2, responses are non-linearly modulated by contrast (Albrecht and Hamilton, 1982;
Levitt et al., 1994; Boynton et al., 1999; Li et al., 2008), but masking the stimulus with
uninterpretable noise has little effect if the total contrast energy of the stimulus is kept constant
(Lerner et al., 2001; Tjan et al., 2006). In the higher-level visual areas, this pattern is reversed:
BOLD response is more invariant to contrast, showing little modulation for contrast great than
10% ((Avidan et al., 2002), but sensitive to noise masking (Tjan et al., 2006; Horner and
Andrews, 2009). In these higher-level visual areas, the differentiation between a meaningful
visual signal and uninterpretable visual noise is prominent at the neural population activity, as
demonstrated by fMRI measurements.
! The response in higher-level ventral visual areas depends not only on the visibility of
image contents, but on what these contents are. Several ventral visual areas show prominent
category specificity, including the fusiform face area (FFA) to faces (Kanwisher et al., 1997), the
parahippocampal place area (PPA) to scenes (Epstein and Kanwisher, 1998), and the extrastriate
39
body area (EBA) to bodies (Downing et al., 2001; Grossman and Blake, 2002). More generally,
the spatial distribution of activities in the ventral visual areas is found to be correlated with
perceptual categorization of the input (Kriegeskorte et al., 2008).
! In addition to stimulus category, the ventral visual areas’ responses are modulated by
attention. Spatial attention can modulate neurons or voxels, whose receptive fields or
population receptive fields overlap the attended location (Moran and Desimone, 1985;
Maunsell, 1995; Reynolds et al., 2000; Buracas and Boynton, 2007; Li et al., 2008). Attention can
also be applied in a non-spatial fashion as in the feature attention (Saenz et al., 2002, 2003;
Serences and Boynton, 2007) and object attention (O'Craven et al., 1999). Feature attention
enhances the neural response to image components in the visual fields that are related to the
attended feature, while object attention enhances the response of the whole object even the
features of the object are not attended (O’Craven et al., 1999).
! Ventral visual areas’ responses to cluttered scenes with multiple objects appear be rather
complex. If we put a neurons’ preferred object in the cluttered images (Sheinberg and
Logothetis, 2001; Rolls et al., 2003) or object pairs (Rolls and Tovee, 1995; Zoccolan et al., 2005;
MacEvoy and Epstein, 2009), the neural responses tend to be smaller than when presenting
isolated, preferred object.
! Although the response properties described thus far appear to be quite complex, we find
that they can be predicted by a simple formula. This formula depends only on the responses to
individual object components. The responses to these individual object components, in turn,
depend only on the object tuning preferences at the relevant area, whether or not the object is
attended, and how visible the object is in the scene. The goal of the current study is to provide
and test this expression for predicting population responses as measured by fMRI.
40
! We conducted three complementary experiments that systematically manipulated
visibility and attention to different image components of a compound image. We measured the
BOLD responses in cortical areas along the ventral visual pathway, starting with V1. The stimuli
consisted of superimposed images of faces, scenes and visual noise. We varied the relative
contrasts of these image components, while keeping the root-mean-square (RMS) contrast of the
stimuli constant. We found that the mean fMRI BOLD amplitude in a visual area can be
expressed as a simple sum of responses to individual image components. The area's response to
a particular image component is linear in the visibility of the component defined as the
proportion of contrast energy of the component to the net contrast energy of the stimuli. The
slope of this liner function is determined by the cortical area’s category specificity and whether
the image component is attended. Our model’s robustness is demonstrated by fitting the model
with two of the three experiments and cross-validating it with the third. We will discuss how
this model of fMRI BOLD response is related to a current model of selective attention.
Results
! In each of the three experiments, subjects performed a face task and a scene task in
separate runs. In the face task, subjects were asked to report whether two consecutive images
were of the same face. In the scene task, they were asked to report whether two consecutive
images were taken from the same scene. The contrast energy of the image components (face,
scene, noise) was manipulated in a rapid event-related design. The time course of BOLD signal
evoked by each conditions in the pre-selected cortical regions of interest (ROIs) were estimated
from the measured fMRI signal using deconvolution. The peak of the BOLD signal time course
was taken to be the amplitude of the BOLD response.!
41
! The contrast energy of the full image (content + noise) was kept constant in all
conditions and experiments while the "signal proportions" of the interpretable contents were
manipulated. Let p
1
and p
2
denote the proportions of contrast energy of face and scene
components, respectively, to that of the full image. The whole image can be expressed as
image= p
1
(face−L
0
)+ p
2
(scene−L
0
)+ 1− p
1
− p
2
(noise−L
0
)+L
0
! (1)
where L
0
is the mean luminance, and face , scene and noise indicate an image of face, scene
and a sample of the pink noise. We refer to p
1
andp
2
as the "signal proportion" of face and
scene respectively, since they represent the proportion of total contrast energy used to render
these particular components of the image.
! In Experiment 1 (Figure 3.1, Exp 1), the images contained only the task-related
component, with the signal proportion set at 0.05, 0.16, 0.34 and 0.45. That is, for the four
conditions of the face task, p
1
= 0.05, 0.16, 0.34 and 0.45, respectively, and p
2
= 0; for the four
conditions of the scene task, p
1
= 0 and p
2
= 0.05, 0.16, 0.34 and 0.45, respectively. In Experiment
2 (Figure 3.1, Exp 2), the images contained both face and scene components. The signal
proportions of face and scene components were kept equal and varied together in four
conditions. In separate runs, subjects were asked to perform either the face task or the scene task
with the same stimuli by attending to the task-relevant components. Thus, for both tasks, p
1
=
p
2
= 0.05, 0.16, 0.34 and 0.45. In Experiment 3 (see Figure 3.1, Exp 3), the images were again
superimposed face and scene images (in addition to pink noise). Unlike Experiment 2, the noise
component was kept the same across conditions, while the relative proportion of the face and
42
scene components varied. For both face and scene tasks (performed in different runs), p
1
= 0.07,
0.22, 0.46 and 0.61, while the corresponding p
2
= 0.61, 0.46, 0.22 and 0.07, respectively. That is,
the total contrast energy of the interpretable contents (scene and face) was kept at 68% of the
total contrast energy of the image.
Figure 3.1 The example stimuli for the experiments.
The four columns represent 4 conditions measured in each experiment. The color bars, which are adjacent
to the images, indicate the amount of component’s contrast energy of the image. The blue color represents
the face component, red color represents the scene component and the gray color represents the noise
component. The length of the specific color bar represents is proportional to the contrast energy of the
corresponding component.
Form-selective area
! For the form-selective areas such as posterior lateral occipital region (LO), the
Exp 2
Face &
scene task
Exp 1
Face task
Scene task
Exp 3
Face &
scene task
Face
Scene
Noise
43
anterior regions in the posterior fusiform sulcus (pFs), FFA, and PPA, BOLD response
amplitude was found to be a linear function of the signal proportion of the task-relevant
component (Figure 3.2A dotted lines). The slopes varied across experiments and tasks, and
variation is highly systematic. We found that we can fully capture the variation with a simple
sum-of-linear-components (SLC) model. Specifically, we can express the measured BOLD
response (R ) as the sum of the BOLD responses evoked by the attended and unattended image
components, with each being a linear function in the respective signal proportion. The slope of
the linear component response depends jointly on the category specificity or preference of a
given visual area and whether attention is directed to the image component. More generally, we
can write

R(

p ,a) = δ
ia
h
i
+ (1−δ
ia
)
˜
h
i ( )
p
i
+ b
k
i=1
n
∑

(2)
where R is the BOLD response, p
i
, the i-th component of


p , is the signal proportion of the

i-th
non-noise component defined as the ratio of the contrast energy of the component to that of the
full stimulus (i p
i
≥0 and p
i
i=1
n
∑
≤1), a indices the attended component, h
i
and

h

i are model
parameters representing the slope of the modulation due to the i -th component when the
component is attended and unattended, respectively, and b
k
is a constant offset that varies with
experiments and tasks. δ
xy
denotes the Kronecker delta, where δ
xy
=1 if and only if x= y . The
model requires that the net contrast energy of the entire stimulus be held constant and
independent of


p .
44
! For our experiments, we manipulated the signal proportions of the face (p
1
) and scene
( p
2
) components. There were four slopes, corresponding to the attended (h
1
,h
2
) and
unattended (


h
1
,

h
2
) face and scene components respectively. In Experiment 1 by construction,
the stimuli only contain the task-related components, so the scene component p
2
=0 in face
task, and the face component p
1
=0 in the scene task. The response for the face task

R(

p,1)=h
1
p
1
+b
1,1
, where the slope of the response ish
1
. Similarly, the slope of the scene task
response

R(

p,2) is h
2
In Experiment 2 where by construction p
1
= p
2
= p , face task responses
was modeled as

R(

p,1)=h
1
p
1
+

h
2
p
2
+b
2,1
= (h
1
+

h
2
)p+b
2,1
, which means the slope of the
response can be expressed as

h
1
+

h
2
. And the slope of the responses can be expressed as


h
1
+h
2
when subjects were doing the scene task. In Experiment 3 where by construction

h
1
+

h
2
= 0.68 ,
the responses

R(

p,1)=h
1
p
1
+

h
2
p
2
+b
2,1
= (h
1
−

h
2
)p
1
+0.68

h
2
+b
3,1
, the slope of the face task can
be written as

h
1
−

h
2
. Similarly the slope of the scene task response can be expressed as

h
2
−

h
1
.
! Thus, the four slope parameters can be uniquely determined with any two of the three
experiments. Data from the third experiment, which was independent from the other two, can
then be used to cross validate the model fit. As seen in figure 3.2 (solid lines), the slopes inferred
from any two experiments provided a very good fit to the data from third experiment. The
average R
2
for cross-validation (leaving one experiment out) is 0.76, 0.86, 0.90 and 0.91 for LO,
pFs, FFA and PPA respectively. Furthermore, the estimated slopes were similar regardless of
which two experiments were used to make the estimations (Figure 3.2B).
45
! Given the high consistency of the estimated parameters based on any two experiments,
we used all three experiments as data to infer the parameters of the SLC model. The model gave
a very good estimate of the response in object-related areas, accounting for 90.4%, 96.2%, 94.9%,
93.4% of the variance in the observed data from the LO, pFs, FFA, and PPA, respectively.
Figure 3.2 The form-selective visual areas’ responses predicted by SLC model
A) The measured response and model’s prediction in the object-relate areas. The blue color and red color
represents the response and model’s result of the face task and scene task , respectively. The solid line
represents the model’s fit when the model fitting use all three experiments’ data. Dash line represents
the model’s prediction, whose coefficients are from the fit based on the other two experiments’ data.
B) The coefficients inferred from two experiments.
Low and mid- level visual areas
! We also investigated the BOLD responses in lower-level visual areas (V1, V2, V3, hV4)
using the same method as we did in the high order visual areas. As seen in Figure 3.3, in
experiments 1 and 2, we did not observer any significant modulation across different conditions
in both face and scene tasks; however, V3 and hV4 show significant modulation in both
experiments. In experiment 3, we did not observe any significant modulation for all the lower-
level visual areas. These results suggest that V1 and V2’s responses are not modulated by the
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.07 0.22 0.46 0.61
0
0.2
0.4
0.6
0.8
0.07 0.22 0.46 0.61
0
0.2
0.4
0.6
0.8
0.07 0.22 0.46 0.61
0
0.2
0.4
0.6
0.8
0.07 0.22 0.46 0.61
0
0.2
0.4
0.6
0.8
LO pFS FFA PPA
0.2
0.4
0.6
LO pFS FFA PPA
0.2
0.4
0.6
LO pFS FFA PPA
0.2
0.4
0.6
Attended Components
Exp 1
Exp 2
Exp 3
BOLD Response
LO pFS FFA PPA Isolated coeffcients based on
Exp 2 and Exp 3
Isolated coeffcients based on
Exp 1 and Exp 3
Isolated coeffcients based on
Exp 1 and Exp 2
Face
task
Scene
task
Measured
Data
Predict from
other two exps
Fit from
three exps
h1
h1
~
h2
h2
~
A) B)
46
external noise or the category of the components. As long as the whole image’s contrast energy
was kept as constant, V1 and V2’s response was constant. However, V3 and hV4’s responses
were modulated with the noise, but not modulated with the components, suggesting that V3
and hV4 differentiate between structure and noise, but do not respond differently to particular
categories of objects.
! The model can account for 37.5%, 35.2%, 54.0%, and 76.9% of the variance for V1, V2, V3,
and hV4, respectively. The relatively lower R
2
values for V1 and V2 are due to the fact that
BOLD response in these regions are relatively insensitive to image content, as long as the total
RMS contrast is held constant (Tjan et al., 2006).
Figure 3.3.Ventral visual areas’s responses and fitted with SLC model
The marker represents the observed data, and line from the model’s fit based on three experiments. Blue
represents the data and fit for face task, and red for the scene task. Error bars represents the within-
subjects error.
Category specificity and attention modulation
! The hallmark of category specificity is the stronger activation of an area in response to
one stimulus category (e.g. faces) than to a number of control categories. In our experiment, by
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.05 0.16 0.34 0.45
0
0.2
0.4
0.6
0.8
0.07 0.22 0.46 0.61
0
0.2
0.4
0.6
0.8
0.07 0.22 0.46 0.61
0
0.2
0.4
0.6
0.8
0.07 0.22 0.46 0.61
0
0.2
0.4
0.6
0.8
0.07 0.22 0.46 0.61
0
0.2
0.4
0.6
0.8
0.07 0.22 0.46 0.61
0
0.2
0.4
0.6
0.8
0.07 0.22 0.46 0.61
0
0.2
0.4
0.6
0.8
0.07 0.22 0.46 0.61
0
0.2
0.4
0.6
0.8
0.07 0.22 0.46 0.61
0
0.2
0.4
0.6
0.8
V1 V2 V3 hV4 LO pFs FFA PPA
Exp 1
Exp 2
Exp 3
Scene task
Face task
Attended Components
BOLD Response
47
comparing the difference between the face component’s modulation and scene component’s
modulation, we can quantify category specificity in the ventral visual areas.
! Based on an SLC model’s fit, which is based on three experiments’ data, we can get four
coefficients,

h
1,

h
2 ,

˜
h
1, and

˜
h
2 . These four coefficients indicate the slope of the relationship
between signal energy proportion and BOLD response for attended faces, attended scenes,
unattended faces, and unattended scenes, respectively. Using these coefficients, we can define
the category specificity under attended condition as

h
1
−h
2, and under unattended condition as

˜
h
1
−
˜
h
2 . If the function value is bigger than 0, it indicates the region responds preferentially to
faces under the specific attention status, and if the value is smaller than 0, it indicates the region
prefers scenes. The bigger the absolute value of the difference is, the greater the category
specificity of that region is.
! As seen in figure 3.4 B), under the ! attended condition, LO and pFs show small and
non-significant preference to the scene category. However, in the unattended condition, the LO
and pFs shows significant preference to the face category. FFA shows stronger preference to face
category regardless attention; In contrast, PPA only shows stronger specificity to the scene
category under the attended condition, and shows no category specificity under the unattended
condition.
! Using the same 4 coefficients, we can also measure how attention modulates the
response to a certain category. We define attention modulation as

h
i
−
˜
h
i, i = 1 or 2 for face, or
scene, respectively. As seen in Figure 3.4 C). In the object-selective area, attention modulation
effect is much smaller for faces than for scenes, which suggest that the face processing is more
involuntary than scene processing.
48
Figure 3.4 All ventral visual areas’ coefficients, category specificity and attention modulation
A) The four model coefficients inferred from each ventral visual areas based on the three experiments.
The errorbar represents the between-subjects SE;
B) Category specificity as quantified by the difference between face modulation and scene modulation
under the two attention states for different visual areas;
C) Attention modulation for the two content categories (face and scene) across the visual areas
Voxel-based analysis with SLC model
! The previous results show that our SLC model can explain nearly all of the category-
based and attention-based modulation in the investigated areas at the ROI-level. This suggests
that the SLC model might explain modulation across the ventral visual areas even at the voxel
level. Voxel-based analysis with our SLC model would provide us with more detailed
information about the visual areas’ response properties.
! One difficulty in investigating data at the voxel level is the lower signal to noise ratio. To
increase the signal to noise ratio without losing the fine scale of the spatial information, the
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Coeffcient
V1 V2 V3 hV4 LO pFs FFA PPA
h1
h1
~
h2
h2
~
ï ï ï 0
0.2
0.4
V1 V2 V3 hV4 LO pFs FFA PPA
Face
Scene
Category Speci!city
Attended
Unattended
ï ï 0
0.1
0.2
0.3
0.4
0.5
Face
Scene
Attention Modulation
V1 V2 V3 hV4 LO pFs FFA PPA
*
*
*
*
* *
*
*
*
A)
B) C)
(h
1
< h
2
)

(h
%
1 < h
%
2)

(h
1
< h
%
1)

(h
2
< h
%
2)
49
functional data were smoothed using a small Gaussian window on the surface, with full width
at half maximum (FWHM) = 3.5mm based on the distance between the center vertex and
surround vertices. Since the analysis is not based on ROI, the coregistration across subjects’
different brains into one space is also a challenge. The poor alignment of subjects’ brains leads
to a loss of sensitivity (Nieto-Castanon and Fedorenko, 2012). Here, we used land-mark based
cortical coregistration (Joshi et al., 2007; Pantazis et al., 2010) to coregister 5 subjects’ brains to
one subject’s. Instead of using anatomical landmarks, such as sulci or gyri, we used V1, V2, V3,
FFA and PPA as cortical landmarks to align the different brain, because of the high reliability of
these regions’ boundaries and smaller variation across different subjects. !
! First, to check whether our model can explain the modulation we observed, the R
2
of
each vertex was projected on the surface (Figure 3.5A). For most vertices in the high visual
areas, our SLC model can explain the variance very well. In general, the more anterior of the
voxels are in, the higher R
2
value are. It could be due the more anteiror regions are modulated
by our stimulus’s manipulation. Also, the inferred coefficient maps (Figure 3.5B) was also
projected on the surface,. As to the category specificity, the vertices sitting in FFA and PPA
exhibit high modulation to the face and scene image, respectively. Also the regions OFA and
TOS also exhibit the high modulation to the face and scene, respectively.
50
Figure 3.5 R
2
and coefficients maps obtained from voxel-based SLC model.
A) The SLC model was fit to all three experiments, and the R2 values associated with each voxel were
mapped to the cortical surfaces.
B) The 4 coefficients inferred from the model’s fit are mapped on the cortical surface.
Discussion
! Based on our three complementary experiments, we built a linear model which shows
that the BOLD response to superimposed images can be expressed as the sum of the responses
to the individual components of that image, where each component’s response is expressed as a
linear function of signal proportion. The slope of this linear function is determined by the area’s
category specificity and whether the image component is attended by the subject or not
SLC model is a normalization model
LO
FFA
pFs
PPA
V1
V2
V3
hV4
LO
FFA
pFs
PPA
V1
V2
V3
hV4
LO
FFA
pFs
PPA
V1
V2
V3
hV4
LO
FFA
pFs
PPA
V1
V2
V3
hV4
V1
V2
hv4
V3
LO
FFA
pFs
PPA
V1
V2
hv4
V3
LO
FFA
pFs
PPA
V1
V2
hv4
V3
LO
FFA
pFs
PPA
V1
V2
hv4
V3
LO
FFA
pFs
PPA
h2 h1
Face, Attended
h1
~
Face, Unattended
Scene, Attended
h2
~
Scene, Unattended
Coe!cient Map
0
0.7
-0.7
LO
FFA
pFs
PPA
V1
V2
V3
hV4
V1
V2
hv4
V3
LO
FFA
pFs
PPA
R
2
Map
Left Hemisphere Right Hemisphere
A
B
0.5
1
0
51
! The SLC model is a simplified version of the divisive-normalization model. The divisive-
normalization model, in which the responses of a neuron are divided by a common factor that
includes the summed activity of a pool of neurons, has been widely used to describe several
suppression effects or mask effects observed in neural responses. When stimulus contrast is the
driving input for a population of neurons, the model can be expressed as:
R=γ
C
j
n
σ
n
+ C
k
n
k
∑
(2)
where response R is driven by the excitatory input C
j
and also modulated by the denominator.
In the denominator, C
k
∑
k
n
reflects the summed activity of a pool of neurons ; σ determines
how responses saturate with only excitatory input C
j
, and n is an exponent that amplifies the
individual input. This model has successfully described V1’s cross-orientation suppression
(Freeman et al., 2002; Busse et al., 2009), surround suppression (Cavanaugh et al., 2002a, b), and
several suppression effects observed in other visual areas, like MT, V4 and IT (Reynolds et al.,
1999; Zoccolan et al., 2005). In our SLC model, the component’s response is a linear function of
the signal proportion, defined as the ratio of the image’s contrast energy to the whole image’s
contrast energy. The component’s response can be expressed as
R=γ
c
2
c
k
2
k
∑
, (2)
! Comparing our SLC model (equation 1) and the normalization model (equation 2), there
are two differences, one is that our model specifies the exponent n to be 2, rather than leaving it
as a parameter. Since most our regions’ responses can be accounted by linear function of signal
52
proportion, suggesting that n should be around 2. The second difference is that SLC model
omits the σ . The σ will affects the response when
C
k
n
k
!
is small. However, in our
experiment, contrast energy is kept as constant and much larger than σ , so whether the model
includes σ or not has little effect with the model‘s output. It also suggests that SLC model have
to put σ into consideration, when the image’s contrast level is small.
! The normalization model also can describe the responses when there are more one
components put drive to the system.In the cross-orientation suppression experiment about V1’s
response (Busse et al., 2009), the mask’s orientation is close to the preferred condition of the
condition, it is reported that the V1‘s responses are weighted sums of the individual grating
responses, and the weight is dependent on the contrast. The response R can be written as:
R=
γ
1
C
1
n
+γ
2
C
2
n
σ
n
+C
1
n
+C
2
n
3 ! This is also with our SLC model. The SLC model describes that the multiple objects’
responses will be the simple-sum of the single component’s response, and the weight is defined
by the ratio of contrast energy of the single object to the whole image’s contrast energy. If we
represent our SLC model in terms of contrast of different component and consider there are
only two components in the image, the response R can be written
R=
γ
1
C
1
2
+γ
2
C
2
2
C
1
2
+C
2
2
Here we also simplify the model as n = 2 and omit the σ .
53
! This model is consistent with some previous findings in the literature. It also predicts
that when the paired objects have similar contrast energy, the response to pairs of objects is the
average of the response evoked by the individual objects, which is consistent with the finding in
human subject’s LOC area (MacEvoy and Epstein, 2009) and individual neuron’s firing rate
recorded in monkey’s inferotemporal cortex(IT) (Zoccolan et al., 2005). Also If one component’s
contrast energy is much bigger than the others, the responses will follow winner-take-all model.
! The SLC model is designed to explain the observed BOLD response in different ventral
visual areas, rather than single-cell firing rates. Neverthess, it is still a normalization model,
which suggests that in the ventral visual areas, normalization is a canonical neural computation.
Attention modulation
! Attention modulation in our SLC model is modeled as a change of the coefficient. In the
previous literature about attention effect on BOLD or neural responses, attention is found to
increase responses by multiplying a fixed response gain factor, called as response gain control.
In some other studies, the attention is found with a change in contrast gain, called as contrast
gain control. The SLC model describing the attention effect as a change of the coefficient agrees
with the response gain control mechanism. It is exactly predicted by the normalization model of
attention (Reynolds and Heeger, 2009). Their model can explain the attention effect as contrast
gain control and response gain control depending on the size of attention field and stimulus
field. Also, their model has a very clear prediction that If the attended components and
unattended components are spatially overlapped, such as when attention is applied on the
feature (Saenz et al., 2002, 2003; Serences and Boynton, 2007) or object (O'Craven et al., 1999;
Serences et al., 2004), regardless of whether the size of attention field is bigger or smaller than
the size of stimulus, the attention effect always behaved as response gain control (Reynolds and
54
Heeger, 2009), and it has been proven by psychophysical experiments (Herrmann et al., 2012).
The attention effect described by our SLC model is consistent with this finding. !
! In conclusion, across all the ventral visual areas we examined, our SLC model is an
extreme simple model, which describes the response of area can be expressed as the sum of the
component’s response. Each component’s response is linear function with signal proportion can
account for the visibility of the object, multiple objects’ representation, and attention
modulation, as all these modulations share the same mechanism of the normalization. With
reasonable assumption about the parameters of normalization, the simplicity of the formula
suggests it could be a powerful tool to study other similar questions.
Methods
Participants
! Six students (4 male) from the University of Southern California participated in this
study. All subjects had normal or corrected-to-normal vision. They gave informed consent prior
to the experiments. The experiments were conducted with the approval of the Institutional
Review Board of the University.
MRI Acquisition
! All (f)MRI imagess were acquired at the Dana and David Dornsife Cognitive
Neuroscience Imaging Center at the University of Southern California. MRI recording used a
standard birdcage head-coil on a Siemens 3T MAGNETON Trio MRI system with TIM. For each
subject, anatomical sagittal images (256 × 256 × 192) of 1-mm3 isotropic spatial resolution were
obtained with a T1-weighted 3D MPRAGE sequence (T1 = 900ms, TR = 2,300 ms, TE = 2.98ms,
55
flip angle = 9°). BOLD activity was measured with a T2*-weighted echo planar imaging
sequence (16 slices acquired perpendicular to the calcarine sulcus, 3 × 3 × 3mm3 voxel
resolution TR = 1,000ms, TE = 25ms, flip angle = 60°, FOV = 192 × 192 mm).
Stimuli
Main experiment
! The stimulus (3.27° × 4.64°) was presented at the center of the screen, where a fixation
point was presented at all times. The scene photographs were selected from the Corel Stock
Photo library and converted into gray-scale images. The face images were selected from the
FERET face database (Philips, et al. 2000) and masked with an ellipse to exclude the hair.
! The whole study contains 3 experiments. In each experiment, subjects were tested with
both a face task and a scene task, and within each task subjects’ BOLD responses were measured
under 4 conditions by using an event-related design. In the face task, subjects were asked to
report whether the two consecutive images were of the same face. In the scene task, they were
required to report whether two consecutive images were taken from the same scene.
! In each condition of each experiment, the images shown to the subjects are the
amalgamation of the noiseless content and noise. The noiseless content could be a face or scene,
or the combination of a face image and a scene image. The noise and the noiseless content were
made to have the same spatial scale so that the noise was likely to survive the noise-reduction
stages in early vision and perturb the later visual processing states effectively (Tjan et al,.
Gober ). We used spatially correlated (“pink”) noise, with the power spectrum of the noise
matched exactly to that of the stimulus, as pink noise will survive the generic noise-reduction
steps in the visual system and could effectively affect higher-level processing, as shown in our
data. The resulting pink noise had the same mean luminance and root mean square (RMS)
56
contrast as the pictures. RMS contrast is defined as the standard deviation of
the pixel intensities.
! The contrast energy of the whole image (content + noise) was kept as same in all
conditions and in all experiments. Let p
1
denote the ratio of contrast energy of face component
to the full image, which we refer to as the signal proportion of the face, and let p
2
denote the
signal proportion of the scene. The whole image can be written as
image= p
1
(face−L
0
)+ p
2
(scene−L
0
)+ 1− p
1
− p
2
(noise−L
0
)+L
0
where L
0
is the mean luminance, and face , scene and noise indicate an image of face, scene
and a sample of the pink noise.
! In Experiment 1 (Figure 3.1, Exp 1), the images contained only the task-related
component, with signal proportion varied at 0.05, 0.16, 0.34 and 0.45. That is, for the four
conditions of the face task , p
1
= 0.05, 0.16, 0.34 and 0.45, respectively, and p
2
= 0; for the four
conditions of the scene task , p
1
= 0 and p
2
= 0.05, 0.16, 0.34 and 0.45, respectively.
! In Experiment 2 (Figure 3.1, Exp 2), the images contained both face and scene
components. Furthermore, the signal proportions of face and scene components were equal and
varied together in four conditions. In separate sessions, subjects were asked to perform either
the face task or the scene task with the same stimuli by attending to the task-relevant
components. Thus, for both tasks, p
1
= p
2
= 0.05, 0.16, 0.34 and 0.45.
! In Experiment 3 (see Figure 3.1, Exp 3), the images were again superimposed face and
scene images (in addition to pink noise). Unlike experiment 2, the noise component was kept
the same across conditions, while the ratio between the face component and scene component
57
varied. For both face and scene tasks (performed in different session), p
1
= 0.07, 0.22, 0.46 and
0.61, while the corresponding p
2
= 0.61, 0.46, 0.22 and 0.07, respectively. The noise component
was kept at 0.32.
! The event-related scans for the main experiment consisted of one epoch of experimental
trials and two 12-s fixation epochs, one at the beginning and one at the end of the scan. Each
scan consisted of 25 experimental trials for each of the four conditions and 25 fixation trials that
were nonperiodically interleaved with the experimental trials. Each trial lasted for 3 s, and in
each trial, two images were presented sequentially for 160 ms with a blank of 250 ms in-between
followed by a blank screen at mean luminance. The order of presentation was counterbalanced
so that trials of each condition, including the fixation condition, were preceded (2 trials back)
equally often by trials of each of the other conditions. For each experiment, there would be 4
scans for each subject. 2 scans for the face task, and 2 scans for the scene tasks, which are
interleaved with each other, and the order of the tasks were counterbalanced across the subjects.
Retinotopy and Localizer scan for defining the visual areas
! Wedges and half rings made of flickering (4 Hz) radially scaled color checker-board
patterns, were used to identify the retinotopic visual areas. For polar angle mapping, a 45°
wedge with an 8.5° radius rotated (jumped) counterclockwise by 11.25° every second, so it
swept the whole visual field in 32 s. For eccentricity mapping, the rings expanded in equal
logarithmic steps from the center of display, where the subjects fixated, and took 20 seconds to
reach the maximum radius of 8.5°. The fixation of the display changed from ‘‘+’’ to ‘‘×’’ or vice
versa randomly between 5 and 10 s (uniform distribution); the subject pressed a button as soon
as a change of the fixation mark was detected.
58
! Two blocked-design localizer scans were run to define LO, pFs, FFA and PPA for each
subject. Each run was composed of 12-s blocks with alternating blocks between intact objects,
scenes, faces, and scrambled images. Each image subtended a visual angel 6° × 6°. Subjected
were asked to passively view the images.
Localizer scan for defining the response region
! Subjects also passively viewed clear noise-free images of face and scene to define the
response areas. The size, contrast and the mean luminance of the images are the same as the
images used in the main experiment. The face and scene stimuli were separately presented in
16s stimulus blocks, interleaved with 16s blank blocks. There are two scans and in each scan,
there are 4 blocks for face images and 4 blocks for scene images.
! All stimuli were displayed using a 3-chip DLP projector with linear gamma. The stimuli
were controlled using MATLAB (version 5.0) and Psychophysics Toolbox with a Mac computer
(Brainard, 1997; Pelli, 1997).
Data Analysis
! Data analysis was performed using BrainVoyager QX (Brain Innovation B.V ., Maastricht,
The Netherlands) (and custom-built Matlab (MathWorks, Massachusetts, USA) code. The
preprocessing of all the functional scan includes head movement correction, high-pass temporal
processing (cutoff: 3 cycles per run) and linear-trend removal. All functional scans were
coregistered to each subject’s anatomical scan and was transformed into Talariach coordinate.
The first 15-18 s of BOLD signals (retinotopy experiment: 15 s, Localizer scan for defining high-
level visual area: 12 s, main experiments: 18s ) were discarded to ensure equilibrium in
longitudinal magnetization and subject state. And the anatomical volume for each subject
59
obtained in the retinotopic mapping session was inflated to generate a model of the cortical
surface.
Defining ROIs
! Rotating wedge and expanding ring stimuli were used for defining subjects’s low-level
visual areas (V1/V2/V3/hV4) followed by a tradition traveling wave method (Engel et al.,
1997).
! LOC was defined by comparing the BOLD activity of the conjunction contrast of object
minus scene, object minus face and object minus scrambled with a t-map threshold at False
Discover Rate (FDR) <0.05. These LOC voxels were divided into 2 subregions, LO and pFs
based on anatomical location (Grill-Spector et al.1999, Kourtzi et al. 2003; Haushofer et al.,
2008). The FFA was defined by the conjunction contrast of face minus scene, face minus object
and face minus scrambled with a t-map threshold at FDR < 0.05 . Since FFA and pFs had
overlap, the pFs mentioned in the paper indicates the regions which sit in the originally defined
pFs, but not in FFA. The PPA was defined by the conjunction contrast of scene minus face, scene
minus object and scene minus scrambled with a t-map threshold at FDR < 0.05.
! A general linear model (GLM), assuming a stereotypical hemodynamic was used to
identify the response areas. The response areas are defined by the voxels that show significant
positive response (FDR <0.05) either in the contrast between face condition and fixation
condition or in the contrast between scene condition and fixation condition. Each ROI is defined
by combining the defined region’s boundary and response regions.
ROI based analysis
! For each experimental scan, the fMRI voxel data were averaged within each ROI and %-
transformed. ROI averaged data were concatenated across scans, and the results were
60
deconvolved against an indicator function formed by placing a Dirac delta function at the on-set
of the stimulus interval of the same trial type while allowing for a constant per-scan offset of no
interest . Motion correction paramters are also put as repressors in the analysis to capture the
noise introduced by the head motion. The deconvolution analysis resulted in one response time
course for each non-fixation trial type, depicting the BOLD response evoked by the condition.
The peak of the time-course is defined as the response of the condition. The responses for each
ROI are averaged across subjects.
Voxel based analysis
! The functional data were projected on the cortical surfaces and smoothed with a
gaussian window (FHWM = 4mm) based on the distance between the center vertex and
surround vertices. Since the surfaces are defined as discrete meshes and can be represented as
graphs with weights corresponding to the lengths of the edges between vertices, the distances
between vertices of the mesh can be approximated with the length of the shortest paths on the
graph by using a Dijkstra algorithm (Dijkstra, 1959). After smoothing the functional data,
deconvolution analysis is applied on each vertex as ROI based analysis. Also The max value
between 4s and 8s of the time-course is defined as the response of the condition.
The coregistration of different subjects’ cortical surfaces are processed by a landmark based
coregistration (Joshi et al., 2007; Pantazis et al., 2010). Instead of using marked sulci or gyri as
landmarks, we used functionally defined regions ( V1, V2, V3, FFA, and PPA) as landmark. The
coregistration was done between 1 selected subject and other 5 remaining subjects.
We mapped other 5 subjects’ responses to the the selected subject based on the vertices’
correspondence obtained from cortical coregistration. Six subjects’ response are averaged on
each vertex and fitted into the model.
61
Model:
For explaining these three experiments data, we used the sum-of-linear-components (SLC)
model to fit the data:

R(

p ,a) = δ
ia
h
i
+ (1−δ
ia
)
˜
h
i ( )
p
i
+ b
k
i=1
n
∑

! where R is the BOLD response, p
i
is the signal proportion of the

i-th non-noise
component defined as the contrast energy of the component to that of the full stimulus (i.e.
p
i
≥0 and p
i
i=1
n
∑
≤1 ) indices the attended component, h
i
and

h

i are model parameters
representing the modulation due to the i -th component when the component is attended and
unattended, respectively, and b
k
is a constant offset that varies with experiments and tasks. δ
xy

denotes the Kronecker delta, where δ
xy
=1 if and only if x= y . The model requires that the net
contrast energy of the entire stimulus be held constant and independent of


p
.
! In our experiment, since we have two non-noise component, so n = 2, based on this
model, we can get 4 parameters,

h
1
,h
2
,h

1,h

2 , which indicates the coefficients of responses of
face and scene under attended condition, and coefficients of responses of face and scene under
unattended condition.
! The model can be fit with the BOLD response from any two experiments or three
experiments. It also was applied on the BOLD response from ROI analysis and surface-based
analysis
62
Chapter 4
Quantifying the relationship between the fMRI BOLD
signal and neural activity with an achiasmic human
subject
Abstract
! Quantifying the relationship between neural activity and the fMRI BOLD signal is hard
because “neural activity” is not a unitary quantity, and precisely measuring neural activity
requires invasive techniques. Attempts to infer this relationship from stimulus-evoked BOLD
response have been frustrated by the complex nonlinearity between stimulus and neural
activity. Here we describe a unique in-vivo model for non-invasively determining the
relationship between neural and BOLD activities. We demonstrated that in the low-level visual
cortex of a human subject who was born without optic chiasm, there are two nearly identical
populations of non-interacting but co-locating neurons, with non-overlapping receptive fields.
By presenting identical stimuli to both of these receptive fields instead of just one, we can
double the local neural activity, regardless of the definition of “neural activity”. Using this in-
vivo model, we found that BOLD response amplitude is proportional to approximately the
square root of the underlying neural activity.
63
Introduction
! While BOLD fMRI has provided unprecedented insights into the workings of the human
brain, the precise relationship between neural signal and BOLD response is not known. Studies
relying on the BOLD signal to measure brain activity have thus far relied on the assumption
that the relationship between the BOLD signal and neural activity is practically linear, despite of
evidence to the contrary (Boynton et al., 1996; Dale and Buckner, 1997; Huettel and McCarthy,
2000). BOLD response is indirectly related to local neural activity through mechanisms
associated with oxygen metabolism, blood volume and blood flow (Davis et al., 1998; Hoge et
al., 1999). Neural activity likewise comprises multiple components that are associated with
information processing. These include the release and re-uptake of neurotransmitters,
generation, propagation and aggregation of postsynaptic potentials, generation and
propagation of action potentials and various maintenance activities. Macroscopic measurements
of neural activity typically include single- and multi-unit spiking activities as well as the power
in the different frequency bands of the local field potential (LFP). While there is demonstrably
close relationship between the BOLD signal and these macroscopic measurements of neural
activity (Logothetis et al., 2001; Mukamel et al., 2005), the quantitative nature of this relationship
has not been well characterized, nor are the specific relationships between the various
components that underlie the neural and BOLD activities known.
! Most fMRI applications bypass neural response entirely and instead rely on establishing
a direct relationship between the BOLD response and the stimulus condition. The general
approach is to assume the BOLD responses evoked at different time and in different stimulus
condition sum linearly. Boynton and colleagues (1996) studied how BOLD signal varied with
stimulus in the striate cortex and found that the system is approximately linear in the sense that
64
the BOLD response evoked by a 12 s stimulus was well approximated by summing the
responses from two consecutive 6 s stimulations. However, predictions using stimulation of
much shorter duration (e.g. 3 s) failed to predict the long-duration stimulus. This and similar
studies (Cohen, 1997; Dale and Buckner, 1997; Heckman et al., 2007) have been used to justify
the broad application of the general linear model in most of fMRI data analyses. While the
neural response is not explicitly involved in this type of approaches, it is always in the
background -- any nonlinearity observed in the BOLD response, e.g. in surround suppression
(Zenger-Landolt and Heeger, 2003) or adaption (Grill-Spector and Malach, 2001; Kourtzi and
Huberle, 2005; Larsson and Smith, 2012), is often attributed to the underlying neural response.
The implicit assumption in the common practice is that the relationship between the BOLD
response and the neural response is essentially linear, a view that is widespread (Logothetis and
Wandell, 2004) and under-examined.
! An extensive set of biophysical models have been proposed to express either the steady-
states (Davis et al., 1998; Griffeth and Buxton, 2011) or the dynamics (Buxton et al., 1998;
Mandeville et al., 1999; Feng et al., 2001; Toronov et al., 2003; Blockley et al., 2009) of the BOLD
response in terms of the more basic physiological components, such as blood flow, blood
volume, and oxygen saturation and extraction fraction in different vascular compartments.
While these biophysical models are foundational in our understanding of the BOLD signal, they
do not provide any explicit and quantitative linkage between the neural response and the
physiological components that are the inputs to these models. Friston et al (2000) proposed a
linkage between the evoked neural activity with the blood-flow parameter of the Balloon model
by Buxton et al (1998, also see Stephan et al., 2007). While the resulting model is a powerful tool
65
for inferring effective connectivity between brain regions from the BOLD signal, direct empirical
support for this specific linkage is largely absent.
! How could one empirically determine the quantitative relationship between the BOLD
response and the neural activity, and do so when the constituents of the neural activity are not
comprehensively defined? Consider an in-vivo model that consists of two distinct groups of
neurons that share the same local blood supply and vascular control. Further suppose that these
two populations of otherwise identically-functioning neurons are independent, non-interacting,
and with non-overlapping population receptive fields (pRFs), allowing independent stimulatoin
to each population. Such a system would be ideal for mapping the relationship between neural
and BOLD responses. Presenting identical stimuli to both pRFs doubles the local neural activity
relative to presenting the stimulus to just one of pRFs, even though we do not know the
constituents of neural activity. Measuring BOLD responses under these conditions allows us to
not only directly test for linearity but also quantitatively characterize the relationship between
neural activity and BOLD response up to a scaling factor. This approach does not require us to
know the constituents of neural activity.
! We found such an in vivo system in the low-level visual cortex of a human subject who
was born without an optic chiasm. This rare condition, known as achiasma or non-decussating
retinal-fugal fibre syndrome, prevents the normal crossing of optic nerve fibers from the nasal
hemi-retina to brain hemisphere contralateral to the eye. This results in a full representation of
the entire visual field in each cerebral hemisphere (Williams et al., 1994; Victor et al., 2000;
Hoffmann et al., 2012; Davies-Thompson et al., 2013). In particularly, the representations of two
visual hemifields are superimposed on the low-level visual areas ipsilateral to each eye. Two
points symmetrically located across the vertical meridian are mapped to the same point on the
66
cortex (Hoffmann et al., 2012). In other words, there are two pRFs for every point on this
person’s low-level visual cortex. The pair of pRFs are symmetrical located across the vertical
meridian. Most critically for our purpose, we showed here that the two neural populations that
underlie these pRFs, while co-locating, do not interact.
! To determine the relationship between the BOLD signal and neural activity, we
measured BOLD responses in cortical areas V1-V3 of our achiasmic subject by presenting the
luminance-defined stimuli of varying contrast to either one or both pRFs of each region of
interest. From this data set, we used a model-free non-parametric method to estimate the
functional relationship between BOLD (B) signal and neural activity (Z), which we called the
BvZ function. We found that the BvZ function is well approximated by a power function with
an exponent close to 1/2. The exponent stays the same for short (1 s) and long (6 s) stimulation.
We cross-validated this result by converting the BOLD respnose from our and nine other fMRI
studies of contrast response to neural activity using the BvZ function and compared the inferred
neural response to the average single-unit response obtained from primates.
Results
The achiasmic subject (S) and his retinotopy
Our primary research subject (S) was a 24-year-old achiasmatic Caucasian male. S has
congenital nystagmus and was diagnosed with isolated foveal hypophasia. His visual acuity
was 0.7 (OS) and 0.5 (OD) logMAR. We confirmed his absence of an optic chiasm using MRI
(Figure 5.1 A). We obtained high-resolution monocular hemifield retinotopic maps from each of
his eyes. The retinotopy was well-defined and of high-quality. Within the retinotopically defined
visual areas V1-V3, the subject’s retinotopic representation of the ipsilateral visual field is a
67
mirror image of its contralateral field representation (Figure 4.1 B). The two are superimposed
on the hemisphere ipsilateral to the stimulated eye. Two points symmetrically placed across the
vertical meridian are mapped to the same point on the cortex. This left-right mirror mapping is
precise at the level of individual voxels – 3x3x3 mm3 (Figure 4.1 C). These findings were
consistent with previous reports on human achiasma (Hoffmann et al., 2012).

Subject S: Right eye presentation
left viusal !eld,
right hemisphere
right visual !eld,
right hemisphere
1 2
e! e"
θ!
V1 V2 V3
θ"
0 90 180
0
90
180
0 90 180
0
90
180
0 90 180(deg)
0
90
180
0 3 6 9
0
3
6
9
0 3 6 9
0
3
6
9
0 3 6 9 (deg)
0
3
6
9
e!
e"
A B
C
Subject S: MRI Images
Eccentricity (e)
Polar Angle (θ)
Figure 4.1.Retinotopy of the achiasmic subject S
A) The MRI image of S shows the lack of the optic chiasm.
68
B) S has well-defined retinotopy. Both left and right visual fields are mapped to the hemisphere
ipsilateral to the stimulated eye (data from right-eye stimulation are shown). Eccentricity and polar
angle map are organized orderly for both visual hemifields. Visual area boundaries are identified at
where polar angle reverses.
C) A schematic of each voxel’s two population receptive fields (pRFs), situated at the mirror-symmetric
locations about the vertical meridian. The polar angles relative to the vertical meridian (upper panels)
and eccentricity of each of the pRFs for a set of randomly sampled voxels in V1-V3 are plotted against
each other. Most all of values are near identity line suggesting that the two pRFs of each voxel are at
the mirror location across vertical meridian.
Co-localized but non-interacting neurons
The results from retinotopy showed that each voxel in the visual cortex of S has two pRFs. We
next tested if the neural populations that underlie these pRFs interact with each other.
Behaviorally, S does not make any obvious confusion between visual hemifields. Most tellingly,
S is an avid reader, which strongly suggest that the co-locating neural populations do not
interact functionally. To quantify this, we measured whether the contrast threshold for S to
detect a 45 degree Gabor grating could be affected by a noise mask of the same spatial
frequency placed at a location symmetrically from the target either across the vertical meridian,
which projects to the same cortical location as the target, or across the horizontal meridian,
which does not project to the same cortical location as the target (Figure 4.2 A). S performed this
sensitive threshold task monocularly (with his right eye) in front of a calibrated CRT. We found
that the placement of the mask did not affect detection threshold, and that the threshold is
within the normal range without mask (Figure 4.2 B). This and other demonstrations of a total
lack of anomalous interactions between the two pRFs (Victor et al., 2000) strongly suggest that
the two underlying neural populations are functionally independent.
! To test if the two neural populations are physiologically independent, we measured
long-duration fMRI adaptation effect (Fang et al., 2005; Fang et al., 2007). We first established
that identical stimuli presented to each of the pRFs separately yielded nearly identical BOLD
response (Figure 4.6). We then measured the effect of adaptation when S was monocularly
69
performing a highly demanding fixation task. The adapting and testing patterns were four
counter-flickering Gabors (Figure 4.2 C). Related to the adaptors, the test patterns could be
either at the same spatial locations or at the mirrored locations that project to the same points on
the visual cortex of S. The orientation of the test could be either the same as or orthogonal to the
adaptors. As expected, when the adaptors and test patterns were at the same spatial locations,
we observed strong BOLD adaptation when they were of the same orientation and a large
release from adaptation when they were of orthogonal orientations. This result has been taken
to mean that vertically and horizontally tuned neurons are of distinct groups, replicating Fang
et al.. For our purpose, the critical conditions are when the adapting and testing patterns were at
different spatial locations but projected to the same points on the cortex of S. In these
conditions, we found a large release from adaptation equal or greater in amplitude than in
same-location cross-orientation condition. We take this result to mean that the co-locating neural
populations that underlie the two pRFs on either side of the vertical meridian do not interact
physiologically with each other.
70
Mask and Target
project to
SAME voxels
ï ï ï ï ï ï ï ï ï ns
ns
Log Contrast Threshold
A
Mask and Target
project to
DIFFERENT voxels
Pre-adaptation
20 s
T op-up
T est stimulus
1 s
one of four
conditions
Same
Orientation
Orthogonal
Orientation
Same
Location
Mirror
Location
one
trial
5 s
C
B D
Contrast detection experiment Long-term fMRI adaptation
5 ï 5 ï 5 ï Time (s)
% Signal Change
V1 V2 V3
Figure 4.2 Behavioral and physiological evidences for non-interactivity of the co-locating neural
populations.
A) Schematic of the psychophysical experiment of contrast detection with contralateral noise mask. The
target Gabor patch was presented in either the lower-right (shown) or upper-left quadrant. An
isotropic noise mask within the same spatial frequency band as target was presented at the mirror
location of target across either the vertical or horizontal meridian. S was to identify one of the two
temporal intervals when target appeared.
B) Regardless of target locations, contrast detection thresholds were essentially identical for the two
mask positions. Error bars denote ±SE across different blocks.
C) Schematic of the fMRI adaptation experiment. Each block of trials was preceded with 20 s of pre-
adaptation. Each trial began with 5 s presentation of the adapting stimulus (“top-up” adaptation) and
followed by one of the four test stimuli. Relative to the adapting stimulus, the test stimulus could
either be at the same or mirror location and have the same or orthogonal orientation. Subjects’
attention was controlled with a demanding central fixation task.
D) Time courses of fMRI responses in V1-V3 to the 4 test conditions in the fMRI adaptation experiment.
Error bars denote ±SE across trials. Test stimuli presented at the mirror location, regardless of
orientation, evoked response of equal or higher amplitude than that evoked by test stimulus at the
same location as the adaptor but with orthogonal orientation.
A pure BOLD summation experiment
! Our data suggest that each fMRI voxel in the visual cortex of S contains two
independent populations of neurons, each with a distinct pRF. We performed a pure BOLD
71
summation experiment with this unique in-vivo model in order to infer the relationship
between neural activity and the BOLD signal. We stimulated each pRF separately with spatially
disjoined stimuli A and B that are mirror images of each other (Figure 4.3 A); we also stimulated
them together by presenting A and B simultaneously (A+B).
! Each stimulus consisted of four counter-flickering black-and-white checkerboards. The
Weber contrast of the stimuli ranged from 0.05 to 1.0 in four equal log steps (five values). In
separate experiments, a stimulus was presented for either 1 s or 6 s, followed by a 16 s blank.
We estimated the full time course of the evoked BOLD response and its amplitude. In V1-V3
and across the contrast levels, the BOLD response to stimulus A+B was significantly higher than
what could be produced by doubling the contrast of stimulus A or B. More importantly, the
response to the combined stimulus (A+B) was significantly lower than the sum of the responses
to stimuli A and B (V1: z = 2.74, p<0.01, V2:, z = 7.24, p<10-12; V3: z = 6.04, p<10-8). This latter
finding shows that BOLD summation, in the absence of neural nonlinearity (assuming that the
A+B stimulus simultaneously excited two independent population of neurons), is itself
nonlinear, with a saturating (compressive) nonlinearity. This demonstration of a nonlinear
relationship between neural activity and BOLD response challenges the prevailing linearity
assumption (cf. Logothetis, 2001).
72
Figure 4.3 Results of the pure BOLD summation experiment obtained from the achiasmic subject with
6 s stimulus duration.
A) The stimuli used in the BOLD summation experiment. Stimulus types A and B are single-sided
stimuli, while type A+B are two-sided stimuli.
B) Estimated time courses from V1-V3 for the three stimulus types at five different contrast levels. Red
and magenta represent responses (lines) and ±SE (bands) to the single-sided stimuli, and green
represent responses to the two-sided stimuli. Black dash lines represent the predictions of linear
BOLD summation, which overestimated the measured response (green band).
C) The contrast response functions of V1-V3 as defined by the amplitude of time course. The amplitude
of a time course was taken to be the average responses between 8-10 seconds post stimulus onset. The
red lines represent the averaged single-sided response amplitude as a function of luminance contrast.
The green lines represent the two-sided response. The dark bands represent the predicted response
10
ï 10
0
10
ï 10
0
10
ï 10
0
10
ï 10
0
10
ï 10
0
10
ï 10
0
0 10 20
ï 0
1
0 10 20
ï 0
1
0 10 20
ï 0
1
0 10 20
ï 0
1
0 10 20
ï 0
1
0 10 20
ï 0
1
0 10 20
ï 0
1
0 10 20
ï 0
1
0 10 20
ï 0
1
0 10 20
ï 0
1
0 10 20
ï 0
1
0 10 20
ï 0
1
0 10 20
ï 0
1
0 10 20
ï 0
1
0 10 20
ï 0
1
A+B A B
linear summation
contrast summation
A
Contrast = V1
V2
9 %Signal Change
Contrast Time (s)
%Signal Change
B C
73
(and 68.2% confidence interval) of evoked with the two-sided stimuli under the assumption of linear
BOLD summation, while the magenta bands represent the prediction of contrast summation – the
equivalent contrast of a two-sided stimulus being twice that of the corresponding single-sided
stimulus. The measured two-sided responses were significantly lower than the predictions of linear
BOLD summation and higher than that of contrast summation.

Inferring the quantitative relationship between BOLD amplitude and neural activity
! We defined the peak of BOLD time course to be the BOLD amplitude. We sought to
express BOLD amplitude (B) as a function of neural activity (Z), where Z refers to the local
aggregate of neural activity with unspecified constituents. At each cortical region of interest, the
BOLD summation experiment provided five levels of Z, corresponding to the five levels of
luminance contrast for the single-sided stimuli (A, B). (The single-sided stimuli A and B resulted
in nearly identical time courses. We therefore use their averages in the analysis.) For each level
of Z evoked by a single-sided stimulus, the level of neural activity evoked by the corresponding
two-sided stimulus (A+B) is doubled under the assumption of non-interactivity, which we have
empirically demonstrated. We therefore have five pairs BOLD measurements that correspond to
Zi and 2Zi (Figure 4.4A). We do not know the values of Zi, but can reasonably assume that these
levels of neural activity were ordered Z
1
≥…≥Z
5 by stimulus contrast. Had we known these
values, we would have 10 data points on the BvZ function, relating BOLD amplitude to neural
activity.
! Without loss of generality, we can set Z
1
to 1. We are left with four unknowns 1≥Z
2
≥…
≥Z
5
. It is reasonable to assume that the BvZ function is monotonic and smooth. So we estimated
the four values Z
2
≥…≥Z
5
such that the ten data point (Z
1
. B
1,1
),…, (Z
5
. B
1,5
),(2Z
1
. B
2,1
),…, (2Z
5
.
B
2,5
) can be best described, in a least-squares sense, by a monotonic and maximally smooth
function (see Methods). This procedure amounts to shifting horizontally on log-scaled abscissa
74
the pairs of data points <(Z
i
. B
1,i
), (2Z
i
. B
2,i
)> relatively to one another until all ten points (five
pairs) fall on a smooth monotonic curve. We do not assume a priori any functional form of this
curve, but allow it to be any cubic spline function that is constrained to be monotonic. For cubic
spline functions, maximum smoothness corresponds to the minimum number of control points
needed to describe the data.
! Applying this stitching procedure to BOLD amplitude data from each of the achiasmic
subject’s three visual cortical areas of interest resulted in three BvZ functions (Figure 4.4 B) for
V1, V2, and V3 respectively. The stitching procedure does not assume any functional form, but it
is clear that the inferred BvZ functions can be well fitted by a power-law function (Figure 4.4 B):
B=kZ
γ
, where k is an arbitrary scaling factor related to the unit of Z. Its indeterminacy reflects
the fact that we do not know the constituents of neural activity. The exponent (γ) is the critical
parameter. We found that for 6 s stimulus presentation γ = 0.76 [95% CI: 0.66, 0.87], 0.54 [0.48,
0.62], and 0.55 [0.46, 0.67] for V1, V2, and V3, respectively. If the hemodynamics is essentially
the same in V1-V3, we would expect γ to be the same across these areas. Instead, the exponents
were statistically indistinguishable between V2 and V3. However, the value for V1 was outside
and higher than the 95% confidence interval of the values for V2 and V3 (Figure 4.4 C). We
suspected that the γ

value of V1 was artifactually biased to unity (linearity) because the neural
populations associated with the two pRFs are not truly co-locating in V1. The two populations
may each occupy distinct columns – with hemifield dominance columns replacing the normal
ocular dominance columns, as we shall elaborate in Discussions. We thus excluded the γ value
obtained in V1 from our construction of a generic BvZ function. Combining the data from V2
and V3, we estimated that the BvZ function for low-level visual areas (V1-V3) is a power
function with an exponent equal to 0.54±0.05 for the 6 s stimulus.
75
! Changing stimulus duration changes the time course of neural activity. Nevertheless, we
found that the BvZ function is remarkably stable over stimulus duration. Reducing the stimulus
duration from 6 s to 1 s resulted in only small changes in exponent of the BvZ function, with γ
0.56 [0.51,0.62], 0.49 [0.44,0.54], and 0.43 [0.37,0.50] for V1, V2, and V3, respectively (Figure 4.4D,
E, F). As with the 6 s stimulus, the γ from V1 is biased towards a significantly higher value, and
those from V2 and V3 are not significantly different from each other. Combining data from V2
and V3 yield γ = 0.47±0.03. A six-fold change in stimulus duration did not resulted in any large
change in the nonlinearly, although shorter stimulus duration seems to cause the BOLD
response to saturate sooner as indicated by a slightly lower γ.
1 2
10
ï 10
0
1 2
10
ï 10
0
1 2
10
ï 10
0
1 2
10
ï 10
0
1 2
10
ï 10
0
1 2
10
ï 10
0
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.8
1
% BOLD signal Change % BOLD signal Change
γ
A
B
C
D
E
F
V1 V2 V3 V1 V2 V3
6 s - stimulation
1 s - stimulation
10
0
10
1
10
ï 10
0
V1
V2
V3
10
0
10
1
10
ï 10
0
V1
V2
V3
V1 V2 V3 V1 V2 V3
Neuronal activity (a.u.) Neuronal activity (a.u.)
Figure 4.4 fMRI BOLD response as a function of neural activity
(A & D) Five pairs of BOLD repsonse amplitudes evoked in V1-V3 with the single-sided and two-sided
stimulations with two stimulus duration, 6 s (A) and 1 s (B). If the neural activity evoked by a single-
76
stimulus is Z i the evoked neural activity by the corresponding two-sided stimulus will be 2Z i assuming
non-interactivity.
(B & E) The BOLD vs. neural activity (BvZ) functions for V1-V3 inferred by the stitching procedure with
two stimulus duration. The inferred functions can be well fitted with power-law functions (i.e. straight
lines in log-log). These functions are nonlinear, with a log-log slope significantly shallower than unity (the
background gray lines).
(C & F) The exponents (γ) of the power-law fit of the BvZ functions for V1-V3. Error bars denote 95% CI. γ
estimated from V2 and V3 (γ ~ 0.5) were not significantly different, while that from V1 was artifactually
biased towards significantly higher values, due to a violation of the co-locating assumption in V1 (see
text).
Comparing BOLD amplitude and spiking activity
! The inferred BvZ function relates the BOLD response amplitude (B) to an aggregate
quantity of local neural activity (Z). We do not know the constituents of neural activity, but
since Z is a summary measure, it can be expected to correlate with other summary measure of
neural activity, as such spike rate. To test this and to provide a reality check, we used the
inferred BvZ function to estimate the neural activity Z from the BOLD amplitude data of the
single-sided conditions in the BOLD summation experiment, which by themselves were generic
contrast response measurements. The inferred neural activity in V1 for both the 6 s and 1 s
stimuli matched extremely well with the average primate V1 contrast response function
measured in terms of single-unit spiking activity by Albrecht (1995) (Figure 4.5 A). Contrary to
an earlier report relative to the same single-unit data (Heeger et al., 2000), linearly scaling our
BOLD amplitude data does not fit the single-unit data.
! Our results from the achiasmic subject are general. We considered nine sets of published
data on the contrast response function measured with fMRI BOLD in human V1, including the
data set (Boynton et al., 1999) used in Heeger et al. (2000). We inferred the neural activity using
either the average BvZ function we obtained from our 6 s and 1 s experiments (γ) or a linear
function (BvZ function with γ = 1). We matched the results from the two models to the single-
77
unit spiking data of Albrecht (1995), allowing arbitrary scaling, and compared the goodness-of-
fit quantified in terms of normalized by the degrees of freedom in the residual. The nonlinear
BvZ model provides better fit to the spiking data for all but two data sets (Figure 4.5 B and C).
The fits are not just better relatively to the linear model but good in the absolute sense (R
2
= 0.91
to 0.96).
2 4 6 8 10 12
2
4
6
8
10
12
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
Contrast
Normalized Neuronal Response
0 0.2 0.4 0.6 0.8 1
0
5
10
15
20
25
30
35
40
Linear scaling of BOLD amplitude
Neuronal activity (Z) inferred from BOLD amp.
Electrophysiological Data (V1; Albrecht, 1995)
6s 1s
Contrast
Spikes/Sec
.35
.30
.25
.20
.15
.10
.05
Z
A
B C
Zenger-Landolt & Heeger 2003
Avidan et al. 2002
Boynton et al. 1999
Brucas & Boynton 2007
Olman et al. 2004
Moradi & Heeger 2009
Park et al. 2008
Pestilli et al. 2011 (Unttended condition)
Li et al. 2007 (Unattended condition)
Linear BvZ Model (χ
2
/ν)
Nonlinear BvZ Model (χ
2
/ν)
Figure 4.5. Cross studies comparison between spiking activity and neural activity inferred from the
BvZ function
A) Neural contrast response function. The black dots are the average single-unit responses (spikes/s) as
a function of luminance contrast recorded in macaque V1 (replotted from Albrecht, 1995) and the
black line represents the best fitting Naka-Rushton function. The blue open triangle and filled triangle
78
represent the linearly scaled BOLD contrast response data with 6s and 1s stimulus duration,
respectively, measured from the achiasmic subject with single-sided stimulation. The red open circle
and filled circle represent the BvZ-inferred neural contrast response with 6s and 1s stimulus duration,
respectively, from the same data set. The BvZ inferred neural responses are good fits to the single-unit
spiking data, whereas the BOLD responses are not.
B) The comparison between single-unit spiking data of macaque V1 (black curve, replotted from A) and
the inferred neural activities from nine sets of published BOLD contrast responses data measured in
human V1. Different symbols represent different sets of data. Blue symbols (two data sets) represent
linearly scaling the BOLD reseponse provides a better fit to the single-unit data, where red symbols
(seven data set) represent the BvZ-inferred response (γ=0.5) provides the better fit.
C) Comparison of the goodness-of-fits (defined as χ
2
/ν, where ν is the degree of freedom in the residual)
between the single-unit data and the inferred neural activities from the nine BOLD data set from B,
using two models: linear or BvZ with γ=0.5. The goodness-of-fits using the nonlinear BvZ model are
uniformally good for all nine data set (mean = 1.60), whereas the linear model cannot fit at least five
of data sets (with χ
2
/ν > 4). Only two out of the nine data sets achieved numerically (but not
statistically) better fit with the linear model.
Discussion
! We found that fMRI BOLD response amplitude is proportional to roughly the square
root of local neural activity. We reached this conclusion by measuring fMRI BOLD response at
five levels of neural activity and again at two times these levels. Our critical ability to double the
neural activity in a local region on the cortex relies on the presence of two co-locating but non-
interacting populations of neurons in the low-level visual cortex of an achiasmic human subject,
which are equally-excitable, and each population subserving a distinct population receptive
field. We demonstrated co-location and equal excitability with fMRI retinotopy and localized
stimulation. We used a sensitive contrast detection task and long-duration fMRI adaptation to
demonstrate independence. We showed that the visual cortex of an achiasmic human subject
provides a versatile in vivo model for investigating the relationship between neural activity and
fMRI BOLD response.
! The relationship between the BOLD signal and the underlying neural activity is
unresolved. Several studies try to manipulate the stimuli to elicit changes in both
electrophysiological and BOLD signals or its components such as cerebral blood flow (CBF),
79
volume (CBV), and oxygen metabolism rate (CMRO
2
). The measurements of
electrophysiological and hemodynamic signals were either done simultaneously (Brinker et al.,
1999; Ngai et al., 1999; Logothetis et al., 2001; Devor et al., 2003; Sheth et al., 2004; Hoffmeyer et
al., 2007; Huttunen et al., 2008; Magri et al., 2011) or in separate experiments (Hewson-Stoate et
al., 2005; Nangini et al., 2008; Liu et al., 2010).
! Several studies suggested that BOLD signal or its component (e.g. cerebral blood flow)
was linearly correlated with neural firing rate (Smith et al., 2002) or synchronized synaptic
activity as manifested in the local field potentials (LFP) (Ngai et al., 1999; Logothetis et al., 2001;
Martindale et al., 2003), electroencephalographic (EEG) (Brinker et al., 1999; Arthurs et al., 2000),
and magnetoencephalographic (MEG) activities (Ou et al., 2009). However some of these
studies (Ngai et al., 1999; Nangini et al., 2008) appear underpowered for detecting departures
from linearity in the presence of considerable measurement noise. Insufficient range of
measurements (Logothetis et al., 2001) or insufficient sampling points within a range (Smith et
al., 2002) also limited the specificity of the conclusion. Equally critical, some of linear fits (Ngai
et al., 1999; Logothetis et al., 2001) had a significant non-zero intersect, which would require a
paradoxical non-zero change in the BOLD signal at baseline – the non-zero intersect thus
implicates that the underlying function is nonlinearr.
! A growing number of studies suggested that the relationship between BOLD signal and
neural activity is nonlinear (Devor et al., 2003; Sheth et al., 2004; Hewson-Stoate et al., 2005;
Hoffmeyer et al., 2007; de Zwart et al., 2009; Liu et al., 2010; Magri et al., 2011). Magri et al
(2011), for example, found the visually evoked fMRI BOLD responses as a function of band-
limited neural signals (LFP and MUA) is a sublinear, consistent with our finding. However,
some of these studies (Devor et al., 2003; Sheth et al., 2004; Hewson-Stoate et al., 2005) reported
80
supra-linear relationships between the neural activity and hemodynamic responses measured
with intrinsic signal optical imaging and spectroscopy. This discrepancy could be due to the
nonlinear relationships between the BOLD signal and its physiological components (CBF, CBV ,
CMRO
2
) that were measured. A biophysical model relating the BOLD response to its
constituents (Buxton et al., 1998; Davis et al., 1998; Hoge et al., 1999; Griffeth and Buxton, 2011)
would be necessary to resolve these discrepancies.
! Several studies (Huettel and McCarthy, 2000; Liu et al., 2010) use the temporal
summation to investigate the relationship between neural activity and the BOLD response. This
is a popular approach that does not involve invasive simultaneous recording of neural signal.
The results, however, can be confounded with neural adaptation effect (Huettel and McCarthy,
2000). To avoid neural adaptation, the summation stimuli must be temporally separated with
sufficient lag time, and thus precluding these studies from gauging response nonlinearity with
little or no temporal offsets.
Our approach
! Our approach is unique in that it allows linear mixing of neural activity across any time
interval, including zero lag. The method bypasses the nonlinearity between stimulus and neural
response. It also avoids having to define the constituents of neural activity. These unique
abilities of our approach stem from the unique configuration of the visual cortex in human
achiasma, that there are two equal but non-interacting populations of neurons co-locating in the
lower-level cortex region and sharing and control the same local blood supplies. Each of these
populations is associated with a distinct population receptive field. If stimulating one
population resulted in some level of neural activity, then stimulating both populations by
81
placing the identical stimuli in each of the two population receptive fields will by definition
doubles the local neural activity (because there are two active populations instead of one).
Notice that we do not know the amount of neural activity we evoked with one stimulus (e.g.
checkerboard of a particular contrast), but we know we can double it by presenting the same
stimulus at both of the population receptive fields. The fact that we can double the local neural
activity but cannot specify the amount of neural activity is not a critical handicap. Given a set of
pairwise doubling measurements, the nonparametric stitching algorithm that we developed can
be used to infer the BvZ function, which relates the BOLD response amplitude to neural activity
up to an arbitrary scaling factor. The fact we are successful in using the inferred BvZ functions
to relate a large set of BOLD contrast response measurements to an average neural (firing rate)
contrast response function demonstrates the validity of our approach.
Amplitudes vs. time courses
! BOLD response and neural activity are both time-varying functions. Formally speaking,
the BvZ function, as is constructed, relates the amplitude of BOLD response to the amplitude of
the underlying neural activity (or any quantities that are linearly related to these amplitudes),
given a fixed but arbitrary time-varying basis function for each. The similarity in BvZ functions
obtained from the 6 s and 1 s experiments suggest that the relationship between the amplitudes
is not particularly sensitive to the underlying bases. That is, even though the shapes of the time
courses for the neural activity differed substantially for the 6 s and 1 s stimulations, and the
measured time courses of the BOLD responses also differed, the relationship between the
amplitudes of these time courses stayed similar. This is an important finding because it suggest
that the underlying nonlinearity is mathematically well behaved and relatively simple. It is
conceivable that a similar model-neutral approach similar to what we have used here to infer
82
the BvZ function can be used to infer the relationship between the time courses of BOLD
response and neural activity.
Conclusion
! We showed that the unique organization of the lower-level visual cortex in human
achiasma provides a versatile in vivo model for non-invasive studies of the relationship
between neural activity and the evoked fMRI BOLD response. The presence of two non-
interacting neural populations with non-overlapping receptive fields at the same cortical
location allows for independent controls with stimuli of two sources of local neural activity. By
measuring the fMRI BOLD response to the doubling of local neural activity, we found that the
amplitude of the evoked BOLD response is proportional to approximately the square root of the
sum-total of the evoked neural activity.
Methods
Contrast detection (behavioral) experiment
! For the behavioral experiment of contrast detection, the subject detected with his right
eye a Gabor target presented at one of two locations (upper left and lower right) on a calibrated
and gamma-linearized CRT display. The display had 10 bits of luminance resolution after
gamma linearization. A task-irrelevant contrast “mask” was simultaneously displayed with the
target at a mirror symmetric location from the target, either about the vertical meridian, such
that the target and the mask would activate the same cortical location, or about the horizontal
meridian, such that the target and mask would activate two different cortical locations. The
83
target was a 45° oriented Gabor with spatial frequency of 1 cycle/° and a space constant ( 2σ )
of 1.4°. The task-irrelevant mask is an isotropic noise patch that has spatial extent and
bandwidth as the target Gabor. Two-interval-alternative-forced-choices paradigm (judging
which interval containing the target) controlled with the adaptive procedure QUEST(Watson
and Pelli, 1983) was used to measure the contrast threshold for 75% correction detection. For
each trial, the stimulus were presented for 150 ms for each interval with 500 ms between the
stimuli. Feedbacks was given after a response. A new trial began 500 ms after the subject had
responded. For each target location, the subject completed 4 blocks of 50 trials.
FMRI data acquisition and preprocessing
! The fMRI data were collected using a 3-Tesla Siemens TIM Trio scanner with a 32-
channel head coil using a T2*-weighted echo planar imaging sequence (TE/TR/flip angle = 25
ms/1 s/60°). 19 slices with 3x3x3 mm3 isotropic voxels were prescribed to be perpendicular to
the calcarine sulcus and covering the occipital pole. A high-resolution T1-weighted anatomical
data set (3D MPRAGE; 1x1x1 mm3 isotropic voxels, TE/TR/flip angle/TI = 2.98ms/2300ms/
9°/900ms) was collected in the same session before the functional runs and used to assist
prescription of the functional slices.
! Data analysis was performed using BrainVoyager QX (Brain Innovation B.V ., Maastricht,
The Netherlands) and custom-built Matlab (MathWorks, Massachusetts, USA) code. The
anatomical volume obtained in the retinotopic mapping session was segmented and inflated to
generate a model of the cortical surface. Functional volumes were preprocessed, which included
3D motion correction, linear trend removal, and high-pass (> 0.0118 Hz) filtering. The functional
images were aligned to one another and to the anatomical volume. The first 15-22 s of BOLD
84
signals (retinotopy experiment: 15 s, long-duration adaption experiment: 22 s, BOLD
summation experiments: 22 s for 6s presentation and 17 s for 1 s presentation) were discarded to
ensure equilibrium in longitudinal magnetization and subject state.
Retinotopy
! Wedges and half rings made of flickering (4 Hz) radially scaled color checker-board
patterns, presented to one eye at a time, were used to identify the retinotopic visual areas in
subject S. For polar angle mapping, a 45° wedge with an 8.5° radius rotated (jumped)
counterclockwise by 11.25° every second, so it swept the whole visual field in 32 s. For
eccentricity mapping, the half rings were presented in the subject’s right or left visual field in
alternating blocks, expanded in equal logarithmic steps from the center of display, where the
subject fixated, and took 20 seconds to reach the maximum radius of 8.5°. The fixation of the
display changed from ‘‘+’’ to ‘‘×’’ or vice versa randomly between 5 and 10 s (uniform
distribution); the subject pressed a button as soon as a change of the fixation mark was detected.
Defining regions of interest (ROIs) for the fMRI data analyses
! A common set of regions of interest (ROIs) in V1-V3 was used to analyze data from both
the adaptation and BOLD summation experiments. Stimuli used for the adaptation experiments
(Gabor patches, see next subsection), set at 100% contrast, were used to define these ROIs using
data collected from scans independent of the main experiments. The stimuli consisted of four
Gabor patches arranged in configurations resembling a forward or backward slash (Figure 4.2
C). Each scan consisted of 13 fixation-only blocks interleaved with 12 stimulus blocks. The
fixation-only blocks lasted for 16 s and the stimulus blocks lasted for 6 s. One stimulus
85
configuration was presented in each stimulus block, and configurations alternated between
blocks. During the stimulus block, the Gabor patches counter-flickered at 1Hz and alternated
their orientations (horizontal or vertical) at 0.5Hz. The subject attended the fixation mark to
perform a change-detection task at fixation.
! A general linear model (GLM), assuming a stereotypical hemodynamic was used to
identify the ROIs. The ROIs for backward- and forward-slash arrangements were defined
separately, which corresponded to the areas that responded more strongly to the ROI-defining
stimulus than blank interval (False Discovery Rate (FDR) < 0.05) and were within areas V1-V3 at
the expected eccentricity (based on retinotopy scans). Since two sets of ROIs, one for each
stimulus arrangement, were very similar (Figure 4.6 A), we define the stimulus response areas
as the conjunction of the two sets at FDR < 0.05. Since the subject S has nystagmus (horizontal
amplitude generally less than ±2.4°, peak-to-peak), we further restricted the ROIs to include
only the responsive voxles that responded to the central 2° of two outer patches, which were at
an eccentricity of 7° The restricted stimulus response areas were further partitioned into ROIs in
V1-V3 based on the subject’s retinotopy.
86
Stimulus A
0 0.5 1 1.5
0
0.5
1
1.5
0 0.5 1 1.5
0
0.5
1
1.5
0 0.5 1 1.5
0
0.5
1
1.5
V1 V2 V3
% signal change for S B
% signal change for S A
A
B
Stimulus B
Figure 4.6. Responses evoked with ROI-defining stimuli.
A) ROI-defining stimuli in the backward-slash (Stimulus A) and forward-slash (Stimulus B)
configurations and the corresponding areas of significant activation (FDR<0.05) depicted on the
flattened visual cortex of the right hemisphere of the achiasmic subject. These stimuli were used to
defined the ROIs for the adaptation experiment (Figure 4.2) and the BOLD summation experiment
(Figure 4.3). White lines delineate the boarders of low-level visual areas as determined with
retinotopic mapping.
B) BOLD response amplitudes to Stimulus A versus that to Stimulus B for a set of randomly selected
voxels in areas V1-V3. The stimuli evoked nearly identical response in each voxel, suggesting that (1)
each voxel has two pRFs and (2) the neural populations underlying each pRF are nearly identical.
Long-duration fMRI adaptation experiment
! The adapting and test stimuli consisted of four luminance-contrast defined Gabor
patches in two spatial configurations, resembling either a backward slash or a forward slash
(Figure 4.2 C). The two outer Gabor patches were at an eccentricity of 7°, with a carrier
frequency of 1 cycle/° and a space constant of 1.4°. The two inner Gabor patches were at an
87
eccentricity of 3.5°, with a carrier frequency of 2 cycle/°, and a space constant of 0.7 °. The four
Gabor patches could be oriented either horizontally or vertically.
! Across different scans, the adapting stimuli could be in one of the four possible
conditions: either in a forward- or backward-slash configuration, and oriented either
horizontally or vertically. Relative to those of the adapting stimulus, the four Gabor patches of
the test stimulus were either at the same or different (mirrored) spatial locations and of the same
or orthogonal orientation. There were 16 scans. Each scan consisted of one adapting stimulus
and 51 trials. A scan began with 20 s of pre-adaptation. Each trial started with a 5 s top-up
adaptation, followed by one of the five event types: four types of test stimulus (2 configurations
x 2 orientations) or a fixation interval. In a non-fixation trial, after 0.4 s of a blank interval after
the top-up adaptation, the test stimulus was presented for 0.3 s, followed by 0.3-s blank. In a
fixation trial, no stimulus was presented for 1 s after the top-up adaptation. The central fixation
mark was present for all trial types (including the blank trial). The fixation mark randomly
changed from ‘‘+’’ to ‘‘×’’ for about 400 ms before returning to “+” in random intervals
uniformly distributed between 3 and 7 s. The subject was instructed to attend to the fixation
mark throughout a scan and press a key as soon as he detected a change of the fixation mark.
The trial types were counterbalanced to ensure that the frequency of the immediately preceding
trial types was equal and the same for each of the 5 trial types.
! For each experimental scan, the fMRI voxel data were %-transformed and averaged
within each ROI. ROI averaged data were concatenated across scans, and the results were
deconvolved against an indicator function formed by placing a Dirac delta function at the on-set
of the stimulus interval of the same trial type while allowing for a constant per-scan offset of no
interest Motion correction parameters are also put as repressors in the analysis to capture the
88
noise introduced by the head motion. The deconvolution analysis resulted in one response time
course for each non-fixation trial type, depicting the BOLD response evoked by the test
stimulus.
BOLD summation experiment
! The BOLD summation experiment had three types of stimuli. For stimulus types A and
B, there were 4 circular patches of black-and-white checkerboards arranged in either a
backward-slash (A) or a forward-slash (B) configuration. For stimulus type A+B, the stimuli of
type A and B were presented simultaneously, resulting in 8 checkerboard patches on the display.
All patches in a stimulus were of the same contrast. We tested five (Weber) contrast levels from
0.05 to 1.0 in equal logarithmic steps. The outer patches were centered at an eccentricity of 7°
and of radius 2°. The inner patches were centered at an eccentricity of 3.5° and of radius 1°. Each
scan tested a specific contrast level. Within a scan, 19 stimulus blocks of 6 s each (or 1 s for a
separate experiment) were interleaved with 19 16-s blank blocks. For each stimulus block, a
stimulus of given type (A, B, or A+B) was presented, counter-flickering at 2 Hz. For each scan,
the stimuli were arranged in an otherwise random sequence with the constraint that each of
three stimulus types was immediately preceded by every stimulus type equally often.
Throughout a scan, the central fixation mark changed from ‘‘+’’ to ‘‘×’’ for about 400ms and
back at randomly intervals uniformly distributed between 3 s and 7 s. When the fixation was
changed to “×”, it was titled ±5° from upright. The subject was to attend the fixation mark and
report the direction of tilt as soon as the “x” appeared by pressing one of the two keys.
89
! The time course of the evoked BOLD response for each stimulus condition was inferred
using deconvolution. For the experiment with the 6 s stimulus duration, the data analysis was
essentially identical to that used for the long-duration adaptation experiment.
! For the experiment with 1 s stimulus duration, the signal-to-noise ratio (SNR) of the data
was low. To improve SNR, we used a GLM based denoising method described in Kay. et al
(2013). The method assumes that noise across the measured voxels in the brain is correlated in
the sense that it resides in a low dimensional space. The method estimates this space in terms of
its principle components from voxels that are not driven by the stimulus and enters these
components into the GLM design matrix for the experiment as regressors of no interest in order
to estimate and remove the influence of noise from each voxel. To ensure that this method does
not introduce confounds, we tested this method with data from the 6-s experiment, which are of
high SNR, and found that results were essentially the same as the ones obtained with the
generic deconvolution method (see figure 4.7).
90
Figure 4.7 Results of the BOLD summation experiment (with 6 s stimulus duration) with and without
removing the global noise component.
The experiment with 1 s stimulus duration does not have sufficient signal-to-noise ratio to yield reliable
results with conventional deconvolution analysis (using finite-impulse-response basis). The GLM
denoising method of Kay et al.,(see http://kendrickkay.net/GLMdenoise/) was used to estimate and
remove the most prominent principle components of noise that were shared across voxels. To confirm
that this denoising method did not lead to systematic bias, we applied the same denoising method to the
same experiment with 6 s stimulus duration, for which conventional deconvolution analysis is also
applicable (Figure 4.3). The red and magenta colors represent the single-sided responses and the green
color represents the double-sided stimuli’s resonses. (see also Figure 4.3). The solid line represents the
time courses estimated using conventional deconvolution analysis without adding the estimated global
noise components as regressors (see methods). The square symbols represents the time course inferred
with the global noise components as regressors of no interest. The two methods yielded virtually identical
results.
Model-neutral data stitching algorithm used to infer the BvZ
! Given a n pair of BOLD response amplitudes

B
1,i
,B
2,i
,i=1…n , corresponding to the n
(n=5 in our experiment) one-sided, two-sided measurement, we sought to estimate n-1
parameters,

1≥Z
2
≥…≥Z
n
such that the 2n points {(1,B
1,1
), (2,B
2,1
), (Z
1
,B
1,2
), (2Z
2
,B
2,2
) ,...,
(Z
n
,B
1,n
) , (2Z
n
,B
2,n
)} are maximally consistent, in the least squares sense, with a monotonic
and maximally smooth function. The multiplier 2 represents our assumption that the two-sided
condition generated twice the neural activity of the corresponding one-sided condition. To be
model-neutral, we considered the entire family of cubic spline functions with k control points
(k≥2) that are constrained to be monotonic. A cubic spline function with k control point has 2k
degrees of freedom. We iteratively estimated the n-1 values of Z
i
and the 2k parameters of the
cubic spline function by minimizing the normalized chi-square (sum of squared errors between
the 2n points and the cubic spline function, divided by the degrees of freedom of the residual).
The degrees of freedom of the residual are n+1−2k , and k has to be at least 2. In our case, n=5,
meaning that the residual would have no degree of freedom for k>2. As a result, we considered
91
only the family of cubic spline functions with two control points, situated at (1,B
1,1
) and
(2Z
5
,B
2,5
) , respectively.
The parameter estimation started with an arbitrary choice of , which was
iteratively optimized by using the fminsearch function of MATLAB (The MathWorks). In the
inner loop, we used the convenient SLM MATLAB toolbox by John D’Erro to fit a monotonic
cubic spline function to the 2n points {(1,B
1,1
), (2,B
2,1
) , (Z
1
,B
1,2
), (2Z
2
,B
2,2
) ,..., (Z
n
,B
1,n
),
(2Z
n
,B
2,n
) }. We computed normalized chi-square, which we used as the objective function for
fminsearch to choose the next set of Z
i
until convergence.
92
Chapter 5
Conclusion and Future Direction
! In my first project, We found that the strongest correlations in spontaneous activity are
between points on the cortex with functional receptive fields at the same eccentricity. We
explained this correlation pattern by using spatially diffused feedbacks model, which suggests
that the correlation of resting state doesn’t always mean the direct anatomical connection. My
second study found that the BOLD response of a cortical area is a simple sum of the responses
to all of the components and we further confirmed this linear model is a simplified
normalization model, which suggested the normalization is a canonical computation
mechanism for all the ventral visual areas. My third study is to understand the relation between
the neural response and BOLD response and we found that BOLD response amplitude is
proportional to approximately the square root of the underlying neural activity. More
important, we discovered achiasmatic subject as an ideal system to explore the relationship
between the observed BOLD signal and its underlying neural mechanism.
! Below I outline several lines of research that emerge from my current projects.
Applying SLC model on human MT (hMT+)
! In our chapter 3, our results suggest that the ventral visual areas’ BOLD responses can
be described by our simplified normalization model, SLC model. Whether this model can be
also used to describe other visual areas, such as the hMT+ in the dorsal visual areas, is still
unknown. There are several studies suggesting the normalization computation exists in hMT+
93
(Heeger et al., 1996; Simoncelli and Heeger, 1998), which might imply that SLC model also can
be used to describe hMT+.
! The whole study is composed of three experiments and moving random dot will be used
as stimuli. The whole experiment’s paradigm is like the experiment we described in chapter 3.
Face components are replaced as the dots, which are moving in 45°, scene components are
replaced as the dots, which are moving in 135°, and noise components are replaced as the dots
whose moving direction is random. Across different conditions in different experiments, the
number of dots are kept as constant, just like the whole contrast energy of the image described
in chapter 3 is kept as the same. Subjects are asked to attend 45° or 135° in different scans by
asking them to do a speed discrimination task. First question is whether MT’s response can be
explained by our SLC model or not? If the model can explain our data, the fitting coefficients
will let us to know the MT direction selectivity, and how attention modulate interact with the
direction selectivity.
Differentiate neural suppression effect versus nonlinear
BOLD effect with an Achiasmatic subject
! The response of a visual neuron to a target in its receptive field can be reduced in the
presence of surround stimuli (Hubel and Wiesel, 1968; Cavanaugh et al., 2002b). A similar
effect has been observed with BOLD fMRI (Kastner et al., 2001; Zenger-Landolt and Heeger,
2003). As with signal summation, the exact sources of BOLD signal suppression are unknown.
The system such as the low-level visual areas in the achiasmatic subject, which contains two
groups of independent and non-interacting neurons but sharing the same blood supply, offers
us an unprecedented opportunity to differentiate the observed nonlinear summation BOLD
94
effect is due to the nonlinearity of the neural responses or the nonlinearity of the neuro-vascular
coupling.
! The experiment will include two parts. One part is about behavioral measurement of
achiasmatic subject’s surround suppression effect. First, it will display when the flankers and
target are adjacent to each other in the visual space, the subject’s contrast detection threshold for
the target will be effected by the high-contrast flanker due to the surround suppression effect.
Second, it will show that when the flanker and target are in the symmetric side, subject’s
contrast detection threshold for the target will not be affected by surround suppression. The
second part is the fMRI scan, which will include 4 conditions, target-only condition, flanker-
only condition, target-flanker are in the same-side, target-flanker are in the symmetric side. If
the observed nonlinearity mainly from the neural suppression, we would expect the responses
of the third condition and the fourth condition would have larger discrepancy. However if the
observed nonlinearity mainly from the nonlinearity of the relationship between the BOLD to the
neural activity, we would expect to see the response of target would be around the same no
matter where the flanker is in the same side or in the different side. And this response should be
smaller than the linear summation of the flanker’s responses and the target’s response.
95
Reference
Aertsen AMHJ, Gerstein GL, Habib MK, Palm G (1989) Dynamics of Neuronal Firing
Correlation - Modulation of Effective Connectivity. Journal of neurophysiology
61:900-917.
Albrecht DG (1995) Visual-Cortex Neurons in Monkey and Cat - Effect of Contrast on
the Spatial and Temporal Phase-Transfer Functions. Visual Neurosci
12:1191-1210.
Albrecht DG, Hamilton DB (1982) Striate Cortex of Monkey and Cat - Contrast
Response Function. Journal of neurophysiology 48:217-237.
Arthurs OJ, Williams EJ, Carpenter TA, Pickard JD, Boniface SJ (2000) Linear coupling
between functional magnetic resonance imaging and evoked potential amplitude
in human somatosensory cortex. Neuroscience 101:803-806.
Avidan G, Harel M, Hendler T, Ben-Bashat D, Zohary E, Malach R (2002) Contrast
sensitivity in human visual areas and its relationship to object recognition.
Journal of neurophysiology 87:3102-3116.
Bandettini PA, Ungerleider LG (2001) From neuron to BOLD: new connections. Nature
neuroscience 4:864-866.
Beckmann CF, DeLuca M, Devlin JT, Smith SM (2005) Investigations into resting-state
connectivity using independent component analysis. Philos T Roy Soc B
360:1001-1013.
Belliveau JW, Kennedy DN, Jr., McKinstry RC, Buchbinder BR, Weisskoff RM, Cohen
MS, Vevea JM, Brady TJ, Rosen BR (1991) Functional mapping of the human
visual cortex by magnetic resonance imaging. Science 254:716-719.
Birn RM, Diamond JB, Smith MA, Bandettini PA (2006) Separating respiratory-
variation-related neuronal-activity-related fluctuations in fluctuations from fMRI.
Neuroimage 31:1536-1548.
Birn RM, Smith MA, Jones TB, Bandettini PA (2008) The respiration response function:
the temporal dynamics of fMRI signal fluctuations related to changes in
respiration. NeuroImage 40:644-654.
Biswal B, Yetkin FZ, Haughton VM, Hyde JS (1995) Functional connectivity in the motor
cortex of resting human brain using echo-planar MRI. Magnetic resonance in
medicine : official journal of the Society of Magnetic Resonance in Medicine /
Society of Magnetic Resonance in Medicine 34:537-541.
Biswal BB, Van Kylen J, Hyde JS (1997) Simultaneous assessment of flow and BOLD
signals in resting-state functional connectivity maps. NMR in biomedicine
10:165-170.
Blockley NP , Francis ST, Gowland PA (2009) Perturbation of the BOLD response by a
contrast agent and interpretation through a modified balloon model.
Neuroimage 48:84-93.
96
Bluhm RL, Miller J, Lanius RA, Osuch EA, Boksman K, Neufeld RWJ, Theberge J,
Schaefer B, Williamson P (2007) Spontaneous low-frequency fluctuations in the
BOLD signal in schizophrenic patients: Anomalies in the default network.
Schizophrenia Bull 33:1004-1012.
Boynton GM, Engel SA, Glover GH, Heeger DJ (1996) Linear systems analysis of
functional magnetic resonance imaging in human V1. J Neurosci 16:4207-4221.
Boynton GM, Demb JB, Glover GH, Heeger DJ (1999) Neuronal basis of contrast
discrimination. Vision Res 39:257-269.
Brinker G, Bock C, Busch E, Krep H, Hossmann KA, Hoehn-Berlage M (1999)
Simultaneous recording of evoked potentials and T*(2)-weighted MR images
during somatosensory stimulation of rat. Magnet Reson Med 41:469-473.
Buckner RL, Vincent JL (2007) Unrest at rest: Default activity and spontaneous network
correlations. Neuroimage 37:1091-1096.
Buracas GT, Boynton GM (2007) The effect of spatial attention on contrast response
functions in human visual cortex. Journal of Neuroscience 27:93-97.
Busse L, Wade AR, Carandini M (2009) Representation of Concurrent Stimuli by
Population Activity in Visual Cortex. Neuron 64:931-942.
Buxton RB, Wong EC, Frank LR (1998) Dynamics of blood flow and oxygenation
changes during brain activation: The balloon model. Magnet Reson Med
39:855-864.
Cavanaugh JR, Bair W, Movshon JA (2002a) Nature and interaction of signals from the
receptive field center and surround in macaque V1 neurons. Journal of
neurophysiology 88:2530-2546.
Cavanaugh JR, Bair W, Movshon JA (2002b) Selectivity and spatial distribution of
signals from the receptive field surround in macaque V1 neurons. Journal of
neurophysiology 88:2547-2556.
Chang C, Cunningham JP , Glover GH (2009) Influence of heart rate on the BOLD signal:
the cardiac response function. NeuroImage 44:857-869.
Cohen MS (1997) Parametric analysis of fMRI data using linear systems methods.
Neuroimage 6:93-103.
Cordes D, Haughton VM, Arfanakis K, Wendt GJ, Turski PA, Moritz CH, Quigley MA,
Meyerand ME (2000) Mapping functionally related regions of brain with
functional connectivity MR imaging. Am J Neuroradiol 21:1636-1644.
Dale AM, Buckner RL (1997) Selective averaging of rapidly presented individual trials
using fMRI. Human brain mapping 5:329-340.
Damoiseaux JS, Rombouts SA, Barkhof F, Scheltens P , Stam CJ, Smith SM, Beckmann CF
(2006) Consistent resting-state networks across healthy subjects. P Natl Acad Sci
USA 103:13848-13853.
97
Davies-Thompson J, Scheel M, Lanyon LJ, Barton JJS (2013) Functional organisation of
visual pathways in a patient with no optic chiasm. Neuropsychologia
51:1260-1272.
Davis TL, Kwong KK, Weisskoff RM, Rosen BR (1998) Calibrated functional MRI:
mapping the dynamics of oxidative metabolism. P Natl Acad Sci USA
95:1834-1839.
de Zwart JA, van Gelderen P , Jansma JM, Fukunaga M, Bianciardi M, Duyn JH (2009)
Hemodynamic nonlinearities affect BOLD fMRI response timing and amplitude.
Neuroimage 47:1649-1658.
Devor A, Dunn AK, Andermann ML, Ulbert I, Boas DA, Dale AM (2003) Coupling of
total hemoglobin concentration, oxygenation, and neural activity in rat
somatosensory cortex. Neuron 39:353-359.
DeYoe EA, Carman GJ, Bandettini P , Glickman S, Wieser J, Cox R, Miller D, Neitz J
(1996) Mapping striate and extrastriate visual areas in human cerebral cortex.
Proceedings of the National Academy of Sciences of the United States of America
93:2382-2386.
Dougherty RF, Koch VM, Brewer AA, Fischer B, Modersitzki J, Wandell BA (2003)
Visual field representations and locations of visual areas V1/2/3 in human visual
cortex. J Vision 3:586-598.
Downing PE, Jiang Y, Shuman M, Kanwisher N (2001) A cortical area selective for visual
processing of the human body. Science 293:2470-2473.
Duncan RO, Boynton GM (2003) Cortical magnification within human primary visual
cortex correlates with acuity thresholds. Neuron 38:659-671.
Engel AK, Kreiter AK, Konig P , Singer W (1991) Synchronization of Oscillatory
Neuronal Responses between Striate and Extrastriate Visual Cortical Areas of the
Cat. P Natl Acad Sci USA 88:6048-6052.
Engel SA, Glover GH, Wandell BA (1997) Retinotopic organization in human visual
cortex and the spatial precision of functional MRI. Cerebral Cortex 7:181-192.
Engel SA, Rumelhart DE, Wandell BA, Lee AT, Glover GH, Chichilnisky EJ, Shadlen
MN (1994) Fmri of Human Visual-Cortex. Nature 369:525-525.
Epstein R, Kanwisher N (1998) A cortical representation of the local visual environment.
Nature 392:598-601.
Fang F, Murray SO, He S (2007) Duration-dependent fMRI adaptation and distributed
viewer-centered face representation in human visual cortex. Cerebral Cortex
17:1402-1411.
Fang F, Murray SO, Kersten D, He S (2005) Orientation-tuned fMRI adaptation in
human visual cortex. Journal of neurophysiology 94:4188-4195.
Felleman DJ, Van Essen DC (1991) Distributed Hierarchical Processing in the Primate
Cerebral Cortex. Cerebral Cortex 1:1-47.
98
Feng CM, Liu HL, Fox PT, Gao JH (2001) Comparison of the experimental BOLD signal
change in event-related fMRI with the balloon model. NMR in biomedicine
14:397-401.
Fox MD, Raichle ME (2007) Spontaneous fluctuations in brain activity observed with
functional magnetic resonance imaging. Nat Rev Neurosci 8:700-711.
Fox MD, Snyder AZ, Vincent JL, Raichle ME (2007) Intrinsic fluctuations within cortical
systems account for intertrial variability in human Behavior. Neuron 56:171-184.
Fox MD, Snyder AZ, Vincent JL, Corbetta M, Van Essen DC, Raichle ME (2005) The
human brain is intrinsically organized into dynamic, anticorrelated functional
networks. Proceedings of the National Academy of Sciences of the United States
of America 102:9673-9678.
Fransson P (2005) Spontaneous low-frequency BOLD signal fluctuations: An fMRI
investigation of the resting-state default mode of brain function hypothesis.
Human brain mapping 26:15-29.
Freeman TCB, Durand S, Kiper DC, Carandini M (2002) Suppression without inhibition
in visual cortex. Neuron 35:759-771.
Friston KJ, Frith CD, Liddle PF, Frackowiak RSJ (1993) Functional Connectivity - the
Principal-Component Analysis of Large (Pet) Data Sets. J Cerebr Blood F Met
13:5-14.
Friston KJ, Mechelli A, Turner R, Price CJ (2000) Nonlinear responses in fMRI: The
balloon model, volterra kernels, and other hemodynamics. Neuroimage
12:466-477.
Gilbert CD, Wiesel TN (1983) Clustered Intrinsic Connections in Cat Visual-Cortex.
Journal of Neuroscience 3:1116-1133.
Goodale MA, Milner AD (1992) Separate Visual Pathways for Perception and Action.
Trends Neurosci 15:20-25.
Greicius MD, Srivastava G, Reiss AL, Menon V (2004) Default-mode network activity
distinguishes Alzheimer's disease from healthy aging: Evidence from functional
MRI. Proceedings of the National Academy of Sciences of the United States of
America 101:4637-4642.
Griffeth VEM, Buxton RB (2011) A theoretical framework for estimating cerebral oxygen
metabolism changes using the calibrated-BOLD method: Modeling the effects of
blood volume distribution, hematocrit, oxygen extraction fraction, and tissue
signal properties on the BOLD signal. Neuroimage 58:198-212.
Grill-Spector K, Malach R (2001) fMR-adaptation: a tool for studying the functional
properties of human cortical neurons. Acta psychologica 107:293-321.
Grill-Spector K, Malach R (2004) The human visual cortex. Annu Rev Neurosci
27:649-677.
99
Grill-Spector K, Kushnir T, Hendler T, Edelman S, Itzchak Y, Malach R (1998) A
sequence of object-processing stages revealed by fMRI in the human occipital
lobe. Human brain mapping 6:316-328.
Grossman ED, Blake R (2002) Brain Areas Active during Visual Perception of Biological
Motion. Neuron 35:1167-1175.
Hagmann P , Cammoun L, Gigandet X, Meuli R, Honey CJ, Wedeen V , Sporns O (2008)
Mapping the structural core of human cerebral cortex. Plos Biol 6:1479-1493.
Hampson M, Peterson BS, Skudlarski P , Gatenby JC, Gore JC (2002) Detection of
functional connectivity using temporal correlations in MR images. Human brain
mapping 15:247-262.
Hasson U, Levy I, Behrmann M, Hendler T, Malach R (2002) Eccentricity bias as an
organizing principle for human high-order object areas. Neuron 34:479-490.
Haxby JV , Grady CL, Horwitz B, Ungerleider LG, Mishkin M, Carson RE, Herscovitch P ,
Schapiro MB, Rapoport SI (1991) Dissociation of Object and Spatial Visual
Processing Pathways in Human Extrastriate Cortex. Proceedings of the National
Academy of Sciences of the United States of America 88:1621-1625.
Heckman GM, Bouvier SE, Carr VA, Harley EM, Cardinal KS, Engel SA (2007)
Nonlinearities in rapid event-related fMRI explained by stimulus scaling.
Neuroimage 34:651-660.
Heeger DJ, Simoncelli EP , Movshon JA (1996) Computational models of cortical visual
processing. Proceedings of the National Academy of Sciences of the United States
of America 93:623-627.
Heeger DJ, Huk AC, Geisler WS, Albrecht DG (2000) Spikes versus BOLD: what does
neuroimaging tell us about neuronal activity? Nat Neurosci 3:631-633.
Herrmann K, Heeger DJ, Carrasco M (2012) Feature-based attention enhances
performance by increasing response gain. Vision Res 74:10-20.
Hewson-Stoate N, Jones M, Martindale J, Berwick J, Mayhew J (2005) Further
nonlinearities in neurovascular coupling in rodent barrel cortex. Neuroimage
24:565-574.
Hoffmann MB, Kaule FR, Levin N, Masuda Y, Kumar A, Gottlob I, Horiguchi H,
Dougherty RF, Stadler J, Wolynski B, Speck O, Kanowski M, Liao YJ, Wandell BA,
Dumoulin SO (2012) Plasticity and Stability of the Visual System in Human
Achiasma. Neuron 75:393-401.
Hoffmeyer HW, Enager P , Thomsen KJ, Lauritzen MJ (2007) Nonlinear neurovascular
coupling in rat sensory cortex by activation of transcallosal fibers. J Cerebr Blood
F Met 27:575-587.
Hoge RD, Atkinson J, Gill B, Crelier GR, Marrett S, Pike GB (1999) Investigation of
BOLD signal dependence on cerebral blood flow and oxygen consumption: The
deoxyhemoglobin dilution model. Magnet Reson Med 42:849-863.
100
Honey CJ, Kotter R, Breakspear M, Sporns O (2007) Network structure of cerebral cortex
shapes functional connectivity on multiple time scales. P Natl Acad Sci USA
104:10240-10245.
Horner AJ, Andrews TJ (2009) Linearity of the fMRI Response in Category-Selective
Regions of Human Visual Cortex. Human brain mapping 30:2628-2640.
Hubel DH, Wiesel TN (1968) Receptive fields and functional architecture of monkey
striate cortex. The Journal of physiology 195:215-243.
Huettel SA, McCarthy G (2000) Evidence for a refractory period in the hemodynamic
response to visual stimuli as measured by MRI. Neuroimage 11:547-553.
Huk AC, Heeger DJ (2002) Pattern-motion responses in human visual cortex. Nature
neuroscience 5:72-75.
Hupe JM, James AC, Payne BR, Lomber SG, Girard P , Bullier J (1998) Cortical feedback
improves discrimination between figure and background by V1, V2 and V3
neurons. Nature 394:784-787.
Huttunen JK, Grohn O, Penttonen M (2008) Coupling between simultaneously recorded
BOLD response and neuronal activity in the rat somatosensory cortex.
Neuroimage 39:775-785.
Joshi AA, Shattuck DW, Thompson PM, Leahy RM (2007) Surface-constrained
volumetric brain registration using harmonic mappings. Ieee T Med Imaging
26:1657-1669.
Kanwisher N, McDermott J, Chun MM (1997) The fusiform face area: a module in
human extrastriate cortex specialized for face perception. The Journal of
neuroscience : the official journal of the Society for Neuroscience 17:4302-4311.
Kastner S, De Weerd P , Pinsk MA, Elizondo MI, Desimone R, Ungerleider LG (2001)
Modulation of sensory suppression: implications for receptive field sizes in the
human visual cortex. Journal of neurophysiology 86:1398-1411.
Kennedy H, Bullier J (1985) A Double-Labeling Investigation of the Afferent
Connectivity to Cortical Areas V1 and V2 of the Macaque Monkey. Journal of
Neuroscience 5:2815-2830.
Kobatake E, Tanaka K (1994) Neuronal Selectivities to Complex Object Features in the
Ventral Visual Pathway of the Macaque Cerebral-Cortex. Journal of
neurophysiology 71:856-867.
Koch MA, Norris DG, Hund-Georgiadis M (2002) An investigation of functional and
anatomical connectivity using magnetic resonance imaging. Neuroimage
16:241-250.
Konkle T, Oliva A (2012) A Real-World Size Organization of Object Responses in
Occipitotemporal Cortex. Neuron 74:1114-1124.
Kourtzi Z, Huberle E (2005) Spatiotemporal characteristics of form analysis in the
human visual cortex revealed by rapid event-related fMRI adaptation.
Neuroimage 28:440-452.
101
Kriegeskorte N, Mur M, Ruff DA, Kiani R, Bodurka J, Esteky H, Tanaka K, Bandettini
PA (2008) Matching Categorical Object Representations in Inferior Temporal
Cortex of Man and Monkey. Neuron 60:1126-1141.
Kwong KK (1995) Functional Magnetic-Resonance-Imaging with Echo-Planar Imaging.
Magn Reson Quart 11:1-20.
Kwong KK, Belliveau JW, Chesler DA, Goldberg IE, Weisskoff RM, Poncelet BP ,
Kennedy DN, Hoppel BE, Cohen MS, Turner R, Cheng HM, Brady TJ, Rosen BR
(1992) Dynamic Magnetic-Resonance-Imaging of Human Brain Activity during
Primary Sensory Stimulation. Proceedings of the National Academy of Sciences
of the United States of America 89:5675-5679.
Larsson J, Smith AT (2012) fMRI repetition suppression: neuronal adaptation or
stimulus expectation? Cereb Cortex 22:567-576.
Legge GE, Foley JM (1980) Contrast Masking in Human-Vision. J Opt Soc Am
70:1458-1471.
Legge GE, Kersten D, Burgess AE (1987) Contrast Discrimination in Noise. J Opt Soc
Am A 4:391-404.
Leopold DA, Murayama Y, Logothetis NK (2003) Very slow activity fluctuations in
monkey visual cortex: Implications for functional brain imaging. Cerebral Cortex
13:422-433.
Lerner Y, Hendler T, Ben-Bashat D, Harel M, Malach R (2001) A hierarchical axis of
object processing stages in the human visual cortex. Cereb Cortex 11:287-297.
Levitt JB, Kiper DC, Movshon JA (1994) Receptive-Fields and Functional Architecture of
Macaque V2. Journal of neurophysiology 71:2517-2542.
Levy I, Hasson U, Harel M, Malach R (2004) Functional analysis of the periphery effect
in human building-related areas. Human brain mapping 22:15-26.
Li X, Lu ZL, Tjan BS, Dosher BA, Chu W (2008) Blood oxygenation level-dependent
contrast response functions identify mechanisms of covert attention in early
visual areas. P Natl Acad Sci USA 105:6202-6207.
Liu Y, Liang M, Zhou Y, He Y, Hao YH, Song M, Yu CS, Liu HH, Liu ZN, Jiang TZ (2008)
Disrupted small-world networks in schizophrenia. Brain 131:945-961.
Liu ZM, Rios C, Zhang NY, Yang L, Chen W, He B (2010) Linear and nonlinear
relationships between visual stimuli, EEG and BOLD fMRI signals. Neuroimage
50:1054-1066.
Logothetis NK (2003) The underpinnings of the BOLD functional magnetic resonance
imaging signal. The Journal of neuroscience : the official journal of the Society for
Neuroscience 23:3963-3971.
Logothetis NK (2008) What we can do and what we cannot do with fMRI. Nature
453:869-878.
Logothetis NK, Wandell BA (2004) Interpreting the BOLD signal. Annual review of
physiology 66:735-769.
102
Logothetis NK, Pauls J, Augath M, Trinath T, Oeltermann A (2001) Neurophysiological
investigation of the basis of the fMRI signal. Nature 412:150-157.
Lowe MJ, Dzemidzic M, Lurito JT, Mathews VP , Phillips MD (2000a) Correlations in
low-frequency BOLD fluctuations reflect cortico-cortical connections.
Neuroimage 12:582-587.
Lowe MJ, Dzemidzic M, Lurito JT, Mathews VP , Phillips MD (2000b) Correlations in
low-frequency BOLD fluctuations reflect cortico-cortical connections.
Neuroimage 12:582-587.
Lu ZL, Dosher BA (1999) Characterizing human perceptual inefficiencies with
equivalent internal noise. J Opt Soc Am A 16:764-778.
MacEvoy SP , Epstein RA (2009) Decoding the Representation of Multiple Simultaneous
Objects in Human Occipitotemporal Cortex. Current Biology 19:943-947.
Magri C, Logothetis NK, Panzeri S (2011) Investigating static nonlinearities in
neurovascular coupling. Magn Reson Imaging 29:1358-1364.
Mandeville JB, Marota JJ, Ayata C, Moskowitz MA, Weisskoff RM, Rosen BR (1999) MRI
measurement of the temporal evolution of relative CMRO(2) during rat forepaw
stimulation. Magnetic resonance in medicine : official journal of the Society of
Magnetic Resonance in Medicine / Society of Magnetic Resonance in Medicine
42:944-951.
Martindale J, Mayhew J, Berwick J, Jones M, Martin C, Johnston D, Redgrave P , Zheng
Y (2003) The hemodynamic impulse response to a single neural event. J Cerebr
Blood F Met 23:546-555.
Maunsell JH (1995) The brain's visual world: representation of visual targets in cerebral
cortex. Science 270:764-769.
Mignard M, Malpeli JG (1991) Paths of Information-Flow through Visual-Cortex.
Science 251:1249-1251.
Moradi F, Heeger DJ (2009) Inter-ocular contrast normalization in human visual cortex. J
Vision 9.
Moran J, Desimone R (1985) Selective Attention Gates Visual Processing in the
Extrastriate Cortex. Science 229:782-784.
Mukamel R, Gelbard H, Arieli A, Hasson U, Fried I, Malach R (2005) Coupling between
neuronal firing, field potentials, and fMR1 in human auditory cortex. Science
309:951-954.
Nangini C, Tam F, Graham SJ (2008) A novel method for integrating MEG and BOLD
fMRI signals with the linear convolution model in human primary
somatosensory cortex. Hum Brain Mapp 29:97-106.
Ngai AC, Jolley MA, D'Ambrosio R, Meno JR, Winn HR (1999) Frequency-dependent
changes in cerebral blood flow and evoked potentials during somatosensory
stimulation in the rat. Brain Res 837:221-228.
103
Nieto-Castanon A, Fedorenko E (2012) Subject-specific functional localizers increase
sensitivity and functional resolution of multi-subject analyses. Neuroimage
63:1646-1669.
Nir Y, Mukamel R, Dinstein I, Privman E, Harel M, Fisch L, Gelbard-Sagiv H,
Kipervasser S, Andelman F, Neufeld MY, Kramer U, Arieli A, Fried I, Malach R
(2008) Interhemispheric correlations of slow spontaneous neuronal fluctuations
revealed in human sensory cortex. Nature neuroscience 11:1100-1108.
O'Craven KM, Downing PE, Kanwisher N (1999) fMRI evidence for objects as the units
of attentional selection. Nature 401:584-587.
Ogawa S, Lee TM, Kay AR, Tank DW (1990a) Brain magnetic resonance imaging with
contrast dependent on blood oxygenation. P Natl Acad Sci USA 87:9868-9872.
Ogawa S, Lee TM, Nayak AS, Glynn P (1990b) Oxygenation-sensitive contrast in
magnetic resonance image of rodent brain at high magnetic fields. Magnetic
resonance in medicine : official journal of the Society of Magnetic Resonance in
Medicine / Society of Magnetic Resonance in Medicine 14:68-78.
Olman CA, Ugurbil K, Schrater P , Kersten D (2004) BOLD fMRI and psychophysical
measurements of contrast response to broadband images. Vision Res 44:669-683.
Ou WM, Nissila I, Radhakrishnan H, Boas DA, Hamalainen MS, Franceschini MA
(2009) Study of neurovascular coupling in humans via simultaneous
magnetoencephalography and diffuse optical imaging acquisition. Neuroimage
46:624-632.
Pantazis D, Joshi A, Jiang JT, Shattuck DW, Bernstein LE, Damasio H, Leahy RM (2010)
Comparison of landmark-based and automatic methods for cortical surface
registration. Neuroimage 49:2479-2493.
Park JC, Zhang X, Ferrera J, Hirsch J, Hood DC (2008) Comparison of contrast-response
functions from multifocal visual-evoked potentials (mfVEPs) and functional MRI
responses. J Vision 8.
Pestilli F, Carrasco M, Heeger DJ, Gardner JL (2011) Attentional Enhancement via
Selection and Pooling of Early Sensory Responses in Human Visual Cortex.
Neuron 72:832-846.
Raichle ME, Mintun MA (2006) Brain work and brain imaging. Annu Rev Neurosci
29:449-476.
Raichle ME, MacLeod AM, Snyder AZ, Powers WJ, Gusnard DA, Shulman GL (2001) A
default mode of brain function. Proceedings of the National Academy of Sciences
of the United States of America 98:676-682.
Reynolds JH, Heeger DJ (2009) The Normalization Model of Attention. Neuron
61:168-185.
Reynolds JH, Chelazzi L, Desimone R (1999) Competitive mechanisms subserve
attention in macaque areas V2 and V4. The Journal of neuroscience : the official
journal of the Society for Neuroscience 19:1736-1753.
104
Reynolds JH, Pasternak T, Desimone R (2000) Attention increases sensitivity of V4
neurons. Neuron 26:703-714.
Riesenhuber M, Poggio T (1999) Hierarchical models of object recognition in cortex.
Nature neuroscience 2:1019-1025.
Rolls ET, Tovee MJ (1995) The responses of single neurons in the temporal visual cortical
areas of the macaque when more than one stimulus is present in the receptive
field. Experimental brain research Experimentelle Hirnforschung
Experimentation cerebrale 103:409-420.
Rolls ET, Aggelopoulos NC, Zheng F (2003) The receptive fields of inferior temporal
cortex neurons in natural scenes. The Journal of neuroscience : the official journal
of the Society for Neuroscience 23:339-348.
Rombouts SARB, Barkhof F, Goekoop R, Stam CJ, Scheltens P (2005) Altered resting
state networks in mild cognitive impairment and mild Alzheimer's disease: An
fMRI study. Human brain mapping 26:231-239.
Rombouts SARB, Damoiseaux JS, Goekoop R, Barkhof F, Scheltens P , Smith SM,
Beckmann CF (2009) Model-Free Group Analysis Shows Altered BOLD FMRI
Networks in Dementia. Human brain mapping 30:256-266.
Saenz M, Buracas GT, Boynton GM (2002) Global effects of feature-based attention in
human visual cortex. Nature neuroscience 5:631-632.
Saenz M, Buracas GT, Boynton GM (2003) Global feature-based attention for motion and
color. Vision Res 43:629-637.
Salvador R, Suckling J, Coleman MR, Pickard JD, Menon D, Bullmore E (2005)
Neurophysiological architecture of functional magnetic resonance images of
human brain. Cerebral Cortex 15:1332-1342.
Schwartz E (1985) On the Mathematical Structure of the Visuotopic Mapping of
Macaque Striate Cortex. Science 227:1065-1066.
Schwarz C, Bolz J (1991) Functional Specificity of a Long-Range Horizontal Connection
in Cat Visual-Cortex - a Cross-Correlation Study. Journal of Neuroscience
11:2995-3007.
Serences JT, Boynton GM (2007) Feature-based attentional modulations in the absence of
direct visual stimulation. Neuron 55:301-312.
Serences JT, Schwarzbach J, Courtney SM, Golay X, Yantis S (2004) Control of object-
based attention in human cortex. Cereb Cortex 14:1346-1357.
Sheinberg DL, Logothetis NK (2001) Noticing familiar objects in real world scenes: the
role of temporal cortical neurons in natural vision. The Journal of neuroscience :
the official journal of the Society for Neuroscience 21:1340-1350.
Sheth SA, Nemoto M, Guiou M, Walker M, Pouratian N, Toga AW (2004) Linear and
nonlinear relationships between neuronal activity, oxygen metabolism, and
hemodynamic responses. Neuron 42:347-355.
105
Shmuel A, Leopold DA (2008) Neuronal correlates of spontaneous fluctuations in fMRI
signals in monkey visual cortex: Implications for functional connectivity at rest.
Human brain mapping 29:751-761.
Shmuel A, Yacoub E, Pfeuffer J, Van de Moortele PF, Adriany G, Hu XP , Ugurbil K
(2002) Sustained negative BOLD, blood flow and oxygen consumption response
and its coupling to the positive response in the human brain. Neuron
36:1195-1210.
Shmueli K, van Gelderen P , de Zwart JA, Horovitz SG, Fukunaga M, Jansma JM, Duyn
JH (2007) Low-frequency fluctuations in the cardiac rate as a source of variance
in the resting-state fMRI BOLD signal. Neuroimage 38:306-320.
Shulman RG, Hyder F, Rothman DL (2001) Cerebral energetics anti the glycogen shunt:
Neurochemical basis of functional imaging. Proceedings of the National
Academy of Sciences of the United States of America 98:6417-6422.
Shulman RG, Rothman DL, Behar KL, Hyder F (2004) Energetic basis of brain activity:
implications for neuroimaging. Trends Neurosci 27:489-495.
Sibson NR, Dhankhar A, Mason GF, Behar KL, Rothman DL, Shulman RG (1997) In vivo
C-13 NMR measurements of cerebral glutamine synthesis as evidence for
glutamate-glutamine cycling. Proceedings of the National Academy of Sciences
of the United States of America 94:2699-2704.
Sibson NR, Dhankhar A, Mason GF, Rothman DL, Behar KL, Shulman RG (1998)
Stoichiometric coupling of brain glucose metabolism and glutamatergic neuronal
activity. Proceedings of the National Academy of Sciences of the United States of
America 95:316-321.
Simoncelli EP , Heeger DJ (1998) A model of neuronal responses in visual area MT.
Vision Res 38:743-761.
Smith AJ, Blumenfeld H, Behar KL, Rothman DL, Shulman RG, Hyder F (2002) Cerebral
energetics and spiking frequency: The neurophysiological basis of fMRI. P Natl
Acad Sci USA 99:10765-10770.
Stephan KE, Weiskopf N, Drysdale PM, Robinson PA, Friston KJ (2007) Comparing
hemodynamic models with DCM. Neuroimage 38:387-401.
Tjan BS, Lestou V , Kourtzi Z (2006) Uncertainty and invariance in the human visual
cortex. Journal of neurophysiology 96:1556-1568.
Tootell RBH, Tsao D, Vanduffel W (2003) Neuroimaging weighs in: Humans meet
macaques in "primate" visual cortex. Journal of Neuroscience 23:3981-3989.
Tootell RBH, Reppas JB, Kwong KK, Malach R, Born RT, Brady TJ, Rosen BR, Belliveau
JW (1995) Functional-Analysis of Human Mt and Related Visual Cortical Areas
Using Magnetic-Resonance-Imaging. Journal of Neuroscience 15:3215-3230.
Toronov V , Walker S, Gupta R, Choi JH, Gratton E, Hueber D, Webb A (2003) The roles
of changes in deoxyhemoglobin concentration and regional cerebral blood
volume in the fMRI BOLD signal. Neuroimage 19:1521-1531.
106
van den Heuvel MP , Pol HEH (2010) Specific Somatotopic Organization of Functional
Connections of the Primary Motor Network During Resting State. Human brain
mapping 31:631-644.
Victor JD, Apkarian P , Hirsch J, Conte MM, Packard M, Relkin NR, Kim KHS, Shapley
RM (2000) Visual function and brain organization in non-decussating retinal-
fugal fibre syndrome. Cerebral Cortex 10:2-22.
Vincent JL, Patel GH, Fox MD, Snyder AZ, Baker JT, Van Essen DC, Zempel JM, Snyder
LH, Corbetta M, Raichle ME (2007) Intrinsic functional architecture in the
anaesthetized monkey brain. Nature 447:83-U84.
Wandell BA, Dumoulin SO, Brewer AA (2007) Visual field maps in human cortex.
Neuron 56:366-383.
Watson AB, Pelli DG (1983) Quest - a Bayesian Adaptive Psychometric Method. Percept
Psychophys 33:113-120.
Williams MA, Dang S, Kanwisher NG (2007) Only some spatial patterns of fMRI
response are read out in task performance. Nature neuroscience 10:685-686.
Williams RW, Hogan D, Garraghty PE (1994) Target Recognition and Visual Maps in the
Thalamus of Achiasmatic Dogs. Nature 367:637-639.
Zenger-Landolt B, Heeger DJ (2003) Response suppression in v1 agrees with
psychophysics of surround masking. The Journal of neuroscience : the official
journal of the Society for Neuroscience 23:6884-6893.
Zoccolan D, Cox DD, DiCarlo JJ (2005) Multiple object response normalization in
monkey inferotemporal cortex. Journal of Neuroscience 25:8150-8164.
107

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Functional magnetic resonance imaging characterization of peripheral form vision

PDF

Computational models and model-based fMRI studies in motor learning

PDF

Characterization of visual cortex function in late-blind individuals with retinitis pigmentosa and Argus II patients

PDF

Selectivity for visual speech in posterior temporal cortex

PDF

Explicit encoding of spatial relations in the human visual system: evidence from functional neuroimaging

PDF

The neural correlates of creativity and perceptual pleasure: from simple shapes to humor

PDF

Sensitivity and dynamic range of rod pathways in the mammalian retina

PDF

Perceptual and computational mechanisms of feature-based attention

PDF

Spatial anomalies of visual processing in retinitis pigmentosa and a potential treatment

PDF

Cortical and subcortical responses to electrical stimulation of rat retina

PDF

Functional properties of the superficial cortical interneurons

PDF

Mode of visual perceptual learning: augmented Hebbian learning explains the function of feedback and beyond

PDF

The neural correlates of face recognition

PDF

Modeling the development of mid-level visual cortex

PDF

The brain's virtuous cycle: an investigation of gratitude and good human conduct

PDF

Dynamic functional magnetic resonance imaging to localize activated neurons

PDF

Pattern detection in medical imaging: pathology specific imaging contrast, features, and statistical models

PDF

Value-based decision-making in complex choice: brain regions involved and implications of age

PDF

Contextual modulation of sensory processing via the pulvinar nucleus

PDF

Individual differences in heart rate response and expressive behavior during social emotions: effects of resting cardiac vagal tone and culture, and relation to the default mode network

Asset Metadata

Creator Bao, Pinglei (author)

Core Title Functional models of fMRI BOLD signal in the visual cortex

School College of Letters, Arts and Sciences

Degree Doctor of Philosophy

Degree Program Neuroscience

Publication Date 04/22/2014

Defense Date 12/06/2013

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag fMRI,Models,OAI-PMH Harvest,visual cortex

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Tjan, Bosco S. (committee chair), Grzywacz, Norberto M. (committee member), Hirsch, Judith A. (committee member)

Creator Email pbao@usc.edu,plbao.usc@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-382903

Unique identifier UC11295354

Identifier etd-BaoPinglei-2396.pdf (filename),usctheses-c3-382903 (legacy record id)

Legacy Identifier etd-BaoPinglei-2396.pdf

Dmrecord 382903

Document Type Dissertation

Format application/pdf (imt)

Rights Bao, Pinglei

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA