University of Southern California Dissertations and Theses

Encoding of natural images by retinal ganglion cells

(USC Thesis Other)

Encoding of natural images by retinal ganglion cells

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

ENCODING OF NATURAL IMAGES BY RETINAL GANGLION CELLS

by
Xiwu Cao

A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(BIOMEDICAL ENGINEERING)

May 2010

Copyright 2010 Xiwu Cao

ii

Dedication
to my great parents for their great love, to my beautiful wife for her joyful presence
and support, and to my lovely daughter and son for the great fun they bring me.

iii

Acknowledgements
I would like to express my deep thanks to my advisor, Norberto M. Grzywacz. His
mentorship and guidance greatly expanded and enhanced my professional knowledge.
More than that, his wisdom and spirits rooted in his heart profoundly inspire me; his
passion, enthusiasm and happiness in the research intensely motivate me; his efforts,
expertise, patience, and dedication throughout my whole Ph.D study provided a model of
being the best advisor and mentor. He encouraged and supported me in every step of my
study, moving me forward and making me with increasingly better insight. His influence
will always shine on me in my future life.
I am also highly indebted to my co-advisor, David K. Merwine, for the enormous
amount of time and efforts he has devoted to me. Every step I have made could not be
achieved without his help and assistance, from successfully completing my first
experiment to writing my first paper.
I am especially grateful to my other three committee members, Bartlett W. Mel,
Judith A. Hirsch, and James D. Weiland, for their invaluable expert suggestions and
sound advices on my studies, as well as the insightful and enjoyable interactions during
my Ph.D studies.
My special gratitude also goes to those diligent and intelligent fellows in the Visual
Processing Laboratory, namely, Eun-Jin Lee, Mónica Padilla, Susmita Chatterjee,
Junkwan Lee, Joaquín Rapela, Jeff Wurfel, Nadav Ivzan, Arvind Iyer, Yerina Ji and
Denise Steiner, for their substantive assistance and so many happy moments we shared.

iv

I would like to gratefully acknowledge the tutoring of those outstanding instructors
I was fortunate to meet at USC, including but not limited to Martha J. Townsend, Bart
Kosko, Gandhi Puvvada, and James Polk, for their exemplification of excellence in
teaching.
Finally, I want to express my thanks to all those other instructors, fellows, and
friends who have always stayed with me and supported me with their kindness,
generosity, and consideration.

v

Table of Contents
Dedication ii
Acknowledgements iii
List of Figures vii
List of Tables ix
Abstract x
Chapter 1: Introduction 1
1.1 Temporal and spatial characteristics of natural images 1
1.2 Response properties of retinal ganglion cells 2
1.3 Problems and our contributions 3
Chapter 2: Methods 6
2.1 Physiological preparation 6
2.2 Electrophysiological recording 7
2.3 Visual stimulation 7
2.4 Statistics of natural images 10
2.5 Contrast-response-function estimation 11
2.6 Receptive-field-center estimation 11
2.6.1 RF estimate from moving a square-wave grating in four
directions 11
2.6.2 RF estimate from natural images based on spike-triggered
average (STA) 14
2.6.3 RF estimate from natural images based on cross-correlation
between responses and putative RFs 15
2.7 Receptive-field-surround estimation 19
2.8 Spike-triggered contrast histogram (STCH) 20
2.9 Mixture-of-exponentials model 21
2.10 Contrast gain control, slow and fast contrast adaptation 23
Chapter 3: Estimation of the Receptive-Field Center and Surround 25
3.1 Polarity of RF with natural-image stimulation 25
3.2 Estimation of RF-center diameter with natural-image stimulation 27
3.3 Estimation of RF-surround diameter with natural-image stimulation 31
Chapter 4: The Center and Surround Responses to Natural Images 34
4.1 Strong center excitation with natural stimuli 34
4.2 Weak surround inhibition with natural stimuli 39
4.3 Division-like and subtractive surround inhibition 41

vi

Chapter 5: Asymmetric Responses to the Onset and Offset of Natural Images 46
5.1 Asymmetric responses to the onset and offset of natural images 46
5.2 Response asymmetries are due to the asymmetry of intensity
distribution in natural images 51
Chapter 6: Dependence of RGC’s Responses on the Local Visual Textures of
Natural Images 56
Chapter 7: Nonlinear-Linear Model 60
Chapter 8: Adaptation of RGCs to Natural-Image Contrast 66
8.1 Multiple temporal components in RGC’s adaptation to natural-image
contrast 66
8.2 Diversity of RGC’s contrast adaptation to natural stimuli 73
Chapter 9: Dependence of RGC’s Contrast Adaptation on the Contrast
Difference of Natural Images 79
9.1 Dependence of RGC’s responses on the contrast difference of natural
stimuli 79
9.2 Dependence of RGC’s contrast adaptation on the contrast difference
of natural stimuli 83
Chapter 10: Conclusions 86
10.1 Estimation of RF with natural-image stimulation 86
10.2 Strong center excitation and weak surround inhibition with natural-
image stimulation 87
10.3 Subtractive and division-like surround inhibition 90
10.4 Asymmetric responses to the onset and offset of natural images 92
10.5 Dependence of RGC’s responses on natural-image visual textures 94
10.6 Nonlinear-linear model 96
10.7 RGC’s contrast adaptation to natural images 97
10.8 Diversity of RGC’s contrast adaptation to natural stimuli 99
10.9 Dependence of RGC’s contrast adaptation on the contrast
difference of natural images 101
10. 10 Future directions 102
References 104

vii

List of Figures
Figure 2.1 Fast sketch of an On brisk transient RGC’s RF with both a drifting
square-wave grating and STA of natural images. 13
Figure 2.2 Methods for determining the RF center and surround size with natural-
image stimulation. 18
Figure 3.1 Example of an On brisk transient RGC’s responses to natural images. 26
Figure 3.2 Using natural images to estimate RF center size and classify surround
behaviors. 28
Figure 3.3 Comparison of RF-center-size estimation with three different methods,
one with artificial and two with natural images. 30
Figure 3.4 Distribution of the RF center and surround sizes. 31
Figure 4.1 Response and contrast-frequency distribution for an example Off brisk
transient RGC when stimulated with 1000 natural images. 35
Figure 4.2 Responses of an Off brisk transient RGC to 1000 natural images as a
function of the center mean contrasts. 37
Figure 4.3 Examples of four cells to illustrate the sigmoidal relationship between
mean responses and center mean contrasts of natural images, and fits
of a cumulative-Gaussian-distribution model (Eq. 2.5). 38
Figure 4.4 Distribution of surround-inhibition strength elicited by natural images. 40
Figure 4.5 Classification of surround effects into subtractive and division-like
inhibition. 42
Figure 5.1 Asymmetric responses to the onset and offset of natural images in both
histogram and linear RF for an On brisk transient cell (A-D) and an Off
brisk transient cell (E-H). 48
Figure 5.2 RGCs’ response asymmetries across the population. 50
Figure 5.3 Asymmetric distribution of pixel intensities in natural images. 51
Figure 5.4 Examples of negative and histogram-equalized natural images. 53
Figure 5.5 Comparison of RGCs’ response asymmetries when using positive and
negative natural images. 54

viii

Figure 5.6 Disappearance of RGCs’ response asymmetries when using histogram-
equalized natural images. 55
Figure 6.1 Dependence of RGC’s responses on the local visual textures of natural
scenes. 58
Figure 7.1 Nonlinear-linear model. 62
Figure 7.2 A validity test of the nonlinear-linear model with high-resolution plaids. 65
Figure 8.1 Contrast adaptation of RGCs to both checkerboards and natural images. 68
Figure 8.2 Multiple temporal components in RGC’s contrast adaptation to the
contrast change of natural images. 72
Figure 8.3 Diversity of RGC’s contrast adaptation to natural stimuli. 74
Figure 8.4 Distribution of time constants for four different types of contrast
adaptation. 75
Figure 9.1 Contrast adaptation of an RGC to three different natural-image contrast
steps. 80
Figure 9.2 Dependence of RGCs’ responses on the contrast difference of natural
images. 82
Figure 9.3 Dependence of the appearance of RGC’s contrast adaptation on the
contrast difference of natural images. 84
Figure 9.4 Dependence of the time constant of RGC’s contrast adaptation on the
contrast difference of natural images. 85

ix

List of Tables
Table 9.1 Distribution of different contrast-adaptation types in terms of RGCs’
polarities (On, Off and On-Off). 77
Table 9.2 Distribution of different contrast-adaptation types in terms of RGCs’
basic temporal properties (transient and sustained). 78

x

Abstract

Natural scenes have many special statistical properties that have shaped our visual system
through natural evolution. It may thus be necessary to use natural images directly to
examine retinal-ganglion-cells’ (RGCs) properties, rather than to extrapolate their
properties from artificial stimuli. In this study, we first inspected what the most
important visual property determining the responses of an RGC to natural images is. A
new method was developed to estimate with natural images the sizes of the receptive-
field (RF) center and surround. We showed that the center sizes estimated with our
method were similar to those obtained with standard artificial stimuli. Furthermore, the
temporal mean contrast of the center of the RFstrongly dominated the RGC’s responses,
while surround contrast mostly showed a weak and division-like (as opposed to
subtractive) inhibition. We then asked whether the RGCs’ responses also depend on the
local visual textures of natural images, or the luminance variation from the mean. We
observed that RGCs responded asymmetrically between the transition from homogeneous
backgrounds to natural images (onset), and the reverse transition (offset), even if both
transitions had the same local temporal mean contrast. Furthermore, the negative of the
natural images inverted this asymmetry, and their histogram equalization eliminated it.
Hence, the response asymmetry arose from the asymmetric intensity distribution in the
natural images. We further developed a method, spike-triggered contrast histogram
(STCH), to demonstrate that a natural image with strong visual texture tended to elicit
larger responses than one with weak texture. To account for these results, a nonlinear-

xi

linear model was developed. It included multiple subunits of nonlinear inputs, each
covering a sub-region of the RF. Finally, we investigated whether the RGCs’ responses
adapt to the spatial and temporal contrasts of natural images. We found that RGCs
displayed a variety of contrast-adaptation types across the population when responding to
a step mean-contrast change of natural images, and that their contrast adaptation
depended on the natural-image contrast difference. Thus, multiple biophysical
mechanisms might be involved in RGCs’ contrast adaptation, and different RGC types
might use a distinct subset of these mechanisms for different visual tasks.

1

Chapter 1
Introduction
1.1 Temporal and spatial characteristics of natural images
Scenes encountered in nature have many complex statistical properties. For instance,
natural scenes usually contain a high degree of spatial and temporal correlation (Field,
1987; Ruderman and Bialek, 1994; Balboa and Grzywacz, 2003); their power spectra
tend to fall with the square of spatial frequency (Burton and Moorhead, 1987; Tolhurst et
al., 1992; van der Schaaf and van Hateren, 1996; Balboa and Grzywacz, 2003); their
contrast distribution falls exponentially with contrast and has a peak at zero (Field, 1987;
Ruderman and Bialek, 1994; Balboa and Grzywacz, 2003); their intensity distribution has
skewness, with more pixels below the mean intensity and a long tail towards high
intensities (Field, 1987; Ruderman and Bialek, 1994; Brady and Field, 2000; Nuala and
David, 2000; Olshausen and Field, 2000; Simoncelli and Olshausen, 2001).
All these special statistical properties are thought to have had a substantial influence,
through evolutionary adaptation, on the design of our visual system, including the retina
(Enroth-Cugell and Robson, 1966; Atick, 1992; Ahmad et al., 2003; Koch et al., 2006).
For instance, retinas have more Off cells than On cells (Ahmad et al., 2003; Koch et al.,
2006). Moreover, RGCs’ contrast sensitivities increase as the square of spatial frequency,
resulting in a decorrelation of input stimuli (Enroth-Cugell and Robson, 1966; Srinivasan
et al., 1982; Atick, 1992).

2

1.2 Response properties of retinal ganglion cells
The most basic computational structure of a RGC is, arguably, its receptive field (RF). It
has an antagonistic, concentric structure with an excitatory center and an inhibitory
surround (Barlow, 1953; Kuffler, 1953; Rodieck, 1965; Levick, 1967; Enroth-Cugell and
Lennie, 1975; Amthor et al., 1989; Sagdullaev and McCall, 2005). This center-surround
structure has been found in all invertebrate and vertebrate species studied (Rattliff, 1965;
Sagdullaev and McCall, 2005). Several theories attempt to explain this concentric,
antagonistic structure. It may behave as a “whitening filter” to decorrelate responses
(Srinivasan et al., 1982; Atick, 1992), perform “edge enhancement” to help in the
detection of object boundaries (for a brief review, see Balboa and Grzywacz, 2000),
maximize the transmission of information (Laughlin, 1981), help constrain energy
consumption (Vincent and Baddeley, 2003), or reduce the effect of the illuminant on
visual responses (Land, 1959).
In addition to the center-surround RF, a well-studied model of RGC computation is
the linear-nonlinear model. This model comprises a weighted sum of light-stimulus
intensities in the cell’s RF, followed by a static nonlinearity (Reid et al, 1997; Meister
and Berry, 1999; Chichilnisky, 2001; Kaplan and Benardete, 2001; Kim and Rieke, 2001;
Zaghloul et al., 2003, 2005; Carandini et al., 2005; Rust and Movshon, 2005).
Another nonlinear property of RGCs deals with their limited response ranges. This
nonlinearity accommodates responses into a much larger range of light intensity (Rushton,
1965; Barlow, 1981). RGCs are also known to adapt their response sensitivities to both
the mean intensity, termed light adaptation (Barlow and Levick, 1969; Enroth-Cugell and

3

Lennie, 1975; Shapley and Enroth-Cugell, 1984; Walraven et al., 1990; Koutalos and
Yau, 1996; Pugh et al., 1999; Yang and Stevenson, 1999), and the range of intensity
fluctuation around the mean, termed contrast adaptation (Shapley and Victor, 1978;
Smirnakis et al., 1997; Benardete and Kaplan, 1999; Brown and Masland, 2001; Chander
and Chichilnisky, 2001; Kim and Rieke, 2001; Baccus and Meister, 2002; Demb, 2002).
1.3 Problems and our contributions
Although much research has addressed RGC response properties, our knowledge of them
is mainly based on data collected from non-natural stimuli, such as flashing spots,
drifting gratings, and Gaussian white noise (Reid et al, 1997; Meister and Berry, 1999;
Chichilnisky, 2001; Kaplan and Benardete, 2001; Kim and Rieke, 2001; Zaghloul et al.,
2003, 2005; Carandini et al., 2005; Rust and Movshon, 2005), as well as flickering
uniform spatial fields and checkerboards for contrast adaptation (Smirnakis et al., 1997;
Brown and Masland, 2001; Chander and Chichilnisky, 2001; Kim and Rieke, 2001;
Baccus and Meister, 2002).
No direct reports about RGC visual properties in natural environments are available.
Unfortunately, visual processing is often nonlinear (Allman et al., 1985; Merwine et al.,
1995; Kaplan and Benardete, 2001). Therefore, we cannot simply extrapolate neuronal
properties obtained with artificial stimuli and apply them to the natural environment
(Simoncelli and Olshausen, 2001; David et al., 2004; Sterling, 2004; Felsen and Dan,
2005; Mante et al., 2005; Sharpee et al., 2006). For example, some studies have claimed
that there is no solid evidence that the linear-nonlinear model fits RGC responses to
natural stimuli (Carandini et al., 2005); adding a mechanism of slow adaptation also did

4

not improve the model’s prediction to natural stimuli (Mante et al., 2008). To enhance
our understanding of RGC response properties, the use of natural stimuli ensembles is
required (Clifford et al., 2007).
In this dissertation, we asked three main questions about the response properties of
RGCs when stimulated with natural images. We then answered them with our eleven
contributions. Initial reports of RGC response properties have appeared in abstract form
(Cao et al., 2006, 2008, 2009).

Question 1: What is the most important visual property determining an RGC’s responses
to natural images?
Our contributions:
1. A new method was developed to measure the RF center and surround sizes.
2. The center sizes estimated with our method from natural stimuli were similar to those
estimated with standard artificial stimuli.
3. The center area of the RF strongly dominated an RGC’s responses.
4. The surround showed a weak and mostly division-like inhibition.

Question 2: Do the RGC’s responses to natural images depend on their local luminance
variation (local visual textures)?
Our contributions:
5. RGCs displayed asymmetric responses to the onset and offset of natural images.
6. This response asymmetry arose from the asymmetry of natural-image intensity
distribution.

5

7. A method was developed to demonstrate that RGCs’ responses depended on local
visual textures of natural images.
8. A nonlinear-linear model was proposed and tested for RGCs to account for their
response dependence on the visual textures and for the onset/offset asymmetry.

Question 3: Do the RGC’s responses to natural images adapt to their spatial and
temporal contrasts?
Our contributions:
9. RGCs showed three adaptation components with different time courses to step mean-
contrast increase and one adaptation component to step mean-contrast decrease in
response to natural images.
10. Different types of RGCs displayed distinct combinations of these components across
the population. Transient cells had more tendency to adapt than the sustained cells;
Off cells had more tendency to display fast adaptation than On cells.
11. The RGC’s contrast adaptation with natural images depended on the mean-contrast
difference of natural images. As the mean-contrast difference increased, both the
peak and plateau responses also increased, but their ratio remained statistically
constant; in addition, the time constant of slow-adaptation component decreased.
Furthermore, Some RGCs yielded adaptation only to a large mean-contrast difference.

6

Chapter 2
Methods
2.1 Physiological preparation
This preparation was similar to that used elsewhere (Chatterjee et al., 2007). Briefly,
pigmented rabbits weighing 2 - 4 kg were dark adapted for 30 minutes before surgery.
Surgery was conducted under dim red light. Rabbits were initially anesthetized with I.P.
Ketamine (50 mg/kg) and Xylazine (5 mg/kg). Then, sodium pentobarbital (1 ml/kg) was
injected into the marginal ear vein to obtain deep anesthesia. Anesthesia was checked by
testing corneal reflexes and reactions to a paw pinch. After confirming a lack of reflexive
movement, an eye was enucleated. The eyecup was then hemisected, and the animal was
subsequently euthanized with an I.V. overdose of sodium pentobarbital. All these
experimental procedures were approved by the Institutional Animal Care and Use
Committee (IACUC) at the University of Southern California, in accordance with the
Guide for the Care and Use of Laboratory Animals (National Institutes of Health).
The retina was then gently separated from the pigment epithelium and mounted,
RGC side down, over a hole punched in a Whatman Filter paper. The filter paper was
then flipped and mounted in a recording chamber. A metal ring gently held the paper in
place. Once in the chamber, we continuously superfused the retina with oxygenated
bicarbonate-buffered Ames solution (Sigma) at 37
º
C at a flow rate of 3-7 ml/min. The
isolated retina remained healthy for at least 6 hr post isolation.

7

2.2 Electrophysiological recording
Methods of recording from a multi-electrode array (Cyberkinetics) and spike sorting by
using a professional software POWERNAP (Cyberkinetics) were as previously described
(Chatterjee et al., 2007). So were general methods of visual stimulation, determination of
each cell’s response polarity (On or Off), characterization of each cell’s temporal
property (transient or sustained), and measurement of spontaneous firing rate. A total of
19 isolated retinas and 379 RGCs were analyzed in this study. Of those, eight isolated
retinas and 102 cells were used to inspect the RF center and surround structures of RGCs
with natural-image stimulation; seven isolated retinas and 152 cells were selected to
examine the dependence of RGCs’ responses on the local visual textures of natural
images; four isolated retinas and 125 cells were taken to investigate RGCs’ contrast
adaptation to the contrast change of natural images.
2.3 Visual stimulation
The natural images used in our study were obtained from the online calibrated-image
database at http://hlab.phys.rug.nl/archive.html (van Hateren and Ruderman, 1998). We
calibrated them to be linear with respect to our monitor’s luminance. The source images
with 1536 × 1024 pixels were cropped down to either their square central regions or into
eight evenly spaced regions with 300 × 300 pixels. Each region was then down-sampled
into a square image of 120 × 120 pixels, and projected onto a 3 × 3 mm
2
region on the
retina.

8

For our initial experiment, natural images were presented in a random sequence,
each for 1000 ms. When the natural image was removed, the display was held at a
spatially uniform gray for 1000 ms before the next image was presented. The mean
intensities of the gray and natural images were the same (9.10 cd/m
2
) to remove
luminance adaptation. The long periods of presentation of gray and natural images
allowed us to distinguish clearly between responses to the onset and offset of images
(Smyth et al., 2003). We presented from 1,000 to 12,000 images in different experiments.
In addition, two kinds of stimuli, the negative and histogram-equalization of natural
images, were obtained by transforming the original natural images to examine whether
the response asymmetry of an RGC was caused by the asymmetric distribution of natural-
image intensities. The first kind was equivalent to the negative of the image about the
mean. To ensure that no intensity was below zero after inversion, we compressed the
distribution towards the mean by a factor of four. The second kind was a histogram-
equalized version of the image. This version had a uniform intensity distribution, which
we enforced in two steps:
Step 1: calculate the intensity distribution of a natural image.
Step 2: replace the intensity of each pixel with









× =
∑
= pixel
i
j
j i
NUM
INT
N round Int
max
0
, (2.1)
where
max
INT is the maximum intensity in the image,
pixel
NUM is the total number of
pixels, and
j
N is the number of pixels with the j
th
brightest intensity (the lowest being 1).

9

To examine RGC’s adaptation to the spatial and temporal contrasts of natural
images, however, we followed a protocol similar to that used in other studies (Smirnakis
et al., 1997; Brown and Masland, 2001). Series of low-contrast natural images were
alternated with high-contrast ones, every 30 s for 10 or 20 trials, unless otherwise noted.
The contrast of an image was defined as the root-mean-square (RMS) of light intensities
divided by the mean. In most series, image contrasts were in a limited range, e.g., from
0.05 to 0.25 in a low-contrast group or from 0.52 to 0.85 in a high-contrast group. The
image update rate was 30 ms per frame (or 33 Hz). The series of natural images were
either different across trials to inspect the variation of the RGC’s contrast adaptation
across different stimulus sets, or identical to examine the consistency of the RGC’s
contrast adaptation. The mean luminance of stimuli was also kept constant at 9.10 cd/m
2

throughout the experiment to avoid any bias from light adaptation.
Only when we investigated the dependency of contrast adaptation on the contrast
difference of natural images, we controlled natural-image contrast to be equal in a series.
To form a natural-image group with specific contrast, we manipulated each image with
the equation below:
( )
raw
raw
new
raw raw new
M
Std
Std
M I I + ⋅ − = (2.2)
where I
new
is the intensity of a new image with the intended standard deviation Std
new
, I
raw

is the intensity of a raw image from the database with the standard deviation Std
raw
, M
new

is the mean intensity of the new image, and M
raw
is the mean intensity of the raw image.
With the above method, we manipulated the contrast value from Std
raw
to Std
new
.

10

Finally, we also probed the RGC’s contrast adaptation to flickering checkerboards
of 15 ×15 squares for comparison with the original experiments. Each square had a side
of 200 μm and a luminance independently updated from a Gaussian probability
distribution at the same rate as that used in natural stimuli.
2.4 Statistics of natural images
The temporal contrast of a natural image with respect to the RF center and surround area
(to be specified in Methods 2.6.3) were defined respectively by

gray
gray c
c
M
M M
C
−
= (2.3)

grayl
gray s
s
M
M M
C
−
= (2.4)
where
c
M is the mean intensity of the specified center area,
s
M the mean intensity of
the specified surround area, and
gray
M is the intensity of full-field gray or the mean
intensity of the entire natural image. The center and surround areas were determined
with our new method based on cross-correlation (Methods 2.6.3).
The definition of contrast in the investigation of RGCs’ contrast adaptation was
different from the above. We followed a protocol similar to other studies (Smirnakis et
al., 1997; Brown and Masland, 2001), and used a traditional contrast measurement: root-
mean-square (RMS), where the contrast was defined as the ratio between the standard
deviation of pixel intensities of natural images and their mean (Peli, 1990).

11

2.5 Contrast-response-function estimation
To investigate the dependence of each cell’s responses on the temporal mean contrast of
the RF center, we fitted the contrast-response function with a cumulative-Gaussian-
distribution curve (Chichilnisky, 2001). A cumulative Gaussian distribution curve was
defined by












 −
+ =
2
1
2
σ
M C
erf
A
r
c
(2.5)
where A is the maximal firing rate, M is the contrast value of the maximum slope, and σ
is the standard deviation of the Gaussian distribution. A criterion of minimum mean-
squared error was applied to obtain an optimal estimate.
2.6 Receptive-field-center estimation
An accurate measurement of the RF center and surround areas is important for properly
interpreting how each contributes to cell responses. We estimated the RF center of each
cell with three different approaches.
2.6.1 RF estimate from moving a square-wave grating in four directions
We modified a technique employed elsewhere to map the RF with artificial stimuli
(Rodieck and Stone, 1965). In our modification of that technique, 20 trials of a low
spatial-frequency square-wave moving grating (0.14 cycle/mm) were displayed
rightwards, leftwards, upwards, and downwards. The spatial frequency of these images
was so low that each cell saw only one edge at a time.
When an edge with appropriate contrast polarity moves into the RF of a cell, a

12

response begins. That moment minus the response delay indicates the boundary of the
cell’s RF center (Rodieck and Stone, 1965). In turn, this delay is that in response to full-
field stimuli. Hence, the delay is the time difference between the beginning of full-field
step and the cell’s subsequent response. Time delays estimated in our experiments varied
across cells with a maximum of about 200 ms and a mode of 35 ms. Therefore, by
moving a square-wave grating of low spatial frequency across the RF in the four cardinal
directions, we can quickly sketch the RF center. This RF-mapping-with-motion approach
is similar to the “minimum discharge field” of Hubel and Wiesel (1962). The histograms
of one cell’s responses to all four directions are shown in Fig. 2.1A-D. To visualize the
two-dimensional RF obtained from moving the square-wave grating, we used its speed to
scale the histogram from time (ms) into space (μm) (insets of Fig. 2.1A-D). Finally, we
combined the spatial mapping of all directions to get a rough approximation of the cell’s
RF center (Fig. 2.1E).

13

Figure 2.1. Fast sketch of an On brisk transient RGC’s RF with both a drifting square-wave
grating and STA of natural images. A low spatial-frequency square-wave grating (0.14 cycle/mm)
was displayed drifting in four directions: rightwards (A), leftwards (B), upwards (C), and
downwards (D). The post-stimulus time histogram of responses for each grating direction for 20
trials was scaled into the spatial position by taking grating velocity and response delay into
account as shown in the insets. E. Receptive-field approximation from the combination of the
insets in A-D. F. Receptive-field estimation of the same RGC as in E from STA of natural
images.

14

2.6.2 RF estimate from natural images based on spike-triggered average (STA)
A rough estimation of the RF from natural stimulation can be obtained by using the
spike-triggered average (STA – Willmore and Smyth, 2003). This estimation would be
valid if we assumed the RGC to be linear and the input to be white noise (Marmarelis and
Marmarelis, 1978). The estimation would also be correct if the system could be modeled
as a linear-followed-by-static-nonlinearity system with either Gaussian white noise
(Chichilnisky, 2001; Willmore and Smyth, 2003) or some other orthogonal stimulus as
input (Reid and Alonso, 1995; Ringach et al., 1997).
However, natural images are neither white nor orthogonal. They have considerable
spatial (and for movies, temporal) correlations. Usually, a further normalization by the
autocorrelation matrix of natural images has to be implemented to correct for these
spatial and temporal correlations, assuming that all are stationary (Willmore and Smyth,
2003). In our experiment, we could only sample a small portion of all possible images;
hence, the autocorrelation matrix of natural stimuli was singular, causing the inverse of
the matrix to be ill conditioned. A few techniques, including pseudo-inverse and
regularized pseudo-inverse with constraint of norm or with constraint of smoothness, had
been used to overcome this obstacle (Theunissen et al., 2000; Smyth et al., 2003). A
detailed explanation of these techniques can be found in the Willmore and Smyth paper
(2003). Nevertheless, these techniques only consider second-order correlations between
pixels in natural images and require these correlations to be stationary. We used these
techniques understanding their limitations. An RF estimated with STA is shown in Fig.
2.1F. Although the RF estimated with STA from natural images is larger than that
derived from the moving grating, their positions match well.

15

2.6.3 RF estimate from natural images based on cross-correlation between
responses and putative RFs
We proposed a new method to estimate the RF center based on one of our discoveries
from natural stimulation and the ubiquitous concentric structures of RGCs. In our
experiments, we found that the center mean contrast dominated a cell’s responses when
stimulated with natural images, even if we varied the assumed center size over a quite
large range. Concomitantly, the surround influence was weak compared to the center.
We know that traditionally, the RF center is defined as the area where one can elicit a
cell’s response, and the surround is the region that can modulate a center-elicited
response. Therefore, in our study, we assumed that the correct center should be the
region where we could obtain the maximal cross-correlation coefficient (or mutual
information) between the cell’s response and the center mean contrast of natural images.
The cross-correlation coefficient is:

( )
( )( )
( ) ( )
∑ ∑
∑
− −
− −
=
2
2
,
r r C C
r r C C
r C C
c c
c c
c (2.6)
where r is the cell’s response to an image, r is the mean of cell’s responses,
c
C is the
center mean contrast,
c
C is the mean of center mean contrasts, and the summations are
over all images (Aertsen et al., 1989).
In turn, the mutual information between the center mean contrasts and the cell’s
responses is:

16

( ) ( )
( )
( ) ( )








=
∑ ∑
r p C p
r C p
r C p r C I
c
c
c
r C
c
c
) ,
log , ,
(2.7)
where ) (
c
C p and ) (r p are the marginal probability distribution functions of
c
C and
r respectively, and ) , ( r C p
c
is the joint probability distribution function of
c
C and r
(Paninski, 2003). To calculate the probability distribution function of r , we divided the
whole range of a cell’s responses into evenly distributed segments, or bins. The response
of each bin was represented by its middle response. We then counted the number of data
points in each bin. The ratio between this number and the number of points in all bins is
the probability density estimated for this bin. Thus, the calculation of mutual information
depends on the number of bins chosen for grouping the cell’s responses. The center
mean contrast displays the same dependence. The estimation of the correlation
coefficient, however, does not depend on these bin-related operations, since its estimation
does not need the probability distributions of
c
C or r (Eq. 2.6). Therefore, to avoid this
dependence, we mainly used the cross-correlation coefficients to estimate the center size.
Nevertheless, our data analysis showed that, over a large range of selected bin sizes,
mutual information gave a similar estimate of center size as cross-correlation.
Figure 2.2 illustrates this method to determine RF center size from natural-image
stimulation. Let us assume that the light gray disc represents the actual RF center and the
black disc represents the actual RF surround. We make various guesses of what the
center may be (white disks). When the guessed center size is small compared to the
actual center size (Fig. 2.2A), the correlation coefficient between the guessed-center
mean contrasts and the cell’s responses will be relatively small. This is because this

17

guessed center only considers a part of the influence from the actual center. As the
guessed center size approaches the correct center size, the cross-correlation coefficient
will increase. When the guessed center size is correctly selected, a maximal correlation
coefficient is reached (Fig. 2.2B). A further increase in the guessed center size will lower
the strength of correlation (Fig. 2.2C). This is because the guessed center will mistakenly
assign some surround influences to the center. Consequently, the trend of the correlation
coefficients should behave as an inverted U-shape function as we continuously vary the
guessed center size from small to large (Fig. 2.2D). The radius that maximizes this
function is our estimate of the radius of the RF center.

18

Figure 2.2. Methods for determining the RF center and surround size with natural-image
stimulation. In this figure, the light gray and black discs represent the true RF center and
surround areas respectively. In turn, the white discs represent the guesses for RF center sizes and
the white annuli represent testing windows for the surround. A. If the guessed RF center size is
small compared to the true center size, the correlation between the center mean contrasts and
responses will not be maximal (D). This is because some areas of the true center are missed. B.
If the guessed RF center size is correct, the correlation is maximal (D). C. If the guessed RF
center size is larger than the true center size, the correlation will also fail to be maximal (D),
because the guessed center includes some areas of the surround. We estimate the RF center size
from Point B of the resulting bell-shape curve (D). After we estimate the true center size, we split
the surround into many mutually exclusive annuli. We then check how the mean contrasts within
these annuli influence responses. For a cell with inhibitory surround, when the positions of the
annulus vary from closest (E) to farthest (G), the surround will show inhibition (H). Near
Position E, inhibition may be weak (H), as some excitatory influence from the RF center
contributes to responses. Near Position G, inhibition may be weak (H), as the surround is ending.
Thus, some intermediate position (F, H) will yield the maximal inhibition. We estimate the
surround size from Position G of the resulting U curve (H).

19

2.7 Receptive-field-surround estimation
While the center region is the place that elicits cell’s responses, the surround region is the
area that modulates the responses, typically, in an inhibitory manner. We took advantage
of this modulation to devise a method to estimate the size of RF surround. After we
estimated the center size, we split the area outside the center into many mutually
exclusive probing annuli (white annuli in Fig. 2.2E-G). For each annulus, we calculated
the surround mean contrast (Eq. 2.4) within this annulus. We then considered those
images with similar center mean contrast (Eq. 2.3). Because the center mean contrast
was the same, any systematic response variations depended on the changes of surround
mean contrast. We then fitted the mean response as a function of surround mean contrast
with a straight line. We took the slope of this linear fit as a measure of the surround
effect. Positive and negative slopes indicate excitation and inhibition respectively. For a
cell with an inhibitory surround, all annuli within the surround area, from the closest
annulus (Fig. 2.2E) to the farthest annulus (Fig. 2.2G), will show inhibition. At one
position, the inhibition strength will reach the maximum negative value (Fig. 2.2F). The
trend of inhibition strength will thus be a U-shape function, as we continuously vary the
probing surround annulus from closest to the center to the farthest (Fig. 2.2H). When the
probing annulus is just too large, this function returns to zero, as the annulus should cause
no inhibition. The surround size is thus determined by that point of this U-shape function,
shown as Point G on Fig. 2.2H.
We determined the strength of the surround only for cells that yield a clear U-shape
behavior as in Fig. 2.2H (39 neurons in our sample). Only for those cells could we be

20

certain where the surround begins and ends. The estimation of the surround strength used
stimuli between Points E and G of Fig. 2.2H. The inner radius would ideally be at the
border of the RF center, not at Point E. However, this point, which comes from the first
annulus eliciting inhibition, has often a larger radius. This is because annuli near the
border of the RF center will have some excitatory contributions from the center. This
contribution is due to the spatial correlation of natural-image stimuli. In other words, we
could classify these annuli as neither center nor surround. Thus, in the estimation of
inhibitory strength, we defined inner radius by the first annulus eliciting inhibition.
In turn, we determined the type of inhibition, i.e., subtractive versus division-like,
from a larger sample of cells (60 neurons). In this sample, we included cells for which
Point G of Fig. 2.2H was not evident. Those were cells with surround sizes larger than
we could measure. Because we could not determine their sizes, measuring the total
strength of inhibition was impossible. However, identifying their type was still possible
and we did so. For this measurement, we used Point G if it was available, or we set the
outer edge to occupy 90% of the image.
The details of the methods used to determine strength and type of inhibition will
appear in Chapter 4.
2.8 Spike-triggered contrast histogram (STCH)
To test whether the local visual textures of natural images contribute to RGCs’ responses,
we developed a new statistical method called spike-triggered contrast histogram (STCH).
In this method, we first built a histogram of pixel contrasts for each image. If the
intensity of a pixel was i (0 ≤ i ≤ INT
max
) and the local intensity mean was M
local
, then its

21

contrast was (i - M
local
)/M
local
. Next, we weighted the contrast histogram for each image
with its elicited differential response. The differential response was R – r(C
c
), where R
was the actual response to the image and r was the predicted response (Eq. 2.5) based on
the local temporal mean contrast C
c
(Eq. 2.3). Finally, we summed all the differential-
response-weighted contrast histograms.
The logic behind STCH is as follows: Imagine that the local temporal mean contrast
is the only variable controlling response. In that case, R – r(C
c
) should be close to zero.
In other words, individual pixel contrasts would not matter. However, if a pixel contrast
was more likely to yield a large response despite the local temporal mean contrast, then
that pixel contrast would yield a positive STCH bin. By the same token, if a pixel
contrast was more likely to yield a small response, then the corresponding STCH bin
would be negative. Consequently, the STCH is similar to the STA, but reveals important
or unimportant pixel contrasts. Hence, the STCH can be used to probe visual texture. On
one hand, images with little variation of intensity, i.e., with weak texture, tend to have
intensities close to the mean and thus pixel contrasts close to zero. On the other hand,
images with strong texture have pixels very different from the mean and thus with
contrasts substantially different from zero. Hence, by comparing the STCH at the bins of
low pixel contrasts with that at the bins of high pixel contrasts, one may be able to tell
whether texture contributes to the response.
2.9 Mixture-of-exponentials model
Following a mean-contrast increase of natural images, the RGC’s response amplitude
decays with multiple time constants. The decay can be fitted with a mixture-of-

22

exponentials model. Depending on whether the decay involves one or two different time
constants, we modeled it as either an exponential function (Eq. 2.8) or a linear
combination of two exponential functions (Eq. 2.9).
The mathematical formulation of the exponential function is

( )
base
tc
t
r p t r + ⋅ =
−
exp ) (
(2.8)
where ) (t r is the response amplitude of an RGC at a given time t , p is the amplitude
before adaptation,
base
r is the baseline response, and tc is the time constant of the
response decay.
The mathematical formulation for the mixture of two exponentials is
base
tc
t
s
tc
t
f
r p p t r
s
f
+ ⋅ + ⋅ =






− 





−
exp exp ) (
(2.9)
where
f
p is the response amplitude in the fast component,
s
p is the response amplitude
in the slow component,
f
tc is the time constant of the fast component, and
s
tc is the time
constant of the slow component.
The parameters of Eq. 2.8 and 2.9 were estimated in four steps. First, the mean
response at each bin was computed by averaging the spikes at that bin across all trials.
Second, the baseline response
base
r , or the plateau response, was calculated as the
medium response of the last three time bins. Third, the baseline response was subtracted
from the cell’s original responses to obtain the relative response decay. Fourth, the
natural logarithm of the relative response decay was fitted by a multi-segments line to
determine the time constants of different temporal components. To determine the

23

number of linear segments needed to fit the data, we first examined whether there was a
downward trend using a technique introduced by Cox and Stuart (1955). Then we
checked whether the slope estimated from the first 6 s of the data was significantly
different from that thereafter.
After stimulus contrast was switched from high to low, i.e., in the opposite direction
from what we have been discussing so far, the response recovery had only one time
constant. Thus, only equation 2.8 was used. However, we subtracted the cell’s responses
from the estimated baseline response because the response amplitude, p (Eq. 2.8), would
be negative.
After fitted with a mixture-of-exponentials model, the peak response was defined as
the firing rate in the first time bin of the fitted response-decaying curve. The ratio of the
peak response to the baseline response was defined as the adaptation index (Albrech et al.,
1984; Brown and Masland, 2001). This index provides an overall quantitative indication
of an RGC’s adaptability. An RGC, showing contrast adaptation, usually has an
adaptation index greater than 1, reflecting a response decay or recovery to the contrast
change of natural stimuli.
2.10 Contrast gain control, slow and fast contrast adaptation
Two different histogram time bins, namely, 1.5 s. and 0.3 s, were used to analyze the
RGC’s contrast adaptation to natural stimuli. The 1.5 s was good for discerning fast and
slow adaptation. However, this bin was not good for contrast gain control, a much faster
temporal component (<0.3 s). To study different time scales, a 0.3-s bin was also used.

24

However, contrast gain control has been known to have a time constant around 100 ms
(Shapley and Victor, 1978; 1979; Enroth-Cugell and Jakiela, 1980; Shapley and Enroth-
Cugell, 1984; Victor, 1987). Therefore, the 0.3-s time bin was usually more than two
times the time constant of the contrast-gain-control component, thus including all its
response decreases. Our method could thus detect contrast gain control, but not analyze
its time constant, as this requires a time bin no larger than 0.025 s. We did not use such a
bin because it is too sensitive to noise, given the insufficient amount of data available in
our study.
For those adaptation components that were much slower than contrast gain control,
either a single-exponential or a mixture-of-exponentials model was used to determine
their time constants. The resistant-line-from-three-group algorithm (Mosteller and Tukey,
1983), which was more robust than least-squares fitting, was applied to slow adaptation.
If contrast adaptation included both fast and slow components, we had to fit the data with
two linear segments. To avoid mutual contamination between the estimations of time
constants of fast and slow temporal components, we first estimated the slow component
using the data after the first 6 seconds, and then estimated the fast component with the
data in the first 6 s. The reason for this two-step procedure was that the fast component
should decay completely after 6 s, leaving the slow component alone. After estimating
the slow component, we subtracted it from the cell’s responses. Then the remaining data
only included the fast-adaptation component. To estimate its time constant, a least-
square technique was used because with only a few available bins, we could not apply the
robust algorithm described above. Some RGCs only exhibited fast adaptation. For these
cells, we calculated the fast component without estimating the slow component.

25

Chapter 3
Estimation of the Receptive-Field Center and
Surround
3.1 Polarity of RF with natural-image stimulation
Our first goal in this work was to study what the most important property determining an
RGC’s responses to natural images is. To undertake this study, we first had to find out
what basic features of a natural image caused RGCs to respond. Based on the
experiments with artificial stimuli, we expected RGCs to respond to spatio-temporal
contrasts. For instance, we expected On and Off RGCs to respond best when the mean
intensity of natural images increased and reduced in their RF center, respectively. As Fig.
3.1 illustrates, RGCs meet this expectation. The polarity (On or Off) of an RGC as
established with artificial stimuli remains the same when one stimulates these cells with
natural images.
An example of On cell (as established with full-field steps) responses to natural
images is shown in Fig. 3.1. A rough estimation of the RF center based on STA
(Methods 2.6.2) using 1,000 natural images appears in the bottom of the figure. The
other ten rows show the responses of this cell to exemplar natural images. This cell
responded strongly to natural images when a bright region stimulated its RF center (Rows
1, 3, 4, 5, 7, and 9). In contrast, the cell responded poorly when images were not bright

26

in the RF center (Rows 2, 6, 8, and 10). We found similar responses with other On cells
and the inverse polarity with Off cells. Figures like this thus lead to the conclusion that
cells keep their polarity with natural stimuli. Moreover, such figures illustrate that the
STA is a rough but simple method to determine the RF center with natural images.

Figure 3.1. Example of an On brisk transient RGC’s responses to natural images. Ten patches of
natural images are shown on the left-hand side of each row, along with their responses on the
right-hand side. The bottom row displays the RF estimated from the STA of 1000 natural images
and the post-stimulus time histogram of the RGC’s responses. This On RGC yielded large
responses when bright regions stimulated its RF (Image Patches 1, 3, 4, 5, 7, and 9), and yielded
low responses otherwise (Image Patches 2, 6, 8, and 10).

27

3.2 Estimation of RF-center diameter with natural-image stimulation
The most commonly employed technique for estimating RFs with complex stimuli is
STA (Willmore and Smyth, 2003). However, STA has three problems when used with
natural images. First, STA requires the system to be linear, or linear-nonlinear
(Chichilnisky, 2001; Willmore and Smyth, 2003). Unfortunately, the visual system
shows complex spatio-temporal nonlinearities, including rectification, saturation,
multiplication, and division (Poggio and Reichardt, 1976; Torre and Poggio, 1978;
Grzywacz and Koch, 1987; Carandini and Heeger, 1994). Second, STA requires stimuli
to be Gaussian white noise. However, natural images are neither white nor Gaussian
(Ruderman and Bialek, 1994; Balboa and Grzywacz, 2003). Third, STA cannot separate
the center and surround contributions to the RF, but mixes them. We thus devised a new
method to estimate RF center size with natural images (Fig. 2A-D). We used this method,
aided by an estimation of the RF middle location with STA. Results appear in Fig. 3.2.

28

Figure 3.2. Using natural images to estimate RF center size and classify surround behaviors.
The RF center was estimated with the method illustrated in Fig. 2.2A-D by using either the cross-
correlation coefficient (A) or mutual information (B). Error bars in Panels A and B stand for one
standard error, based on jackknife analysis with 90% data and 10 groups (Efron and Tibshirani,
1993). With either cross-correlation coefficient or mutual information, the diameter of the RF
center was estimated around 400 μm for the example RGC in this figure (an Off sluggish
sustained RGC). However, the estimation of the RF-center curve with mutual information was
noisier and thus, we used the cross-correlation coefficient for the rest of this study. In turn, we
observed four different types of RF-surround behavior when using the methods of Fig. 2.2E-H. C.
Example of an Off sluggish sustained RGC (same cell as Fig. 3.2A) exhibiting the downward-
upward (U-shape) effect of surround as a function of annulus position predicted in Fig. 2.2E-H.
D. Example of a downward-only function (for an On sluggish transient RGC). E. Example of an
upward-downward function (for an On-Off brisk transient RGC). F. Example of an On-Off brisk
transient RGC whose behavior was not classified in one of the above three categories. Of the 102
cells recorded, we had 39 RGCs with downward-upward trends, 21 cells with downward trends,
10 cells with upward-downward trends, and 32 non-classified cells (N values inside panels). See
text for the interpretation of these four types of trends.

29

Fig. 3.2A,B shows for one cell how the cross-correlation coefficient (Eq. 2.6) and
mutual information (Eq. 2.7) between the responses and mean contrasts respectively vary
with the guessed RF center diameter. As predicted in Fig. 2.2A-D, both estimates have a
similar inverted U-shape function. Consequently, both methods yielded a similar
estimation of RF center diameter, namely, around 400 μm for this cell. The only
difference between the two methods is that the estimation from mutual information
showed larger error bars, thus having more uncertainty in its computation. This added
uncertainty, coupled with that given by the choice of histogram-bin sizes in the mutual-
information technique (Methods 2.6.3), led us to use the cross-correlation method in the
rest of this study. All cells reported here showed inverted U-shape functions with the
cross-correlation technique.
To confirm the validity of the cross-correlation technique for RF-center estimation,
we compared it with two alternate methods. The first alternative was STA and the
second used the square-wave gratings moving in four directions (Methods 2.6.1). For
comparison, we selected an On cell (Fig. 3.3A) and an Off cell (Fig. 3.3B) to illustrate
the RF estimated with the three different methods. For each cell, the left panel illustrates
the RF center sketched from the moving-grating technique. In turn, the right panel
illustrates the RF estimated from the STA of natural-image stimulation. The RF center
determined with the cross-correlation method is shown as the red ring in both panels.
One can see that the method based on cross-correlation captures well the center size
estimated with the moving gratings. The center area estimated with STA is not too
different, but as predicted, larger than obtained with the other methods.

30

Figure 3.3. Comparison of RF-center-size estimation with three different methods, one with
artificial and two with natural images. Panels A and B give RF examples for an On RGC and an
Off RGC respectively. The left-hand side of each panel is the RF center sketched from the
moving-grating method (Fig. 2.1E). The right-hand side is the RF estimated from the STA of
natural images (Fig. 2.1F). The RF center estimated from the cross-correlation method with
natural images (Fig. 3.2A) is plotted as the red rings. The method based on cross-correlation
yields similar RF center sizes as those from moving gratings. However, STA overestimates the
RF centers, because of the spatial correlation between neighboring regions in natural images.

With a sample of thirty-three cells, we compared the center sizes estimated from
moving gratings with those estimated from natural images using the cross-correlation
method (Fig. 3.4A). The methods based on artificial and natural stimuli yielded strongly
correlated estimates of center size. The cross-correlation coefficient between the center
sizes estimated with these two methods was 0.87. In conclusion, our cross-correlation
method is a valid, robust technique to estimate RF-center size from natural images.

31

Figure 3.4. Distribution of the RF center and surround sizes. A. Center diameters
estimated with moving artificial gratings versus those estimated with natural images
based on the cross-correlation method. Each point shows a single cell’s RF-center
diameter estimated with both methods. Only those RGCs with clear boundaries in
response to moving gratings are included. The two example cells illustrated in Fig. 3.3
are marked with the open circles. The center diameters estimated with the cross-
correlation method are similar to those estimated with moving gratings, exhibiting a
positive correlation coefficient of 0.87. B. Distribution of the RF center and surround
diameters estimated with natural images based on the cross-correlation method. Here,
only those RGCs with the surround downward-upward trend are considered (Fig. 3.2C).
We do not include the other RGCs, because we cannot determine their surround sizes
based on our method.
3.3 Estimation of RF-surround diameter with natural-image
stimulation
After we obtained the RF center, we estimated the RF surround. The surround is the
region that modulates the cell’s responses from the center. For estimating the RF
surround, we split the area outside the RF center into many mutually exclusively annuli
(Fig. 2.2E-G). We then considered images with fixed center mean contrast and measured
how the surround mean contrast within these annuli influenced the responses (Fig. 2.2H).
Results of our use of this multi-annulus technique appear in Fig. 3.2C-F.

32

For the cells recorded in our study, four different types of surround behavior were
observed. In the first surround type, response went downward and then upward (U-shape)
as a function of the distance of the surround annulus from the RF center (Fig. 3.2C). In
the second type, the response went only downward (Fig. 3.2D). A possible reason for
this downward function is that the surround was so large for these cells that we could not
visualize the upward portion of the behavior with the available stimuli. The third
surround type yielded an upward-downward behavior (Fig. 3.2E). This behavior occurs
for those cells with a facilitatory, instead of an antagonistic, surround. All other cells that
could not be classified into one of the above three categories formed our fourth type. For
these types of behavior, we were not certain whether the surround was facilitatory or
antagonistic. One example is shown in Fig. 3.2F. We think that the uncertainties of these
cells were mostly due to the surround influence being weak and thus easily corrupted by
noise. Hence, we could not observe clear surround behaviors for these cells. In total, we
recorded from 102 cells. From those, we had 39 cells with surrounds displaying the
downward-upward trend, 21 cells with the downward trend, 10 cells with the upward-
downward trend, and 32 non-classified cells. For the 70 classified cells, the ratio
between the number of cells with antagonistic surround (60 cells) and those with
facilitatory surround (10 cells) was 6:1.
With data like those in Fig. 3.2, we were able to measure and compare the
diameters of the RF center and surround. We limited this comparison to these 39 cells
with the downward-upward surround trend (Fig. 3.2C). We chose this constraint because
these cells had clear zero crossing positions in their surround, allowing us therefore to
determine their surround sizes. The 10 cells with facilitatory surrounds were not studied

33

in this way, since they were too few in number. We also did not consider the other two
categories in Fig. 3.2. This is because either the surround regions of these cells were
beyond the available stimuli or the surround effect was too weak to get a clear estimation
of surround boundaries. Results for the comparison of the RF center and surround
diameters appear in Fig. 3.4B.
Figure 3.4B shows that the diameters of the RF centers and surrounds are variable
and did not correlate over the population of RGCs. We conclude that the basic center and
surround structure of the RF revealed by natural images is similar to the structure
revealed by artificial ones. Center diameters for the recorded cells ranged from about
150 to 900 μm. For the same cells, surrounds had diameters between about 1500 to 4000
μm, and the ratios between the surround and center diameters were variable. In the
extremes, while the smallest ratios we found were about two, the largest were almost
twenty.

34

Chapter 4
The Center and Surround Responses to Natural
Images
4.1 Strong center excitation with natural stimuli
With artificial stimuli, the stimulus contrasts at the RF center and surround control rabbit
RGCs’ responses (Merwine et al., 1995). To ask whether the same happens with natural
images, one must first define their contrasts over the given regions of the scenes. Natural
images have complex structures, making multiple definitions possible. The definitions
that we adopted for this investigation can be interpreted as the relative temporal changes
of mean intensity in the center (Eq. 2.3) and in the surround (Eq. 2.4). Alternatively, one
can interpret these definitions as the relative difference of mean intensities between the
center (and between the surround) and the rest of the image. Fig. 4.1A uses a false color
scheme to show how responses varied with mean contrasts for an Off-brisk-transient
RGC. (To make such figures comparable for On and Off cells, we invert the sign of
contrast for the latter in this and all other figures of this dissertation.) To generate this
figure, we first split the entire ranges of center and of surround contrasts into 30 evenly
spaced intervals each. Thus, with two variables, center mean contrast and surround mean
contrast, we got 900 combinations. For every combination, we calculated the mean
response over 1000 natural images.

35

Figure 4.1. Response and contrast-frequency distribution for an example Off brisk transient
RGC when stimulated with 1000 natural images. A. Mean responses as a function of center and
surround mean contrasts. The graph shows that the center mean contrast dominates the cell’s
responses, because if one fixes surround contrast, responses vary strongly with center contrast but
not vice-versa. B. Joint distribution of center and surround contrasts. This joint distribution
peaks at zero and, in our experiments, center and surround contrasts exhibit a negative correlation.
(The negative correlation between center mean contrasts and surround mean contrasts is an
artifact of our choice of images. The mean intensities of our natural images and of the gray
images that preceded them were the same. We did so to control for effects of overall intensity
modulation. Therefore, if, say, the RF-center intensity in a natural image is above the mean, the
peripheral intensity will be below. If the image patches are small, then the surround of the cell
will occupy most of the periphery. Consequently, the surround will have a mean intensity below
the mean.)

Based on the two-dimensional function plotted in Fig. 4.1A, the responses increase
as the RF-center and RF-surround mean contrasts rise and fall, respectively. The center
mean contrast dominates the responses. With fixed surround mean contrast, the cell’s
responses vary considerably if we select images so as to modulate the center mean
contrast from low to high. Responses can vary from zero to the maximal saturated value
that the cell can reach, especially for low surround contrasts. However, if instead we fix
the center mean contrast, the cell’s responses do not change much as surround is varied.

36

Figure 4.1A does not show how many images exhibit each particular combination
of center and surround contrasts. To quantify this point, we plot the joint distribution of
center and surround mean contrasts in Fig. 4.1B. Again, every point represents a
particular combination of center and surround mean contrasts. However, different from
Fig. 4.1A, the gray level in Fig. 4.1B represents how many natural images had the
specified combination of contrasts. The main conclusion from this figure is that most
natural images have both low center mean contrast and low surround mean contrast. One
rarely observes a natural image with high center or surround contrast. These results are
consistent with those of previous studies (Field, 1987; Ruderman and Bialek, 1994;
Balboa and Grzywacz, 2003), but this figure extends the conclusion to contrasts in spatial
scales similar to a RGC’s RF.
Because the center mean contrast controls the cells’ responses, we further
quantified the dependence of responses on this variable. Figure 4.2 shows an example of
this quantification for the same cell as in Fig. 4.1. In this figure, small dots are the
original responses to 1,000 natural images, which produce the indicated mean contrast in
the RF center. Then, we divided the center mean contrasts into 20 equally spaced
intervals. We grouped images with center mean contrasts in these intervals to calculate
their mean responses. The mean responses are plotted as large dots. The trend of mean
responses shows that this cell’s response increases sigmoidally with the center mean
contrast of natural images. In addition, Fig. 4.2 displays the center-mean-contrast
distribution (dotted curve). Comparing this distribution with the response dependence on
contrast reveals that the majority of natural images cause low responses, and only a few
images lead to response saturation.

37

Figure 4.2. Responses of an Off brisk transient RGC to 1000 natural images as a function of the
center mean contrasts. Pairs of responses and center mean contrasts are plotted as the small dots.
We also separated the whole range of center mean contrasts into 20 equal intervals, and
calculated the mean responses (large dots). Finally, the dotted curve shows the distribution of
center mean contrasts for all images (right vertical axis). The RGC’s response increases
sigmoidally with the center mean contrasts of natural images, but the majority of them cause little
response and only rare images cause saturation.

We attempted to fit the sigmoidal behavior of the relationship between responses
and center mean contrasts with a cumulative Gaussian function (Eq. 2.5). The fit proved
to be good, capturing the mean response in each contrast interval (Fig. 4.3). Figure 4.3A
shows the same cell as in Fig. 4.2. Three other examples from Off transient, On-Off
transient, and Off transient cells are respectively shown in Fig. 4.3B-D. As in Fig. 4.2,
the dotted curves in Fig. 4.3 show the distribution of center mean contrasts in natural
images. Their peaks were located around -5% to 5%. In turn, the fitted “means” in the
cumulative Gaussian-distribution functions (Parameter M in Eq. 2.5) were around 25%.

38

Because about half of natural images produced negative center mean contrasts, the
percentage confirmed that cells showed little or no response to natural images.

Figure 4.3. Examples of four cells to illustrate the sigmoidal relationship between mean
responses and center mean contrasts of natural images, and fits of a cumulative-Gaussian-
distribution model (Eq. 2.5). These fits are plotted as solid curves. Large dots show the mean
responses in 20 separate intervals of contrasts. The distributions of center mean contrasts are
plotted as dotted curves. A. The same RGC as in Fig. 4.2. B. Example for an On-Off brisk
transient RGC. C. Example for an On-Off transient RGC. D. Example for an Off brisk transient
RGC. The cumulative-Gaussian-distribution model provides an excellent fit to the response-
contrast curve. The sign of center mean contrast is inverted for Off cells to simplify comparison
with the curves for On cells.

In conclusion, the main variable controlling the responses of RGCs to natural
images is the mean contrast in the RF centers and thus, as this tends to be low, these cells
produce a sparse coding of natural stimuli (Vinje and Gallant, 2000; Olshausen and Field,
2004).

39

4.2 Weak surround inhibition with natural stimuli
With artificial stimuli, contrasts in the RF surround tend to inhibit RGCs’ responses
(Enroth-Cugell and Robson, 1966; Merwine et al., 1995). For example, with annuli, an
80% surround contrast can reduce responses by factors of five or more in the rabbit retina
(Merwine et al., 1995). However, as already observed in Figs. 4.1 – 4.3 and elsewhere
(Introduction 1.1), contrasts in natural images are low. Therefore, one may expect that
stimulation with natural images has much less effect on the surround than typical
artificial stimulation does. Furthermore, averaging out luminance variation over the large
surrounds of RGCs (Fig. 3.4B) may exacerbate their weakness. Consequently, both low
natural-image contrasts and large areas of surround inhibition cause surround effects to
be weak. Figure 4.1A has already illustrated that surround inhibition is weak. For most
center mean contrasts, the figure reveals no effect of the surround. On the other hand, the
best example of a center mean contrast for which we observe an effect of surround is
20%. For this center mean contrast, responses in Fig. 4.1A have yellow and light-blue
colors for surround mean contrasts of -15% and 15%, respectively. This corresponds to a
fall of response by about a factor of two. By taking into account other center mean
contrasts, we expect the effect of the surround to reduce response on average by a much
smaller factor.
We quantified the strength of surround inhibition of each cell as the percentage of
response change over a subset of natural images in the input set. In measuring this
strength, we focused on center mean contrasts between 0 and 50%. The reason not to
pick higher center mean contrasts was that they happen only for a few images (Figs. 4.1 –

40

4.3). Hence, we did not have enough images with high center mean contrasts to obtain
statistically reliable estimates. We also did not use lower center mean contrasts, because
they produced small or no responses. With such small responses, we did not have a large
enough range to observe surround inhibition even if it existed. For these grouped images,
we then fitted the cell’s responses to surround mean contrast with a line. After fitting, we
calculated the maximal response and the response decrease for the observed surround
contrasts. The ratio between the response fall and the maximal response was estimated as
the inhibition strength of the surround. The result of this analysis over the population of
cells appears in Fig. 4.4. This figure includes the 39 RGCs for which we could determine
their inhibitory surround sizes (Fig. 3.4B). We did not include the 10 RGCs with
facilitatory surrounds.

Figure 4.4. Distribution of surround-inhibition strength elicited by natural images. This figure
includes only RGCs that exhibited the downward-upward surround trend (Fig. 3.2C). Most
natural images elicit surround-inhibition strength below 0.3 in RGCs.

41

Figure 4.4 shows that, for natural stimuli, surround inhibition reduced responses by
less than 30% on average. The median reduction was 15% for all the cells in this figure.
In comparison, the response reduction from the surround for artificial stimuli was around
83% (Merwine et al., 1995). Only three cells in Fig. 4.4 (8% of the population) showed
inhibition strengths comparable to those observed with artificial stimuli. We thus
conclude that surround inhibition estimated with artificial stimuli is often a gross over-
estimation of that present when using natural images.
4.3 Division-like and subtractive surround inhibition
Using annuli and spots, the effect of surround on the response-versus-center-contrast
curves of RGCs could be classified for the rabbit retina as one of three types (Enroth-
Cugell and Robson, 1966; Merwine et al., 1995). The first and most common type was
division-like, i.e., inhibition tended to divide the response-contrast curves by a factor
larger than one. The second and least common type was subtractive, i.e., inhibition
tended to shift the curves down by a constant. Finally, the third type was a mixture of
subtractive and division-like effects. These classifications brought forward the question
of what types of surround effects one could observe with natural stimuli. Answering this
question was hard, since inhibition was weak (Fig. 4.4). Nevertheless, we developed a
method to answer this question. In this method, we divided the whole data set for each
cell in Fig. 3.2C,D into two groups based on surround mean contrasts, one with low
contrasts (i.e., below the median contrast – usually negative) and one with high contrasts.
(One reason to divide the data in two groups was to get reliable statistical fits by
increasing the sample sizes. Another reason was that two-group separation was enough

42

to determine surround behavior.) Then, cumulative Gaussian distribution functions were
applied to the response-center-mean-contrast curves for each group. With two
cumulative Gaussian distribution functions estimated for low and high surround contrasts,
we could determine how surround shifted the curves. In particular, we could use the
pairs of functions to inquire if the effect of surround was subtractive or division-like.
Figure 4.5 shows examples of these pairs of functions for three RGCs.

Figure 4.5. Classification of surround effects into subtractive and division-like inhibition.
Whole data set such as in Fig. 4.1A was divided into two groups, one with the lower 50 percentile
of surround mean contrasts and one with the higher 50 percentile. Then, for each group, we fitted
a cumulative-Gaussian-distribution function to the response-center-mean-contrast curve (Fig. 4.3).
The best fit is shown as the solid curve for the group of low surround contrasts and as the dotted
curve for the groups of high surround contrasts. We classified the surround inhibition of RGCs
into three types: A. Out of 60 RGCs, 37 showed the division-like inhibition illustrated in Panel A
for an example Off brisk transient RGC. B. In contrast, 8 RGCs showed more complex, crossing
behavior illustrated in Panel B for an example On-Off sluggish transient RGC. In the text, we
explain that this behavior is consistent with subtractive inhibition. C. Finally, 15 RGCs could not
be classified, as their surrounds elicited too weak effects.

The solid curves in Fig. 4.5 represent the group of low surround contrasts. In turn,
the dotted curve represents the group of high surround contrasts. The three RGCs

43

displayed in Fig. 4.5 show the three kinds of behavior that we observed over the
population of RGCs. The behavior in Fig. 4.5A is typical of division-like inhibition
(Amthor and Grzywacz, 1991, 1993; Merwine et al., 1995). The slopes and saturation
level of the curve in Fig. 4.5A fall with increasing surround contrast. (We used the
bootstrapping technique to test whether the change of slope was significant. In our
application of this technique, we separated the whole data into 10 jackknife sets of 90%
data – Efron and Tibshirani, 1993; Zoubir, 1998.) On the other hand, the behavior in Fig.
4.5B is more complex and difficult to interpret. The slope of the curve in Fig. 4.5B
increases with surround stimulation, but the height of the function becomes lower,
making the curves cross. Hence, this cell had inhibitory and facilitatory surrounds at low
and high contrasts respectively. The reason this cell was inhibitory overall (it had the
behavior in Fig. 3.2C, and appeared in Fig. 3.4B and Fig. 4.4) is that most contrasts in
natural images are low. Finally, the behavior in Fig. 4.5C shows little change with the
increase of surround contrast. This behavior did not allow us to determine the functional
form of inhibition. This failure occurred, because the inhibition was so weak for some
cells (Fig. 4.4) that we could not reliably tell what its types was. A similar failure
happened when the available range of center mean contrasts was too narrow (a problem
for cells with large RFs).
For 15 out of the 60 RGCs for which we could establish an inhibitory surround
(behaviors as in Fig. 2.2C,D), we could not determine their inhibition types. Of the
remaining 45 cells, 37 (82%) had a behavior like that in Fig. 4.5A, i.e., division-like
inhibition. The other eight cells (18%) had the complex behavior of Fig. 4.5B. We will
argue later that this behavior shows signs of subtractive inhibition. Hence, the ratio

44

between the number of cells with division-like inhibition and those with subtractive
inhibition was almost 4:1.
With our dataset, about 20% of the cells displaying surround inhibition showed an
increase in the slope of the contrast-response curve with surround stimulation (Fig. 4.5B).
One cannot easily explain this type of crossing as subtractive or division-like based on
the models previously described (Amthor and Grzywacz, 1991, 1993; Merwine et al.,
1995). However, we now argue that a subtractive mechanism is involved. The increase
of slope was probably not due to mechanisms of surround excitation, because the
surround was inhibitory overall. The simplest alternative is that we failed to isolate the
inhibitory surround completely and caused the estimated surround to contain part of the
center. Thus, stimulation of this center portion would cause injection of some excitation
into the RGC, causing an increase in slope. The modulated response curve with an
increasing slope, however, crosses rather than sits above the control curve (Fig. 4.5B).
This crossing indicates that an inhibitory process lowered the modulated response curve.
This inhibition is not just division-like, because in that case, its effect would be to reduce
the slope back towards the control level or past it. With such reduction alone, the curves
should not just cross. We thus suggest that the crossing indicates a subtractive
component.
In our analysis, we described surround inhibition as subtractive or division-like (Fig.
4.5). Why did we not instead describe inhibition in terms of neural mechanisms?
Synaptic mechanisms, possibly GABAergic (Flores-Herr et al., 2001; Johnson and Vardi,
1998), appear to mediate inhibitory surrounds. Functionally, synaptic inhibition can be
shunting, hyperpolarizing, or both (Torre and Poggio, 1978; Amthor and Grzywacz,

45

1991). Shunting inhibition is division-like, whereas hyperpolarizing inhibition is usually
subtractive. One can identify surround inhibition as subtractive or division-like by
examining how the surround influences the center-mean-contrast-to-response function
(Amthor and Grzywacz, 1993; Merwine et al., 1995). However, this method cannot
discern whether a division-like mechanism is due to shunting inhibition. For example, an
exponential-like function of a subtraction can also be expressible as a division (Amthor
and Grzywacz, 1991). Hence, with our methods, we prefer to speak of subtractive and
division-like behaviors rather than of shunting or hyperpolarizing inhibition.

46

Chapter 5
Asymmetric Responses to the Onset and Offset of
Natural Images
5.1 Asymmetric responses to the onset and offset of natural images
One can determine the polarities of RGCs, On or Off, by their responses to black and
white stimuli. On cells respond well to the transition from black to white stimuli (Fig.
5.1A), but not to the opposite transition. Instead, Off cells respond well only to the
transition from white to black stimuli (Fig. 5.1E). Another approach to determine an
RGC’s polarity is to examine its linear RF. This linear-RF approach is more general and
one can apply it to many types of visual stimulus, including natural images. The linear
RFs of RGCs may be estimated by using STA (Methods 2.6.2).
Our main experiment determined two types of transitions in RGC’s responses to the
alternation of natural images and full-field, gray background. One transition was from
gray to natural images and we called it “onset.” The other was from natural images to
gray and we called it “offset.” Because in our experiments, the mean intensities of all
natural images were constant, the onset and offset of natural images did not cause any
change of overall mean luminance.
The post-stimulus response histograms of example On and Off cells to the
alternation of 1,000 natural images and gray background appear in Fig. 5.1 B,F. Because

47

the mean intensity of each natural image was the same as that of the full-field, gray
background and because the selection of natural images was random, the probability of
contrast increase in the RGC’s RF center was almost same as that of contrast decrease.
We thus expected that RGCs would show similar mean responses to the onset and offset
of natural images. However, we observed asymmetric responses for both On and Off
cells. On cells responded more strongly to the onset of natural images than to the offset.
In turn, Off cells showed similar asymmetries, but with the opposite polarity.
We also calculated the linear RFs of these On and Off cells separately for the onset
and offset of natural images. The linear RFs showed similar response asymmetries as the
post-stimulus response histograms. The magnitude of linear RF of the On cell was high
in response to the onset of natural images (Fig. 5.1C) but low for the offset (Fig. 5.1D).
Instead, the Off cell showed a large magnitude for the offset (Fig. 5.1H) but small for the
onset (Fig. 5.1G).

48

Figure 5.1. Asymmetric responses to the onset and offset of natural images in both histogram
and linear RF for an On brisk transient cell (A-D) and an Off brisk transient cell (E-H). A. The
post-stimulus time histogram of the On cell to alternating white and black full-field stimuli. B.
The post-stimulus time histogram to the alternation of 1,000 natural images and same-luminance,
full-field, gray backgrounds. C-D. The linear RFs of the On cell estimated from STA for the
onset and offset of natural images, respectively. E-H. Similar to A-D but for the Off cell.

49

The linear RFs for both the onset and offset of natural images were calculated for
96 cells, including 69 Off and 27 On RGCs. To visualize response asymmetries for all
cells, we plotted in Fig. 5.2A a point per RGC, with the peak magnitude of its onset linear
RF as abscissa and that for the offset as ordinate. We included in the plot a diagonal line
to indicate equal responses to the onset and offset. If RGCs responded symmetrically to
both onset and offset, the points should have had a symmetric distribution about the
diagonal line. However, 87 cells (91%) were below the equality line. Sixty-three of 69
Off cells had larger peak magnitudes in their offset linear RFs than in their onset
counterparts. In turn, 24 of 27 On cells had stronger magnitudes in their onset linear RFs.
Hence, RGCs showed significantly asymmetric responses to the onset and offset of
natural images, with the Off cell’s polarities being opposite to those of On cells.
To quantify the degree of an RGC’s asymmetry, we defined an asymmetry index as
the ratio of peak magnitudes of linear RFs for the onset and offset of natural images.
Because of the opposite response-asymmetry polarities of On cells and Off cells, we used
the ratio of the onset to offset for On cells, and the ratio of the offset to onset for Off cells.
The asymmetry indexes across the population of recorded RGCs are plotted
logarithmically (base 10) in Fig. 5.2B. Almost all RGCs showed asymmetry indexes
between 0 and 10, with a peak around two (0.3 in log scale).

50

Figure 5.2. RGCs’ response asymmetries across the population. A. Joint distribution of the peak
magnitudes of linear RFs for the onset and offset of natural images for 96 RGCs, including 69 Off
cells and 27 On cells. The two example cells in Fig. 5.1 are marked in red. The diagonal red line
indicates the theoretical equality of onset and offset responses. B. Histogram of the logarithm
(base 10) of asymmetry indexes across the population. Because of the opposite response-
asymmetry polarities of On cells and Off cells, we used the ratio of the onset to offset for On cells,
and the ratio of the offset to onset for Off cells. The asymmetry indexes tend to be positive with a
mode of about two (0.3 in log scale).

51

5.2 Response asymmetries are due to the asymmetry of intensity
distribution in natural images
Why did RGCs show asymmetric responses to the onset and offset of natural images?
The simplest hypothesis arose from the intensity distribution in natural images, which is
asymmetric about the mean (Nuala and David, 2000; Olshausen and Field, 2000;
Simoncelli and Olshausen, 2001). For example, the intensity distribution for the 4,000
natural images in our experiments is asymmetric, as shown in Fig. 5.3. More intensities
are below the mean than above it and those above the mean are often very bright.
Therefore, we decided to test whether the response asymmetry depended directly on the
properties of natural images.

Figure 5.3. Asymmetric distribution of pixel intensities in natural images. For 4,000 natural
images in our study, intensities varied from 0 to 5000 cd/m
2
. More intensities were below the
mean (vertical red line) than above it, and those above the mean were often very bright.

52

To test this natural-image-asymmetry hypothesis, we performed two control
experiments. One experiment was to use the negative of natural images to stimulate
RGCs. An original natural image (positive) and its negative appear in the top panels of
Fig. 5.4A,B, respectively. Because the negatives of natural images had an inverted
asymmetric intensity distribution about the mean intensity (compare the bottom panels of
Fig. 5.4A,B), our hypothesis predicted an inversion of the asymmetry of the RGCs’
responses. That was exactly what we observed. For example, consider the Off cell as
determined by alternating black and white stimuli (Fig. 5.5A). As expected, this cell
showed stronger responses to offsets of natural images than to onsets (Fig. 5.5B), and its
linear RF had a higher magnitude for the offset (Fig. 5.5E) than for the onset (Fig. 5.5D).
However, when stimulated with the negative of the natural images, both its histogram
(Fig. 5.5C) and linear RFs (Fig. 5.5F,G) were inverted. This inversion occurred across
our sample of 27 cells (Fig. 5.5H).

53

Figure 5.4. Examples of negative and histogram-equalized natural images. A,B. A natural
image and its negative (above) with their intensity distributions (below). C,D. A natural image
and its histogram-equalized transformation (above) with their intensity distributions (below).

54

Figure 5.5. Comparison of RGCs’ response asymmetries when using positive and negative
natural images. A,B,D,E. The same as Fig. 5.1E-H but for another Off brisk transient cell.
C,F,G. Similar to B, D, and E but for the negative of the same group of 1,000 natural images. H.
Distribution of response asymmetries from both the positive and negative natural images for a
sample of 27 RGCs. The data for the positive natural images are plotted as stars, while those
from the negative images are plotted as open circles. The example Off cell (A-G) is marked in
red. One observes the inversion of the response asymmetries when using the negative of natural
images.
The other control experiment to test the natural-image-asymmetry hypothesis used
histogram–equalized natural images to stimulate RGCs. Two of these images, one
normal and one histogram equalized, are shown in the top panels of Fig. 5.4C,D.
Because the histogram equalization eliminated the asymmetry of the intensity distribution

55

(bottom panels of Fig. 5.4C,D), our hypothesis predicted the elimination of the
asymmetry of RGCs’ responses. Again, that was exactly what happened. For example,
consider the Off cell in Fig. 5.6A. This cell’s responses to both positive and negative
histogram-equalized natural images are plotted in Fig. 5.6B,C. These responses did not
show any significant asymmetries. One could draw similar conclusions from this cell’s
linear RFs for both the onset (Fig. 5.6D,E) and offset (Fig. 5.6F,G) of natural images.
The same held for a sample of 29 cells (Fig. 5.6H).

Figure 5.6. Disappearance of RGCs’ response asymmetries when using histogram-equalized
natural images. This figure is like Fig. 5.5, except that for B-H, stimuli underwent histogram
equalization. One observes the elimination of response asymmetries when using the histogram-
equalized version of natural images.

56

Chapter 6
Dependence of RGC’s Responses on the Local
Visual Textures of Natural Images
What property of natural images underlies the asymmetric onset/offset responses of
RGCs? Our previous studies have shown that the most important stimulus variable
determining an RGC’s responses to natural images is the local temporal contrast in its RF
center (Chapter 4). Thus, the first candidate explanation for the asymmetric responses of
RGCs would be that the distribution of center mean contrasts of natural images is
asymmetric. However, this distribution is approximately symmetric, despite the
asymmetric distribution of intensities. As an example, we show the distribution of center
mean contrasts of 1,000 natural images for an Off cell (Fig. 6.1A). This distribution is
symmetric because the original (blue) and its inverted (red) curves overlap. Thus, the
property underlying the onset/offset asymmetry could not be the distribution of local
mean contrasts, as it was symmetric.
The simplest asymmetric alternative turns out to be standard deviation, a measure of
the strength of local texture. Studies have shown that the local mean and variance of
intensities in natural images exhibit a positive correlation (Simoncelli and Olshausen,
2001; Frazor and Geisler, 2006). Figure 6.1B illustrates this correlation in the joint
distribution of local intensity mean and standard deviation for a sample of 4,000 of our
natural images. In this figure, the correlation coefficient between the mean and standard

57

deviation is 0.65. Therefore, the distribution of standard deviations is asymmetric in
reference to the mean luminance.
To test whether the local visual texture had an effect on RGC’s responses, we had
to discount the responses accounted by the local temporal contrast first. To do so, we
used the center-mean-contrast-to-response model (Eq. 2.5). One example of the fit of this
model is shown in Fig. 6.1C. For this cell, the correlation coefficient between the data
and fitted model was 0.86. This high correlation coefficient confirms that the local
temporal contrast plays the most important role in determining an RGC’s responses to
natural images.
By discounting the model-predicted responses from the original responses, we
obtained the responses contributed by the intensity variation in the RF center, i.e., the
local visual texture. To quantify its contribution, we used the spike-triggered contrast
histogram (STCH — Methods 2.8). The STCH quantified how often a relative pixel
intensity (the horizontal axis of the histogram) contributed responses larger or smaller
that that predicted by the model. Figure 6.1D shows the STCH of a typical RGC. We
found that the expected relative responses for pixel intensities around the mean were
always negative, whereas the relative responses for those away from the mean were
always positive. Pixels near the mean are most common if images are homogeneous.
Hence, Fig. 6.1D shows that images with weak local texture tend to cause less response
than predicted by the local temporal mean contrast. In contrast, pixels away from the
mean are most common if images have strong local variation. Therefore, Fig. 6.1D
shows that images with strong local texture tend to cause more response than predicted
by the local temporal mean contrast.

58

In summary, RGC’s responses depended on the local visual textures of natural
images, with a preference for strong visual textures.

Figure 6.1. Dependence of RGC’s responses on the local visual textures of natural scenes. A.
Distribution of center mean contrasts of 1,000 natural images for an Off cell. The original curve
is shown in blue while the inversion of the curve is shown in red. That these curves overlap
indicates that the distribution is symmetric and thus, cannot cause the onset/offset asymmetry. B.
The joint distribution of mean and standard deviation of intensities in local circular areas from
4,000 natural images. The sizes of the local areas (approximately, 30-pixel radius) were equal to
that of an On cell’s RF. As the local mean intensity increases, so does the standard deviation,
yielding a positive correlation coefficient of 0.65. C. Center mean contrast–to–response model.
The blue dots represent the original responses of an Off-brisk-transient RGC, while the red
cumulative-Gaussian-distribution curve is the model fit (Eq. 2.5). The fit is excellent, yielding a
correlation coefficient of 0.86 between the data and model despite the large amount of noise.
(We inverted the sign of contrast in this graph to make it comparable for On and Off cells.) D.
Spike-triggered-contrast histogram (STCH). The horizontal axis represents the relative difference
of pixel intensities to the mean. The vertical axis represents the relative difference of response
from that predicted by the model in C. The relative responses are negative for intensities around
the mean and positive for those away from the mean. Consequently, because most intensities
near the mean typically came from relatively homogeneous images, these kinds of images yielded
less response than expected. In contrast, images with strong texture (those with large relative
difference of pixel intensities) yielded more responses than expected. Hence, the local texture
(i.e., inside the RF center) modulated responses significantly.

59

Based on the visual texture preference of RGCs’ responses, we explain why On and
Off RGCs had asymmetric responses to the onset and offset of natural images. Consider
On cells for example. On cells respond to contrast increase. Thus, for the onset of
natural images, only the images of positive center mean contrast could elicit the cell’s
responses. In return, for the offset of natural images, only the images of negative center
mean contrast could elicit the cell’s responses. Compared with the images with negative
contrast, the images with positive contrast had higher mean luminance, thus higher
variation or stronger textures (Fig. 6.1B). Strong textures contributed more responses
than weak textures if both had the same mean (Fig. 6.1D). Therefore, the natural images
with the positive contrast in their onset caused higher responses than those with the
negative in their offset. In other words, On cells preferred the onset of natural images to
their offset. Off cells can be explained similarly. Off cells respond to the contrast
decrease. Thus, only the images with positive contrast in the offset or the images with
negative contrast in the onset could elicit the cell’s responses. However, the natural
images with positive contrast in their offset would have higher luminance, or stronger
textures, thus higher responses. Therefore, Off cells preferred the offset of natural
images to the onset.

60

Chapter 7
Nonlinear-Linear Model
Nonlinearity has been found to predominate neural processing in the retina of many
species (Enroth-Cugell and Robson 1966; Caldwell and Daw, 1978; Shapley and Enroth-
Cugell,

1984; Allman, 1985; Merwine et al., 1995; Demb et al., 1999; Baccus and
Meister, 2002; Chichilnisky and Kalmar, 2002; Zaghloul et al., 2003). Much research
addressed this nonlinearity with a linear-nonlinear model (Shapley and Victor, 1978;
Reid et al., 1997; Meister and Berry, 1999; Chichilnisky, 2001; Kaplan and Benardete,
2001; Willmore and Smyth, 2003; Paninski et al., 2004; Carandini et al., 2005; Rust and
Movshon, 2005). In the model, a RGC’s responses were characterized as a weighted sum
of light stimuli in its receptive field, followed by a static nonlinear function. However,
no report has shown that this model could fit well to natural stimuli (Carandini et al.,
2005) despite its good prediction to artificial stimuli (Chichilnisky, 2001; Kim and Rieke,
2001; Zaghloul et al., 2003, 2005). One explanation is that the mean luminance and
luminance variation in natural scenes change very sharply. To successfully apply the LN
model to natural stimuli, one has to calibrate both its linear RF and nonlinearity for all
different luminances and contrasts (Carandini, 2005).
In addition, the STCH showed a RGC’s responses depended on the intensity
variation of local visual textures in the cell’s RF (Fig. 6.1D). Different relative intensities
had to be assigned different weights to account for their contributions for RGC’s
responses. The LN model with its linear RF as the first stage, however, cannot account

61

for this contribution. Its linear RF is used to weight different locations rather than
different intensities. To characterize RGC’s special response properties, we instead
proposed a nonlinear-linear model (NL). The crucial difference of the NL model from
the LN model is that the nonlinearities have to be put into the first stage in the RGC’s
visual processing. The nonlinearities could come from the outputs of bipolar cells or the
dendritic trees of RGCs (Koch et al., 1983; Burkhardt and Fahey, 1998; Dacey et al.,
2000; Euler and Masland, 2000; Wu et al., 2000; Demb et al., 2001).
For simplicity, let us assume that a retinal ganglion cell connects to only two
bipolar cells (Fig. 7.1B,C), and the stimulus image includes only two pixels. Each
bipolar cell responds to one pixel of the image with a “threshold-like” nonlinear function
(Fig. 7.1A), and the output of this ganglion cell is equal to the sum of the outputs of these
two bipolar cells. With the threshold-like nonlinearity, the response increase at the
output of a bipolar cell will be multiple times the increase of stimuli intensity. Let us
compare two situations. One is that an image has a strong visual texture (Fig. 7.1B). It is
represented by two pixel intensities far away from the mean, shown as the blue and red
dots. The other is that an image has a weak visual texture (Fig. 7.1C). It is represented
by two pixels intensities close to the mean, shown as the green and yellow dots. The
means of the two pixels in both situations are equal, evenly distributed from the mean
(red vertical line) for two pixels in each situation (Fig. 7.1A). Because of the nonlinearity,
the bipolar-cell’s response to the blue pixel is slightly smaller than that to the green pixel,
but its response to the red pixel is much larger than that to the yellow pixel (Fig. 7.1A).
As a result, the added response for the blue and red pixels (Fig. 7.1B) is higher than that
for the green and yellow pixels, even if both have the same mean (Fig. 7.1C). Thus, the

62

RGC shows higher response to the image with strong visual texture than to the image
with weak visual texture.

Figure 7.1. Nonlinear-linear model. A nonlinear-linear model was proposed based on the
nonlinearity of the output of bipolar cells or the inputs at the dendritic tree of the ganglion cell.
The nonlinearity can be characterized as a “threshold-like” function (A). With the same mean
luminance, two different visual textures, respectively strong (B) and weak (C), are represented
respectively by two dots away from the mean (blue and red – B) and close to the mean (green and
yellow – C). Because of the nonlinearity of a threshold-like function, the bipolar cell response to
the blue pixel is slightly smaller than that to the green pixel, but the response to the red pixel is
much larger than that to the yellow pixel. As a result, the response sum for the blue and red
pixels (B) is higher than the sum for the green and yellow pixels (C). Thus, the RGC showed
higher responses to strong visual texture than to weak visual texture.

63

We finally tested the validity of this nonlinear-linear model with a sequence of
plaids. These plaids were constructed with a very high resolution, including only two
different pixel intensities. To simulate the effect illustrated in Fig. 7.1, the mean of these
two pixel intensities was fixed the same as the gray background while their difference
varied from small to large. Thus these two pixel intensities, in reference to the gray
background, had exactly the same contrast but opposite signs: one is positive and the
other is negative. The purpose to construct high-resolution plaids with only two
opposite-contrast intensities was to enforce a zero for the linear summation of these
plaids with any linear RFs if the first stage of visual processing would be linear, such as
the LN model. In addition, using these plaids also helped to eliminate the response
contribution from the intensity mean and to keep the contribution only from the intensity
variation. Two example plaids, respectively representing weak visual texture and strong
visual texture, are respectively shown in Fig. 7.2C,D. Each pixel had a size 150 μm by
150 μm.
Both the positive and negative plaids with varied differences of two intensities were
alternated with the full-field gray background. The responses of an example RGC are
shown in Fig. 7.2A. First, expected from the nonlinear-linear model, the RGC’s
responses increased as the difference of pixel intensities increased. Thus, the RGC
showed a response dependence on the visual texture of stimuli. In addition, we observed
that RGC’s responses to the positive plaids (red) were roughly equal to its responses to
their negative (blue). Both the above response dependence and response equality showed
that the responses of this RGC were completely contributed by the intensity variation of
the plaids, rather than by any inadvertently manipulated mean bias. Any LN model

64

would fail to quantify these response properties, no matter how its linear and nonlinear
functions were selected.
The above properties of RGC’s responses generally held for the plaids of varied
high resolutions, including 100, 150, 200 or 300 μm for each pixel. However, when we
increased the plaid’s resolution to 50 μm for each pixel, we found that this RGC showed
no responses to either weak or strong visual textures (Fig. 7.2B). This result is predicted
by our NL model. In the NL model, the nonlinearities come from the outputs of bipolar
cells or the dendritic trees of a retinal ganglion cell, and it is appropriate to model the
bipolar-cell responses as a linear-nonlinear function. With extremely high-resolution
plaids, the RF of each bipolar cell, although smaller than the RGC’s RF, would also
contain enough opposite-contrast pixels. Their linear summation would be zero, and thus
cause no response at the outputs of bipolar cells and subsequently no response at the
output of the RGC.

65

Figure 7.2. A validity test of the nonlinear-linear model with high-resolution plaids. Each plaid
contained only two different intensities with opposite contrasts. Three plaid examples are shown
respectively with weak visual texture (C), strong visual texture (D), and extremely-high-
resolution visual texture (E). The pixel size is 150 μm by 150 μm for the first two plaids (C,D),
and 50 μm by 50 μm for the third plaid (E). Expected from the nonlinear-linear model, RGC’s
responses increased as the increase of pixel-intensity difference in stimuli, and their responses to
the onset of both the plaids and their negative were equal (A). When the plaid resolution was
extremely high (E), RGC’s responses were zero no matter whether the stimuli had strong or weak
visual texture (B). The zero response in plaids of extremely high resolution implies that a bipolar
cell can be modeled as a linear-nonlinear model.

66

Chapter 8
Adaptation of RGCs to Natural-Image Contrast
8.1 Multiple temporal components in RGC’s adaptation to natural-
image contrast
With artificial stimuli, RGCs show adaptation to both the temporal and spatial contrast of
artificial stimuli (Sakai et al., 1995; Smirnakis et al., 1997; Kim and Rieke, 2001), in both
center and surround of their receptive fields (Brown and Masland, 2001; Solomon et al.
2006). Generally, after the mean contrast of stimuli increases, the sensitivity of an RGC
falls with different time constants. The fastest time constant corresponds to an
“instantaneous” adjustment of sensitivity, occurring within 100 ms after the contrast
increase. This adjustment was termed “contrast gain control” (Shapley and Victor, 1978;
1979; Enroth-Cugell and Jakiela, 1980; Shapley and Enroth-Cugell, 1984; Victor, 1987;
Benardete et al., 1992; Sakai et al., 1995; Zaghloul et al., 2005). A second, much slower
adjustment extends seconds or tens of seconds, and one cannot describe it with a single
exponential function (Smirnakis et al., 1997; Brown and Masland, 2001; Chander and
Chichilnisky, 2001; Kim and Rieke, 2001; Baccus and Meister, 2002; Solomon et al.,
2004). This adaptation can be separated into two distinct kinetic components (Kim and
Rieke, 2001). One is a fast-decaying component, occurring with a time scale of <2 s,
which we termed fast contrast adaptation. The other is a slowly-decaying component
with a time scale >10 s, which we called slow contrast adaptation.

67

Having known the RGC’s contrast adaptation to artificial stimuli, we wanted to
know whether it also adapts to contrasts as distributed in natural images. The contrast
adaptation of two RGCs to both flickering checkerboards and natural images are shown
in Fig. 8.1. Similar to their contrast adaptation with checkerboards (Fig. 8.1B,C), RGCs
also adapted to the contrast of natural images (Fig. 8.1D,E). After a step mean-contrast
increase from low to high, for example, from 0.05-0.25 to 0.52-0.85 for natural images or
from 0.09 to 0.62 for checkerboards, the mean response of an RGC first increased
abruptly, then had an “immediate” decay from its peak, and finally decayed slowly to a
lower, sustained plateau. After a step mean-contrast decrease from high to low, the mean
response first dropped abruptly and then gradually recovered to a new sustained plateau.

68

Figure 8.1. Contrast adaptation of RGCs to both checkerboards and natural images. A. Twenty
image examples (for checkerboards, low contrasts were 0.09 and high contrasts were 0.62; for
natural images, low contrasts ranged from 0.05 to 0.25 and high contrasts ranged from 0.52 to
0.85. B,C. Contrast adaptation of an RGC to the step mean-contrast increase (at t = 30 s) and
decrease (at t = 0 s) change of flickering checkerboards, plotted with two different time bin
widths: 1.5 s (B) and 0.3 s (C). D,E. Contrast adaptation of an RGC to the step mean-contrast
increase (at t = 30 s) and decrease (at t = 0 s) change of natural images, plotted in two different
time bin widths: 1.5 s (D) and 0.3 s (E).

69

Totally, three temporal components with different time scales in RGC’s adaptation
were observed for the contrast increase of natural images. The fastest component is
contrast gain control. As shown in the time bin width 0.3 s (Fig. 8.2A), the RGC’s
response fell a huge amount from the first time bin to the second. This fall of response
did not disappear even when a smaller time bin width 0.1 s was used (Fig. 8.2B),
implying that this contrast-gain-control component had a time constant <0.1 s.
Note that, although some other studies had tried to analyze this component by using
the linear-nonlinear model (Smirnakis et al., 1997; Brown and Masland, 2001; Chander
and Chichilnisky, 2001), we did not apply this model because it proved to be invalid for
natural stimuli (Carandini et al., 2005). However, we could not analyze the time scale of
this component with our mixture-of-exponentials model, but we still termed it as contrast
gain control based on its time scale and our observations with artificial stimuli (Shapley
and Victor, 1978; 1979; Shapley and Enroth-Cugell, 1984; Benardete et al., 1992; Sakai
et al., 1995; Zaghloul et al., 2005). In addition, some studies called this component fast
adaptation (Victor, 1987; Chander and Chichinlsky, 2001), but in our study, the term ‘fast
adaptation’ is used elsewhere. Note that, if we did not observe a huge response fall from
the first bin in the small time bin width, it does not rule out a contrast-gain-control
component in this cell. Because if the contrast gain control was very weak, we would not
observe it with our method.
When this RGC’s response was replotted in a time bin width of 1.5 s (Fig. 8.2C), a
component of continual response decay across the entire time course was observed. The
trend of responses after taking a natural logarithm demonstrates a much slower temporal

70

component in RGC’s contrast adaptation (Fig. 8.2E). This temporal component was slow
adaptation. Two different fitting techniques, resistant-line-from-three-groups fitting and
least-square fitting (Methods 2.10), were used to estimate the time constant of this
temporal component, shown respectively as a straight red line and a dash-dot red line.
The resistant-line-from-three-groups fitting proved to be more robust to the outlier point
circled with the red ring. The predicted responses from this one-term exponential model
with only slow adaptation are plotted as the red curve in Fig. 8.2C. The time constant of
this component is 20.4 s.
Although this one-term exponential accounted for most of the cell’s response decay,
it failed to capture the fast decay in the first six seconds, indicated by a more and more
large difference between the actual response and the fitted response as the time
approaches to the start point of adaptation. Clearly, there was a faster temporal
component in this RGC’s contrast adaptation. We thus used a two-term exponential
function to model the entire time course. After subtracting the predicted responses
contributed by the slow-adaptation component from the actual responses, the relative
responses after logarithm were replotted in Fig. 8.2F. A least-square fit is shown as the
red straight line, and the time constant of this component is 1.95 s. The fitted responses
from the two-term exponential function based on both slow and fast adaptation are shown
as the red curve in Fig. 8.2D. This two-term exponential function accounted very well
the cell’s response decay after a step contrast increase of natural images, having a
correlation coefficient of 0.96 between the fitted and actual responses. Note that some
other studies did not separate these two components, fast and slow contrast adaptation,

71

but mixed them into only slow adaptation (Smirnakis et al., 1997; Brown and Masland,
2001; Chander and Chichilnisky, 2001; Baccus and Meister, 2002).
However, for those RGCs showing well-observed contrast adaptation to the step
contrast increase of natural images, not all of them had a clear recovery time course to the
step contrast decrease. For example, we did not observe a significant response recovery
for the cell shown in Fig. 8.2A-F. In addition, different from the three-temporal-
component decay found in some RGCs’ adaptation to a step contrast increase, the
response recovery to a step contrast decrease, if present, always had only one temporal
component. An example is shown in Fig. 8.2G,H. The after-logarithm responses to the
contrast decrease were well fitted with a straight line. Thus, a one-term exponential
function was sufficient to model the entire time course of the response recovery. The
predicted response curve with this one-term exponential function is shown as the red
curve in Fig. 8.2G, having a correlation coefficient of 0.91 for the fit. The time constant
for this example cell was 14.8 s.

72

Figure 8.2. Multiple temporal components in RGC’s contrast adaptation to the contrast change
of natural images. A-D. The mean responses of an RGC to the contrast change of natural images
in 0.3-s (A), 0.1-s (B), and 1.5-s (C-D) time bins. The huge response difference between the first
and second time bin (A,B) represented the contrast gain control. The red curve in (C) was fitted
by a one-term exponential function from slow adaptation, and the red curve in (D) was fitted by a
two-term exponential function from both the fast and slow adaptation. E. After-logarithm RGC’s
responses to step contrast increase, respectively fitted by robust resistant-line-from-three-groups
(solid red line) method and least-square method (dash-dot red line). Only the data after the first
six seconds were used to fit. The robust fit proved less affected by the outlier (marked by red
circle). F. After-logarithm responses after subtracting the responses from slow-adaptation. The
red line is a least-square fit to the data in the first six seconds. G. Contrast adaptation of an RGC
to step contrast decrease of natural images. The time bin width is 1.5 s, and the fitted responses
based on a one-term exponential function is shown as the red curve. H. After-logarithm
responses to the contrast decrease, and a fit based on the least-square is shown as the red line.

73

8.2 Diversity of RGC’s contrast adaptation to natural stimuli
In total, 125 RGCs were examined for their temporal components in response to the
contrast change of natural images. We found that 43 cells (34.4%) did not show any
adaptation. Of them, 10 cells did not even demonstrate a response difference between the
low and high contrast; the other 33 cells illustrated the response difference, but we did
not observe a clear contrast adaptation.
Eighty-two cells (65.6%) showed adaptation to a step contrast increase. Of them,
twenty cells’ temporal-component types were difficult to determine because of the
response noise. We thus ruled out these cells from further quantitative analyses. Of the
remaining 62 cells, three different contrast-adaptation types were observed: 14 cells
(22.6%) showed only the fast component, which we termed fast-only adaptation (Fig.
8.3A); 25 cells (40.3%) showed only the slow component, which we termed slow-only
adaptation (Fig. 8.3B); finally, 23 cells (37.1%) had both fast and slow components,
which we termed fast-and-slow adaptation (Fig. 8.3C).
To a step contrast decrease, however, only one contrast-adaptation type was
observed from 24 RGCs (19.2%), with only a slow component in their response recovery
(Fig. 8.2G,F). All these cells also demonstrated an adaptation to the step contrast
increase.

74

Figure 8.3. Diversity of RGC’s contrast adaptation to natural stimuli. Top row shows the mean
responses in 1.5-s time bins, and the red curve is the fit by a mixture-of-exponentials model;
bottom row displays after-logarithm responses, and the red line is the fit by a multi-segment
linear function (method 2.10). A. Fast-only adaptation. Of the 62 classified cells, 14 cells had
only fast components. B. Slow-only adaptation. 25 cells had only slow components. C. Fast-
and-slow adaptation. 23 cells had both fast and slow components.

The distribution of time constants for the four classified contrast-adaptation types
described above is shown in Fig. 8.4. Generally, the time constants for the RGCs with
fast-only adaptation varied from 1 to 6 s (Fig. 8.4A). The time constants for the RGCs
with slow-only adaptation varied from 10 to 50 s with a mode around 17 s (Fig. 8.4B).
For the RGCs with fast-and-slow adaptation (Fig. 8.4C), the time constants of slow
component were similar to those found in RGCs with slow-only adaptation (Fig. 8.4B),
but the time constants of fast component were slightly smaller than those in fast-only
adaptation (Fig. 8.4A). One explanation of this slight difference of time constant is that

75

there might be some weak slow-adaptation components in those RGCs with fast-only
adaptation, which were not eliminated, and thus would contaminate the estimate of the
fast-component time constants. In addition, no strong correlation (only 0.09) was found
between the time constants of fast and slow components, implying that fast and slow
contrast adaptation might have different biophysical mechanisms. Finally, for the RGCs
with contrast adaptation to a step contrast decrease, their time constants varied from 10 to
45 s with a mode around 20 s (Fig. 8.4D).

Figure 8.4. Distribution of time constants for four different types of contrast adaptation. A.
Time constants of fast-only adaptation, varying from 1 to 6 s. B. Time constants of slow-only
adaptation, varying from 0 to 50 s with a mode around 17 s. C. Time constants of fast-and-slow
adaptation. The time constants of their slow components were similar to those in slow-only
adaptation (B), and the time constants of their fast components were slightly smaller than those in
fast-only adaptation (A). D. Time constants of RGCs with contrast adaptation to step contrast
decrease, varying from 10 to 45 s with a mode around 20 s.

76

Different types of RGC’s contrast adaptation to the contrast change of natural
images were examined in terms of cells’ polarities: On versus Off. Totally 125 cells were
examined, including 38 On cells, 50 Off cells, 32 On-Off cells, and 5 unclassified cells.
The distribution of the contrast-adaptation types by cell polarities (On, Off, or On-Off) is
summarized in Table 9.1. Five groups were tested, including having adaptation or not
(group 1), having fast adaptation or not (group 2), having slow adaptation or not (group
3), having fast-and-slow adaptation or not (group 4), and having adaptation to contrast
decrease or not (group 5). Note that both fast-only adaptation and fast-and-slow
adaptation have the component of fast adaptation. We found that Off cells (20 from 25
cells) had a higher percentage of showing fast adaptation than On cells (7 from 19 cells).
Their difference was statistically significant (χ2 test, p<0.01), as marked as star. No other
groups were found to have a significant difference (p<0.01) between On and Off cells.

77

Group Type On Off On-Off
1
Adapt 22 33 25
Non-adapt 16 17 7
2
Fast 7 * 20 * 10
Non-Fast 12 * 5 * 7
3
Slow 15 17 15
Non-slow 4 8 2
4
Fast and slow 3 12 8
Non-FaSl 16 13 9
5
Contrast Decrease 3 11 7
Non-contDecr 35 39 25

Table 9.1. Distribution of different contrast-adaptation types in terms of RGCs’ polarities (On,
Off and On-Off). Five groups were tested, including having adaptation or not, having fast
adaptation or not, having slow adaptation or not, having fast-and-slow adaptation or not, and
having adaptation to contrast decrease or not. Statistically, Off cells had a higher percentage (20
from 25 cells) of showing fast adaptation than On cells (7 from 19 cells) with p<0.01 (χ2 test).

Different types of RGC’s contrast adaptation were also inspected in terms of the
basic temporal properties of cells’ responses to full-field black and white stimuli:
transient versus sustained. With total 96 transient cells and 23 sustained cells, the
distribution of different contrast-adaptation types by their temporal response properties is
summarized in Table 9.2. Six cells’ temporal properties could not be determined because
of the noise of response. We found that transient cells (69 from 96 cells) had a
statistically significantly higher possibility (χ2 test, p<0.01) of showing contrast

78

adaptation than sustained cells (10 from 23 cells: 43.5%), as marked in star. No other
groups were found to have a significant difference between transient and sustained cells.

Group Types Trans Sustain Unsure
1
Adapt 69 * 10 * 3
Non-adapt 27 * 13 * 3
2
Fast 31 5 1
Non-fast 21 3 1
3
Slow 39 7 2
Non-slow 13 1 0
4
Fast and slow 18 4 1
Non FaSl 34 4 1
5
Contrast Decrease 18 3 3
Non-contDecr 78 20 3

Table 9.2. Distribution of different contrast-adaptation types in terms of RGCs’ basic temporal
properties (transient and sustained). Similar to table 9.1, five groups were tested. Statistically,
transient cells had a higher percentage of showing contrast adaptation than sustained cells
(χ2 test, P<0.01).

79

Chapter 9
Dependence of RGC’s Contrast Adaptation on the
Contrast Difference of Natural Image
9.1 Dependence of RGC’s responses on the contrast difference of
natural stimuli
Having observed a diversity of contrast adaptations across the population of RGCs, we
next wanted to answer whether contrast adaptations were dependent on the contrast
difference of the natural images. Natural images with specific contrasts were created as
described in Methods 2.3. Five examples of natural images under four different contrasts,
such as 0.05, 0.25, 0.45, and 0.70, are plotted in Fig. 9.1A. All twenty images had the
same mean luminance.
The contrast adaptation of an RGC in three kinds of contrast difference, including
from 0.05 to 0.25 (small difference), from 0.05 to 0.45 (medium difference), and from
0.05 to 0.70 (large difference), are plotted in Fig. 9.1B with two different time bin widths.
As the contrast difference increased from small to large, both the peak and plateau
response, which followed the step contrast increase of natural images, also increased, i.e.,
respectively from 53 to 64 spikes and from 40 to 44 spikes when in 1.5-s time bins (left).
In contrast, the trough response, which followed the step contrast decrease of natural
images, decreased from 27 to 20 spikes. The above trend was also evident when plotted

80

in 0.3-s time bins (Fig. 9.1B).

Figure 9.1. Contrast adaptation of an RGC to three different natural-image contrast steps. A.
Five natural-image examples with the same mean luminance under four different contrasts,
respectively 0.05, 0.25, 0.45, and 0.70. B. Contrast adaptations of an RGC to three different
contrast steps of natural image, including from 0.05 to 0.25 (small), from 0.05 to 0.45 (medium),
and from 0.05 to 0.70 (large), plotted in two kinds of time bins, 1.5 s (left) and 0.3 s (right).

81

To quantify the dependence of RGC’s contrast adaptation on the contrast difference,
we first examined their peak and plateau responses, which were estimated based on the
mixture-of-exponentials model (Methods 2.9). We also calculated the adaptation index,
defined as the ratio of peak response to plateau responses. For comparison, all
parameters in three different contrast steps (small, medium, and large) were normalized
by their estimates in the small contrast difference (from 0.05 to 0.25), and then a natural
logarithm was taken to compress their ranges. Thus, each parameter in the small contrast
difference (from 0.05 to 0.25) was constantly zero after normalization and the logarithm
taken (Fig. 9.2A). Overall, 35 RGCs were examined. We found that, as the contrast
difference increased, statistically both the peak response (Fig. 9.2A, left) and plateau
response (Fig. 9.2B, left) increased. This trend is also demonstrated by the increased
mean responses (red line) on the increased contrast differences.
We also determined the trend of each cell’s responses over theses three different
contrast steps, and counted the sign of this trend: positive or negative (Fig. 9.2A,B, right).
In terms of the peak response, thirty-four of 35 cells (97%) showed positive trends and
only one cell (3%) showed negative trends; with respect to the plateau responses, 33 cells
(94%) showed positive trends and only 2 cells (6%) showed negative trends. The
dominance of positive trends over negative trends also demonstrated that both the peak
and plateau responses increased as the contrast difference increased. However, for
adaptation index (Fig. 9.2C), no significant positive or negative trend was found across
the population. Twenty-two cells (63%) showed positive trends and 13 cells (37%)
showed negative trends. Compared with the dominance of positive trends shown in peak

82

and plateau responses, we conclude that the adaptation indexes statistically keep
unchanged as the contrast difference of natural images increases. The adaptation indexes
for all 35 RGCs in the medium contrast increase (from 0.05 to 0.45) are plotted in Fig.
9.2D. The adaptation indexes varied from 1 to 3 with a mode of 1.5.

Figure 9.2. Dependence of RGCs’ responses on the contrast difference of natural images. A.
Left side: peak responses of 35 RGCs over three different contrast steps, respectively small,
medium, and large, after normalized by the estimates in the small contrast step and taken a
natural-logarithm function. The red line is the mean responses; right side: statistics of signs of
response trends, positive or negative, over increased contrast steps for all RGCs. B. Similar to
(A), but for plateau responses. C. Similar to (A) but for adaptation indexes. As the contrast
difference increased, both the peak and plateau responses increased, but the adaptation indexes
kept unchanged. D. Histogram of adaptation indexes in the medium contrast increase from 0.05

83

9.2 Dependence of RGC’s contrast adaptation on the contrast
difference of natural stimuli
We observed two different types of contrast adaptation based on how RGCs responded to
different contrast steps of natural images. One type was that RGCs showed contrast
adaptation to all three different contrast steps (small, medium and large). An example is
shown in Fig. 9.3A. This RGC had a similar temporal component in its contrast
adaptation to all three different contrast steps of natural images, from large contrast
difference to small contrast difference, although its peak response, plateau response, and
time constant were slightly different from each other. Overall 24 cells (69%) had this
type of contrast adaptation. The other was that RGCs showed contrast adaptation only to
large contrast differences. In other words, this type of RGC would lose their fast or slow
component when the contrast difference of natural images was small. One example is
plotted in Fig. 9.3B. A clear contrast adaptation was observed from this RGC to the
medium and large contrast differences but not to the small difference. Totally 11 cells
(31%) lost their contrast adaptations to the small contrast difference. We thus conclude
that the contrast adaptations of RGCs are dependent on the contrast difference of natural
images.

84

Figure 9.3. Dependence of the appearance of RGC’s contrast adaptations on the contrast
difference of natural images. A. Twenty-four RGCs showed contrast adaptation to all contrast
steps, from the small difference to the large difference. B. Eleven RGCs lost their contrast-
adaptation components, fast or slow component, to the small contrast difference.

We finally examined the time constants of the 24 cells that showed adaptation to the
above three different contrast steps. Only their slow components were analyzed. Their
time constants in three different contrast steps, after normalized by the estimate in the
large contrast difference and taken a natural logarithm (similar to Fig. 9.2), are plotted in
Fig. 9.4A. The mean time constant is shown in red. We found that the time constants of

85

the majority of RGCs decreased as the contrast difference increased. Overall 19 cells
(79%) showed negative trends, and only 5 cells (21%) showed positive trends (Fig. 9.4B).

Figure 9.4. Dependence of the time constant of RGC’s contrast adaptations on the contrast
difference of natural images. A. The time constant of slow components for RGCs’ contrast
adaptations to three different contrast increases: small (from 0.05 to 0.25), medium (from 0.05 to
0.45), and large (from 0.05 to 0.70) after normalized to the estimate of the large contrast step and
taken a natural logarithm function. B. statistics of signs of time-constant trends, positive or
negative, over increased contrast steps for a sample of 24 RGCs.

86

Chapter 10
Conclusions
10.1 Estimation of RF with natural-image stimulation
The RF center, also called the classical RF, is defined as the region of visual space from
where one can elicit responses with light stimuli (Kuffler, 1953). Although this
definition is clear, the estimation of the RF center is dependent on the stimuli used. For
instance, Barlow et al. (1957) showed that the inhibitory surround disappears at low mean
background intensities.
In this study, we first developed a technique to estimate RF center boundaries. The
main assumption of our technique is that the RF center dominates responses with
relatively weak modulation from the surround. This assumption was corroborated by our
experiments (Fig. 4.1 and Fig. 4.4). Another assumption is that the RFs of ganglion cells
are roughly circular, which seems to be correct (Kuffler, 1953; Rodieck, 1965; Levick,
1967). With these assumptions, our method yielded a center size similar to that captured
by the traditional method with moving square-wave gratings (Fig. 3.3 and Fig. 3.4A). In
contrast, STA tended to over-estimate the size of the RF center. One possible
explanation for this over-estimation is that the correlation between the RF center and its
neighboring area in natural scenes may tend to make the neighboring area artifactually
provide a positive contribution to RGCs' responses. Our method avoids this artifact. The

87

success of this method suggests that we can redefine the RF as the region where one
obtains the maximal correlation between responses and mean contrasts.
We also devised a method to estimate the RF surround size by seeking those areas
outside the center that statistically modulate an RGC's responses, typically in an
inhibitory manner (Fig. 2.2E-H). An alternate method would have been STA if, for
example, the interaction between the RF center and surround were linear (e.g., a
difference-of-Gaussians), followed by a static nonlinearity (Rodieck and Stone, 1965;
Enroth-Cugell and Robson, 1966). However, the center-surround interaction is nonlinear
(Enroth-Cugell and Robson, 1966; Carandini and Heeger, 1994; Merwine et al., 1995).
Our method does not need to assume a linear interaction between center and surround.
However, our technique also has some limitations. First, our method has to assume that
the surround is circular, which is only an approximation of the shape of RGC dendritic
trees (Rockhill et al., 2002). Second, we estimate only the size of the RF center and
surround, rather than the RF itself as STA does. Third, our method ignores the RF spatial
weighting function, which may also potentially cause a bias in our estimation.
10.2 Strong center excitation and weak surround inhibition with
natural-image stimulation
When RGCs were stimulated with natural images, most natural images caused little or no
responses, and only rare natural images had sufficiently high center mean contrasts to
cause cells’ responses to saturate (Figs. 4.1 – 4.3). The retina thus produces a sparse
coding of natural stimuli (Vinje and Gallant, 2000; Olshausen and Field, 2004).

88

In addition, the RF center dominated RGCs’ responses and the surround inhibition
elicited by natural images was weak. For the seventy cells for which we could classify
the surround (Fig. 3.2C-E), sixty showed inhibition and only ten showed excitation. The
ratio of RGCs exhibiting an inhibitory surround to those exhibiting a facilitatory surround
was approximately 6:1. Reduction of responses due to inhibition was typically below
30% (Fig. 4.4). This is much less than the values of 80% or more obtained with artificial
annuli (Merwine et al., 1995). Such weak inhibition with natural images explains why
the RF center dominated cells’ responses (Fig. 4.1). A caveat to the weak-inhibition
results is that our method allows studying mechanisms of surround inhibition only
outside the RF center. Therefore, strong inhibitory mechanisms may still exist if they are
hidden in the center. Nevertheless, we emphasize that our conclusion of much weaker
inhibition with natural images than with artificial stimuli holds, because we compare both
outside the RF center (Merwine et al., 1995).
Why is the surround inhibition with natural images weak? One possibility to
consider is that perhaps our experiments used relatively low mean luminance. Barlow et
al. (1957) showed that surround inhibition becomes weak and even disappears when the
mean luminance falls. However, our stimulation setup was similar to that of Merwine et
al. (1995). Their studies showed with artificial annuli that a strong surround inhibition
was elicited using a similar mean luminance level (of about 10 cd/m
2
). Hence, our
photopic background illumination was high enough to elicit strong surround inhibition.
An alternative was because we presented stationary images. Our experiments simulated
something similar to saccades in the sense that stationary images appeared suddenly.

89

Will RGCs show stronger inhibition when stimulated with natural movies rather with
saccade-like stimuli? Although we cannot presently rule out this possibility, we argue
against it. Our experiments used similar stimulation protocols as Merwine et al. (1995),
whose only difference was to show stationary annuli instead of stationary natural images.
They got strong surround inhibition with their stationary stimuli, indicating that motion is
not necessary for strong surround inhibition.
Consequently, we looked for an explanation for the weakness of surround inhibition
that had to do with the statistics of natural images rather than with our experimental
conditions. We propose that the weakness of surround inhibition with natural images was
simply due to the surround area of the RF seeing a preponderance of low contrasts
(Ruderman and Bialek, 1994; Balboa and Grzywacz, 2003). In addition, surround
regions were large, and thus saw much visual texture in natural images. The dark and
bright intensities in these textures tended to cancel out, thus causing even lower mean
contrasts in the surround than in the center. Low contrast makes surround inhibition
weak (Barlow et al. 1957; Merwine et al., 1995; Devries and Baylor, 1997). Therefore,
in natural-image stimulation, most cells showed a weak surround inhibition. The
mechanism of surround itself was not weak, as shown by experiments with artificial
stimuli. However, in natural conditions, the surround is weakly, or less effectively
stimulated than in artificial conditions.
This explanation for the weakness of surround inhibition might seem disappointing,
because it would not teach us anything new about retinal mechanisms. The explanation
might not be even surprising at face value, as we knew that natural contrasts were low
and surround size large. However, we argue that knowing that surround inhibition is

90

typically weak has profound implications for the understanding of early vision. All of the
functions proposed for surround inhibition so far have been based on it being strong.
These proposals include the removal of correlation in the input stimuli (Atick, 1992),
predictive coding (Srinivasan et al., 1982), and detection of sharp image transitions
(Ratliff, 1965). None of these proposals works well if surround inhibition is weak. As an
alternative, we propose that the weakness of the surround, although not optimal for the
above visual functions, may be necessary for maintaining information about the absolute
reflectance of large objects or regions in scenes. A strong surround, however, would
make points in these large regions inhibit mutually, causing responses to become sub-
threshold. As a minimum, we would have to modify the theories above, developed by
assuming strong surround inhibition, to allow for compromises imposed by it being weak.
Another compromise may be to take both energy consumption and information-
transformation efficiencies into account, since strong surround inhibition will introduce a
strong metabolic cost (Laughlin, 1981; Vincent and Baddeley, 2003).
10.3 Subtractive and division-like surround inhibition
In our experiments with natural images, we found two kinds of behavior when analyzing
the RGCs’ inhibitory surround: First, about 80% of the cells showed a behavior with the
slope of the response-center-contrast curve falling with surround stimulation (Fig. 4.5A).
Such a fall in slope is the hallmark of division-like inhibition (Amthor and Grzywacz,
1991, 1993; Merwine et al., 1995). Second, about 20% of the cells displaying surround
inhibition showed an increase in the slope of the contrast-response curve with surround
stimulation (Fig. 4.5B). One cannot easily explain this type of crossing as subtractive or

91

division-like based on the models described above. However, we now argue that a
subtractive mechanism was involved. The increase of slope was probably not due to
mechanisms of surround excitation, because the surround was inhibitory overall. The
simplest alternative is that we failed to isolate the inhibitory surround completely, and
this failure caused the estimated surround to contain part of the center. Thus, stimulation
of this center portion would cause the injection of some excitation into the RGC, causing
an increase in slope. The modulated response curve with an increasing slope, however,
crossed, rather than sit above the control curve (Fig. 4.5B). This crossing indicated that
an inhibitory process lowered the modulated response curve. This inhibition was not just
division-like, because in that case, its effect would be to reduce the slope back towards
the control level or past it. With such reduction alone, the curves should not just cross.
We thus suggest that the crossing indicates a subtractive component. .
Given its ubiquity, one must consider what may be the role of a weak, division-like
surround inhibition of responses. The mathematical models developed in the context of
removal of correlation in the input stimuli (Atick, 1992) and predictive coding
(Srinivasan et al., 1982) were linear, thus not being consistent with a division-like
inhibition. In contrast, some models for detection of sharp image transitions (Ratliff,
1965) have used division-like inhibition (Balboa and Grzywacz, 2000; Grzywacz and
Balboa, 2002). Another possible role of division-like inhibition is to remove the effect of
the illuminant, thus obtaining the real reflectance of objects (Land, 1959). The idea is
that illumination is common to an object and to its neighbors in the scene. Therefore, as
the illumination increases, all their intensities increase proportionally. The large
surround mechanisms may then estimate a quantity that is proportional to the mean

92

reflectance times the illumination. As the RF center estimates the local reflectance times
the same illumination, then a division-like mechanism would eliminate illumination,
yielding the relative reflectance. Division-like surround inhibition is not just important
for RGCs but also for the early visual system in general (Allman et al., 1985; Krieger and
Zetzsche, 1996; Fitzpatrick, 2000; Vinje and Gallant, 2000). Roles postulated for non-
retinal division-like inhibition include normalization (Grossberg, 1982; Geisler and
Albrecht, 1992; Heeger, D.J., 1992; Gaudry and Reinagel, 2007) and decorrelation
(Simoncelli and Olshausen, 2001; Zhou and Mel, 2008). A question for future research is
whether the brain can fulfill the requirements of these computations despite weak
inhibition.
10.4 Asymmetric responses to the onset and offset of natural images
Statistically, the onset and offset of natural images did not cause equal responses for
retinal ganglion cells. On cells responded more to the onset of natural images than to the
offset; Off cells responded in an opposite way, with a preference for the offset of natural
stimuli. These asymmetries were observed by using both the histograms and linear RFs
of RGCs (Fig. 5.1 and Fig. 5.2).
Several known mechanisms of RGCs might be possible to explain these asymmetric
responses. One is that RGCs have abilities to adapt to the temporal and spatial contrasts
of light stimuli (Smirnakis et al., 1997; Chander and Chichilnisky, 2001; Kim and Rieke,
2001; Baccus and Meister, 2002). In our experiments, the initial states of the onset and
offset of natural images were different. The onset started with full-field gray background
including no contrasts, but the offset started with natural images having various contrasts.

93

If the initial contrast-adaptation state played an essential role in determining RGCs’
responses to the alternation of natural images and gray, we would observe asymmetries in
RGCs’ responses. Our results using the negative natural images, however, ruled out this
possibility. With the negative of natural images, we did not alter the initial contrast
adaptation states of the onset and offset. However, we observed opposite asymmetries.
Thus, the response asymmetries did not result from the difference of initial contrast-
adaptation states between the onset and offset.
The second possible explanation might be that the positive and negative center
mean contrasts in the RFs of RGCs for natural stimuli could be asymmetric. We know
from previous studies that the determinant parameter for RGC’s responses is the intensity
mean or weighted mean of the light stimuli in RGC’s RF center. If the mean intensities
in the RF center were asymmetric in reference to the gray background, or the center mean
contrasts were asymmetric between the onset and offset, we would observe asymmetric
responses from RGCs. However, although the distribution of pixel intensities in natural
images was asymmetric (Fig. 5.3), their center mean contrasts were approximately
symmetric (Fig. 6.1A). This symmetry could be explained by the central limit theorem
that the mean of any non-Gaussian distributions tends to be symmetric Gaussian
distribution (Rice, 1995). We thus eliminated this explanation by the mean intensity in
the RF center.
We finally proved that RGCs’ asymmetric responses to the onset and offset of
natural images were caused by the asymmetric distribution of pixel intensities of natural
images. This conclusion was drawn based on two control experiments that the negative
of natural images caused the response asymmetries to be inverted (Fig. 5.5), and

94

performing histogram equalization on natural images eliminated their response
asymmetries (Fig. 5.6). We further demonstrated that the asymmetric distribution of
pixel intensities caused an asymmetric distribution of the intensity variation in the RF of
RGC, characterized by their standard deviation (Fig. 6.1B). Finally, based on the STCH
(Fig. 6.1D), we concluded that the asymmetry of intensity variation caused RGCs’
response asymmetry between the onset and offset of natural stimuli.
10.5 Dependence of RGCs’ responses on natural-image visual textures
It is well known that the distribution of pixel intensities in natural stimuli is asymmetric,
with more pixels below the mean intensity and a longer tail towards high intensities
(Brady and Field, 2000; Olshausen and Field, 2000; Simoncelli and Olshausen, 2001).
Psychology studies have also demonstrated that humans are sensitive to these
asymmetries of intensity distribution. For example, human subjects could use the
skewness of reflected light to estimate objects’ surface qualities (Motoyoshi, 2007), and
use a ‘blackshot’ mechanism to sense the prevalence of the darkest intensities (Chubb et
al., 2004). Our studies confirmed the above experiments, and showed that very early in
visual processing, retinal ganglion cells respond to the asymmetric intensity distribution
in natural stimuli.
The asymmetric intensity distribution causes a positive correlation between the
local mean luminance and luminance variation (Brady and Field, 2000; Simoncelli and
Olshausen, 2001; Frazor and Geisler, 2006). In our study, this positive correlation (Fig.
6.1B) is crucial in explaining the asymmetric responses of RGCs. Some other study,
however, claimed that the local luminance and variation (they called contrast) were

95

independent (Mante, 2005). Their claim of independence was due to the difference in the
definition of variation. In our study, we used the root-mean-square (RMS - Peli, 1990),
but they used the RMS divided by the mean luminance. Therefore, their claim has
expressed the same property of natural images as ours, and both are caused by the
asymmetric distribution of pixel intensities in natural stimuli.
To quantify the dependence of a RGC’s responses on the variation of pixel
intensities, we used the histogram itself to represent the variation, rather than some
parameters extracted from the histogram, such as standard deviation or entropy. Standard
deviation is only sufficient to quantify the variation of Gaussian-distributed stimuli, but
not for non-Gaussian-distributed stimuli, like natural images. Entropy is also not a good
parameter here since it fails to distinguish the difference between the distribution of high
and low intensities in natural stimuli, which is actually critical in our analysis.
With histogram to represent the intensity variation, we developed a method, called STCH,
and showed that RGCs preferred visual textures to homogenous full fields for natural
stimuli (Fig. 6.1D). Strong visual textures, having most pixel intensities away from the
mean, elicit more responses than weak visual textures that instead have most pixel
intensities close to the mean. The results from STCH are very inspiring. However,
several limitations of the STCH might constrain its further quantitative analysis. First, it
neglects the response contribution from the RF surround, although weak in natural
stimuli. Second, it assumes that an RGC has the same histogram preference for all
different center mean contrasts. Third, it does not differentiate different positions in a
RGC’s RF, since the same intensity histogram but different position distribution might
have different response contributions. Finally, the positive correlation between

96

neighboring pixels in natural stimuli (Olshausen and Field, 2000) may influence the
STCH results.
10.6 Nonlinear-linear model
To characterize the different contributions from different relative intensities (Fig.
6.1D), we proposed a nonlinear-linear model (NL). The nonlinearity was put into the
first stage of the RGC’s visual processing (Fig. 7.1). This nonlinearity could come from
the outputs of bipolar cells or the dendritic trees of RGCs (Koch et al., 1983; Burkhardt
and Fahey, 1998; Dacey et al., 2000; Euler and Masland, 2000; Wu et al., 2000; Demb et
al., 2001). Our model is consistent with some other studies. For example, some claimed
that transient ganglion cells integrate subregions of their receptive field nonlinearly,
resulting from the nonlinearities at the output of bipolar cells (Demb et al., 2001).
With this NL model, we successfully account for the dependence of RGC’s
responses on the visual textures of natural images. The validity of the model was tested
and confirmed by two control experiments (Fig. 7.2): RGC’s responded equally to both
the positive plaids and their negative; RGC’s responses increased as the intensity
variation increased. The model also explains why RGCs’ responses disappeared when
extremely high-resolution plaids were used.
Any linear-nonlinear (LN) model, however, would fail to quantify these response
properties, no matter how its linear and nonlinear functions were selected. The LN model
is a well-developed model to characterize RGC’s responses (Shapley and Victor, 1978;
Reid et al., 1997; Meister and Berry, 1999; Chichilnisky, 2001; Kaplan and Benardete,
2001; Willmore and Smyth, 2003; Paninski et al., 2004; Carandini et al., 2005; Rust and

97

Movshon, 2005,). However, no report has shown that this model could fit well to natural
stimuli (Carandini et al., 2005), regardless of its good prediction to artificial stimuli
(Chichilnisky, 2001; Kim and Rieke, 2001; Zaghloul et al., 2003, 2005). One possible
explanation is that the mean luminance and luminance variation in natural scenes change
very sharply. To successfully apply this model to natural stimuli, we have to calibrate its
linear RF and nonlinear function for all different luminances and contrasts (Carandini,
2005).
10.7 RGCs’ contrast adaptation to natural images
We directly inspected RGCs’ contrast adaptation with natural-image stimulation. We
found that, similar to the contrast adaptation RGCs showed to artificial images, RGCs
also had contrast adaptation to natural images (Fig. 8.1). This contrast adaptation was a
combination of the adaptation to both temporal contrasts and spatial contrasts in both the
center and peripheral region of a RGC’s RF.
To determine the temporal components of RGCs’ contrast adaptation to natural
stimuli, a mixture-of-exponentials model (Methods 2.9) was applied. Totally, three
different contrast-adaptation components were found in the contrast increase of natural
images. The fastest was the contrast gain control, observed when a small time bin width
0.3 s was used. It occurred as brief as tens or hundreds of milliseconds (Fig. 8.1E). The
second component was fast contrast adaptation, which was slower than contrast gain
control and extended to a couple of seconds (Fig. 8.2D,E). The slowest was slow
contrast adaptation and extended to tens of seconds (Fig. 8.2C,E).

98

In contrast, RGCs only showed one temporal component in the contrast decrease of
natural images (Fig. 8.2G,H). No significant contrast-gain-control and fast-adaptation
components were observed. Thus, the contrast adaptation of RGCs to the contrast
increase of natural images is different from that to their contrast decrease. This
asymmetry is consistent with other studies (Smirnakis et al., 1997; DeWeese and Zador
1998; Brown and Masland, 2001; Kim and Rieke 2001), and can be explained by how
quickly the contrast change could be detected (Meister and Berry, 1999; DeWeese and
Zador, 1998). For example, an RGC easily discerns a sudden contrast increase from low
to high by immediately detecting any one or a few large intensities beyond the low-
contrast range. However, the cell cannot identify a contrast decrease by just detecting
one or a few low intensities because there are many low-contrast intensities even in a
high-contrast environment. The identification of a contrast decrease needs a statistical
measurement of sufficient low intensities, requiring a much longer time for pooling. The
difference between detecting contrast increase and contrast decrease has been predicted
by a Kalman theory (Grzywacz and de Juan, 2003).
In addition, we found that the number of cells showing contrast adaptation to the
contrast decrease (19%) was much smaller than the number to the contrast increase
(66%). An explanation is that RGCs usually did not respond well to low-contrast stimuli.
Thus, when the natural stimuli was switched to low-contrast, we would sometimes not
observe contrast adaptation even if the cell did it.
Contrast adaptation with different temporal components can play different
functional roles in retinal signal processing. For example, contrast gain control with the
fastest time course can instantaneously change a RGC’s sensitivity and prevent the

99

saturation of its response as the eye moves (Victor, 1987; Baccus and Meister, 2002); fast
contrast adaptation can adjust a cell’s response to moving objects across its receptive
field (Berry et al., 1999); slow contrast adaptation can adjust a cell’s visual sensitivity to
the overall temporal and spatial structures in a new environment (Smirnakis et al., 1997;
Truchard et al. 2000; Chander and Chichilnisky 2001). In addition, that RGCs showed
contrast adaptation to natural stimuli has perceptual correlates in human psychophysics
and is important for the early vision system (Blakemore and Campbell, 1969; Schieting
and Spillmann, 1987; Anstis 1996, DeMarco et al 1997, Freeman and Badcock 1999).
For example, it can help optimize some specific tasks, such as the localization of edges
(Grzywacz and Balboa, 2002).
10.8 Diversity of RGCs’ contrast adaptation to natural stimuli
RGCs across the population showed a variety of contrast-adaptation types when
stimulated with natural images (Fig. 8.3). Of a total of 125 cells, only 82 cells (66%) had
contrast adaptation. With those 62 cells that we could determine their contrast-adaptation
types, 14 cells (23%) had fast-only adaptation, 25 cells (40%) had slow-only adaptation,
and 23 (37%) had fast-and-slow adaptation.
Multiple mechanisms might be involved in RGCs’ contrast adaptation, resulting
from the variety of RGCs’ morphology and retinal circuitry (Sakai et al., 1995; Kim and
Rieke, 2001). The contrast adaptation occurs at multiple sites within the retinal circuitry,
including bipolar cells (Brown and Masland 2001; Rieke, 2001; Manookin and Demb,
2006) and ganglion cells (Kim and Rieke, 2001; Zaghloul et al., 2005; Beaudoin et al.,
2007). If individual retinal inter-neurons, including bipolar cells and amacrine cells,

100

perform contrast adaptation independently, RGCs with different retinal circuits will show
different time courses in their contrast adaptation to light stimuli (Brown and Masland,
2001; Kim and Rieke 2001; Baccus and Meister, 2004).
In addition, we also inspected whether transient cells and sustained cells adapted
differently to the contrast change of natural images (Table 9.1). We found that transient
cells had a higher chance to show contrast adaptation than sustained cells (χ2 test,
p<0.01). This result is consistence with previous findings (Solomon et al., 2004). We
know that contrast adaptation shows an RGC’s ability in adjusting its sensitivity to the
contrast of outside environment. Transient cells with the quick adjustment of responses
thus have stronger capability of showing contrast adaptation than sustained cells.
Furthermore, we found that the percentage of RGCs showing the fast components
for Off cells was higher than that for On cells (χ2 test, p<0.01). Contrast adaptation is
known to occur in bipolar cells (Kim and Rieke, 2001; Rieke, 2001), where the pathways
of On cells and Off cells are separated. Because On and Off cells have different
properties of anatomy and function (Rodieck, 1998), their contrast adaptation can be
different. Since Off cells tend to have faster kinetic responses than On cells (Chander
and Chichilnisky, 2001; Rieke, 2001), Off cells could have a higher percentage of
showing fast contrast adaptation than On cells.

101

10.9 Dependence of RGCs’ contrast adaptation on the contrast
difference of natural images
We tested RGCs’ contrast adaptation with three different natural-image contrast steps,
including from 0.05 to 0.25, from 0.05 to 0.45, and from 0.05 to 0.7. We found that,
similar to using spatially uniform flickering stimulation (Smirnakis et al., 1997), both the
peak and plateau responses also increased after following a step of contrast increase as
the contrast step enlarged, but the adaptation indexes kept the same (Fig. 9.1 and Fig. 9.2).
The increased peak and plateau responses to the increased contrast difference confirm
that RGCs might encode the contrast of light stimuli as a sigmoidal function; the same
adaptation indexes across different contrast steps imply that RGCs might follow some
divisive or normalization mechanism in their contrast adaptation (Smirnakis, et al., 1997).
In addition, different RGCs showed different types of contrast adaptation in
response to different contrast changes. Of a total of 35 RGCs, 11 RGCs lost their
contrast-adaptation components when the contrast change of natural images was small
(Fig. 9.3). The disappearance of contrast adaptation demonstrates that a minimum
contrast difference is required for some RGCs to show contrast adaptation. One possible
explanation is that RGCs’ contrast adaptation might require a pooling mechanism
(Baccus and Meister, 2004), which could be modeled as a Kalman filter (Grzywacz and
de Juan, 2003). As the difference of contrast change was reduced, the difficulty of
detecting contrast change would increase. RGCs thus needed a longer time to discern the
change by pooling the statistics of stimuli. When the contrast difference was too small,
the long-time integration made it impossible for RGCs to show contrast gain control and

102

fast adaptation, and difficult for us to observe a weak slow contrast adaptation in thirty
seconds. This long-time integration also explains why the time constant of RGCs’
contrast adaptation decreased when the contrast difference increased (Fig. 9.4).
10. 10 Future directions
The main direction of our future study is to combine together all the findings we have on
RGCs’ response properties in this work to build a quantitative model to predict RGC’s
responses to natural stimuli.
As we have discussed, although the linear-nonlinear (LN) model was used
extensively in characterizing RGCs’ responses to artificial stimuli, it has been proved to
be inaccurate for natural stimuli. Although some studies tried to apply this LN model for
natural stimuli by calibrating its linear RF and nonlinear function in all possible contrast
levels, we show that it failed to capture the results of our experiments (Fig. 7.2).
Our work has brought forward a doable way to reach this goal. To predict a RGC’s
responses to natural stimuli, we could use a nonlinear-linear (NL) model, which we
describe qualitatively in Chapter 7. However, to develop this NL model into a
quantitative one, we have two crucial questions to answer.
The first question is how many bipolar cells we should virtually put in the first stage
of the NL model. Could we use a same number of virtual bipolar cells for different
RGCs? If we could, how many should we choose? If not, how can we determine the
number of virtual bipolar cells for different RGCs? To answer this question, we can take
the knowledge that we have on the anatomy of the retina, and check the range of real
bipolar-cell numbers that are connected to an RGC. If we have to use a different number

103

of virtual bipolar cells for different RGCs, a couple of methods are available to help
determine their number, including independent component analysis, neural network, or
some statistical test by which an optimal number of bipolar cells will be obtained.
The other question is how we determine the subregions of bipolar cells, as well as
the nonlinearities at their outputs. In other words, what are the shapes of these
subregions? Should we separate the whole RF exclusively into subregions or can they
overlap each other? Further, what techniques can we apply to quantify their nonlinear
functions?

104

References
Aertsen, A.M., Gerstein, G.L., Habib, M.K. and Palm, G. (1989). Dynamics of neuronal
firing correlation: modulation of "effective connectivity". Journal of Neurophysiology
61(5), 900-917.
Ahmad, K.M., Klug, K., Herr, S., Sterling, P., Schein, S. (2003). Cell density ratios in a
foveal patch in macaque retina. Visual Neuroscience 20(2), 189-209.
Allman, J., Miezin, F. and McGuiness, E. (1985). Stimulus-specific responses from
beyond the classical receptive field: neurophysiological mechanisms for local-global
comparisons on visual neurons. Annual Review of Neuroscience 8, 407-430.
Amthor, F.R. and Grzywacz, N.M. (1991). Nonlinearity of the inhibition underlying
retinal directional selectivity. Visual Neuroscience 6, 197-206.
Amthor, F.R. and Grzywacz, N.M. (1993). Inhibition in ON-OFF directionally selective
ganglion cells of the rabbit retina. Journal of Neurophysiology 69, 2174-2187.
Amthor, F.R., Takahashi, E.S. and Oyster, C.W. (1989). Morphologies of rabbit retinal
ganglion cells with concentric receptive fields. Journal of Comparative Neurology 280(1),
72-96.
Anstis, S. (1996). Adaptation to peripheral flicker. Vision Research 36(21), 3479–3485.
Atick, J.J. (1992). Could information theory provide an ecological theory of sensory
processing? Network: Computation in Neural Systems 3, 213-251.
Baccus, S.A. and Meister, M. (2002). Fast and slow contrast adaptation in retinal
circuitry. Neuron 36(5), 909–919.
Baccus, S.A and Meister, M.(2004). Retina versus cortex; contrast adaptation in parallel
visual pathways. Neuron 42(1), 5-7.
Balboa, R.M., Grzywacz, N.M. (2000). The role of early lateral inhibition: more than
maximizing luminance information. Vision Research 17, 77– 89.
Balboa, R.M. and Grzywacz, N.M. (2003). Power spectra and distribution of contrasts of
natural images from different habitats. Vision Research 43(24), 2527-2537.

105

Barlow, H.B. (1953). Summation and inhibition in the frog’s retina. Journal of
Physiology-London 119, 69–88.
Barlow, H. B. (1981). Critical limiting factors in the design of the eye and visual cortex
(The Ferrier Lecture, 1980). Proceedings of the Royal Society of London Series B-
Biological Sciences 212, 1−34.
Barlow, H.B., Fitzhugh, R. and Kuffler, S.W. (1957). Change of organization in the
receptive fields of the cat’s retina during dark adaptation. Journal of Physiology-London
137, 338–354.
Barlow, H.B. and Levick, W.R. (1969). Changes in the maintained discharge with
adaptation level in the cat retina. Journal of Physiology 202(3), 699–718.
Beaudoin, D.L., Borghuis, B.G. and Demb, J.B. (2007). Cellular basis for contrast gain
control over the receptive field center of mammalian retinal ganglion cells. Journal of
Neuroscience 27, 2636–2645.
Benardete, E.A. and Kaplan, E. (1999). The dynamics of primate m retinal ganglion cells.
Visual Neuroscience 16, 355-368.
Benardete, E.A., Kaplan, E. and Knight, B.W. (1992). Contrast gain control in the
primate retina: P cells are not X-like, some M cells are. Visual Neuroscience 8, 483-486.
Berry, M.J., Brivanlou, I.H., Jordan, T.A. and Meister, M. (1999). Anticipation of
moving stimuli by the retina. Nature 398, 334–338.
Blakemore, C. and Campbell, F.W. (1969). On the existence of neurones in the human
visual system selectively sensitive to the orientation and size of retinal images. Journal of
Physiology . 203(1), 237–260.
Brady, N., and Field, D.J. (2000). Local contrast in natural images: Normalization and
coding efficiency. Perception 29, 1041–1055.
Brown, S.P. and Masland, R.H. (2001). Spatial scale and cellular substrate of contrast
adaptation by retinal ganglion cells. Nature Neuroscience 4, 44-51.
Burkhardt, D.A. and Fahey, P.K. (1998). Contrast enhancement and distributed encoding
by bipolar cells in the retina. Journal of Neurophysiology 80, 1070-1081.
Burton, C.J. and Moorhead, I.R. (1987). Color and spatial structure in natural scenes.
Applied Optics 26, 157–170.

106

Cao, X., Merwine, D.K. and Grzywacz, N.M. (2006). Weakness of surround inhibition
with natural-image stimulation. Journal of Vision 6(13), 43a.
Cao, X., Merwine, D.K. and Grzywacz, N.M. (2008). Asymmetric responses of retinal
ganglion cells to the onset and offset of Natural Images. Annual Meeting of the
Association for Research in Vision and Ophthalmology. E-3860/A403.
Cao, X., Merwine, D.K. and Grzywacz, N.M. (2009). Adaptation of retinal ganglion cells
to natural-image contrast. Annual Meeting of the Association for Research in Vision and
Ophthalmology. E-1415/A241.
Caldwell, J.H. and Daw, N.W. (1978). New properties of rabbit retinal ganglion cells.
Journal of Physiology 276, 257–278.
Carandini, M., Demb, J.B., Mante, V., Tolhurst, D.J., Dan, Y., Olshausen, B.A. Gallant,
J.L. and Rust, N.C. (2005). Do we know what the early visual system does? Journal of
Neuroscience 25, 10577–10597.
Carandini, M. and Heeger, D. (1994). Summation and division by neurons in primate
visual cortex. Science 264, 1333-1336.
Chander, D. and Chichilnisky, E.J. (2001). Adaptation to temporal contrast in primate
and salamander retina. Journal of Neuroscience 21, 9904-9916.
Chatterjee, S., Merwine, D.K., Amthor, F.R. and Grzywacz, N.M. (2007). Properties of
stimulus-dependent synchrony in retinal ganglion cells. Visual Neuroscience 24(6), 827-
843.
Chichilnisky, E.J. (2001). A simple white noise analysis of neuronal light responses.
Network: Computation in Neural Systems 12, 199–213.
Chichilnisky, E.J. and Kalmar R.S. (2002). Functional asymmetries in On and Off
ganglion cells of primate retina. Journal of Neuroscience 22(7), 2737-2747.
Chubb, C., Landy, M.S. and Econopouly, J. (2004). A visual mechanism tuned to black.
Vision Research 44, 3223–3232.
Clifford, C.W., Webster, M.A., Stanley, G.B., Stocker, A.A., Kohn, A., Sharpee, T.O.
and Schwartz O. (2007). Visual adaptation: neural, psychological and computational
aspects. Vision Research 47(25), 3125-3131.
Dacey, D., Packer, O.S., Diller, L., Brainard, D., Peterson, B. and Lee, B. (2000). Center
surround receptive field structure of cone bipolar cells in primate retina. Vision Research
40, 1801-1811.

107

David, S.V., Vinje, W.E. and Gallant, J.L. (2004). Natural stimulus statistics alter the
receptive field structure of V1 neurons. Journal of Neuroscience 24(31), 6991-7006.
DeMarco, P. J. Jr., Brigell, M.G. and Gordon, M. (1997). The peripheral flicker effect:
desensitization of the luminance pathway by static and modulated light. Vision Research
37, 2419–2425.
Demb, J.B. (2002). Multiple mechanisms for contrast adaptation in the retina. Neuron 36,
781-783.
Demb, J.B., Haarsma, L, Freed, M.A. and Sterling, P. (1999). Functional circuitry of the
retinal ganglion cell's nonlinear receptive field. Journal of Neuroscience 19, 9756–9778.
Demb, J.B., Zaghloul, K., Haarsma, L. and Sterling, P. (2001). Bipolar cells contribute to
nonlinear spatial summation in the brisk-transient (Y) ganglion cell in mammalian retina.
Journal of Neuroscience 21, 7447-7454.
Devries, S.H. and Baylor, D.A. (1997). Mosaic arrangement of ganglion cell receptive
fields in Rabbit Retina. Journal of Neurophysiology 78(4), 2048-2060.
DeWeese, M. and Zador, A. (1998). Asymmetric dynamics in optimal variance
adaptation. Neural Computation 10, 1179–1202.
Efron, B. and Tibshirani, R. (1993). An Introduction to Bootstrap. New York, NY:
Chapman & Hall.
Enroth-Cugell, C. and Jakiela, H. G. (1980). Suppression of cat retinal ganglion cell
responses by moving patterns. Journal of Physiology-London 302, 49–72.
Enroth-Cugell, C. and Lennie, P. (1975). The control of retinal ganglion cell discharge by
receptive field surrounds. Journal of Physiology-London 247, 551–578.
Enroth-Cugell, C. and Robson, J.G. (1966). The contrast sensitivity of retinal ganglion
cells of the cat. Journal of Physiology-London 187(3), 517-552.
Euler, T. and Masland, R.H. (2000). Light-evoked responses of bipolar cells in a
mammalian retina. Journal of Neurophysiology 83, 1817-1829.
Felsen, G. and Dan, Y. (2005). A natural approach to studying vision. Nature
Neuroscience 8, 1643–1646.
Field, D.J. (1987). Relations between the statistics of natural images and the response
properties of cortical cells. Journal of the Optical Society of America A-Optics Image
Science and Vision 4(12), 2379–2394.

108

Fitzpatrick, D. (2000). Seeing beyond the receptive field in primary visual cortex.
Current Opinion in Neurobiology 10, 438-443.
Flores-Herr, N., Protti, D.A. and Wässle, H. (2001). Synaptic currents generating the
inhibitory surround of ganglion cells in the mammalian retina. Journal of Neuroscience
21(13), 4852-4863.
Frazor, R.A. and Geisler, W.S.. (2006). Local luminance and contrast in natural images.
Vision Research 46, 1585-1598.
Freeman, A.W. and Badcock, D.R. (1999). Visual sensitivity in the presence of a
patterned background. Journal of the Optical Society of America A-Optics Image Science
and Vision 16, 979–986.
Gaudry, K.S. and Reinagel, P. (2007). Benefits of contrast normalization demonstrated in
neurons and model cells. Journal of Neuroscience 27, 8071–8079.
Geisler, W.S. and Albrecht, D.G. (1992). Cortical neurons: isolation of contrast gain
control. Vision Research 8, 1409-1410.
Grossberg, S. (1982). Adaptation and gain normalization: a comment on Ullman &
Schechtman. Proceedings of the Royal Society of London Series B-Biological Sciences
219(1217), 471-473.
Grzywacz, N.M. and Balboa, R.M. (2002). A Bayesian framework for sensory adaptation.
Neural Computation 14, 543–559.
Grzywacz, N.M. and de Juan, J. (2003). Sensory adaptation as Kalman filtering: theory
and illustration with contrast adaptation. Network: Computation in Neural Systems 14(3),
465-482.
Grzywacz, N.M. and Koch, C. (1987). Functional Properties of Models for Direction
Selectivity in the Retina. Synapse 1, 417-434.
Heeger, D.J. (1992). Normalization of cell responses in cat striate cortex. Visual
Neuroscience 9(2), 181–197.
Hubel, D.H. and Wiesel, T.N. (1962). Receptive fields, binocular interaction and
functional architecture in the cat’s visual cortex. Journal of Physiology-London 160, 106-
154.
Johnson, M.A. and Vardi, N. (1998). Regional differences in GABA and GAD
immunoreactivity in rabbit horizontal cells. Visual Neuroscience 15(4), 743-753.

109

Kaplan, E. and Benardete, E. (2001). The dynamics of primate retinal ganglion cells.
Progress in Brain Research 134, 17-34.
Kim, K.J. and Rieke, F. (2000). Temporal contrast adaptation in the salamander retina.
Investigative Ophthalmology and Visual Science [Suppl] 41, 937.
Kim, K.J. and Rieke, F. (2001). Temporal contrast adaptation in the input and output
signals of salamander retinal ganglion cells. Journal of Neuroscience 21, 287-299.
Koch, C., Poggio, T. and Torre, V. (1983). Nonlinear interactions in a dendritic tree:
localization, timing, and role in information processing. Proceedings of the National
Academy of Sciences of the United States of America 80(9), 2799–2802.
Koch, K., McLean, J. and Segev, R. (2006). How much the eye tells the brain. Current
Biology 16, 1428–1434.
Koutalos, Y., Yau, K.W. (1996). Regulation of sensitivity in vertebrate rod
photoreceptors by calcium. Trends in neurosciences 19, 73-81.
Krieger, G. and Zetzsche, C. (1996). Nonlinear image operators for the evaluation of
local intrinsic dimensionality. IEEE Transactions on Image Processing 5(6), 1026–1041.
Kuffler, S.W. (1953). Discharge patterns and functional organization of the mammalian
retina. Journal of Neurophysiology 16, 37–68.
Land, E.H. (1959). Color vision and the natural image part II. Proceedings of the
National Academy of Sciences of the United States of America 45(4), 636–644.
Laughlin, S.B. (1981). A simple coding procedure enhances a neuron’s information
capacity. Eitschrift Fur Naturforschung C – Journal of Biosciences 36(9-10), 910–912.
Levick, W.R. (1967). Receptive fields and trigger features of ganglion cells in the visual
streak of the rabbit’s retina. Journal of Physiology-London 188, 285-307.
Manookin, M.B. and Demb, J.B. (2006). Presynaptic mechanism for slow contrast
adaptation in mammalian retinal ganglion cells. Neuron 50, 453–464.
Mante, V., Bonin, V. and Carandini, M. (2008). Functional mechanisms shaping lateral
geniculate responses to artificial and natural stimuli. Neuron 58(4), 625-638.
Mante, V., Frazor, R.A., Bonin, V. , Geisler, W.S. and Carandini, M. (2005).
Independence of luminance and contrast in natural scenes and in the early visual system.
Nature Neuroscence 8, 1690–1697.

110

Marmarelis, P. and Marmarelis, V. (1978). Analysis of Physiological Systems: The White
Noise Approach. Plenum Press, New York.
Merwine, D.K., Amthor, F.R. and Grzywacz, N.M. (1995). Interaction between center
and surround in rabbit retinal ganglion cells. Journal of Neurophysiology 73, 1547-1567.
Meister, M. and Berry II, M.J. (1999). The neural code of the retina. Neuron 22, 435–450.
Mosteller, F., and Tukey, J.W. (1983). Understanding Robust and Exploratory Data
Analysis. In D. C. Hoaglin (Ed.), Wiley sSeries in probability and mathematical statistics.
New York: John Wiley & Sons Inc.
Motoyoshi, I., Nishida, S., Sharan, L. and Adelson, E.H. (2007). Image statistics and the
perception of surface qualities. Nature 447, 206-209.
Nuala, B. and David, J.F. (2000). Local contrast in natural images: normalisation and
coding efficiency. Perception 29, 1041-1055.
Olshausen, B.A. and Field, D.J. (2000). Vision and the coding of natural images.
American Scientist 88(3), 238-245.
Olshausen, B.A. and Field, D.J. (2004). Sparse coding of sensory inputs. Current Opinion
in Neurobiology 14(4), 481-487.
Paninski, L. (2003). Estimation of entropy and mutual information. Neural Computation
15, 1191–1253.
Paninski, L., Pillow, J.W. and Simoncelli, E.P. (2004). Maximum likelihood estimation
of a stochastic integrate-and-fire neural encoding model. Neural Computation 16, 2533-
2561.
Peli, E. (1990). Contrast in complex images. Journal of the Optical Society of America A-
Optics Image Science and Vision 7(10), 2032-2040.
Poggio, T. and Reichardt, W.E. (1976). Visual control of orientation behaviour in the fly:
Part 11: Towards the underlying neural interactions. Quarterly Reviews of Biophysics 9,
377-438.
Pugh, E.N., Nikonov, S. and Lamb, T.D. (1999). Molecular mechanisms of vertebrate
photoreceptor light adaptation. Current Opinion in Neurobiology 9, 410–418.
Ratliff, F. (1965). Mach Bands: Quantitative Studies on Neural Networks in the Retina.
San Francisco, california: Holden-Day.

111

Reid, R.C. and Alonso, J.M. (1995). Specificity of monosynaptic connections from
thalamus to visual cortex. Nature 378, 281–284.
Reid, R.C., Victor, J.D. and Shapley, R.M. (1997). The use of m-sequences in the
analysis of visual neurons: linear receptive field properties. Visual Neuroscience 14,
1015-1027.
Rice, J.A. (1995). Mathematical Statistics and Data Analysis (Second ed.), Duxbury
Press.
Rieke, F. (2001). Temporal contrast adaptation in salamander bipolar cells. Journal of
Neuroscience 21(23), 9445-9454.
Ringach, D.L., Sapiro, G. and Shapley, R. (1997). A subspace reverse-correlation
technique for the study of visual neurons. Vision Research 37(17), 2455-2464.
Rockhill, R.L., Daly, F.J., MacNeil, M.A., Brown, S.P. and Masland, R.H. (2002). The
diversity of ganglion cells in a mammalian retina. Journal of Neuroscience 22(9), 3831–
3843.
Rodieck, R.W. (1965). Quantitative analysis of cat retinal ganglion cell response to visual
stimuli. Vision Research 5, 583–601.
Rodieck, R.W. (1998). The first steps in seeing. Sunderland, MA: Sinauer.
Rodieck, R.W. and Stone, J. (1965). Analysis of receptive fields of cat retinal ganglion
cells. Journal of Neurophysiology 28(5), 832-849.
Ruderman, D.L. and Bialek,W. (1994). Statistics of natural images: scaling in the woods.
Physical Review Letters 73(6), 814–817.
Rushton, W.A. (1965). Visual adaptation (The Ferrier Lecture, 1962). Proceedings of the
Royal Society of London Series B-Biological Sciences 162(986), 20-46.
Rust, N.C. and Movshon, J.A. (2005). In praise of artifice. Nature Neuroscience 8, 1647
– 1650.
Sagdullaev, B.T. and McCall, M.A. (2005). Stimulus size and intensity alter fundamental
receptive-field properties of mouse retinal ganglion cells in vivo. Visual Neuroscience
22(5), 649-659.
Sakai, H.M., Wang, J.L. and Naka, K. (1995). Contrast gain control in the lower
vertebrate retinas. Journal of General Physiology 105, 815-835.

112

Schieting, S. and Spillmann, L. (1987). Flicker adaptation in the peripheral retina. Vision
Research 27, 277−284.
Shapley, R.M. and Victor, J.D. (1978). The effect of contrast on the transfer properties of
cat retinal ganglion cells. Journal of Physiology-London 285, 275-298.
Shapley, R.M. and Victor, J.D. (1979). Nonlinear spatial summation and the contrast gain
control of cat retinal ganglion cells. Journal of Physiology-London 290, 141–161.
Shapley, R.M. and Enroth-Cugell, C. (1984). Visual adaptation and retinal gain controls.
In: Progress in Retinal Research (Osborne N, Chader G, eds.), pp 263-346. London:
Pergamon.
Sharpee, T.O., Sugihara, H., Kurgansky, A.V., Rebrik, S.P., Stryker, M.P. and Miller,
K.D. (2006). Adaptive filtering enhances information transmission in visual cortex.
Nature 439, 936–942.
Simoncelli, E.P. and Olshausen, B.A. (2001). Natural image statistics and neural
representation. Annual Review of Neuroscience 24, 1193-1216.
Smirnakis, S.M., Berry, M.J., Warland, D.K., Bialek, W. and Meister, M. (1997).
Adaptation of retinal processing to image contrast and spatial scale. Nature 386, 69-73.
Smyth, D., Willmore, B., Baker, G.E., Thompson, I.D. and Tolhurst, D.J. (2003). The
receptive-field organization of simple cells in primary visual cortex of ferrets under
natural scene stimulation. Journal of Neuroscience 23, 4746–4759.
Solomon, S.G., Lee, B.B. and Sun, H. (2006). Suppressive surrounds and contrast gain in
magnocellular-pathway retinal ganglion cells of macaque. Journal of Neuroscience 26,
8715–8726.
Solomon, S.G., Peirce, J.W., Dhruv, N.T., and Lennie, P. (2004). Profound contrast
adaptation early in the visual pathway. Neuron 42, 155–162.
Srinivasan, M.V., Laughlin, S.B. and Dubs, A. (1982). Predictive coding: a fresh view of
inhibition in the retina. Proceedings of the Royal Society of London Series B-Biological
Sciences 216, 427-459.
Sterling, P. (2004). How retinal circuits optimize the transfer of visual information. In:
The Visual Neurosciences, Vol. 1 (Chalupa LM, Werner JS, eds), pp. 234–259,
Cambridge, MA: MIT.

113

Theunissen, F.E., Sen, K. and Doupe, A.J. (2000). Spectral-temporal receptive fields of
nonlinear auditory neurons obtained using natural sounds. Journal of Neuroscience 20,
2315–2331.
Tolhurst, D.J., Tadmor, Y. and Chao, T. (1992). Amplitude spectra of natural images.
Ophthalmic and Physiological Optics 12, 229–232.
Torre, V. and Poggio, T. (1978). A synaptic mechanism possibly underlying directional
selectivity to motion. Proceedings of the Royal Society of London Series B-Biological
Sciences 202, 409-416.
Truchard, A.M., Ohzawa, I. and Freeman, R.D. (2000). Contrast gain control in the visual
cortex: monocular versus binocular mechanisms. Journal of Neuroscience 20, 3017–3032.
van der Schaaf, A. and van Hateren, J.H. (1996). Modeling the power spectra of natural
images: Statistics and information. Vision Research 36, 2759–2770.
van Hateren, J.H. and Ruderman, D.L. (1998). Independent component analysis of
natural image sequences yields spatio-temporal filters similar to simple cells in primary
visual cortex. Proceedings of the Royal Society of London Series B-Biological Sciences
265, 2315–2320.
Victor, J.D. (1987). The dynamics of the cat retinal X cell centre. Journal of Physiology-
London 386, 219-246.
Vincent, B.T. and Baddeley, R.J. (2003). Synaptic energy efficiency in retinal processing.
Vision Research 43, 1283-1290.
Vinje, W.E. and Gallant, J.L. (2000). Sparse coding and decorrelation in primary visual
cortex during natural vision. Science 287(5456), 1273-1276.
Walraven, J., Enroth-Cugell, C., Hood, D.C., MacLeod, D.I.A. and Schnapf, J.L. (1990).
The control of visual sensitivity. In: Visual Perception: the Neurophysiological
Foundations (Spillmann L, Werner SJ, eds), pp 53-101. San Diego: Academic.
Willmore, B. and Smyth, D. (2003). Methods for first-order kernel estimation: simple-
cell receptive fields from responses to natural scenes. Network: Computation in Neural
Systems 14, 553–577.
Wu, S.M., Gao, F. and Maple, B.R. (2000) Functional architecture of synapses in the
inner retina: segregation of visual signals by stratification of bipolar cell axon terminals.
Journal of Neuroscience 20, 4462-4470.

114

Yang, J. and Stevenson, S.B. (1999). Post-retinal processing of background luminance.
Vision Research 39, 4045–4051.
Zaghloul, K.A., Boahen, K. and Demb, J.B. (2003). Different circuits for On and Off
retinal ganglion cells cause different contrast sensitivities. Journal of Neuroscience 23,
2645-2654.
Zaghloul, K.A., Boahen, K. and Demb, J.B. (2005). Contrast adaptation in subthreshold
and spiking responses of mammalian Y-type retinal ganglion cells. Journal of
Neuroscience 25, 860-868.
Zhou, C. and Mel, B.W. (2008). Cue combination and color edge detection in natural
scenes. Journal of Vision 8(4), 1–25.
Zoubir, A.M. and Boashash, B. (1998). The bootstrap and its application in signal
processing. IEEE Signal Processing Magazine 15(1), 55-76.

Abstract (if available)

Abstract Natural scenes have many special statistical properties that have shaped our visual system through natural evolution. It may thus be necessary to use natural images directly to examine retinal-ganglion-cells’ (RGCs) properties, rather than to extrapolate their properties from artificial stimuli. In this study, we first inspected what the most important visual property determining the responses of an RGC to natural images is. A new method was developed to estimate with natural images the sizes of the receptive-field (RF) center and surround. We showed that the center sizes estimated with our method were similar to those obtained with standard artificial stimuli. Furthermore, the temporal mean contrast of the center of the RF strongly dominated the RGC’s responses, while surround contrast mostly showed a weak and division-like (as opposed to subtractive) inhibition. We then asked whether the RGCs’ responses also depend on the local visual textures of natural images, or the luminance variation from the mean. We observed that RGCs responded asymmetrically between the transition from homogeneous backgrounds to natural images (onset), and the reverse transition (offset), even if both transitions had the same local temporal mean contrast. Furthermore, the negative of the natural images inverted this asymmetry, and their histogram equalization eliminated it. Hence, the response asymmetry arose from the asymmetric intensity distribution in the natural images. We further developed a method, spike-triggered contrast histogram (STCH), to demonstrate that a natural image with strong visual texture tended to elicit larger responses than one with weak texture. To account for these results, a nonlinear-linear model was developed. It included multiple subunits of nonlinear inputs, each covering a sub-region of the RF. Finally, we investigated whether the RGCs’ responses adapt to the spatial and temporal contrasts of natural images.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Dependence of rabbit retinal synchrony on visual stimulation parameters

PDF

Characterization of visual cells using generic models and natural stimuli

PDF

A probabilistic model predicting retinal ganglion cell responses to natural stimuli

PDF

Spatial anomalies of visual processing in retinitis pigmentosa and a potential treatment

PDF

Learning contour statistics from natural images

PDF

Manipulation of RGCs response using different stimulation strategies for retinal prosthesis

PDF

Spatial and temporal precision of inhibitory and excitatory neurons in the murine dorsal lateral geniculate nucleus

PDF

Towards a high resolution retinal implant

PDF

Functional models of fMRI BOLD signal in the visual cortex

PDF

Exploring sensory responses in the different subdivisions of the visual thalamus

PDF

Human visual perception of centers of optic flows

PDF

Functional magnetic resonance imaging characterization of peripheral form vision

Asset Metadata

Creator Cao, Xiwu (author)

Core Title Encoding of natural images by retinal ganglion cells

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Biomedical Engineering

Publication Date 04/18/2010

Defense Date 08/24/2009

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag adaptation,contrast,natural image,nonlinear model,OAI-PMH Harvest,retinal ganglion cell,spatial context,surround inhibition,visual receptive field

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Grzywacz, Norberto M. (committee chair), Hirsch, Judith A. (committee member), Mel, Bartlett W. (committee member), Merwine, David K. (committee member), Weiland, James D. (committee member)

Creator Email caoxiwu@gmail.com,xiwucao@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-m2935

Unique identifier UC186619

Identifier etd-Cao-3168 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-313816 (legacy record id),usctheses-m2935 (legacy record id)

Legacy Identifier etd-Cao-3168.pdf

Dmrecord 313816

Document Type Dissertation

Rights Cao, Xiwu

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Repository Name Libraries, University of Southern California

Repository Location Los Angeles, California

Repository Email cisadmin@lib.usc.edu