Modeling the integration of salamander vision and behavior

PDF

Download

Open document

Flip pages

Summarize

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content MODELING THE INTEGRATION OF SALAMANDER VISION AND
BEHAVIOR
by
Jerey Robert Begley
A Dissertation Presented to the
FACULTY OF THE GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulllment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(COMPUTER SCIENCE)
May 2009
Copyright 2009 Jerey Robert Begley
Acknowledgments
I would like to thank my advisor and committe chair, Michael Arbib, for the many stim-
ulating opportunities he has provided at USC, and for his very patient and wise guidance
and motivation. I would like to thank my defence committee, Laurent Itti and Bartlett
Mel, for their patience, insight, and guidance. I would like to thank Wei-Min Shen for
serving on my guidance committee. Thanks to Christof von der Malsberg for insightful
conversations and guidance, to David Merwine for many interesting conversations about
and wide variety of topics, especially the retina. Thanks to Joaquin Rapela, David Mer-
wine, and Norberto Grzywacz for organizing USC's Vision Journal Club, to Norberto for
organising the annual Vision Symposium, and to Laurent Itti for bringing the Joint Sym-
posium on Computational Neuroscience to USC. I would especially like to thank Auke
Ijspeert for introducing me to the topic, for allowing the use of his code for the neurome-
chanical CPG model, for interesting and fruitful conversations, and for his encouragement
and guidance. Thanks to Northrop Grumman, and to folks in the lab: Nathan Mund-
henk, Javier Bautista, Jimmy Bonaiuto, and Salvador Marmol, wherever you are. And
special thanks to Sook Yee Leong.
ii
Acknowledgment: chapter 2 has been previously published, in substantially the same
form, as Network: Computation in Neural Systems 18(2):101{128 (Begley & Arbib, 2007).
It is included with kind permission of the publisher.
jrb
Los Angeles, California
February 2009.
iii
Table of Contents
Acknowledgments ii
List Of Tables vi
List Of Figures vii
Abstract x
Chapter 1: Introduction 1
1.1 Why study the salamander? . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Roadmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Chapter 2: Retina Model 12
2.0 Chapter abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 Model description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.3 Natural image simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.3.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.4 Simplied model simulation . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.5 Striped pattern OMS protocol simulations . . . . . . . . . . . . . . . . . . 44
2.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Chapter 3: Modeling Salamander Visual and Locomotor System Interac-
tion 57
3.0 Chapter abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.1.1 Ijspeert's CPG model . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.1.2 Locomotion control physiology . . . . . . . . . . . . . . . . . . . . 58
3.1.3 Salamander retina . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.1.4 Modeling background . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.1.5 Simulation method . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
iv
3.1.6 Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.1.7 Roadmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.2 Model description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.2.2 Brainstem model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.3 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.4 Dynamical analysis of empirical kinematics . . . . . . . . . . . . . . . . . 92
3.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3.7 Appendix: Retina model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.8 Appendix: Brainstem model . . . . . . . . . . . . . . . . . . . . . . . . . . 109
3.9 Appendix: Animated graphical simulation system . . . . . . . . . . . . . . 116
Chapter 4: Modeling Salamander Numerical Preference 120
4.0 Chapter abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
4.2 Model description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
4.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
4.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
4.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Chapter 5: Salamander Graphical Simulation Software 147
5.1 User Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
5.2 Results Data Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
5.3 Code Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
5.4 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
5.5 Simulation Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Bibliography 155
v
List Of Tables
2.1 Simulation image le details . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.1 Key results of Uller et al. (2003) reproduced in simulation. . . . . . . . . . 136
4.2 Weak surround inhibition improves model's numeric discrimination. . . . 137
4.3 Speed or size equalization abolish model's numeric discrimination. . . . . 140
4.4 Changing tube length changes model's numeric discrimination. . . . . . . 141
vi
List Of Figures
2.1 The full correlation-based motion detector model . . . . . . . . . . . . . . 19
2.2 Simulated wriggling objects on the natural image backgrounds . . . . . . 27
2.3 Synchronization of wriggler motion and global image motion . . . . . . . 30
2.4 Full retina images of 3 consecutive snapshots . . . . . . . . . . . . . . . . 37
2.5 Time series plot of model RGC activity . . . . . . . . . . . . . . . . . . . 38
2.6 Eect of sigmoidal nonlinear transformation of ltered motion detector
input on ganglion cell output . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.7 Simulation results for a simplied correlation-based motion detector model 43
2.8 Illustrations of simulation of experimental protocols . . . . . . . . . . . . 47
2.9 Results of striped pattern simulations . . . . . . . . . . . . . . . . . . . . 49
2.10 Simulations run without the sigmoidal transformation . . . . . . . . . . . 50
3.1 Modeled brain regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.2 The turn command and the weighted sum of the retinal salience vector . . 75
3.3 The simulation display . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.4 Paths taken during walking and swimming runs of the two models of visual
updating of locomotion steering at \image xation" times . . . . . . . . . 79
vii
3.5 Paths taken during walking and swimming runs of the simulation model
of continual visual updating of locomotion steering . . . . . . . . . . . . . 80
3.6 Distance to target during trotting and swimming runs . . . . . . . . . . . 82
3.7 Turn command and vector sum during simulations of trotting to the left-
ward target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
3.8 Turn command and vector sum (identical in this model) during simulations
of trotting to the leftward target using the continual visual steering model 85
3.9 Body segment joint motoneuron output of the spinal CPGs during simu-
lations of trotting to the leftward target . . . . . . . . . . . . . . . . . . . 87
3.10 Turn command during simulations of swimming straight ahead . . . . . . 88
3.11 Left body motoneuron output of the spinal CPGs during simulations of
swimming straight ahead . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.12 Paths to targets at arbitrarily chosen initial target bearings at 700 mm. . 90
3.13 Distance between head positions at successive reversals of rotational mo-
tion for simulations of swimming straight ahead to the forward target . . 91
3.14 Return maps of the distance between head positions at successive reversals
of rotational motion for simulations of swimming straight ahead to the
forward target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.15 Distance between head positions at successive reversals of rotational mo-
tion for simulations of walking straight ahead to the forward target, and
return maps of same . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.16 The correlation-based motion detector model of motion sensitive retinal
ganglion cells (RGCs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
3.17 The salience direction vector . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.1 The weighted sum of the salience vector . . . . . . . . . . . . . . . . . . . 129
4.2 Snapshot of the real-time simulation display showing the transition of
salience from the leftward group of
ies to the rightward . . . . . . . . . . 130
viii
4.3 Snapshot of the real-time simulation display showing the salamander ap-
proaching the virtual tube with the greater number of
ies . . . . . . . . . 132
4.4 Snapshot of the simulation display just after the start of a demonstration
run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
4.5 Three dierent dierence of Gaussian (DoG) spatial RGC lters modeling
the classical receptive eld (RF) . . . . . . . . . . . . . . . . . . . . . . . 139
ix
Abstract
This thesis examines the behavioral consequences of motion sensitivity of salamanders'
visual neurons. The analysis is centered on a correlation-based motion detector model
of motion-sensitive retinal ganglion cells (RGCs). This model is then integrated with
Ijspeert's model (Ijspeert, 2001) of spinal central pattern generators (CPGs) for
salamander walking and swimming. The integration in the salamander brain is studied
by means of a model of a sensorimotor pathway centering on the tectum. Vision is used
to control locomotion, while locomotion strongly aects vision.
Salamander undulatory locomotion may involve substantial side-to-side head movement
(Frolich & Biewener, 1992; Gillis, 1997; D'Aout & Aerts, 1997). Chapter 2 explores the
eect of the side-to-side movements on the activity of motion-sensitive RGCs. In
chapter 3, we consider the eects on locomotion gaits. We hypothesize that visual
control of the direction of salamander motion occurs when the side-to-side head
movement changes direction, a time of relatively low background motion motion. I.e.,
we propose that visual control of locomotion is suppressed at times of high retinal slip.
This hypothesis is supported by our simulation results and related analysis.
x
In chapter 4, we demonstrate how the model of chapter 3 can generate behavior that
has been interpreted as numerical cognition (Ansari, 2008). Our results support the
hypothesis that the observed numerical preferences of salamanders are emergent
properties of the early visual system, rather than evidence of higher cognitive ability.
xi
Chapter 1
Introduction
1.1 Why study the salamander?
The salamander has the simplest vertebrate nervous system, with a total of approximately
10
6
neurons. Living salamanders are thought to have evolved in the direction of neural
simplicity from the common ancestor of terrestrial tetrapods (Roth et al., 1997), i.e., their
nervous systems have devolved. We may think of them as \minimal" vertebrates.
Nevertheless, salamanders have been an evolutionary success. They have adapted and
thrived through many millions of years. In studying the salamander nervous system, we
are studying not only an extremely simple system, but an eective one. We presume it
must eciently use its limited neural resources to accomplish the tasks of animal survival.
The eciency and ecacy of the salamander nervous system are evident in the control of
their multimodal (aquatic and terrestrial) locomotion. This thesis oers a contribution
toward understanding sensory control of this behavior.
1
Salamander locomotion control has been adapted to the control of an amphibious
robot (Ijspeert et al., 2007, 2005; Ijspeert, 2008). This type of locomotion may be of use
in reconguable robotics (Shen et al., 2006), where veratile, possibly autonomous robots
join in a salamander-like conguration. Results of this thesis, in addition to those of
Ijspeert (2001); Ijspeert et al. (2007); Ijspeert (2008), indicate that the modules would
need a simple function processing just 2 input control signals to function properly in a
coherently moving chain of segmental modules performing collective directed locomotion.
The results of this thesis contribute to understanding how the salamander's visual system
may be used to control this locomotion.
The salamander nervous system has been studied extensively (e.g., Roth (1987); Roth
et al. (1998)). In particular, the salamander retina has been studied as a model vertebrate
retina (e.g.,

Olveczky et al. (2003); Baccus et al. (2007); Teeters et al. (1997)). With this
thesis, we hope to contribute to an understanding of how aspects of the nervous system
are integrated to produce behavior (e.g., locomotion and more complex behavior such as
that studied in Uller et al. (2003)).
In the more distant future, the study of a remarkable aspect of the salamander nervous
system, its unique ability to regenerate after injury (e.g., Cherno et al. (2003); Okamoto
et al. (2007)) may contribute to the development of regenerative technology for the human
brain. Perhaps the regenerative ability of the salamander may simplify somewhat the
interface of neural prosthetics with live neural tissue. In that case, the simplicity of the
salamander brain and the relative ease of working with it may make it an advantageous
2
model for solving certain problems in the development of neural prosthetics. One may
hope that this could lead to replacements for rather substantial brain regions, and/or
assistive devices closely integrated with the brain. If so, brain simulation is likely an
important early step on this path.
1.2 Background
The biological background of the modeling presented in this thesis includes retinal mo-
tion sensitivity, midbrain control of central pattern generator (CPG) -based locomotion,
and numerical preference. Here, we discuss the primary biological motivations for the
contribution of this thesis, and the modeling literature which motivated and inspired our
work. Subsequent chapters also survey the modeling context and the biological grounding
for work they describe.

Olveczky et al. (2003) identied what they called \object motion sensitive" (OMS)
RGCs in isolated salamander retinas. \Object motion sensitivity" refers to the response to
grating stimulus movement, where the grating movement in the surround diers from the
movement in the cell's receptive eld (RF) center (the excitatory RF, ERF). The authors
have oered models of the cells' behavior, based on simple, classical linear-nonlinear (LN)
techniques. Their empirical results suggested the need to consider the eects of motion
opponency in the classical center-surround RF on biological functions that involve vision.
The results of

Olveczky et al. (2003), expanded upon in Baccus et al. (2007), involved very
slow random movements about a visual xation point, as might be induced by salamander
3
respiration. In this thesis, we consider the mutual eects of this visual processing and
the salamander's motivated locomotion. In chapter 2, we present a correlation model of
motion detection in the salamander retina, testing this model with the side-to-side head
movements, which are often seen in the motivated locomotion of salamanders.
One approach to modeling the retina is to attempt to reproduce known retinal ganglion
cell (RGC) response properties with models incorporating what is known about the low-
level properties of the retinal cells from the photoreceptors on out to the RGCs. Examples
of this are found in the work of Teeters (Teeters et al., 1997; Teeters & Arbib, 1991;
Teeters et al., 1993). One diculty with this approach is that behaviorally relevant RGC
properties may not arise directly from currently known low-level properties of retinal cells.
In particular,

Olveczky et al. (2003) suggested center-surround motion opponency as an
operating principle, although the detailed neural substrate of its implementation had yet
to be worked out. Despite all that has been discovered about the retina, a complete
account is not yet available { research continues.

Olveczky et al. (2003) presented a model which reproduced their object motion sen-
sitivity (OMS) results. Their model was composed of very simple models of retinal
processing { this approach is minimalist in the sense that the model uses the least detail
required to produce the experimental results. The problem with this approach, for our
purpose, is that their experimental results were actually rather limited { salamander be-
havior will produce retinal stimuli that tax the generality of the model. Inadequacies in
the model were revealed by continued research. Complications were added to the model
4
in Baccus et al. (2007), but the results are still quite limited compared with the range of
stimuli to which a behaving animal is exposed.
Woesler (2001), in his object segmentation model inspired by the salamander, used
a very simple rule to model the RGCs. This approach gave the minimum functionality
required for that particular task. Previous to that, models of visually-guided amphibian
behavior likewise used minimal retinal models specically tailored to the problem at
hand (Eurich et al., 1997, 1995; Wang & Arbib, 1992, 1990; Liaw & Arbib, 1993). In our
modeling, we sought a more general, and more robust approach.
Our approach was to explore the behavioral consequences of the general operating
principle suggested by

Olveczky et al. (2003), center-surround motion opponency. In neu-
ral models of motion detection in 2-dimensional images, there are two general techniques:
gradient-based and correlation-based methods Borst (2007); Hildreth & Koch (1987).
Gradient-based models are unusual, but have been used (Srinivasan, 1990). These mod-
els tend to be complicated { since the basic model involves division by the space derivative
of luminance, the computation is undened when the local spatial contrast is 0 at the
input to the motion detector. The basic correlation detector computation, on the other
hand, is always dened. Correlation-based motion detection has been used extensively
in models of invertebrate vision (Zeil & Zanker, 1997; Reichardt et al., 1989; Egelhaaf
& Reichardt, 1987), and (to a lesser degree) in models of human vision (Zanker, 2004,
1996). It has been argued that correlation-based motion detectors have subtle properties
that make them more useful than gradient-based methods (Borst, 2007; Zanker, 1996).
5
Correlation-based detectors have contrast as well as movement sensitivity (Zanker & Zeil,
2005), as do salamander RGCs (Roth, 1987; Burkhardt et al., 1998;

Olveczky et al., 2003;
Baccus et al., 2007).
Segev et al. (2006) provided much of the primary data grounding our retina model
of chapter 2. Among many other interesting results, they were unable to nd direction-
selective (DS) RGCs in the salamander retinas studied. Although

Olveczky et al. (2003)
did not specically test for motion diering in direction between center and surround,
generalization of the object motion sensitivity concept leads to the hypothesis that sala-
mander RGCs might respond when motion in the inhibitory surround diers in direction
from motion in the central ERF. Investigating this possibility, we propose that direction
selectivity of retinal interneurons (e.g., amacrine cells) is lost at the RGC layer through
summation across the DS channels. If this is so, then center-surround opponency may be
the only vestige of DS in salamander RGC responses.
Ijspeert (2001) developed an articial neural network model of the salamander spinal
CPGs. Visual control of locomotion using this model is the core of this thesis. Ijspeert's
CPG network was developed using a genetic algorithm variant. The network evolved in
stages. First, a core oscillator evolved. Then Ijspeert evolved interconnections between
these oscillators, arranged as a spinal cord. These oscillators control body segments in
anguilliform swimming, the type of locomotion observed in sh. Finally, coupling evolved
between limb oscillators and the body segment oscillators, thus developing terrestrial-
style locomotion. Ijspeert (2001) did not simulate the biological evolution of vertebrate
6
locomotion. The tness criteria rewarded the oscillation patterns actually observed in
salamanders { the evolution was teleological, where the detailed result was intended from
the outset. Furthermore, the evolution occurred over a very few generations, compared
with the length of the history of the evolution of the contemporary salamanders, whose
locomotion is simulated by the result of the evolutionary algorithm. The network so
developed was mechanically coupled to the environment with a mechanical simulation of
the physics of locomotion. Ijspeert (2001) found that the left-right direction of locomotion
could be controlled by applying dierent top-down tonic drive inputs to the left and
right oscillators. This is the starting point of chapter 3, where model visual control of
locomotion based on Ijspeert's model.
The means by which the direction of motion is controlled in this model is consistent
with more recent biological ndings. Fagerstedt et al. (2001) reported nding reticu-
lospinal neurons that appeared to be associated with steering in the lamprey that we
nd to be consistent with the steering mechanism in the salamander model of Ijspeert
(2001). Cabelguen et al. (2003) found that MLR stimulation in the salamander elicits
locomotion, and that varying stimulus intensity could vary the speed of the induced lo-
comotion movement and could induce gait transitions between stepping and swimming.
In Ijspeert's model, we use a higher tonic drive level for swimming, compared with the
level for walking { the relative intensities are consistent with the ndings of Cabelguen
et al. (2003).
7
The tectum is the salamander's primary visual brain region (Roth, 1987). Saitoh et al.
(2007) found that tectal electrostimulation in a limited region could induce locomotion,
as well turning and orienting movements. These experiments could not distinguish the
projections that delivered the tectal motion control to the spinal cord: the tectum might
steer via projections to the MLR, or it may operate through direct projections to the
reticulospinal tract. In our model of visual steering of CPG-based locomotion, presented
in chapter 3, we do not formally include the MLR, but the model does not depend on the
pathway by which the steering modulation is delivered to the spinal CPGs. The visual
input to this model is based on the retina model of chapter 2. Our simulations of visual
control of locomotion are accomplished with a relatively simple model of the salamander
body and environment in which the salamander swims in water or walks on land. A
simple moving prey-like object, somewhat abstracted from the visual stimuli of Schuelert
& Dicke (2005) and Sch ulert & Dicke (2002), provides the attractive target toward which
the salamander steers.
Cobas & Arbib (1992) introduced the \motor heading map" concept in their discussion
of prey-approach and predator-avoidance behavior in the frog. Their purpose was to
model the two basic schemas sharing a common neural substrate. Liaw & Arbib (1993)
built upon this in an avoidance model. The frog's locomotion choice emerged from the
interaction of these schemas. In contrast, our original purpose was to model precise visual
steering of locomotion. The salamander response to aversive stimuli is to turn way (Azizi
& Landberg, 2002), which we think of as less demanding of steering precision. Thus,
8
approach behavior is the test of steering eectiveness, the visually salient stimuli are
considered attractive, and winner-take-all is the principle by which the steering direction
is determined. Thus, our 1-dimensional \salience map" in chapter 3 has characteristics of
the motor heading map. We do not model the behavioral choice of avoidance; however,
a complete account of salamander behavior would require such a choice mechanism.
An open question for Roth (1987) was the accuracy of salamander snapping and
orientation movements, given the large extent of individual tectal neuron receptive elds.
Simulander (Eurich et al., 1995) was a simulation system that, with a network of 100
model large RF tectal neurons, achieved orientation movement accuracy comparable to
that of live salamanders. These ndings were veried analytically in Eurich & Schwegler
(1997). Simulander II (Eurich et al., 1997) extended these ndings to 3 dimensions in
a binocular vision simulation of projectile tongue control. These papers constitute the
answer to Roth's question.
Simulander I & II are models of visual control of movement toward nearby targets,
while in chapter 3 we control movement toward distant targets. In chapter 4, we illustrate
the need for an integration of these models. The simulations end when the salamander
comes close enough to a target for short-range visual mechanisms. A complete account of
visually guided behavior in salamanders requires both short-range and long-range vision.
In chapter 4 we oer a new interpretation of the ndings of Uller et al. (2003) that
salamanders tend to walk to the larger of two groups of
ies under certain specic ex-
perimental conditions. Ansari (2008) interpreted these results as salamander sensitivity
9
to the integral number of objects. However, we were able to reproduce the results of
Uller et al. (2003) by manipulating the relative strength of the RGC inhibitory surround.
Furthermore, we found that the limitations on numerical discrimination were sensitive to
the conguration of the stimuli seen by the simulated salamander. Our results support
an interpretation of the results of Uller et al. (2003) as emerging from the properties
of low-level visual processes, rather than as evidence of primary sensitivity to integral
numbers.
An in
uential model of numerical cognition was reported in Dehaene & Changeux
(1993). This was a model of nonlinguistic numeric processing in humans and higher
vertebrates. These authors distinguish the cognitive processing that they are modeling
from counting. However, their model (as well as that of Domijan (2004)) does produce
an internal representation of the number of items it sees. We distinguish this from our
modeling in chapter 4, the results of which might be supercially interpreted as numeric
processing, but which in fact involves no representation of number at all. In this, our
modeling follows in the spirit of Cli & Noble (1997), in which the results of Cli et
al. (1993) were interpreted as wholly lacking internal neural representations. This was
a loosely biologically motivated robotics experiment involving the evolution of articial
neural networks for guidance of simple robots, rather than a model of a specic biological
system. Without endorsing the argument of Cli & Noble (1997) in full, we join these
authors in warning against drawing undue inferences from observed behavior.
10
1.3 Roadmap
Chapter 2 fully describes the retina model, and presents ndings on the model's proper-
ties. Chapter 3 describes our model of visual steering of locomotion, and the implications
of this model. This chapter has an appendix brie
y summarizing the retina model of
chapter 2. Chapter 4 presents our ndings concerning numerical preference in the sala-
mander vision model of chapter 3. This chapter includes an appendix brie
y summarizing
the model of chapter 3.
11
Chapter 2
Retina Model
2.0 Chapter abstract
We report on a computational model of retinal motion sensitivity based on correlation-
based motion detectors. We simulate object motion detection in the presence of retinal
slip caused by the salamander's head movements during locomotion. Our study oers
new insights into object motion sensitive ganglion cells in the salamander retina. A sig-
moidal transformation of the spatially and temporally ltered retinal image substantially
improves the sensitivity of the system in detecting a small target moving in place against
a static natural background in the presence of comparatively large, fast simulated eye
movements, but is detrimental to the direction-selectivity of the motion detector. The
sigmoid has insignicant eects on detector performance in simulations of slow, high con-
trast laboratory stimuli. These results suggest that the sigmoid reduces the system's
noise sensitivity.
12
This chapter has been previously published, in substantially the same form, as Net-
work: Computation in Neural Systems 18(2):101{128 (Begley & Arbib, 2007). It is in-
cluded here with kind permission of the publisher.
2.1 Introduction
In studying the interaction of salamanders' motion sensitive neurons with the side-to-side
head movements often observed in these animals' locomotion, we developed a simple nat-
ural image simulation protocol. We sinusoidally scan a static natural image in order to
simulate the retinal slip generated by these head movements. We accept the limitations
of simulating visual motion with a
at image in order to capture a part of natural image
structure that the salamander eye evolved to handle { specically, the static structure of
these images. This method does not consider dynamic phenomena of a 3-D visual envi-
ronment (e.g., motion parallax, illumination changes, optic
ow perspective, expansion
and contraction, and the motion of scene elements at varying distances).
Salamanders present interesting opportunities to study visual motion. They eectively
use vision during locomotion, both swimming and walking (Roth, 1987), yet possess
very few neurons, compared with other vertebrates (Roth et al., 1998, 1995). Their
locomotion and vision seem to have interesting interactions. Salamanders' eyes do not
move with respect to the head (Roth, 1987). They may experience large horizontal head
movement during locomotion (Frolich & Biewener, 1992; Azizi & Landberg, 2002; D'Aout
& Aerts, 1997; Gillis, 1997; Ijspeert, 2001), directly causing retinal slip. It would be useful
13
for the animal, while swimming or walking, to be able to distinguish moving objects
from wide-eld motion percepts induced by head movements. This chapter reports on a
computational investigation of this possibility.
Although the salamander retina has fewer neurons than, say, the mammalian retina,
it's structure and function are remarkably similar (Roth et al., 1998; Dowling, 1987),
so the salamander is often used as a model organism in retina research. For example,

Olveczky et al. (2003) reported nding object motion sensitive (OMS) retinal ganglion
cells (RGCs) in the salamander retina. OMS neurons respond when motion in the re-
ceptive eld (RF) center diers from motion in the RF surround. These results suggest
that such neurons could be of use in distinguishing self-motion from the motion of in-
dependently moving objects in the visual environment.

Olveczky et al. (2003) used low
magnitude sporadic random jitter movements of high contrast articial stimuli, thereby
eliminating contrast sensitivity as a variable factor in their results. In terms of animal
behavior, their results were applicable to the jitter of the eye when the animal is not
intentionally moving. In the case of the salamander, the animal's eye jitters somewhat
due to such causes as respiration (Manteuel et al., 1977).
The work of

Olveczky et al. (2003) left open two question that we address in this
chapter: How might such a mechanism respond to natural images? And, furthermore,
could such a mechanism distinguish self-motion from independently moving object motion
when the animal is more active, particularly during locomotion?
14

Olveczky et al. (2003) proposed a model of OMS responses in which motion sensitivity
was a property of temporal and spatial lter units, nonlinearly transformed and spatially
pooled. The object motion response of the model was based on subtraction of the pooled
wide-eld responses from the responses in the receptive eld center. The sensitivity to
dierential motion between the center and surround in their model depends on dierent
time courses of jitter in the RF center and surround { the model is not sensitive to
dierences in motion direction (i.e., matching velocity time courses in center and surround
will synchronize excitation and inhibition, even when the motion directions dier). In
their model, the apparent lack of directionality of salamander RGCs (Segev et al., 2006)
is a fundamental property of the basic mechanisms of motion detection.
Amacrine cells are a very diverse class of inner retina neurons (see Baccus (2007)
for a review).

Olveczky et al. (2003) demonstrated the involvement of glycinergic wide-
eld amacrine cells in generating the responses of OMS RGCs. The specic role of
these amacrine cells seems to be surround suppression { they spatially pool the motion
sensitive responses and inhibit the OMS RGC. Other work has found involvement of
amacrine cells in motion discrimination. The starburst amacrine cell, which releases
both the inhibitory neurotransmitter GABA as well as the excitatory neurotransmitter
acetylcholine (ACh) (O'Malley et al., 1992), has been shown to play an essential role in
retinal direction selective (DS) motion detection (Yoshida et al., 2001; Amthor et al.,
2002). The fundamental DS computational units are individual starburst cell dendrites,
which have both pre- and post-synaptic sites (Euler et al., 2002; Fried et al., 2002; Lee
15
& Zhou, 2006). Grzywacz et al. (1998) found that cholinergic synaptic transmission is
necessary for direction selectivity to textured image motion, but not to moving striped
patterns. Salamanders seem to lack DS RGCs (Segev et al., 2006), and, to date, starburst
cells have not been found in salamanders (V gh et al., 2000), but Zhang & Wu (2001)
recently discovered cholinergic salamander amacrine cells. In our modeling, we propose
that direction selectivity in the salamander inner retina is ultimately integrated away at
the ganglion cell output.
The object motion sensitivity in the model of

Olveczky et al. (2003) may have de-
pended on the time interval between the intermittent jitter movements, during which time
no movement occurred. The suppression of the response to coherent motion (throughout
the receptive eld) resulted from the temporal overlap of the transient motion signals in
the excitatory center and inhibitory surround of the RGC RF. Assuming that the orien-
tation sensitive units that gave the transient responses were also contrast sensitive, we
suppose that their model results also depended on having commensurate contrast distri-
butions in the center and surround. Given the sparseness of motion signals in natural
environments (Zanker & Zeil, 2005), continuous movement in a natural visual environ-
ment may not preserve this synchrony.
The correlation-based motion detector (Reichardt, 1961), also called the the Reichardt
elementary movement detector (EMD), has been commonly used in modeling biological
visual motion sensitivity for 50 years (Borst & Egelhaaf, 1989; Cliord & Ibbotson, 2003).
Delbr uck (1993) used used a correlation-based motion detector in a hardware retina
16
model. Reichardt et al. (1989) used the correlator to model biological gure/ground seg-
mentation, using center/surround opponency of pooled motion-sensitive neural signals for
object motion detection (cf. the model of

Olveczky et al. (2003), in which non-directional
motion sensitive signals were pooled for object motion detection). An advantage in many
applications is the numerical robustness of correlation-based motion detectors, even when
the local image motion is undened (i.e., when their is no luminance change) (Sarpeshkar
et al., 1996).
The Reichardt detector { commonly used to model direction-selective motion sensitiv-
ity { is highly sensitive to visual motion direction, but is considered relatively insensitive
to image speed (Sarpeshkar et al., 1996). Dror et al. (2001), showed that correlator perfor-
mance tends toward an average speed sensitivity, although the output on ner time-scales
is rather erratic.

Olveczky et al. (2003) did not report direction selectivity in the salamander OMS
RGCs they studied, while Segev et al. (2006) reported nding no direction selective
salamander RGCs at all after a diligent search. Apparently, any direction sensitivity
that the RGCs may use to distinguish center-surround motion dierences is eectively
balanced at the cells' axonal output.
In this chapter, we propose to model object motion sensitivity with correlation-based
motion detectors. Such detectors generally model direction selective motion sensitivity.
We nd that a neurally plausible nonlinear transformation of the input to the correlator
will reduce the directionality, but improve the object motion sensitivity, and, in some
17
cases, give an acceptable match to some of the results of

Olveczky et al. (2003). The
directionality remaining after this transformation is then completely washed out by pool-
ing of nominally directional subunits before the ganglion cell output. Grzywacz & Koch
(1987) showed that 2
nd
order motion detectors, particularly correlation-based detectors,
have characteristic responses to the spatio-temporal frequencies of their input when the
input is linearly ltered. Zanker et al. (1999) analyzed this in detail for Reichardt detector
responses to sinewave gratings. This has made it possible to experimentally and analyt-
ically characterize such motion detectors, but the vertebrate retina includes nonlinear
processes that might make it dicult to identify a 2
nd
order detector.
We have developed a simulation model of OMS RGCs using pooled correlation-based
motion detectors. We use this model to investigate an extension of object motion sensi-
tivity to object identication during self-motion. Inx2.3, we model short time periods of
the salamander's locomotion-induced retinal slip by sinusoidally scanning static images.
We investigate details of the model's performance inx2.4 by simulating a simple protocol
on a system that isolates the temporal preltering parts of the model, and, inx2.5, by
simulating protocols comparable to those of

Olveczky et al. (2003).
2.2 Model description
Figure 2.1 shows our correlation-based motion detector model. The two multiplications
(g. 2.1(g)) followed by subtraction (g. 2.1(h)) in the lower part of the diagram are the
essence of the motion detector. Each ltered input is multiplied by a time-delayed version
18
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(j)
(k)
(m)
(i)
(l)
Figure 2.1: The full correlation-based motion detector model. (a) 2-D photoreceptor layer,
simulated as image pixels; (b) 3 3 convolution mask (small DoG); (c) rectication; (d) Biphasic
bandpass temporal lter; (e) Sigmoidal transformation, S
0
{ empirical investigation of the eect
of this nonlinearity is a major part of this paper; (f) Low-pass lter for temporal correlation;
(g) Multiplication (correlation); (h) Subtraction; (i) Adaptive sigmoid, S
1
; (j) Rectication of
rightward and leftward motion selective channels; (k) 39 39 convolution mask (big DoG); (l)
Rectication; (m) RGC output is the sum of directional channels.
19
of the other input. The delay is eected with a low pass lter (g. 2.1(f)). The time
constant of this lter is not crucial to the results we obtained; tests done with wide range
of time constants (5{300 ms.) produced results very similar to those we are reporting.
The simulations reported in this chapter used a delay time constant of 30 milliseconds.
Suppose that the amacrine cell membrane is the basic memory element providing
the temporal asymmetry. Coleman & Miller (1989) found amacrine cell membrane time
constants generally greater than 30 ms. { we presume that cells in active animals are
livelier than cells in isolated tissue preparations.
The input to the correlator consists of linear-nonlinear (NL) lters, a spatial lter
(g. 2.1(b,c)), followed by a temporal lter (g. 2.1(d,e)). The linear spatial prelter is
a 3 3 convolution mask with positive center and negative surround (in eect, a small
\Dierence of Gaussians" (DoG)) (g. 2.1(b)). The relative weight of the center equals
the negative of the sum of the surround relative weights, so this lter removes the DC
component of the pixel intensity signals.
1
The spatially ltered pixels are rectied (g. 2.1(c)). Vertebrate retinas rectify On
and O channels very early. Although we do not distinguish these dierent channels in
the present work, it is important to be able to model this fundamental property of the
retina.
2
1
The relative strengths of center and surround in this model are well within the range for salamander
bipolar cells found in Fahey & Burkhardt (2003). (Inx2.3 we further discuss how this receptive eld model
relates to receptive elds of retinal cells in the salamander, in terms of the specic simulation protocol.)
2
Note that this nonlinearity precludes the spatio-temporal stimulus-response analyses of Zanker et al.
(1999), Ibbotson & Cliord (2001) and Grzywacz & Koch (1987) { the ltered sinewave will no longer be
sinusoidal.
20
The linear temporal prelter is a biphasic bandpass lter(g. 2.1(d)). This lter
produces transient responses. Our biphasic lter is sine-symmetric with a period of 100
ms. It has a positive early response, and a negative response in the later, refractory phase.
The steady state response of the biphasic prelter to constant input is 0, re
ecting the
view that the eye responds to visual change.
Ibbotson & Cliord (2001) found that a biphasic temporal prelter caused tripha-
sic unbalanced Reichardt motion detector responses. An unbalanced correlation-based
detector is one in which products are multiplied by unequal weights before subtraction
(g. 2.1(h)). Inx2.4, our empirical results for a very dierent correlation-based motion
detector system are comparable.

Olveczky et al. (2003) used a similar temporal prelter in their model of OMS RGCs.
Biphasic ltering has been reported in the salamander outer retina (Armstrong-Gold &
Rieke, 2003). Segev et al. (2006) demonstrated biphasic responses in salamander RGCs.
Of particular interest to us is the response prole of what Segev et al. (2006) term
\biphasic o" cells.

Olveczky et al. (2003) found that their salamander OMS cells were
of the \fast o" type. We consider these two types a match. (100 ms. is shorter than
the period we nd in Segev et al. (2006). We are modeling sensorimotor interactions
behaving animals. We expect cells in such animals to respond faster than cells in isolated
tissues.)
We applied a sigmoidal nonlinear transformation to the biphasic temporal bandpass
lter output (g. 2.1(e)). The sigmoidal function is S
0
(x) = (1 +exp(a(xb)))
1
.
21
The parameters for this sigmoid were chosen so that S
0
(0) = 0:01, S
0
(1) = 0:99, and
S
0
(
1
2
) =
1
2
. It asymptotically approaches the upper bound of 1 and the lower bound of 0.
Negative bandpass lter output gives sigmoid output between 0 and 0.01, so the neuron
ring rate can be depressed, in a refractory period, from the nominal background (steady-
state) level of 1% of the maximum rate. Positive bandpass lter outputs are between 0
and 1. The sigmoid symmetrically maps this range onto a large portion of itself, 0.01 to
0.99. The goal of these choices was that, to the greatest possible extent, the correlator
output with and without this sigmoid may be compared. The sigmoidal transformation
of the correlator input has a considerable eect on the model responses, as we detail in
subsequent sections.
The output of the temporal lter constitutes the input to the correlator (g. 2.1(f{h)),
described above.
Zanker & Zeil (2005) found that motion signals caused by self-motion in a natural
visual environment have a highly kurtotic distribution (i.e., they are very sparsely dis-
tributed { the distribution is sharply peaked near zero, and has \fat tails" away from
zero), so they applied a threshold maximum of 25% of the actual maximum.
Our approach is inspired by Zanker & Zeil (2005) { we apply a sigmoidal nonlinearity
to the result of the correlator (g. 2.1(i)). Our sigmoid achieves much the same eect {
we, too, found very sparsely distributed motion signals in our natural image simulations.
The functional form of our sigmoid is S
1
(d
i;j
) = tanh(d
i;j

arctanh(0:99)
0:25max
k;l
(jd
k;l
j)
), where d
i;j
is the detector output at 2-D location (i;j), and max
k;l
(jd
k;l
j) is the maximum absolute
22
value of detector output at all retinal locations.
3
This function is nearly horizontal for
inputs of absolute value above 25% of the maximum (so it is squashing in this part of its
range). Thus, it compresses the dynamic range of the motion signals in a manner similar
to the threshold maximum of Zanker & Zeil (2005). In addition, this sigmoid tends to
broaden the distribution around 0 (so it is expansive in this part of its range), allowing
greater resolution in the region where the highly kurtotic distribution has greatest weight.
Finally, this particular sigmoid retains the motion signal symmetry about zero { the sign
of the motion signals indicate the opposing motion directions. Throughout most of the
range of motion in the natural image motion simulations, the absolute value of the sigmoid
produces images in which contours and textures of the scene are recognizable to human
observers. Substituting other choices for the 0:25 in the formula were likely to result in
either images with so many saturated pixels that image detail was lost, or else in images
with such sparely distributed luminance that, again, image detail was lost.
This particular sigmoid models extremely fast adaptation { the slope of the sigmoid
is a function of the maximum absolute input at every timestep. We will suggest later,
inx2.6, that this is a conservative assumption, given our ndings. That is, the system
could, with much slower adaptation, tune to adequate object motion sensitivity under
the circumstances in which it was observed in our experiments.
The nonlinearly transformed correlator dierence is rectied into leftward and right-
ward motion signals. These two rectied channels form direction selective layers, to which
3
Note that this function is dierent from the temporal lter sigmoid of gure 2.1(e),S0, re
ecting the
dierent purposes of the two functions.
23
a nal linear-nonlinear (LN) spatial lter is applied. A 39 pixel diameter DoG spatial
lter, followed by rectication, is applied at every 4
th
cell in both dimensions. This mask
is inspired by previous work in amphibian vision modeling (Teeters & Arbib, 1991; Wang
& Arbib, 1990), with a parameter change so that the mask integral is 0. This DoG gives
center-surround opponency. The mask, applied to direction selective channels, is intended
to implement object motion sensitivity. If the motion in the center is the same as the
motion in the surround, at a particular location, the value of the convolution at that
location is 0. The lter output is non-zero only when the motion signals in the center
in the preferred direction exceed those in the inhibitory surround. (Inx2.3 we further
discuss this receptive eld model.)
Segev et al. (2006) found that the \biphasic o" cell type was the only RGC type
completely tiling the retina. The spacing of our simulated RGCs leads to overlap of
the excitatory receptive elds (ERFs) (i.e., the positive parts of the DoGs), i.e., our
simulatated RGC array overtiles the retina.
The ltered signals for the direction selective channels are rectied and summed to
give the simulated RGC output. Thus, the RGC output is sensitive to dierent motion
velocities in the center and surround, but does not have a preferred motion direction.
One caveat is that we model motion sensitivity in the horizontal direction only, but
the locally measured motion direction of a moving edge is normal to the edge, regardless
of the actual motion direction { the \aperture problem" (Hildreth & Koch, 1987). A
model sensitive to motion in planar images requires a basis set of detectors for at least
24
two directions, even when the actual global image motion is in one dimension only (Zeil
& Zanker, 1997; Zanker, 2004; Zanker & Zeil, 2005). However, the direction-selective
channels are independent { although the model may miss some motion signals, the ones
that do occur in the simulations would be unaected by the missing channels.
Our model of retinal motion sensitivity is quite simple compared to the actual retina.
But even this simplied model contains a number of nonlinear processing steps. Formal
analysis would be very dicult; so, in the next section, we seek insight through systematic
simulation experiments
2.3 Natural image simulations
2.3.1 Methods
Our natural image simulation protocol is inspired by the side-to-side head movements
often seen in salamander locomotion. The horizontal rotational components of these
head movements directly give rise to retinal slip; the translational components
4
cause
more complex retinal motion patterns which we do not consider in this study (seex2.1).
Our method for simulating this retinal slip is to sinusoidally scan a static natural gray-
scale image. In this scan, at each simulation time step, the position of the retina (the
photoreceptor array) is computed; the image pixels at that location form the motion
detector input. The position is rounded to an integral pixel location { pixel values are
not interpolated (cf.x2.4).
4
There will be a translational component of the eye movement even for pure head rotations, since the
horizontal rotational axis cannot be located in both retinas.
25
Our intent is to capture some of the image structure that the salamander eye evolved
to handle { in this study, we consider only the static structure of natural images. Our
simulations will operate in the contrast and spatial correlation range of natural images.
In order to test object motion sensitivity, we superimposed a small \wriggling" object
on the scanned image. Figure 2.2 shows the \wriggler" in the image contexts at three
levels of detail. The wriggler consists of 3 horizontal segments, stacked vertically. Each
segment consists of a light and dark stripe. Each segment moves horizontally back and
forth a few pixels. The segment oscillations are sinusoidal. The top and bottom segments
are perfectly aligned throughout the movement cycle, while the movement of the middle
segment is 180
o
out of phase with that of the other two. The size of the wriggler is such
that it ts entirely within an excitatory receptive eld of the 39 39 pixel DoG (see
x2.2). In all simulations, each wriggler segment was 6 pixels wide by 3 pixels high. The
total peak-to-trough range of motion was 4 pixels { giving a maximum oset of 4 pixels
between the top/bottom segments and the middle segment.
The gray scale intensities of the light and dark stripes are based on local image
statistics. We compute the mean, , and standard deviation, , of the image pixel
intensities in the 3939 pixel RGC receptive eld (RF) minimum bounding box centered
on the wriggler. In the simulations presented in this chapter, the stripe intensities are

1
2
.
26
(a) (b)
(c) (d)
Figure 2.2: Simulated wriggling objects on the natural image backgrounds used in the scanning
experiments reported in this paper. For each of the 4 images, the full image is shown (left),
along with a 39 39 pixel model RGC RF minimum bounding box centered on the wriggler (top
right), and a detailed view of the wriggler (bottom right). The wriggler is shown at maximum
displacement, i.e., the top and bottom segments are at the right-most position in their horizontal
range, while the middle segment is at the left-most position. The wriggling object is framed with
a small white rectangle in the full image, and with black rectangle in the top right-hand view.
The only dierence between 2.2(b) and 2.2(c) is the wriggler position; some RGCs that scan the
wriggler in 2.2(b) also spend substantial time scanning the bright, high contrast
owing water.
Each of the 12 images is shown with scaled brightness; for the smaller images in the right-hand
columns, the absolute intensity range is much lower than it appears. The two wriggler brightness
levels are the mean of the background in the RF bounding box
1
2
of the standard deviation.
The full images can be downloaded at http://hlab.phys.rug.nl/imlib/index.html (van Hateren &
van der Schaaf, 1998). See table 2.1 for le names.
27
File name Window Wriggler ctr Wriggler illum. Full img mean, sdev, max, min Figures
imk00347.iml (601:1193,400:800) (400, 221) 0.182, 0.143 0.219, 0.105, 1, 0.0261 2.2(a) 2.6(a)
imk00877.iml (601:1193,400:800) (400, 181) 0.0357, 0.0241 0.0859, 0.124, 1.00, 0.0123 2.2(b) 2.6(b)
imk00877.iml (601:1193,400:800) (400, 121) 0.0475, 0.0359 0.0859, 0.124, 1.00, 0.0123 2.2(c) 2.5 2.4
imk04208.iml (435:1027,125:525) (400, 101) 0.200, 0.170 0.205, 0.0821, 0.678, 0.0218 2.2(d) 2.6(c)
Table 2.1: Simulation image le details. See text and referenced gures for explanation.
28
In gure 2.2, the top right-hand detailed view of each image shows the wriggler in its
local image context { within the minimum RGC RF bounding box. Each image in this
gure is scaled according to the intensity range for that image. Generally, the gray-scale
pixel illumination intensities are in the range (0::: 1). Table 2.1 gives the actually pixel
values for the wrigglers' light and dark stripes. It also shows illumination statistics for
the entire image window { in general, the absolute contrast is low, in the context of the
entire image window. Object motion sensitivity is a local property { we will show in
x2.3.2 that the wriggler contrast is sucient for very good motion detection. Figure 2.2
shows the image window, with the (very small) wriggler superimposed and bordered by
a small white rectangle { the white-bordered regions in the gure 2.2 left-hand full image
views correspond to the black-bordered regions in the top right-hand detailed views. The
challenge is to detect the low magnitude movement of a low-contrast object within this
very small window, and to detect it in the presence of visual motion signals, caused by
self-motion, over the entire image.
A range of frequencies for both the wriggler and the scan were simulated, as was a
range of scan pixel distance ranges. In the simulations presented in detail in this chapter,
the scan range, side-to-side, was 90 pixels, the scan frequency was 2.5 Hz and the wriggle
frequency was 2 Hz
5
. Thus, the total scan motion was 450 pixels per second, while the
total wriggle motion was 16 pixels per second. These parameters were chosen for both
biological plausibility, and to challenge the system. The scan motion is at the high end of
5
These parameters may have some behavioral relevance { a salamander walking or swimming with a
2.5 Hz locomotion oscillation might well be interested in, say, a caterpillar wriggling at 2 Hz.
29
of what a salamander would normally experience during locomotion. (2.5 Hz is fast for
a salamander terrestrial gait, and within the range of a normal aquatic gait. Using RF
sizes detailed below, we nd the range of the head movement implied by the 90 pixel scan
is over 30
o
, large for locomotion. (Frolich & Biewener, 1992; Azizi & Landberg, 2002;
D'Aout & Aerts, 1997; Gillis, 1997; Ijspeert, 2001)) The challenge is to detect the slow
wriggle movement embedded in the retinal slip generated by the much faster scan.
0 500 1000 1500
0
10
20
30
40
50
60
70
80
90
0 500 1000 1500
0
10
20
30
40
50
60
70
80
90
Figure 2.3: Synchronization of the 2 Hz wriggler motion and the 2.5 Hz global image scan motion.
Right, single pixel wriggler movements shown as steps. Left: single pixel wriggler movement times
indicated with vertical lines. The y-axis shows scan position in pixels from the left.
Figure 2.3 shows the relationship between the simulated head movement and the
wriggler movement. The wriggler movement is shown as steps (g. 2.3, left), or bars (g.
2.3, right), each indicating a single pixel movement. It will turn out that the wriggler
was detected only at the scan reversals (seex2.3.2) { the motion detector must be very
sensitive to detect the very small wriggler movements that occur at these times.
An OMS RGC layer dimension of 116 91 cells was used in the natural image simu-
lations
6
. Allowing a border for convolutions and the single pixel correlator oset, as well
6
Since wriggler motion only aects RGCs whos RFs include at least one row of the wriggler's pixels,
most of the RGCs in this array are, eectively, experimental controls.
30
as the 90 pixel horizontal scan, we get an input image dimension of 593 401 pixels. A
window of this size was selected for each image le. Within this window, a location was
selected for the wriggler. Figure 2.2 shows the image window, with the wriggler bordered
by a small white rectangle. Table 2.1 shows the window coordinates, along with the
wriggler coordinates. References to the gures showing these images are provided.
The wriggler locations in the simulations reported in this chapter were chosen arbi-
trarily. Other locations gave results similar to those we are reporting { we are reporting
results for locations that seem \typical" for the image, yet challenging for the model.
We avoided locations with high contrasts, and locations in or near atypical features (e.g.,
water). In selecting locations arbitrarily, we did not nd any that had to be rejected as
having too little pixel intensity variation for our method. The brighter locations were
avoided { these regions tend to have higher contrasts. Simulations in regions with higher
pixel intensity variance gave stronger responses than the locations reported in this chap-
ter, but seemed to be a lesser challenge for the model, because our method computes the
wriggler intensity based on this variance. Finally, we made sure that the location chosen
was in the eld of view throughout the scan.
The dimensions for the arrays of simulated RGCs and photoreceptors are in the range
of the number of such cells found in the natural salamander retina (Zhang et al., 2004).
The visual eld of a salamander eye may range up to very nearly 180
o
(Roth, 1987). Thus,
in these simulations, the excitatory receptive eld (ERF) of of the bipolar cells as modeled
with the 33 convolution mask is somewhat less than 0:4
o
, while the total receptive eld
31
of a model bipolar cell has a diameter of 1 1:5
o
(the imprecision is caused by the very
coarse pixelation at this scale). We nd that the RGC ERF diameter is about 6
o
{ this is
realistic(Roth, 1987). Each wriggler stripe is 3 pixels wide, or just over 1
o
. The wriggler
moves in a 10 pixel wide box, corresponding to less than 4
o
. The photograph portions
used in these simulations clearly encompass a visual angle of well under 180
o
{ in fact,
their resolution is about 2 arc minutes per pixel (van Hateren & van der Schaaf, 1998), an
order of magnitude ner resolution than our discussion of the simulated salamander retina
suggests. Thus, the validity of our use of these images as backgrounds in simulations of
the salamander visual system, which has much lower resolution than the camera which
acquired those images, depends on sucient robustness in the scaling of image statistics,
at least in the higher spatial frequencies relevant to local motion detection.
Comparing the sizes of our wriggler and the stimuli used in behavioral experiments on
salamanders, we nd similarity. For example, the smallest dimension of cricket dummies
used as stimuli in the experiments reported in Schuelert & Dicke (2005) was 0.5 cm at a
distance of 9 cm, giving a visual angle of 3
o
. Our wriggler is close to this size. Thus, the
wriggler size corresponds to a prey-sized object at a distance from which the salamander
might eect a capture.
Images were selected from the online archive of Hans van Hateren's Lab
7
(van Hateren
& van der Schaaf, 1998). We used \linear" image les from this archive { these images
\are the raw images produced by the camera ... slightly blurred by the point-spread
7
The images can be downloaded at http://hlab.phys.rug.nl/imlib/index.html (accessed Febrary 15,
2007).
32
function of the camera"
8
. We selected linear images, rather than \deblurred" images,
because the deblurring procedure did not seem to enhance the biological realism of the
images considered as simulated retinal input. However, we believe it unlikely that this
choice aected our results. The images were selected more or less arbitrarily, with the
subjective criterion that the images could plausibly be scenes from natural salamander
habitats. See table 2.1 for the les used in the simulations.
The simulations were coded in Matlab and run under Linux. The simulation duration
was 1.5 seconds. Since our correlation-based motion detector computes correlations for
adjacent pixels only, a time step of 0.5 milliseconds was used, ensuring a maximum image
shift of one pixel in any time step, thus avoiding aliasing.
2.3.2 Results
In this section, we rst demonstrate that the model (gure 2.1), simulated under the
conditions described inx2.3.1, is sensitive to object motion at wide eld motion direction
changes. We then demonstrate that the prelter sigmoid, S
0
(g. 2.1(e)), substantially
enhances this sensitivity.
The simulations described inx2.3.1 were run with a variety of parameters { the wriggle
frequency ranged up to 31 Hz, the scan frequency ranged down to 1 Hz, and the scan
oset for some runs was as low as 10 pixels. In none of these runs was the wriggler visible
throughout the run. Visibility is indicated by high activation of an RGC whose excitatory
receptive eld contains the wriggler.
8
See http://hlab.phys.rug.nl/imlib/merits.html (accessed Febrary 15, 2007).
33
However, it was very often observed that the wriggler \
ashed" into visibility at the
slow phases of the scan, near where the scan direction changes. This was conspicuous in
movies made of the RGC array every 10
th
simulation millisecond. At slower scan speeds
and higher wriggle frequencies, the duration of the
ash was longer. All such
ashes were
associated with the wriggler { we did not observe similar model behavior at non-wriggler
locations.
Figure 2.5(a), left, shows a time series plot of two simulated RGCs. Also plotted are
the mean RGC activation, and a sinusoidal plot of the scan position, where low is the
leftmost scan position, and high is rightmost. The parameters for this run are given in
row 3 of table 2.1. The key features of this plot are the spikes that occur soon after
the direction reversals. (In this model, the main source of delay is the biphasic temporal
prelter.) There is a high level of activity just after the fastest part of the scan; however,
the background activity, indicated by the plot of the mean, is very high then (we see that
the right RGC is very active at these times). When spikes occur about 50 milliseconds
after the scan extremes, they are quite salient against the low mean background activity.
The excitatory receptive eld (ERF) of the spiking RGC contains at least most of the
wriggler at the time of the immediately previous scan motion reversal. Before the motion
simulation, we computed the indices of the simulated RGCs which would contain the
wriggler at these times, and saved the data for plotting. Figure 2.5(a) shows examples
of these plots. Since the RGC ERFs overlap in these experiments, when spikes were
observed, they could be seen in multiple RGCs.
34
The least intense of these direction-change spikes in gure 2.5(a), left, occur after the
2
nd
, 3
rd
, and 7
th
direction reversals at times 400, 600, and 1400 milliseconds. Viewing this
gure in light of the incremental wriggler movement timing shown in gure 2.3, we see that
these are the scan extremes with low coincidence of wriggler movement. Furthermore, the
most intense direction-change spike occurs after the 5
th
direction reversal at 1000 ms.;
gure 2.3 shows that this is the scan extreme with the greatest coincidence of wriggler
movement.
Figure 2.5(a), right, shows a detailed view of the most intense spike. The mean
activity is rising as the left RGC spikes as a result of dwelling on the wriggler. The mean
activity continues to rise because of the increasing scan speed, after the spike subsides.
Figure 2.5(a) suggests that object motion related spikes are associated with global
image motion direction reversals; on the other hand, these experiments leave open the
possibility that the spikes may simply be associated with low background motion speed.
In order to answer this question, we ran one-way scan simulations. In these experiments,
the speed was modulated sinusoidally, but the direction did not change. The results of
these experiments indicate that the object motion depends on the change of direction;
low background speed in not sucient. Figure 2.5(b) shows results for a one-way scan
simulation that uses the initial conditions of the simulation of g. 2.5(a); the initial part
of the scan is the same as in g. 2.5(a) (The scaling of the scan position in the two plots
are dierent; the actual scans are the same up to just beyond
9
200 ms., the end of the
rst peak in 2.5(a), left, and the rst plateau in 2.5(b).) The RGC activity plotted in
9
The sinusoidal scan dwells for 18.5 ms. at the reversals.
35
gure 2.5(b) is for two adjacent RGCs with overlapping ERFs; one of these is the same
as the "RGC left" of gure 2.5(a), left. The rst spike seen in 2.5(a), left, after the peak
at 200 ms., is absent in 2.5(b) after the plateau at 200 ms.
This dierence was observed in movies made during the simulations as well. The
spikes observed as
ashes in the runs with scan reversals were not seen in one-way runs.
That the spiking was associated with the wriggler was strongly demonstrated by
the movies made of the retinotopically organized RGC array. Spikes were seen as salient

ashes in these movies. These
ashes were not observed at non-wriggler locations. Movies
were also made of the the sigmoidally (S
1
) transformed motion signals in a retinotopic
array (the result at g. 2.1(i)). The
ashes were clearly seen in these movies, too; again,

ashes were not seen at non-wriggler locations.
Figure 2.4 shows 3 consecutive frames taken every 10
th
millisecond. The images are
scaled so that the maximum value in the image is shown at maximally bright white,
and the minimum value is shown as black. The left column shows the motion detector
layer, where gray is 0, dark is negative activity, and light is positive activity. The right
column shows the RGC layer, where dark is 0. The basic texture of the original image is
apparent. In the bottom image, features are emerging from the background as the scan
speed increases. In the middle images, corresponding to simulation time 1050 milliseconds
(g. 2.5), the activity caused by the transient salience of the wriggler is circled. Notice
that there is no visible activity at that location 10 milliseconds before or after the spike.
36
50 100 150 200 250 300 350 400 450
50
100
150
200
250
300
350
10 20 30 40 50 60 70 80 90 100 110
10
20
30
40
50
60
70
80
90
50 100 150 200 250 300 350 400 450
50
100
150
200
250
300
350
10 20 30 40 50 60 70 80 90 100 110
10
20
30
40
50
60
70
80
90
50 100 150 200 250 300 350 400 450
50
100
150
200
250
300
350
10 20 30 40 50 60 70 80 90 100 110
10
20
30
40
50
60
70
80
90
Figure 2.4: Full retina images of 3 consecutive snapshots taken during the simulation that
generated the timeseries shown in gure 2.5. The left is the motion detector view, while the
right is the ganglion cell (RGC) view. Each image is scaled. In the motion detector views, zero
activation is the middle intensity; dark and bright pixels indicate activation of the motion detectors
for opposite directions. Bright pixels tend to be associated with nearby dark ones, because the
sigmoidal transformation of the ltered input confounds the directionality. The top images are
at simulation time 1040 milliseconds, before the
ash indicating wriggler detection. The middle
images are at time 1050 ms., soon after the peak (see gure 2.5) { still clearly showing the
ash,
which is circled. The
ash has subsided completely at 1060 ms., the bottom images. As the scan
accelerates, background image features cause more motion detector activity, while the wriggler
disappears into the background.
37
0 500 1000 1500
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
imk00877.iml (501:1093,300:700), target −− (400,121)
RGC left
RGC right
RGC mean
scaled scan position
1020 1030 1040 1050 1060 1070 1080
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
RGC left
RGC right
RGC mean
scaled scan position
(a)
0 50 100 150 200 250 300 350 400 450 500
0
0.01
0.02
0.03
0.04
0.05
0.06
imk00877.iml (501:1093,300:700), target −− (400,121)
RGC 1
RGC 2
RGC mean
scaled scan position
(b)
Figure 2.5: 2.5(a), right: Time series plot of 2 model RGCs, along with the mean for all RGCs,
and the movement displacement from the left. The left RGC spikes when it detects the wriggler
at the rightmost part of the scan, while the right RGC spikes when it detects the wriggler at the
leftmost part of the scan. The image is scanned over a range of 90 pixels at 2.5 Hz, for a total
of 450 pixels per second. The wriggler oscillates at 2 Hz, with a range of movement of 4 pixels,
giving a total movement of 16 pixels per second. The y-axis shows neural ring rate, where the
maximum is 1. For all plots in this gure, he x-axis is time in milliseconds. 2.5(a), left: Detail
of a spike. 2.5(b): A one-way scan, with the same sinusoidal speed modulation as in 2.5(a), but
without the direction reversals. We see the activity of 2 model RGCs with adjacent, overlapping
ERFs. The absence of spikes demonstrates the dependence of the spiking phenomenon on global
motion direction changes. \RGC 1" is the same as \RGC Left" of 2.5(a). The scaling of the scan
position is dierent from that in 2.5(a), but the actual speed modulation is the same. In this run,
the wriggle frequency was 7 Hz { spikes were absent at 2 and 31 Hz, also. (See gure 2.2(c) for
the image with the wriggler at 3 levels of detail.)
38
0 500 1000 1500
0
0.02
0.04
0.06
0.08
0.1
0.12
imk00347.iml (601:1193,300:700), target −− (400,221)
0 500 1000 1500
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
imk00347.iml (601:1193,300:700), target −− (400,221)
(a)
0 500 1000 1500
0
0.01
0.02
0.03
0.04
0.05
0.06
imk00877.iml (501:1093,300:700), target −− (400,181)
0 500 1000 1500
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
imk00877.iml (501:1093,300:700), target −− (400,181)
(b)
0 500 1000 1500
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
imk04208.iml (435:1027,125:525), target −− (400,101)
0 500 1000 1500
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
imk04208.iml (435:1027,125:525), target −− (400,101)
(c)
Figure 2.6: Eect of sigmoidal nonlinear transformation of ltered motion detector input on
ganglion cell output. See gure 2.5 for the legend. The left column show the results of omitting
the sigmoidal transformation, while the right show the results with the sigmoid. In every case,
the detection of the target, indicated by brief spikes, is substantially better with the sigmoid. In
2.6(b), there are essentially no detectable spikes without the prelter sigmoid. 2.6(a) shows data
for a scan of the image shown in 2.2(a); 2.6(b), 2.2(b); and 2.6(c), 2.2(d).
39
Figure 2.6 shows the results of several runs that were made to test the eect of
sigmoidal transformation of the motion detector preltered input. The left column shows
the results without the sigmoid, while the right column shows the results of the same
run made with the nominal system, with the sigmoid. The parameters for the top runs,
2.6(a), are given by the 1
st
row of table 2.1, 2.6(b) by the 2
nd
row, and 2.6(c) by the 4
th
row of table 2.1. Notice that, with or without the sigmoid, the left RGC of the top runs
is strongly activated by the high contrast
owing water.
The sigmoid clearly enhances the sensitivity to the wriggling object motion. In 2.6(b),
without the sigmoid, no visible activation is caused by dwelling on the wriggler. In
2.6(a), the sigmoid enhances the response to both the wriggler and to the high contrast
stream, but the wriggling motion is enhanced much more, thereby enhancing the wrig-
gler's salience. In 2.6(c), the weaker wriggler peaks (at simulation times 449.5 and 649.0
ms.) are relatively enhanced compared to the background, and compared to the stronger
peaks. Also in 2.6(c), we notice double peaks, 18.5 milliseconds apart, in the left RGC
response without the sigmoid; the double peaks do not occur with the sigmoid.
Object motion saliency has been demonstrated for this model in response to very
small incremental wriggler movements. In gure 2.3, we see that for the third scan
motion reversal, only one incremental wriggler pixel movement occurs while the spiking
ganglion receptive eld contains the wriggler. This excellent motion detection might
indicate possibly excessive noise sensitivity; inx2.6, we will argue that, on the contrary,
40
the system may be very robust in the presence of natural levels of noise. The salamander
may indeed have both high motion sensitivity and remarkably low noise sensitivity.
This section presented the evidence for this chapter's major nding: that the sigmoidal
transformation of the preltered input to the correlation-based motion detector enhanced
moving object detection during a sinusoidal scan of natural images, where the scan motion
greatly exceeds the object motion. The object was only detected near the motion direction
changes of the scan; during the high speed portions of the scan, object motion was not
detected.
2.4 Simplied model simulation
The main result this section is that the temporal prelter sigmoid reduces the direc-
tionality of the motion detector, compared with correlation-based motion detectors with
linearly ltered inputs. We also show that simulating continous motion with small, frac-
tional pixel movements results in somewhat stronger motion detector responses than the
integral pixel movements ofx2.3.
To further investigate the temporal characteristics of the model, we performed simu-
lations of abstract stimuli on a simplied version of our model (see gure 2.7(a)). This
model includes just two temporally varying spatial inputs, abstracting away the spatial
prelter (gure 2.1(a-c)). We examine the output of the correlator (at gure 2.1(h))
without the integration of the motion sensitive signals into the model RGC output, i.e.,
without post-ltering and without rectication into channels of opposite directionality.
41
Comparing the results with and without the sigmoid following the the linear temporal
prelter, we nd that the sigmoid compromises the directionality of the motion detector.
These runs started with the application of a dark stimulus to both inputs of the
simplied motion detector. Starting at simulation time 525 milliseconds, a bright stimulus
moved across the two detector inputs, with dierent speeds for dierent runs. First one
input was set to the bright level; after a specied dwell time, the second input was
set bright and the rst was reset to the dark level. After another interval of the same
duration, the second input was reset to the dark background level. The simulations were
run for 1 second. In the simulations reported here, the bright pixel level was 0.9, while
the dark level was 0.1.
We show the results in gure 2.7. Dwell times for the bright stimulus of 0.5, 5, and 10
milliseconds were used. Each case was run with and without the sigmoidal transformation
of the temporally preltered input to the correlator. A timestep of 0.5 ms. was used for
the rst case, 1 ms. for the longer dwell times.
One observation is that the detector responses increased with decreasing stimulus
speed in the range of dwell times tested
10
.
Another observation is that with the prelter sigmoid, multiphasic responses are ob-
served, while the responses are monophasic when the prelter is just a biphasic linear
temporal lter. Compare Ibbotson & Cliord (2001), using an unbalanced Reichardt
detector (where the products are multiplied by unequal weights before subraction (g.
10
See Dror et al. (2001) for a discussion of the average speed tuning of Reichardt detectors.
42
(a)
500 550 600 650 700
−1
0
1
2
3
4
5
x 10
−6
500 550 600 650 700
−4
−2
0
2
4
6
x 10
−7
(b)
500 550 600 650 700
−1
0
1
2
3
4
x 10
−3
500 550 600 650 700
−0.5
0
0.5
1
1.5
2
2.5
x 10
−4
(c)
500 550 600 650 700
−0.005
0
0.005
0.01
0.015
0.02
0.025
0.03
500 550 600 650 700
−0.5
0
0.5
1
1.5
2
2.5
3
3.5
x 10
−3
(d)
Figure 2.7: Simulation results for a simplied correlation-based motion detector model (2.7(a)).
The input is simply a pulse at one input, then then other, after which the input returns to the
background level. In the displayed results, the input pulse was 0.9 against a background intensity
of 0.1. The pulse starts at simulation time 525 milliseconds. For each run, the time to dwell on
each input pixel is: 2.7(b), 0.5 ms.; 2.7(c), 5 ms.; 2.7(d), 10 ms. For the left column, the sigmoidal
transformatiion of the preltered motion detector input was omitted. Without the sigmoid, the
detector output is strictly directional; with the sigmoid, the detector output crosses zero. In these
runs, the sigmoid decreases the absolute magnitude of the output, as well as introducing some
directional ambiguity.
43
2.1(h))), in which a biphasic linear temporal prelter was associated with triphasic motion
detector responses.
Finally, we note that the sigmoidal transformation of the preltered input somewhat
confounds the direction selectivity of the detector. In all cases, the greater weight is still in
the same direction as the linear prelter case, but the directionality of the detector is now
ambiguous. Synchrony between pooled detectors can result in large scale unit responses
in the opposite direction to the true local image motion. Since salamanders seem to
have few, if any, direction selective RGCs (Segev et al., 2006), such a retinal model can
be considered. (We have seen (x2.3.2) that this nonlinearity can have desirable saliency
enhancing eects.)
Not shown are runs with a time step of 0.01 ms.; their was no signicant dierence
in the results. Also not shown are runs in which the stimulus is linearly interpolated, so
that the stimulus moves so that it is fractionally blended between the detector inputs.
This gives somewhat larger magnitude responses than the discrete case, but the response
plots appear qualitatively the same.
2.5 Striped pattern OMS protocol simulations
We performed simulations based on the protocols reported in

Olveczky et al. (2003). In
their work, the retina was stimulated with randomly jittering patterns of alternating light
and dark stripes covering the receptive eld (excitatory and inhibitory) of the recorded
ganglion cell. The jitter directions were perpendicular to the stripes.
44
In their \eye + object" case, the jitter in the central region was independent of
the jitter in the surround. In their \eye only" case, the center and surround jittered
coherently, i.e., a single jittering pattern covered the entire receptive eld. OMS neurons
responded strongly to the \eye + object" and \object only" stimuli; they did not respond
to the \eye only" stimulus. A variant of the \eye only" case was coherent motion with
stripe pattern reversed between center and surround (i.e., a dark stripe in the surround
was light in the center, and vice versa). The OMS neurons did not respond to this variant,
indicating that they respond to motion dierences, but not to instantaneous dierences
in the static pattern (which intermittent jitter must cause).
Three other cases were reported in

Olveczky et al. (2003). (1) A constant drift was
added to the \eye + object" case. The speed of drift was very close to the average
absolute jitter speed, but the drift direction was constant. In this case, the OMS neurons
responded. (2) The the \eye only" jitter was used with opposite stripe patterns in the
center and surround. OMS neurons had no signicant activity in response, as with \eye
only". Finally, (3) they jittered the pattern so that the central part of the RF always
moved in the opposite direction of the surround pattern. Again, the OMS cells had no
signicant activity in response, as with \eye only". Thus, OMS cells do not respond
directly to the motion dierence.
In the simulations reported here, the stripes were 2 pixels wide. The pixel intensity
of the dark stripes was 0.1, and of the light stripes, 0.9. The central \object" region
corresponded closely, but not exactly, to the RGC excitatory RF (the positive region of
45
the DoG). The simulation timestep was 1 millisecond. Following the protocol of

Olveczky
et al. (2003), jitter was performed every 15 ms. For each jitter step, a random number
was generated for the direction computation, with both directions equally probable (2
independent random numbers were needed for the incoherent case). We derived the
incremental jitter magnitude (in pixels) from the methods of

Olveczky et al. (2003) as
6:7
133

s
, where
s
= 4 is the stripe pattern period in pixels. Since this increment is a small
fraction of a pixel, we linearly interpolate a gray level when a stripe boundary occurs at
a pixel (i.e., when the boundary is in the simulated photoreceptor RF).
The drift speed was
450
133

s
pixels per second. We ran 2 dierent subcases of \eye +
object drift".

Olveczky et al. (2003) did not report how the drift was implemented, so we
ran both an incremental drift case, where movements occurred only every 15 milliseconds,
as well as a simulated continuous movement case, where a proportionately smaller move
was made at every simulation time step.
See gure 2.8 for illustrations of the visual input to a gangion cell receptive eld for
these cases.
In the model used to simulate these protocols, we omitted the adapting sigmoidal
transformation of the correlator subtraction, S
1
(x2.2 and gure 2.1(i)). Sigmoid S
1
was
used to investigate how the retina might respond to natural image motion. This is not a
consideration for the striped stimuli { the distribution of motion signals here in no way
resembles the distribution in natural images. The striped stimuli cover a single RGC
RF, while the S
1
parameterization depended on a maximum motion signal magnitude
46
5 10 15 20 25 30 35 40
5
10
15
20
25
30
35
40
(a)
5 10 15 20 25 30 35 40
5
10
15
20
25
30
35
40
(b)
5 10 15 20 25 30 35 40
5
10
15
20
25
30
35
40
(c)
5 10 15 20 25 30 35 40
5
10
15
20
25
30
35
40
(d)
5 10 15 20 25 30 35 40
5
10
15
20
25
30
35
40
(e)
Figure 2.8: Illustrations of simulation of experimental protocols like those described in

Olveczky
et al. (2003). The basic pattern is alternating vertical light and dark bars, each 2 pixels (pho-
toreceptor receptive elds (RFs)) wide. The gray bars in this gure result from blending light
and dark when the edge is in the photoreceptor RF. The images are retinal input in a mini-
mal bounding box enclosing the entire RF of a single model retinal ganglion cell (RGC). 2.8(a)
illustrates the \eye + object" protocol { the pattern in the RF center and that in the RF sur-
round move independently, i.e., they jitter incoherently. Another protocol is illustrated by the
same snapshot: center & surround jitter incoherently, and a constant drift is added to the center
movement. 2.8(b) illustrates the \eye only" protocol { the randomly moving pattern covers the
entire excitatory and inhibitory RF, i.e., the movement in the center and in the surround are
coherent. This is the only case in which there is no dierence in pattern nor movement between
the RF center and surround. 2.8(c) illustrates the \object only" protocol { the pattern in the RF
center jitters, but there is no movement, and no features by which movement could be detected
in the RF surround. 2.8(d) illustrates a protocol in which the center and surround patterns jitter
coherently, but the illumination polarities are opposed, i.e., dark in the center correspond to light
in the surround, and vice versa. 2.8(e) illustrates a protocol in which the center and surround
jitter in opposition, i.e., every incremental move in the center is accompanied by an equal and
opposite move in the surround.
47
over thousands of samples. Furthermore, the striped stimuli cause much repetition of
stereotyped responses within the single receptive eld, further reducing the signicance
of statistics over the striped images. This is a degenerate case of image statistics { it
reveals that one of the limitations of our model of adaptation to natural image motion
signal distributions is that it is not clear how the model applies to unnatural images. In
omitting the adaptive nonlinearity, we suggest that the eye defaults to linear behavior at
the level of motion signal rectication.
The results of these simulations are illustrated in gures 2.9 and 2.10. Figure 2.9 shows
results for the \nominal model", with sigmoidal transformation of preltered inputs to
the correlator. Figure 2.10 shows results for the model in which the prelter sigmoid is
omitted. The \eye + object" (2.9(a) & 2.10(a)) and the \object only" (2.9(c) & 2.10(c))
cases show strong responses, and the \eye only" (2.9(b) & 2.10(b)) cases show very weak
responses. The opposite motion cases (2.9(g) & 2.10(g)) shows a weak response, an order
of magnitude less intense than the strong responses, but comparatively much stronger
than the \eye only" cases. We consider theses results consistent with those of

Olveczky
et al. (2003).
The strongest responses in our simulations were for the object drift cases, gures
2.9(d), 2.9(e), 2.10(d), and 2.10(e).

Olveczky et al. (2003), however, do not indicate that
this case produces the strongest responses
11
. The jittery steady drift cases (2.9(e) &
2.10(e)) showed the highest peaks of all the cases, with the spikily intermittent behavior
11
In fact, in

Olveczky et al. (2003, gure 2d) the object drift case response appears weaker than for
\eye + object" and \object only" cases. However, this may not have been signicant enough to claim
that OMS neurons respond less to drift than to stationary jitter.
48
0 5000 10000 15000
0
0.5
1
x 10
−3
0 5000 10000 15000
−20
0
20
(a)
0 5000 10000 15000
0
2
4
6
x 10
−8
0 5000 10000 15000
−10
−5
0
5
(b)
0 5000 10000 15000
0
0.2
0.4
0.6
0.8
1
1.2
x 10
−3
0 5000 10000 15000
−1
0
1
2
3
4
5
(c)
0 5000 10000 15000
0
0.5
1
x 10
−3
0 5000 10000 15000
0
200
400
(d)
0 5000 10000 15000
0
1
2
x 10
−3
0 5000 10000 15000
0
1
2
x 10
−3
0 5000 10000 15000
0
200
400
(e)
0 5000 10000 15000
0
2
4
6
8
x 10
−5
0 5000 10000 15000
−2
0
2
4
6
(f)
0 5000 10000 15000
0
1
2
3
4
x 10
−4
0 5000 10000 15000
−6
−4
−2
0
2
(g)
Figure 2.9: Results of striped pattern simulations (see gure 2.8). The plots show time series
simulation results for a single model RGC. The model RGC activation plot is jagged and always
drops to near 0 (so it looks like a bar graph), with the scale on the left horizontal axis. Note
the scale multiplier at the upper left corner. The dotted lines are a jitter position indicator with
an interpretation depending on the run; the scale is on the right horizontal axis. 2.9(a) shows
data for a run of the \eye + object" protocol. The dotted line is the dierence in osets of the
pattern position from the starting position (in pixels) between the RF excitatory center and the
inhibitory surround. 2.9(b) shows data for a run of the \eye only" protocol. The dotted line is
the oset of the pattern position (in pixels) from the starting position. 2.9(c) shows data for a
run of the \object only" protocol. For this and all following plots, the dotted line is the oset, in
pixels, of the central (\object") pattern from the starting position. 2.9(d) & 2.9(e) shows results
of runs in which a constant drift is added to the central object motion. Center and surround also
have incoherent random jitter. 2.9(d) shows the response when the drift in small increments at
every simulation time step, a simulation of continuous motion; 2.9(e) shows the \jittery drift"
case, where the drift is added to the jitter the jitter every 15 ms. 2.9(f) shows data for a run in
which the center and surround patterns move coherently but have opposing illumination polarity.
2.9(g) shows data for a run in which the center and surround move in opposition.
49
0 5000 10000 15000
0
0.005
0.01
0.015
0.02
0.025
0.03
0.035
0.04
0 5000 10000 15000
−12
−10
−8
−6
−4
−2
0
2
4
(a)
0 5000 10000 15000
0
1
2
3
x 10
−9
0 5000 10000 15000
−10
−5
0
5
(b)
0 5000 10000 15000
0
0.02
0.04
0 5000 10000 15000
−5
0
5
(c)
0 5000 10000 15000
0
0.01
0.02
0.03
0 5000 10000 15000
0
100
200
300
(d)
0 5000 10000 15000
0
0.01
0.02
0.03
0.04
0.05
0 5000 10000 15000
0
50
100
150
200
250
(e)
0 5000 10000 15000
0
2
4
x 10
−3
0 5000 10000 15000
−5
0
5
(f)
0 5000 10000 15000
0
0.01
0.02
0 5000 10000 15000
−5
0
5
(g)
Figure 2.10: The simulations of gure 2.9 run without the sigmoidal transformation of the motion
detector input.
50
typical of correlation-based motion detectors (Dror et al., 2001). The continuous steady
object drift cases (2.9(d) & 2.10(d)) showed very dense activity; the activation level never
dropped to near zero during these runs.
12
It seems unlikely that live RGCs will, in general,
respond very dierently to continuous motion vs. motion incremented at 15 ms. intervals.
A low-pass lter could reduce the dierence between model output in these cases.
The sigmoidal prelter transformation, S
0
, is no help for modeling OMS neuron re-
sponses to these high contrast, low velocity simuli. The relative responses are in the same
order in both cases. The magnitude of the responses for the sigmoid cases are quite low,
but adaptation could amplify these responses.
2.6 Discussion
We used a natural image simulation protocol to study the interaction of salamanders' mo-
tion sensitive neurons with locomotion-induced side-to-side head movements. We scanned
a static natural image in order to simulate the retinal slip generated by these head move-
ments. Our particular implementation of this protocol assumed sucient invariance in
natural image statistics over an order of magnitude of scale. Our results demonstrated
that the salamander may be very sensitive to object motion at the extremes (the direction
reversals) of the head movement.
12
Inx2.4, we noted that simulating continous movement of the stimulus spot across the detector by
linearly combining the portions of the 2 pixel values shared by a detector input resulted in somewhat more
intense detector responses than moving the spot in discrete, full pixel value steps. The comparison of the
continuous vs. jittery steady drift cases extends this observation: here, the magnitude of the responses
in the \continuous" case were somewhat lower, but their was a non-zero
oor under the activity level.
51
Sigmoidal transformation of the preltered motion detector input enhanced the de-
tectability of the wriggling object superimposed on a natural image; it had no clear eect
on the perfomance of the system in simulations of protocols similar to the laboratory
experiments of

Olveczky et al. (2003). It may be that the natural system is closer to
the linear temporal prelter case in performance; in this case, the salamander may seem
suboptimal in detecting moving objects during locomotion. Also, the linear temporal
prelter case is compatible with very strong directionality, over a large range of stimulus
velocities, in the fundamental motion detection mechanism (say, at the level of amacrine
cell direction-selective dendrites). On the other hand, the salamander's actual perfor-
mance may be closer to the sigmoidally transformed prelter case; our results suggest
that in this case, the fundamental motion detection mechanism may have somewhat lim-
ited directionality, especially at high stimulus velocity. Finally, the retina may adapt to
retinal slip by modifying a prelter nonlinearity according to slip velocity.
The model's sensitivity (at the self-movement extremes) to low magnitude object
movement suggests the possibility of high vulnerability to noise. However, the results of
x2.4 { where the sigmoid did not signicantly aect the response to the random jitter
dierential motion { suggest that the sigmoid may relatively reduce the sensitivity to
noise, and increase the sensitivity to object motion. Here, we interpret the random jitter
as noise. The results of

Olveczky et al. (2003) imply that the other noise sources in their
experiments had less impact on the neural responses than did the random jitter. From
52
these results, we infer that our nonlinear transformation has the desirable eects both of
increasing the detection rate (x2.3.2), and of decreasing the false alarm rate.
That we were unable to extend object motion sensitivity to conditions of high retinal
slip tends to conrm and extend the results of Zanker & Zeil (2005). They found very
sparse, highly kurtotic distributions of motion signals caused by self-motion in natural
environments. A local motion signal opponency scheme is based on the presumption
that during global movement, a motion signal in the receptive eld center is likely to be
balanced by a similar signal in the surround { but the sparseness of these signals suggests
otherwise. This is a diculty for schemes based on local motion signal opponency. We
indeed encountered this diculty, as reported inx2.3.2. At high scan speeds, the sparse
motion signals were such that the coarse texture of the scene was apparent to human
observers in the array of center-surround motion opponent retinal ganglion cells; this
indicates to us that the local motion signals were far from balanced. Only at low scan
speeds was the balance between center and surround such that small object movements
could pop out.
We encountered this situation despite the adaptive sigmoidal transformation of the
detector output,S
1
. S
1
adapted according to the easily computed maximum signal mag-
nitude. An issue for further investigation is the possibility that an adaptive system more
directly tuned to normalize the moments of the distribution of motion signals might have
better balanced local activity. It seems likely, however, that the luminance distribution
53
in natural scenes will preclude this, i.e., the sparseness of motion signals in natural im-
ages may arise from luminance sparseness. This suggests another possibility for further
research: a dierent preltering scheme. The variance modied image in Ruderman
(1994, gure 14) (see also Ruderman & Bialek (1994)) could perhaps give a less sparse
distribution of motion signals. However, Ruderman (1994) did not suggest that such a
representation occurs in the retina. Such preltering might seem to contradict current
ideas about the function of retinal processing. Balboa & Grzywacz (2000) argue that kur-
tosis maximization cannot account for observed characteristics of bipolar cell responses,
but the bipolar cell representation is nevertheless sparse. Assuming that with robust
scaling of a variety of natural image statistics (Ruderman & Bialek, 1994; Ruderman,
1994) comes at least rough, sparseness preserving scaling of the motion signal kurtosis,
we hypothesize that salamander RGCs will respond somewhat erratically to motion. This
seems to be the case even for highly unnatural visual stimuli, as we see when we examine

Olveczky et al. (2003, gure 2): in a 15 second \object only" experiment, the represen-
tative RGC seems to have produced about 20 bursts of ring. This probably overstates
the sparsity of response that the living animal's neurons would produce, given the sala-
mander's reputation for eectively responding to visual motion (Roth et al., 1998; Roth,
1987). Our model's reliable responses to object motion during simulated locomotion may
help to understand how the salamander can approach moving prey.
Our adaptive sigmoidal transformation of the detector output, function S
1
inx2.2,
tends to give human recognizable image features during most of the range of motion of
54
the simulations described inx2.3. It adapts instantly to the global maximum motion
detector output. This normalization of the correlator output leads to similar conditions
regardless of the speed of motion. It did not obscure the wriggling object at the slow
phase; nor did it enable object detection at the fast phase. Thus, for object motion
detection, we think it likely that the motion detection system parameters will adapt to
conditions at the slow phase, leading to lower acuity at the faster phases. We suggest
that the recognizable image features we observed during the fast portions of the natural
image scans would be unlikely to occur in the more slowly adapting natural system (the
salamander retina). Thus, our results predict that OMS RGCs are likely to be eectively
blind during fast self-motion induced retinal slip. This suggests an analogy with saccade
blindess.
During the natural image scan simulations, our model reliably detected small object
movement as the average RGC activity level was increasing from its minimum, just after
the direction reversals at the 2.5 Hz locomotion extremes. In the model of

Olveczky et al.
(2003), this time duration to change retina directions (just over their experimental inter-
jitter period) might be sucient to desynchronize the inhibition cause by the scan from
the excitation caused by the object movement. So their model may be able to produce
an OMS response at the locomotion extremes.
The interpretation of the output of our model as neural activity raises issues. RGCs
re at most a few hundred spikes per second. Owing to the well-known chaotic behavior of
correlation-based motion detectors (Dror et al., 2001), our model's output often changes
55
considerably over very short time steps. Thus, a more realistic neural model might include
a low-pass lter (perhaps in the form of a leaky integrator), supporting the interpretation
of the output as a neural ring rate.
13
Furthermore, such smoothing might reduce or
eliminate the unrealistic dierence between responses to continous motion vs. rapidly
intermittent motion (seex2.5 and gures (2.9(e) & 2.9(d)).
13
Another possibility might be a band-pass lter, with amplitude normalization, perhaps supporting
the interpretation of spikes in the output as action potentials.
56
Chapter 3
Modeling Salamander Visual and Locomotor System
Interaction
3.0 Chapter abstract
We report on a computational investigation of visual steering of salamander aquatic and
terrestrial locomotion. A tectum model determines the direction toward which to move
using input from motion sensitive retinal ganglion cells. We examine the in
uence of the
timing of visual locomotor control signals on the ecacy of the direction selection, and
on the dynamics of the undulatory movement. This builds on a previous CPG model of
salamander locomotion, with top-down control from the brainstem via a very small num-
ber of brainstem inputs. We show that undulatory locomotion of an elongate vertebrate
can be controlled in head-centered coordinates. Our results support the hypothesis that
use of the visual image is inhibitted at times of high background motion.
57
3.1 Introduction
3.1.1 Ijspeert's CPG model
Ijspeert's connectionist model of vertebrate central pattern generator (CPG)-based loco-
motion (Ijspeert, 2001) shows realistic transitions between terrestrial and aquatic gaits,
and is steerable with the biologically realistic method of applying a higher ring rate
to a tonic drive input to the CPGs on the side of the spinal cord toward which a turn
is planned { walking or swimming, the steering method is the same. Ijspeert embed-
ded the CPG model in a neuromechanical model with a simulation of the interaction of
neurally-controlled body with the environment. This simulation allows evaluation of the
behavioral consequences of neural mechanisms (Cli, 2003).
3.1.2 Locomotion control physiology
The understanding of the physiology of control of spinal CPG-based locomotion has
progressed in recent years (see Ijspeert (2008) and Grillner et al. (2008) for reviews).
Midbrain areas involved in the modulation of CPG-based gaits include the mesencephalic
locomotor region (MLR) (Cabelguen et al., 2003; Kagan & Shik, 2003) and the tectum
(Saitoh et al., 2007). The tectum appears to be the primary locomotion steering brain
area (Grillner et al., 2008; Saitoh et al., 2007). The tectum is salamanders' most im-
portant visual brain region (Roth, 1987), with substantial projections to motor control
areas (Dicke, 1999; Roth et al., 1999). Turning movements in CPG-based locomotion are
associated with increased reticulospinal neuron ring rates on the side to which a turn is
58
made (Fagerstedt et al., 2001). These dierential ring rates are modulated by eerents
from the tectum.
Fagerstedt et al. (2001) found abundant reticulospinal neurons whose ring rates were
directly correlated with the direction of turning movements. Saitoh et al. (2007) produced
turning movements and locomotion with electrical stimulation at specic locations in
a circumscibed region of the tectum. In this lamprey study, tectal electrostimulation
generally produced eye movements, in addition to specic locomotor responses to some
stimulation at some tectal sites. As we attempt to generalize these results to salamander,
we must note that salamanders' head movements directly imply retinal slip. In lamprey,
tectal electrostimulation-induced locomotion is accompanied by eye movements that seem
to stabilize the retinal image by compensating for the undulatory locomotion-induced
head movements, but this compensation is not seen in salamanders (Roth, 1987).
A number of simulation models of CPG activity in locomotion have contributed to
understanding of control of CPG-based locomotion (Ijspeert, 2001; Ijspeert et al., 2005;
Ijspeert & Kodjabachian, 1999; Kozlov et al., 2002, 2007; Wall en et al., 1992). These
studies have contributed to a theoretical understanding of such issues as the capability of
the underlying neuronal architecture; gait transitions, e.g., between terrestrial and aquatic
locomotion in salamanders (Ijspeert, 2001; Ijspeert et al., 2005); the eects of network
topology (Ijspeert et al., 2005); the eects of physiological manipulations (Kozlov et al.,
2007); the eects of coupling neural activity with the environment (Ijspeert, 2001; Wall en
59
et al., 1992; Ijspeert et al., 2005); the eects of proprioceptive feedback (Ijspeert et al.,
2005); and mechanisms of steering (Ijspeert, 2001; Kozlov et al., 2002).
Open issues regarding steering of CPG-based locomotion include possible organiza-
tion of the tectal control regions, and the eects of visual system properties on visually
controlled locomotion. These properties include visual system delays, and blurring due
to the locomotion itself (see chapter 2, also Begley & Arbib (2007)). There may be ef-
fects on the eectiveness of the locomotion, e.g., time to reach a target and eciency of
movement.
3.1.3 Salamander retina
There are substantial delays in the salamander visual system, including in the retina
(Segev et al., 2006) and tectum (Roth et al., 1999; Roth, 1987). Chapter 2 (Begley &
Arbib, 2007), modeling the fast o retinal ganglion cells (RGCs) of Segev et al. (2006),
included a biphasic temporal lter with a period of 100 milliseconds (less than the period
suggested by the peristimulus time histograms of Segev et al. (2006)). Given that the
salamander's head may oscillate back and forth at 4 Hz during fast swimming (Delvolve
et al., 1997), retinal delays alone are a substantial portion of the cycle time. This suggests
the question: must the visuomotor coordination system compensate for these delays, or
is the locomotor control system suciently robust that such compensation is not needed?
Salamander RGCs have shown repeatable millisecond-level precision in the timing
of responses to visual stimuli (Berry et al., 1997). This timing has been analyzed in
terms of information theory and data transmission coding eciency (Balasubramanian
60
& Berry, 2002; Warland et al., 1997). Questions of the possible eects of this precision
in sensorimotor coordination have so far received less attention.

Olveczky et al. (2003) reported on salamander RGCs that respond when motion in
the receptive eld center diers from motion in the surround. Very slow background
(surround) motion was used in their study, similar to that produced by eye movement
of a quiescent salamander (e.g., movement produced by respiration). Locomotion may
produce much greater speeds of movement of the retinal background image. In chapter 2
(Begley & Arbib, 2007), our modeling showed that object motion sensitivity was evident
only when the background image was relatively still. This occurred after the reversal of
the locomotion-induced horizontal oscillatory head movement, with a neural processing
delay. We suggested that object motion sensitivity could be used at these times in the
undulatory locomotion cycle, and suggested suppressing the visual input during the times
of high background motion. In the present work, we develop this idea for locomotion
steering, and test it in simulation, comparing it with models of continual visual control
throughout the locomotion cycle, and with non-biological steering perfectly coupled to
the true bearing to the target.
3.1.4 Modeling background
The model of salamander saccade generation of Manteuel & Roth (1993) was a model of
tectal control of directed movement. Instead of controlling locomotion with 2 descending
signals, they controlled head movement with 4 neck muscles. There may be overlap
in tectal regions controlling neck movements and locomotion. In Manteuel & Roth
61
(1993) visual input was simplied based on the observation that the movements were
not sensitive to the strength of the retinal input. In the present work, we use a more
realistic, but still abstract, retina model; nevertheless, as in Manteuel & Roth (1993),
the commanded movements are based on the retinal location of the target, and not the
strength of the retinal activation (nor other saliency measures) produced by the target;
strength of RGC activity is used for determining the direction toward which to move, but
not for determining the speed of the turn.
Didday & Arbib (1975) proposed a model in which visual salience is an eye movement
guidance criterion, used to build a feature-based representation of the visual scene. Koch
& Ullman (1985) proposed a model of human visual attention based on two fundamental
concepts: the saliency map, and the winner-take-all (WTA) network. Winner-take-all
(WTA) networks (Amari & Arbib, 1977) have long been used in amphibian tectum mod-
els (Didday, 1976; Manteuel & Roth, 1993). The saliency map, i.e., a topographic neural
structure combining location-dependent features so that the neural activation is a quan-
titative assessment of the visual \conspicuity" at that location, has not been explicitly
adapted for amphibian brain modeling, although one could argue that it has been implic-
itly used (e.g., Woesler (2001)). We note that, although a visual saliency map is normally
considered a 2-dimensional retinotopic map, this dimensionality is not required by the
denition in Koch & Ullman (1985).
There have been several models of short range visuomotor coordination in amphib-
ians. Simulander (Eurich et al., 1995) modeled orienting movements and snapping, It
62
was used to demonstrating the eectiveness of large overlapping visual receptive elds
by producing a system level precision ner than the size of the individual receptive elds
(Eurich et al., 1997). Lamb (1997) proposed and simulated a model of the involvement of
the nuclei isthmi in amphibians' snapping. Liaw & Arbib (1993) modeled neural control
of frogs' ballistic escape movement; this work built upon the motor heading map hypoth-
esis introduced in Cobas & Arbib (1992), in which it was proposed that the direction of
motion was based on interaction among the sensory signals converging on this 1-D topo-
graphically organized neural assembly. These models include tectum-based visual control
of locomotion, but focused on discrete movements rather than continuing locomotion.
The motor heading map hypothesis is particularly relevant to the present work. That
hypothesis was proposed to explain how a single network could command either approach
toward or
ight away from any direction, depending on the nature of the stimulus in that
direction. The present work models the path from motor heading to the CPGs.
3.1.5 Simulation method
We investigate the interaction of salamander locomotion and motion sensitive visual neu-
rons by building on Ijspeert's neuromechanical model of salamander locomotion in an
articial world (Ijspeert, 2001; Ijspeert & Arbib, 2000a,b). OpenGL
r
graphics generate
the moving salamander and its world, and simulate the constantly changing photoreceptor
input to the salamander visual system. This study demonstrates that an elementary, but
neurally plausible, visual guidance system can be used to steer central pattern generator
(CPG)-based locomotion.
63
This work builds on Ijspeert's model of the salamander spinal central pattern gen-
erator (CPG) (Ijspeert, 2001). This work developed a simulated CPG that produced
realistic swimming and trotting movements in a biologically plausible manner. Ijspeert
developed an animated simulation system which displayed this locomotion in the context
of simulated prey pursuit. This simulation used a simple, non-biological visual system
for guidance. The present work enhances this system with a biologically-inspired motion-
sensitive visual guidance system.
Ijspeert's model of CPG-based locomotion included modulation of the direction of
motion using two signals descending from the brainstem to the left and right spinal
CPGs, respectively. These 2 activation levels represent the population activity of the
hindbrain neurons that are modulated during steering (Saitoh et al., 2007).
3.1.6 Hypotheses
We hypothesize that the visual and locomotor systems have co-evolved as interactive
systems well adapted to each other. The present study continues our work of chapter
2 (Begley & Arbib, 2007) in modeling synergies of these systems. In that work we
introduced a model of retinal motion sensitivity, and explored the model's responses to
locomotion-induced head/eye movement in simple simulations using natural images for
photoreceptor input. That investigation did not address the question of how these RGC
responses could be of use in guiding locomotion in a dynamic environment.
64
Anatomically, the tectum is retinotopically organized (Roth, 1987). We note that
primate arm movements are coded in the superior colliculus (homologous to the amphib-
ian tectum) in gaze-based coordinates (Stuphorn et al., 2000); tectal motor control using
such coordinates may be an evolutionarily conserved brain operating principle. However,
it is not obvious that undulatory locomotion can be eectively controlled in gaze-based
coordinates. Undulation can cause large head movements (Frolich & Biewener, 1992;
Gillis, 1997; D'Aout & Aerts, 1997), leading to a constantly changing gaze direction. An
alternative hypothesis might use body-centered coordinates based on some center of the
undulation, or on some general direction of motion. Our modeling tests the hypothesis
that gaze-based coordinates are adequate to control undulatory locomotion, though the
reference frame is in constant motion.
The salamander has on the order of 70,000 RGCs (Zhang et al., 2004; Roth, 1987).
Although this is a rather small number, it represents a very large proportion of the sala-
mander's neurons, compared with other vertebrates. The optic tectum (OT) is the most
important visual area in the salamander's brain (Roth, 1987). The OT contains between
35,000 and 150,000 neurons, depending on species (Roth et al., 1995). Generalizing the
lamprey ndings of Saitoh et al. (2007) to salamanders suggests that a minority of these
neurons may be involved in locomotion steering. We infer that data compression and
dimensionality reduction at the level of the OT is essential.
The undulatory movements characteristic of salamander swimming and terrestrial
trotting can show a wide range of rotational head velocities in the horizontal plane. The
65
undulatory gait makes it dicult to dene the salmander's instantaneous direction of
motion. Since the salamander's eyes do not move with respect to the head (Roth, 1987),
head movement directly implies eye movement. This suggests that guiding the visual
frame toward a goal may be a good strategy for adaptively controlling locomotion.
In chapter 2 (Begley & Arbib, 2007), we showed that, during periods of high self-
motion induced background motion in the retina image, the activity of motion-sensitive
RGCs may be erratic. We demonstrated the possibility of acute motion sensitivity in
salamanders at the extremes of their locomotion-induced side-to-side head movements,
after neural delay. These correspond to periods of relatively low self-motion induced
retinal slip. In this chapter, we investigate this further, proposing a mechanism by which
the relatively high sensitivity may be used in guiding locomotion during foraging. We
propose that salamanders may have a neural mechanism somewhat analogous to saccadic
suppression in higher vertebrates (Goldberg & Wurtz, 1972; Diamond et al, 2000). The
visual scene with minimal background motion at the extremes of locomotion-induced
head oscillations may be so much more useful for locomotion control that the salamander
brain would disable visual control during times in the locomotion cycle of high background
motion.
3.1.7 Roadmap
x3.2 describes the model, with implementation details provided in the appendices. Sim-
ulation results are described inx3.3. The kinematics produced by our simulations are
66
analyzed inx3.4. The context and implications of our results are discussed inx3.5, and
brie
y recapitulated inx3.6.
3.2 Model description
3.2.1 Overview
Figure 3.1 shows the connectivity of the brain regions simulated in our model. The
tectum receives input from a number of other brain regions; in the current model, we
focus on direct sensory input from the retina (appendix 3.7), integrated with important
information provided by an idealized model of tegmental/pretectal self-motion estimates
(x3.2.2) (Manteuel & Naujoks-Manteuel, 1990).
The challenge is to identify a visually salient target during rapid undulatory loco-
motion. We will present evidence that the retina model provides its best information
just after the extremes of the locomotion-induced head undulation, i.e., after the head
movement direction reversals. In our model, visual processing occurs a xed time delay
after the direction reversal, as detected by the idealized self-motion estimation system.
Inx3.2.2 we describe how the 2-dimensional retinal image is converted to a scalar
motion direction. This section describes three variations on the model, each characterized
by the timing of the visual control of locomotion. Two are characterized by visual update a
xed delay after the direction reversal of the rotational component of locomotion-induced
undulatory head movment. This delay is based on neural delays, not the speed of the
movement. At each reversal, the tectum's desired direction of movement is updated to
67
salience vector
WTA
tectum
tegmentum
pretectum
head rotation
reversal
head
orientation
turn direction
desired
heading
spinal CPGs via
brainstem
left push right pull right push left pull
retina via
optic nerve
Figure 3.1: Modeled brain regions. In our model, the tectum converts the 2-D retinal motion
signals into a 1-D salience vector, then converts this to a commanded direction of motion, relative
to the head orientation. This direction determines CPG excitatory signals. These tonic signals
are transmitted to the spinal central pattern generators (CPGs) via the brainstem reticulospinal
pathway. The spinal cord commands limb and body muscles, generating undulatory locomotion
(swimming or walking). See gure 3.16 for details of the retina model. A simplied, ideal,
head-orientation meter is shown; inputs from the tegmentum and/or the pretectum may be used
by the salamander tectum for this purpose. This ideal orientation detector has two functions
in this model: 1) it determines the time at which the desired motion direction command is
visually updated, and 2) it updates the motion direction command according to the current
head orientation. Retinal delays and a related delay in the enablement of the desired heading
computation after the head rotational movement direction reversal synchronize the visual system
and locomotor control.
68
the direction indicated by the tectum model's 1-D visual \salience" vector. Between these
times, the tectum's desired direction of movement is continually updated by subtracting
the actual head movement, as determined by idealized dead reckoning. One of these
models updates the turn command to the visually salient direction at an instant (the
\instantaneous" model), while the other continues visual control for a brief interval (the
\duration" model). A third model uses continual visual control, even during periods of
high retinal background motion (the \continual" model).
Further, inx3.2.2 we explain how the desired direction of motion is used to control the
direction of motion. This control uses very few neural signals to control the spinal central
pattern generator (CPG), as described and analyzed in Ijspeert (2001). In this model,
co ordinated movement of the undulatory body movement in both swimming and walking,
and of the limbs, is provided through a network of spinal neural central pattern generators
(CPGs). Each spinal segmental neural oscillator is coupled with nearby segments, with
the strenth of the coupling descreasing with increasing distance along the spinal cord.
Each spinal pattern generator receives a scalar input from a descending projection from
the brainstem. It is through this input that the brain controls the speed and direction of
movement.
The steering model is a 2-D model; the salamander may turn to the left or right. This
is obvious for terrestrial locomotion. It is also appears to be a basic property of vertebrate
CPG-based undulatory swimming (Ijspeert, 2001; Fagerstedt et al., 2001; Saitoh et al.,
69
2007; Kozlov et al., 2002). The steering direction is controlled in visual gaze-based, head-
centered coordinates { this work tests the hypothesis that such a coordinate system,
constantly moving, is adequate to control undulatory locomotion.
Salamander locomotion is generated by a spinal CPG consisting of 40 segmental neural
oscillators. Nearby segments are interconnected, and thereby coupled, resulting in the
characteristic salamander locomotion spinal wave patterns (standing waves for swimming,
travelling waves for trotting). The physics are simplied, especially the hydrodynamics
of swimming. The simulated world is 2-dimensional: motion is in the horizontal plane.
In the current implementation, only two of the descending inputs to the CPGs from
the brainstem are signicant for the control of the direction of motion: the tonic inputs
to the left and right body CPGs. Ijspeert & Arbib (2000b) demonstrated that the CPG
model can be robustly controlled by continually varying these two signals, and that a
neuromechanical simulation model of the salamander can be reliably guided toward a
target with this method. The computation of these two scalar signals,tonic
r;l
, is described
inx3.2.2.
The dynamics of the muscle and body movement in the simulations reported in this
chapter are 2-D simplications of the 3-D mechanical simulation described in Ijspeert
& Arbib (2000b,a); Ijspeert (2000). The muscles are modeled as idealized springs and
dampers, based on Ekeberg (1993).
70
The retina model is that of chapter 2 (Begley & Arbib, 2007). A sparser RGC array
in the present work is closer to the biologically observed density
1
. See appendix 3.7 for
full details of the retina model. See appendix 3.9 for details of the animated graphical
articial world.
3.2.2 Brainstem model
The model of sensorimotor integration and control in the central nervous system (CNS)
is shown in gure 3.1. This model receives visual input from the retina (appendix 3.7),
and sends output signals to the spinal CPGs ((Ijspeert, 2001)). Details of the simulation
implementation of this model are provided in Appendix 3.8.
The rst step in the CNS model is dimensionality reduction, from the 2-D retinal
image to a 1-D salience direction vector. The term \salience" emphasizes the relationship
of visual features to the locomotion direction. In our salience vector, the activity of the
most active location is greatly enhanced { the basic idea is a winner-take-all function
applied over all visible directions, to the resolution of the retina. For the purpose of
this model, we use the raw RGC activity { this is a simplication. Past models have
demonstrated subtle visual prey discrimination (Wang & Arbib, 1990), obstacle avoidance
(Arbib & House, 1987), and predator avoidance (Liaw & Arbib, 1993). These phenomena
are all part of of a complete account of amphibian visuomotor coordination, but we
abstract these out of this model for the purpose of studying sensory-motor integration in
1
Chapter 2 was an investigation of the properties of the essentially local center-surround motion
opponency operating principle, so the denser array provided more data for comparison. Here, we are
interested in the retina as an integrated part of the animal, so we study the system performance with the
relatively sparse array.
71
CPG-based locomotion steering. This simplication preserves the basic spatio-temporal
features of the neural processing of visual information. We note that these RGCs are
sensitive to both contrast and motion, basic visual features to which salamanders are
sensitive (Schuelert & Dicke, 2005; Sch ulert & Dicke, 2002).
Before applying the winner-take-all, we rst sum the activation in each column corre-
sponding to a particular turning direction. We assume that a linear summation of RGC
activity over the retinal locations having a particular preferred locomotion direction gives
a reasonable estimate of the desirability of moving in that direction. This avoids the less
likely alternative of winner-take-all over the entire retina, which would require global in-
terconnectivity over an extensive tectal retinotopic structure. However, our model does
not preclude WTA over the entire retina { the key is to select a head-centered (and hence,
eye-centered) direction.
Considering this part of our model as an adaptation of the framework of Didday &
Arbib (1975) and Koch & Ullman (1985), the RGC array is a feature map. Rather than
projecting projecting this map onto a 2-dimensional saliency map from which an atten-
tional locus may be determined, we project onto a 1-dimensional motor map from which a
direction of locomotion may be determined. In our case, the topographic correspondence
is with the head-centered horizontal directions. The 1-dimensional data structure is small
enough that global inhibition is feasible, permitting a fast and simple neural realization
of a winner-take-all (WTA) network
2
.
2
In the attentional model of Koch & Ullman (1985), such global interconnectivity was considered
implausible.
72
This model can be considered an instance of the Liaw & Arbib (1993) interpretation
of the motor heading map (introduced in Cobas & Arbib (1992)) as selecting the direction
of motion through the interaction of converging sensory signals at the 1-D topographical
map. In our case, the interaction is a winner-take-all enhancement of the most salient
direction, determined by the sum of the activity of motion-sensitive RGCs whose ERF
locations correspond to the preferred direction of each neuron in the 1-D map.
The preferred direction of movement is computed as a weighted sum of the salience
vector neural activity levels. The weights represent the strength of the synapses from the
salience-encoding units to the summing neuron. These synaptic strengths are roughly
related the angle of the bearing, in head-centered coordinates, to objects whose images
project into the RGC RFs; these RGCs are then themselves projected onto the salience
map. It is the 1-D salience map that is neurally summed in a bearing direction weighted
manner. Thus, the 2-dimensional retinal images have been reduced to a scalar direction
value for motor control. For locomotion control, the use of this direction may depend on
the timing of the undulatory locomotion cycle.
In chapter 2 (Begley & Arbib, 2007), we found that object motion detection is very
sensitive to background motion motion eects. When the salamander's locomotion in-
volves oscillatory head movement, background motion will be minimized when the head
movement reverses direction. This suggests that important retinal information is avail-
able at these times, after a neural processing delay. We simulate an idealized model of the
tecta's self-motion estimation. It has been suggested that the pretectum and tegmentum
73
provide self-motion information in their inputs to the tectum (Manteuel & Naujoks-
Manteuel, 1990; Woesler & Roth, 2003). We model this as a perfect estimate of head
orientation. This model knows, with no delay, the head orientation at every simulation
step. The times in the undulatory locomotion cycle when the rotational component of
head movement changes direction are considered times of relative retinal image stability,
with minimal artifacts of self-motion.
In order to investigate the eects of sensorimotor neural precision on steering CPG-
based locomotion, we simulated two dierent models of updating the desired direction of
locomotion a xed delay after the reversal of the rotational component of head movement.
In the instantaneous model, the desired heading is updated to the visually computed value
at an instant 32 milliseconds after the reversal. This interval is within the range of tectal
interneuron latencies (Roth et al., 1999). In the duration model, the desired heading
is under visual control for a 16 millisecond duration beginning 24 millseconds after the
reversal. We nd that the simulated RGC activity is generally rather stable during this
time interval. During this interval, the descending CPG drive signals are decoupled from
the actual salamander movement, computed from the idealized motion detection system.
At other times, the desired turn is based on the desired heading (computed at the last
visual update), corrected for the rotational movement of the salamander's head { the
desired heading is thus, brie
y, an allocentric direction.
74
We compare these two models with a model of continual visual control of the CPG
drive signals. In this model, the drive signals are never directly coupled with the salaman-
der's actual movement. In what follows, we refer to these models as the instantaneous
model, the duration model, and the continual model.
1950 2000 2050 2100 2150 2200 2250 2300 2350 2400
−80
−60
−40
−20
0
20
40
60
simulation time (milliseconds)
weighted sum / turn command (degrees)
Figure 3.2: The turn command, relative to head orientation (solid curve), and the weighted sum
of the retinal salience vector (dotted line), the basis of direction of motion commands to the CPGs.
The short spike from zero indicates the reversal of the rotational component of head movement.
The long spike from the bottom of the graph indicates the instant that the CPG control signals
are updated, a xed delay after the reversal. This is for a turn to left in a swimming simulation.
The turn command follows the head movement, except at the visual update discontinuities. The
turn command is updated at times of low retinal background motion. Background motion causes
large
uctuations in the weighted sum, compared with the turn command.
An example of the instantaneous model turn command, compared with the retinal
salience-averaged direction, is shown in gure 3.2. The turn command follows the ro-
tational component of head movement except at the instant of visual update. The dis-
continuities in this command are small and regular, compared with
uctuations in the
75
salience dot product (the dotted line in gure gure 3.2). It can be seen in this gure
that the turn direction command discontinuities are largely due to the uncompensated
neural delays.
The observation of abundant reticulospinal neurons with ring rates that increase
coincident with ipsiversive turns and decrease with contraversive turns (Fagerstedt et
al., 2001) suggests the possibility of a push-pull steering mechanism, which would give
temporally precise control of both the increased and decreased ring rates. Although
other mechanisms might account for the observation, the push-pull hypothesis leads to a
compact model consistent with the observation. Higher activation levels in these neurons
produce swimming (Cabelguen et al., 2003). Those authors reported that where elec-
tostimulation of the mesencephalic locomotor region (MLR) with 0.5{3.5A produced a
stepping gait, while 1.8{5.5 A produced swimming. (The overlap is due to individual
dierences between subjects, and perhaps hysteresis.) These push/pull signals from the
tectum are used to control locomotion via tonic input to spinal units.
See Appendix 3.8 for a complete formal description of of our implementation of this
model. See Ijspeert (2001) for a detailed description of how tonic input from the brainstem
to spinal central pattern generators (CPGs) is used to control locomotion.
3.3 Simulation results
We report here on simulations of swimming and walking to the truncated cone stimulus
at locations 90
o
to the left and right, and straight ahead of, the initial position and
76
orientation of the salamander's head. In each case, the target was 1400 millimeters
from the initial head position. This was empirically found to be near the limit beyond
which the salamander does not reliably steer toward the target. The 90
o
turns were used
to investigate whether our visual steering models can successfully orient and guide the
salamander. No turn was needed to reach the forward target; these runs are used to
investigate the eect of visual steering models on the dynamics of locomotion.
Figure 3.3 is a snapshot of the simulation of the instantaneous model in a swim to the
leftward target. This gure illustrates the delays in the visual locomotion control system.
The snapshot was taken at the simulation step in which the turn direction was updated
to the visual input. We observe that the image of the target in the raw \photoreceptor"
display (g. 3.3b) is to the right of the activation it caused in the RGC layer (g. 3.3e)
and in the 2-D salience sum (g. 3.3f). This shift is the eect of locomotion-induced head
movement during the delay period. These delays will tend to enhance the movement by
commanding a larger turn in the direction of the head movement, compared with the turn
that would be commanded by an undelayed system. On the other hand, during the 32
millisecond period between the head rotation reversal and the visual direction command,
the direction from the previous visual xation is still active, which would tend to command
a smaller turn in the head movement direction { this may tend to damp the acceleration
during a slow phase of head movement. To investigate these eects on the locomotion
gait, we will compare our visual control models with such an undelayed system, which
77
a)
b)
c)
d)
e)
f)
Figure 3.3: The simulation display. While this is shown in grayscale, the actual simulation
display is in color. a) Top view of the salamander in its environment; b) Photoreceptor input
to the retina model (g. 3.16a) { the simulation display shows full RGB color, but the model
sums the channels to yield an achromatic visual system; c) Retina prelter (g. 3.16e); d) Retina
motion sensitivity (the sum of the 4 directional channels, 2 of which are shown in g. 3.16e);
e) Retinal ganglion cell (RGC) activity (g. 3.16h); f) Bar graph of one-dimensional salience
map activity. The dark cone is the target, which moves back and forth along its axis line. The
salamander is swimming to the leftward target. This was a simulation of the \instantaneous"
model. This snapshot was taken just after the turn command was updated to the value computed
from the salience vector { this occurs a short delay after the reversal of the locomotion-induced
head rotation. Note the eect of visual system delays { the target image in b) is well right of
its in
uence in f). Note also the eect of perspective distortion; the target, near the center line
where the left and right projections join, appears longer on its axis than the greatest diameter,
but the diameter at the left end is actually slightly greater than the length. We also see the eect
of perspective distortion in the water shading artifact above the target in b).
78
we will call the \ideal" model. This model continually updates the commanded turn to
the true target head-centered direction.
−1500 −1000 −500 0 500 1000 1500
0
500
1000
1500
trot instantaneous
swim instantaneous
trot duration
swim duration
Figure 3.4: Paths taken during walking and swimming runs of the two models of visual updating
of locomotion steering at \image xation" times. Simulation runs start with the salamander facing
\upward".
The paths of the salamander's head during these simulation runs are shown in gures
3.4 and 3.5. Figure 3.4 shows paths for the two models of head rotation reversal-based
updating of the desired movement direction, the \instantaneous" and the \duration"
models. Figure 3.5 shows paths for the continual visual control model. In each simulation,
the salamander reached the target { the greatest minimum distance from the center of
the salamander's head to the target center was 52 millimeters.
There was only one case in which the salmander did not make continual progress
until it reached the target. While walking toward the leftward target, the salamander
79
−1500 −1000 −500 0 500 1000 1500
0
500
1000
1500
swim
trot
Figure 3.5: Paths taken during walking and swimming runs of the simulation model of continual
visual updating of locomotion steering. Each simulation starts with the salamander facing \up-
ward". While turning toward the leftward target, the trotting salmander was distracted by its
tail.
80
was distracted by its own foot, and thereupon commenced chasing its own body (mainly
the tail) for more than two full loops before once again the target won the salience
competition and the salamander continued on to complete the task. This distraction
occurred during the execution of a rather wide turn to the left; the loops were clockwise
turns to the right. It appears that a third loop could have been well under way when the
target nally won the salience competition. In gure 3.5, we see a transitional segment
of the path, in which the side-to-side head movements don't immediately commence after
the second loop, suggesting that the salamander may still be turning right toward its
tail, although the direction of movement is taking it toward the target. Also, we observe
that the salamander makes some progress toward the goal even while pursuing itself; the
second loop is nearer the target than the rst. The target in
uenced the direction of
movement even when the salamander's pursuit of its own body dominated the steering.
Note that the paths to the leftward and rightward targets are relatively smoothly
curved (g. 3.4). The continual model simulation paths do not show this pattern. Among
the paths shown in gures 3.4 and 3.5, the widest turn appears in the swim toward the
rightward target guided by the continual model (gure 3.5). And during the walk to the
rightward target using the continual model (again, gure 3.5), the salamanders appears
to turn too far to the right, then curves to the left to compensate.
The two head rotation reversal dependent visual steering models (the instantaneous
and duration models), by contrast, produce relatively smooth, uneventful paths to the
81
target. The continual model also made uneventful progress to the forward target in both
the swimming and walking simulations.
0 2000 4000 6000 8000
0
500
1000
1500
simulation time (milliseconds)
distance to target (millimeters)
Continual
Duration
Instantaneous
Ideal
0 2000 4000 6000 7000
0
500
1000
1500
simulation time (milliseconds)
distance to target (millimeters)
Continual
Duration
Instantaneous
Ideal
0 2000 4000 6000 8000 10000
0
500
1000
1500
simulation time (milliseconds)
distance to target (millimeters)
Continual
Duration
Instantaneous
Ideal
0 2000 4000 6000 8000
0
500
1000
1500
simulation time (milliseconds)
distance to target (millimeters)
Continual
Duration
Instantaneous
Ideal
0 1000 2000 3000 4000 5000 6000
0
200
400
600
800
1000
1200
1400
simulation time (milliseconds)
distance to target (millimeters)
Continual
Duration
Instantaneous
Ideal
0 1000 2000 3000 4000 5000 6000
0
500
1000
1500
simulation time (milliseconds)
distance to target (millimeters)
Continual
Duration
Instantaneous
Ideal
Figure 3.6: Distance to target during trotting and swimming runs. Four steering methods were
used. \Ideal" steering uses the true bearing to the target to compute the relative descending
control signals to the spinal CPGs. The other 3 use visual steering via the optic tectum. The
\instantaneous" model updates the desired heading 32 milliseconds after each reversal of the
rotational component of locomotion-induced oscillatory head movement; the \duration" model
updates the desired heading for 16 milliseconds, ending 39 milliseconds after each head rotation
reversal. At other times, these 2 models simply steer to that desired heading, using dead reckoning.
The \continual" model constantly steers toward the visually-computed direction. Top { swimming;
bottom { trotting. The left, center, and right columns show data for runs starting with the
target to the left, straight ahead, and to the right of the salamander in the starting position.
While turning toward the leftward target using the continual visual steering model, the trotting
salmander was distracted by its tail.
In gure 3.6, we plot the distance to the target as a function of time. In addition to
the 3 visual steering models whose paths are plotted in gures 3.4 and 3.5, we include a
model in which the desired direction to the target is constantly the true direction; this
direction is used in the computation ofpush
r;l
(x3.2.2 and Appendix 3.8). We call this the
\ideal" model. The data plotted may appear to contradict this name { the ideal model
82
is not consistently the rst to reach the target. And, notwithstanding the anomalies
observed in gures 3.4 and 3.5, the continual model appears to perform comparatively
adequately in all but the walk to the left (gure 3.6, bottom left). This suggests that
the continual model may be adequate for steering, especially when locomotion control is
modulated by other brain regions.
Certain aspects of the present work limit the usefulness of the speed data one can
infer from gure 3.6. First, the physics simulations are highly simplied. Furthermore,
the modeling includes no energy limitations; e.g., muscle fatigue is not modeled. An
inecient gait may therefore be faster than an ecient one. Thus, speed can't be used
quantitatively as a tness discriminator for theses models.
Figure 3.7 shows the turn direction and the direction weighted retinal salience sum for
the two head rotation reversal-based visual steering models, the instantaneous and the du-
ration models, during the turn toward the leftward target. This is shown to demonstrate
the apparently relatively smooth behavior of these models during successful negotiation
of this sharp turn. Due primarily to uncompensated visual system neural delays, there
is a rather large jump in the turn direction when it is updated according to visual input
after the head rotation reversal. Unlike the continual model, these models consistently
selected the target direction.
In contrast, gure 3.8 shows the behavior of these quantities during the interrupted
turn to the left while walking with the continual model. In this model, the two quantities
are the same. The large jumps are due to erratic visual system responses to background
83
500 1000 1500 2000 2500
−40
−30
−20
−10
0
10
20
30
simulation time (milliseconds)
turn command (degrees)
500 1000 1500 2000 2500
−50
−40
−30
−20
−10
0
10
20
30
40
simulation time (milliseconds)
turn command (degrees)
500 1000 1500 2000 2500
−100
−80
−60
−40
−20
0
20
40
simulation time (milliseconds)
weighted sum (degrees)
500 1000 1500 2000 2500
−80
−60
−40
−20
0
20
40
60
simultion time (milliseconds)
weighted sum (degrees)
Figure 3.7: Turn command (top) and vector sum (bottom) during simulations of trotting to the
leftward target. Left { instantaneous visual update model (32 ms. after head rotation reversal).
Right { duration visual update model (16 ms. ending 39 ms. after head rotation reversal).
84
500 1000 1500 2000 2500
−100
−80
−60
−40
−20
0
20
40
60
80
100
simulation time (milliseconds)
turn command (degrees)
1500 2000 2500 3000 3500
−60
−40
−20
0
20
40
60
80
100
120
simulation time (milliseconds)
turn command (degrees)
3000 3500 4000 4500 5000 5500
−100
−80
−60
−40
−20
0
20
40
60
80
100
simulation time (milliseconds)
turn command (degrees)
Figure 3.8: Turn command and vector sum (identical in this model) during simulations of trotting
to the leftward target using the continual visual steering model. During the early part (top) the
salamander started chasing its own body. During the the \tail chasing" episode (middle), the
target brie
y aected the turn direction. Later (bottom), the target again dominated the turn
direction determination, and the salmander made good progress toward the target.
85
motion (cf. chapter 2 and Begley & Arbib (2007)). The salamander in this simulation
run starts the turn to the left, becomes distracted by its own body parts, loops while
chasing its tail, then resumes pursuit of the target.
Figure 3.9 shows CPG output to the body muscles during the walk to the left. The top
row shows a rather regular oscillatory pattern with regular phase lags from in the head-
to-tail direction. This regularity contrasts with our impression of the bottom two rows,
which show activity of the CPGs under the continual visual control model. The middle
row shows activity before and during the \tail-chasing" episode, while the bottom row
shows activity as the salamander emerges from the loop and afterward when it is making
progress toward the target. We observe a clear distinction between tail and torso CPGs
in the continual model. This distinction is much subtler in the top row, \instantaneous"
model data. This may be due to frequent changes in the commanded direction for the
continual model (g. 3.8), which may alter the phase relationship between the front and
hind limbs { in this model of the spinal cord, the torso CPGs are coupled to the front
limb CPGs, while the tail CPGs are coupled to the hind limb CPGs (Ijspeert, 2001).
In gure 3.10, we turn our attention to simulations of swimming to the forward target.
This is a test of the visual steering mechanisms to guide the salamander to a target on
a straight path. During swimming, the salamander keeps its feet back near the body, so
they are unlikely to enter the visual eld, so the salamander is not likely to be distracted
by its own foot, as it was in simulated walking to the leftward target under the continual
model. As we saw in gure 3.6, all models successfully completed this task. All of these
86
500 1000 1500 2000 2500
1
2
3
4
5
6
7
8
9
simulation time (milliseconds)
motoneurons
500 1000 1500 2000 2500
1
2
3
4
5
6
7
8
9
simulation time (milliseconds)
motoneurons
500 1000 1500 2000 2500
1
2
3
4
5
6
7
8
9
simulation time (milliseconds)
motoneurons
500 1000 1500 2000 2500
1
2
3
4
5
6
7
8
9
simulation time (milliseconds)
motoneurons
3500 4000 4500 5000 5500
1
2
3
4
5
6
7
8
9
simulation time (milliseconds)
motoneurons
3500 4000 4500 5000 5500
1
2
3
4
5
6
7
8
9
simulation time (milliseconds)
motoneurons
Figure 3.9: Body segment joint motoneuron output of the spinal CPGs during simulations
of trotting to the leftward target using the instantaneous (top) and continual (bottom 2 rows)
visual steering update models. Left and right columns show data for the left and right CPGs,
respectively. The segments are numbered 1-9, in the head-to-tail direction.
87
0 1000 2000 3000 4000 5000 6000 7000
−60
−40
−20
0
20
40
60
simulation time (milliseconds)
commanded turn (degrees)
0 1000 2000 3000 4000 5000 6000 7000
−60
−40
−20
0
20
40
60
simulation time (millliseconds)
turn command (degrees)
0 1000 2000 3000 4000 5000 6000 7000
−80
−60
−40
−20
0
20
40
60
80
simulation time (milliseconds)
turn command (degrees)
Figure 3.10: Turn command during simulations of swimming straight ahead to the forward target
using the instantaneous (top), duration (middle), and continual (bottom) visual steering update
models.
88
control methods provide stable locomotion. In particular, the rather erratic behavior of
the continual model (g. 3.10, bottom) produced the fastest gait.
2000 2500 3000 3500 4000 4500 5000
1
2
3
4
5
6
7
8
9
simulation time (milliseconds)
motoneurons
2000 2500 3000 3500 4000 4500 5000
1
2
3
4
5
6
7
8
9
simulation time (milliseconds)
motoneurons
Figure 3.11: Left body motoneuron output of the spinal CPGs during simulations of swimming
straight ahead to the forward target using the instantaneous (top) and continual (bottom) visual
steering update models. Cf. g. 3.9
Figure 3.11 shows the left side body CPG motoneuron output. In contrast to our
observations in gure 3.9, the pattern under both intermittent and continual visual control
is quite regular. Some irregularity is evident in the bottom plot (continual model data)
compared with the top (instantaneous model data), but both give the impression of
regularity. Considering that the continual model produced greater speed, it may be
89
signicant that the head-to-tail wave in the pattern of the phase lags appears to be
slower under the continual model. It may be that the slower wave produces a greater
amplitude of undulation, resulting in greater propulsive force. Since the limb CPGs do
not operate during swimming, there is no clear distinction between the torso and tail
patterns, in contrast to the walking patters observed in gure 3.9.
−600 −400 −200 0 200 400 600
−100
0
100
200
300
400
500
swim instantaneous
swim duration
trot instantaneous
trot duration
Figure 3.12: Paths to targets at arbitrarily chosen initial target bearings at 700 mm. The
bearings are42:4
o
and31:7
o
from straight ahead at the initial salamander position. The
intermittent models, instantaneous and duration, reached the targets without incident.
90
0 1000 2000 3000 4000 5000 6000 7000
0
10
20
30
40
50
60
simulation time (milliseconds)
millimeters
0 1000 2000 3000 4000 5000 6000 7000
0
10
20
30
40
50
60
simulation time (milliseconds)
millimeters
0 1000 2000 3000 4000 5000 6000 7000
0
5
10
15
20
25
30
35
40
45
50
simulation time (milliseconds)
millimeters
0 1000 2000 3000 4000 5000 6000 7000
0
10
20
30
40
50
60
simulation time (milliseconds)
millimeters
Figure 3.13: Distance between head positions at successive reversals of rotational motion for
simulations of swimming straight ahead to the forward target. Top left: instantaneous; top right:
duration; bottom left: continual; middle right: ideal.
91
3.4 Dynamical analysis of empirical kinematics
An important hypothesis (x3.1) for our work is that the visual scene with minimal back-
ground motion at the extremes of locomotion-induced head oscillations is so much more
useful for locomotion control that the salamander brain would disable visual control dur-
ing times in the locomotion cycle of high background motion. This hypothesis suggests
that quantitative analysis of the time series of these \visual xation" points may yield
information relevant to locomotion control. One such discrete characterization of the
continuous head movement is the sequence of distances between head positions at suc-
cessive head rotation reversals. These are plotted in gure 3.13 for the four simulations
of swimming to the forward target. Each plot starts at the simulation start and ends at
the closest approach to the target.
Figure 3.13 seems to suggest a ranking of the models in terms of the stability of
this quantity. By this measure, \ideal", non-visual, guidance (the slowest!), produces
the most stable gait, after the gait stabilizes near 2000 milliseconds. The instantaneous
model appears to have a secondary oscillation (in addition to the oscillation that forms
the basis of the computation of this quantity). The duration model seems to produce a
less stable gait by this measure, and the continual model (the fastest!) shows the least
stability.
In gure 3.14, we plot (rst) return maps, also called Poincar e maps, for the distance
between successive head rotation reversal locations. In graphing a return map for this
gait discretization, we plot d
i
on the X-axis and d
i+1
on the Y-axis, where i is the
92
0 10 20 30 40 50
0
5
10
15
20
25
30
35
40
45
50
Figure 3.14: Return maps of the distance between head positions at successive reversals of rota-
tional motion for simulations of swimming straight ahead to the forward target for the instaneous
(`o'), duration (`x'), and continual (`*') visual steering update models.
93
0 1000 2000 3000 4000 5000 6000
0
5
10
15
20
25
30
35
40
simulation time (milliseconds)
millimeters
0 1000 2000 3000 4000 5000 6000
0
5
10
15
20
25
30
35
simulation time (milliseconds)
millimeters
0 1000 2000 3000 4000 5000 6000
0
5
10
15
20
25
30
35
40
45
simulation time (milliseconds)
millimeters
0 1000 2000 3000 4000 5000 6000
0
5
10
15
20
25
30
35
40
simulation time (milliseconds)
millimeters
0 10 20 30 40 50
0
5
10
15
20
25
30
35
40
45
50
Figure 3.15: Top 2 rows: distance between head positions at successive reversals of rotational
motion for simulations of walking straight ahead to the forward target. Top left: instantaneous;
top right: duration; middle left: continual; middle right: ideal. Bottom: return map of the
distance between head positions at successive reversals of rotational motion for simulations of
walking straight ahead to the forward target for the continual visual steering update model.
94
rotation reversal ordinal (increasing with simulation time). Returns maps are useful
for characterizing the dynamics of temporal phenomena (Denton & Diamond, 1991),
including robot movement (Nakamura et al., 1997; Wisse et al., 2005), and human gaits
(Begg et al., 2005). Here, we introduce their use in analyzing our models' simulated
anguilliform swimming.
The periodicity we observe in the plot of the instantaneous model's successive rota-
tional reversal point distances (gure 3.13, top right) is seen as clustering in a loop in
gure 3.14. The same looping structure is not as evident for the duration model, but this
model's return map seems somewhat structured, although more data might be needed to
characterize that structure. This structure suggests the possibility that the gait could be
modulated for stability under these steering models. Seex3.5.
The continual model had more head rotation reversals than the other models. The
head movement had several short duration rotation movements with short distances be-
tween some successive rotation reversals. The undulation of this faster gait appears to
include higer frequency oscillations. We notice that, withd
i
< 13,d
i+1
lies near or above
the d
i+1
= d
i
line, with several points well above that line { when the head movement
distance is in this low range, this distance tends to increase, as we would expect. How-
ever, the range of d
i+1
in the plot is large, with possible clustering evident only near
the d
i+1
= d
i
line { this map does not provide enough information to predict the head
movement. In the region of d
i
> 30, most of the points are below the d
i+1
=d
i
line. In
the middle range of d
i
, there are several points well above and well below the d
i+1
=d
i
95
line. Throughout the plot, the range of d
i+1
is large, with no evident clusters. In the
seeming uniformity of the distribution of points on the return map,it is like that there is
no phase space
ow toward a xed point, nor toward a limit cycle. These observations
suggest that it may be dicult for the salamander brain to modulate the gait for stability,
in this measure, under continual visual steering.
3.5 Discussion
We developed a model of visual guidance of salamander CPG-based locomotion with re-
alistic, unmitigated visual system delays. We investigated possible eects of the duration
of the disinhibition of visual control of the direction of movement. We used instantaneous
visual update of the commanded direction, and updating over a 16 millisecond duration
(the \duration" model). For these two alternatives, the steering control was perfectly
coupled to the head movement when visual control was inhibitted. We compared these
with a model of continual steering toward the visually salient direction, with no inhibition
of visual control (the \continual" model), and with a non-biological model of ideal steer-
ing toward the actual target direction in gaze-based coordinates (the \ideal" model). We
ran swimming and walking simulations with the target 1400 millimeters forward of, and
90
o
to the left and to the right of the initial salamander head position and orientation.
In all but one of these runs, the salamander performed adequately. The success of the
intermittent visual control models (instantaneous and duration) with unmitigated neural
96
delays suggest that compensation for visual system delays is not needed for locomotion
steering.
In the walk to the leftward target, the salamander with continual visual guidance was
distracted by its own body, started pursuing it, and made two full loops to the right
(clockwise) before the target nally dominated the visual salience computation. We ana-
lyzed the gaits during swimming to the forward target as sequences of distances between
head positions at successive rotation reversals. The instantaneous model gave a secondary
head oscillation, which was seen in the return map as clustering. In contrast, the continual
model showed little structure in the successive head rotation distance, suggesting there
may be diculties modulating the gait for stability under this visual guidance model.
With the wide dispersion of the continual model return map of distances between
successive head rotations (gure 3.14), there is little predictability of the gait. On the
other hand, under the head rotation reversal-based models of visual steering direction
update, modulation could stabilize the gait by pushing the locomotion toward the clusters,
where the steering protocol produces a rather stable, predictable gait. When \outlier"
head movement occurs, the gait could be modulated so the head movement will tend
toward regimes where the visual steering is more or less inherently stable.
The simulations presented in this report support the hypothesis that gaze-based coor-
dinates are adequate to control undulatory locomotion. All steering models we tested used
such coordinates. This hypothesis was suggested by the tectum's retinotopic anatomi-
cal organized (Roth, 1987). The use of gaze-based coordinates in the primate superior
97
colliculus (homologous to the amphibian tectum) (Stuphorn et al., 2000) suggests that
gazed-based coordinates in tectal motor control may be evolutionarily conserved.
Werner & Himstedt (1985) reported a tendency for salamander orientation movements
to undershoot the target angle. The curved paths to the left and right targets during
simulations of the intermittent steering (instantaneous and duration) models (g. 3.4)
suggest that that these models also have a tendency to undershoot. In simulations of the
continual model (g. 3.5), in the rightward paths we see examples of both undershoot
and overshoot. The swim to the right shows the widest turn in all of our simulations, but
the walk to the right shows an oversteer with later a compensatory left turn.
The continual model seems not to have a consistent midline bias. It may oversteer
or understeer based on erratic visual phenomena associated with high background mo-
tion. Werner & Himstedt (1985) did not report absolute consistent orienting movement
undershoot; their data showed instances of overshoot.
The midline bias of the visually intermittent models may be sensitive to the gains in
the translation of the visually determined target direction into descending CPG control
signals. We are unaware of data regarding such sensitivity in animals. However, it may
be possible to investigate this sensitivity pharmacologically, e.g., by manipulating the
excitability of the reticulospinal neurons, or perhaps by pharmacologically control tectal
excitability. The basal ganglia may aect this bias through inhibitory projections to the
midbrain and hindbrain (Mar n, 1997; Grillner et al., 2008).
98
Our models of suppression of visual control of motion at times of high voluntary
motion-induced retinal slip can help interpret some of the results of Werner & Himstedt
(1985). Based on head trajectory changes in their \competition" condition, they con-
cluded that the movement was not ballistic. In the competition condition, they changed
the location of the target when the head movement commenced. In some cases, however,
the head trajectory appeared unaected by the change. An interpretation suggested
by our current locmotion control work is that changing the location of the target while
visual control was still enabled would change the head trajectory, making it appear non-
ballistic. Neural integration and smoothing, leading to hysteresis, could explain why the
head movement was not directed toward the later target location. On the other hand,
if the target change occurred when visual control of the movement was suppressed, the
movement would appear ballistic.
The methods of Werner & Himstedt (1985) could help investigate the issues addressed
in the current work. It is likely that those authors' open-loop (in which the visual target
was removed when the head movement started) and competition (in which the target po-
sition changed when the head movement commenced) conditions were implemented with
too little time precision to adequately characterise visual control of movement. Updating
the experimental apparatus might provide some resolution. That is, observing salmander
movements in response to stimuli that are either left in place, or moved or removed with
precise timing relationships to the head movement, could help answer the question of
99
the existence of neural suppression of visual control of movement during periods of high
voluntary movement-induced retinal slip.
Furthermore, these methods could be adapted to directly address questions regard-
ing visual control of locomotion. Visual stimuli could be presented to an animal moving
in a
ow tank or on a treadmill (e.g., Frolich & Biewener (1992)). Observation of the
animal's attempts to turn in response to precisely timed manipulations of those stim-
uli, coordinated with the animals movement, could help characterize visual control of
locomotion.
Final, such an apparatus, combined with the neural recording techniques of Schuelert
& Dicke (2005), could help characterize the neural mechanisms of locomotion steering.
Control of locomotion in a moving head-centered reference frame suggests that neural
steering signals are modulated in coordination with the movement itself. The lack of
such coordinated modulation would suggest inadequacy of the head-centered reference
frame model of locomotion control.
The compensation for perspective distortion in the retinal simulation (g. 3.16(a))
has a midline bias. This explicit bias could aect target selection, but will not aect the
computed direction of a turn toward a single selected target. It may have introduced a
midline bias into the continual model paths, where direction selection was sensitive to
spurious retinal activation at times of high background movement. However, the continual
model paths (g. 3.5) did not show consistent midline bias. We conclude that the artical
100
compensation for articial perspective projection distortion did not substantially aect
our results.
The simulated salamander's distraction during the walk to the leftward target oc-
curred in a minimally distracting environment. It was distracted from the target by the
only other object in the environment, its own body. While live animals probably have
neural mechanisms to avoid this particular distraction, to avoid other distractions in more
natural environments would be a challenge for the salamander's rather limited nervous
system.
We have found that the locomotion induced head movement-induced unpredictable
retinal phenomena (e.g., motion streaks) could behavioral consequences. In our modeling,
we successfully mitigated these eects by the use of visually determined salience at a
xed delay after the direction reversal of the rotational component of head movement {
these two intermittent visual control models perform better at the task of steering to the
target than continual visual updating of the commanded motion direction throughout the
locomotion cycle.
Dealing with self-motion induced visual phenomena is an important task for animal
nervous systems. Activation of CPG-based locomotion through tectal electrostimulation
in lampreys is accompanied by eye movements that appear to compensate for the head
movements of anguilliform locomotion (Saitoh et al., 2007); such eye movements are not
observed in salamanders (Roth, 1987). Toads compensate for self-motion induced back-
ground motion in visual prey localization by closing their eyes while moving toward prey
101
after visually localizing the prey while still; a toad approaching prey may do this more
than once, starting and stopping (Lock & Collett, 1979). If salamanders used a simi-
lar strategy, they might be observed openning and closing their eyes during undulatory
locomotion, with their eyes open only at times of minimal head movement. We are un-
aware of any such observations; indeed, salamander larvae lack eyelids, but still perform
visually-guided anguilliform locomotion (Duellman & Trueb, 1986).
Higher vertebrates may be capable of sophisticated processing of visual motion (Hil-
dreth & Royden, 1998). These vertebrates nevertheless have neural mechanisms sup-
pressing the use of the visual image during periods of high background motion, particu-
larly during saccades (Goldberg & Wurtz, 1972; Diamond et al, 2000). We propose that
salamanders may have a neural mechanism analogous to saccadic suppression in higher
vertebrates, i.e., our instantaneous or duration model.
Sub-millisecond spike timing precision has been demonstrated in isolated salamander
retinas (Berry et al., 1997). To our knowledge, this issue has not been studied in higher
salamander brain regions, such as the tectum. Schuelert & Dicke (2005) recorded the
responses of single tectal neurons in live salamanders exposed to prey-like visual stimuli,
but the results were reported in terms of spike frequency. Their results extended knowl-
edge of the complex structure of tectal receptive elds and salience, but left open the
question of how salience may be encoded by tectal neurons. The simulation results we
report in the present work suggest that precise timing could be very useful for sensory
motor control.
102
Frolich & Biewener (1992) found that during constant speed swimming, the standard
deviation of the maximum lateral displacement was always less than 20% of the mean.
However, their laboratory experiments probably minimized the eect of vision on the
locomotion gait. However, if this regularity is a general feature of salamander locmotion
over relatively long distances, that fact might tend to support the instantaneous model
hypothesis. On the other hand, non-visual brain mechanisms could perhaps modulate the
gait, suppressing some of the irregularity that the steering mechanism induces. However,
as we argued above, the distribution of points in the return map for the continual model
(g. 3.14) suggests that this steering mechanism may be dicult to stabilize. We hope
that development of experimental techniques for recording and stimulating neurons of
active salamanders could answer questions concerning the relationship of sensorimotor
control and the statistics of locomotion kinematics.
Our model requires a neural timer for an interval in the millisecond range. The
32 millisecond delay is well within the range of salamander tectum interneuron latencies
(Roth et al., 1999). On the other hand, this may be handled by a specialized neural circuit
for short-duration timing, suggesting that that the cerebellum may play an essential role
in visual steering of locomotion. In humans, the cerebellum seems necessary for behaviors
that depend on precisely timed short durations. (Spencer et al., 2005; Schlerf et al., 2007).
We did not model the mechanism by which the head movement direction reversals
were detected, but we indicated in gure 3.1 possible involvement of the tegmentum
(Manteuel & Naujoks-Manteuel, 1990) and pretectum (Manteuel, 1989). Pretectal
103
neurons typical have large visual receptive elds (up to 90
o
) and respond to horizontal
movement, often with a preferred direction, and usually responding to speeds less than
10
o
per second (Manteuel, 1989). Our locomotion simulations generally produced much
faster rotational head motion; however, these pretectal neurons might be well suited to
detecting the motion reversal. Further, salamanders sense head orientation with magne-
toception (Phillips, 1986) and non-retinal light polarity detection (Phillips et al., 2001).
We presume that through multimodal sensory integration, the active salamander may be
able to accurately determine when the retinal image may best be used to detect possible
prey moving relative to the background. Furthermore, corollary discharge signals from
spinal cord could be processed with a neural model (in the salamander's brain) of body
responses to those signals, and combined with sensory signals to produce a very accurate
estimate of head movement.
Finally, consider the possibility that intermittent (instantaneous and/or duration) and
continual control may both be used by the salamander. There may be a threshold of head
movement, above which visual control is intermittent, and below which visual control is
continual.
3.6 Conclusion
We have described a model of vision-based steering of salamander locomotion. Sim-
ulations of this model support the hypothesis that salamanders use a head-centered,
vision-based reference frame for locomotion steering. These simulations also support the
104
hypothesis that visual control of locomotion is suppressed at times of high-speed lateral
motion of this frame, i.e., at times of high retinal slip.
3.7 Appendix: Retina model
The retina model is shown is gure 3.16. Here we provides brief description of this model,
rst describing dierences between the current implementation and that described in
chapter 2. Then we provide a detailed description of the current implementation. See
chapter 2 or Begley & Arbib (2007) for a detailed discussion of the retina model and its
properties.
First, there are a some minor dierences in details of the retina model between the
present work and chapter 2. The spatial lter signs have been reversed, implementing
a simple o-channel model. In chapter 2 we noted that the RGC array was more dense
than that of live salamanders; in the present work, we use a sparser array that is closer
to biological reality. The shape of the spatial lter implementing the classical receptive
eld (a dierence of Gaussians) is somewhat dierent in order to preserve retina tiling.
Figure 3.16 shows 2 directionally opponent channels in one dimension, but in this im-
plementation, both vertical and horizontal channels are simulated { in chapter 2 (Begley
& Arbib, 2007), only horizontal direction sensitive channels were used. The four direction
selective channels (up, down, right, and left) are combined in the nal retinal ganglion
cell (RGC) output summation (g. 3.16(m)).
105
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(j)
(k)
(m)
(i)
(l)
OT
(n)
(o)
(p)
Figure 3.16: The correlation-based motion detector model of motion sensitive retinal ganglion
cells (RGCs). This gure shows the processing of two opposing directional sensitivities in a single
dimension; in this work, however, we actually model sensitivity to 4 directions of movement in
two dimensions, horizontal and vertical. (a) 2-D photoreceptor layer, simulated as image pixels
generated with OpenGl's perspective projection; (b) 3 3 convolution mask; (c) rectication;
(d) Biphasic bandpass temporal lter; (e) Sigmoidal transformation, S
0
; (f) Low-pass lter for
temporal correlation; (g) Multiplication (correlation); (h) Subtraction; (i) Adaptive sigmoid, S
1
;
(j) Rectication of opposing motion selective channels (up/down and left/right); (k) 60 60 con-
volution mask { dierence of Gaussians (DoG); (l) Rectication; (m) Sum of directional channels,
actually modeled as 2 vertical and 2 horizontal channels; (n) Perspective distortion horizontal
mitigation; (o) Optic nerve; (p) Optic tectum (gure 3.1). (Adapted from Begley & Arbib (2007).
106
The retina is achromatic. It is modelled as a single, cyclopean unit { bilaterality is
not modelled.
The following is a detailed description of the current implementation.
The rst layer of the retina model is the photoreceptor layer (g. 3.16(a)). see section
3.9 for information on the implementation of this layer.
At each non-boundary pixel we apply a 3 3 convolution mask (g. 3.16(b)) with
a negative center and positive surround. This gives an o polarity, responding strongly
when the center is dark and the periphery is bright. A half-rectier is applied to the
result.
The linear temporal prelter is a biphasic bandpass lter(g. 3.16(d)). This lter
produces transient responses. Our biphasic lter is sine-symmetric with a period of 100
ms. It has a positive early response, and a negative response in the later, refractory phase.
The steady state response of the biphasic prelter to constant input is 0, re
ecting the
view that the eye responds to visual change. The lter shape is given by (1 +) sin(),
where is times the fraction of the half-lter length from the ends of the lter vector.
The lter weights are normalized so that the absolute value of the sums of the weights
in both lter halves are 1. A sigmoidal nonlinearity is applied to the temporal biphasic
lter output (g. 3.16(e)); the sigmoid is given by (1 +exp(24:3239(x 0:18)))
1
,
which was empirically found to be well adapted to the articial visual environment of the
simulations reported here (see appendix 3.9 for more information on that environment).
107
Directional motion sensitivity is modelled with spatio-temporal correlations. For each
direction, a delayed version of the pixel value (g. 3.16(f)) is multiplied (g. 3.16(g))
by the current value of the neighboring pixel in the specied direction. Neighboring
correlations of opposing polarities are subtracted (g. 3.16(h)).
An adaptive sigmoid, symmetric about zero, is applied to the dierences:
tanh

d
i;j

arctanh(0:99)
0:25 max
k;l
(jd
k;l
j)

;
whered
i;j
is the correlation dierence at 2-D pixel location (i;j), and max
k;l
(jd
k;l
j) is the
maximum dierence absolute value at all locations. (See gure 3.16(i).) This value is then
rectied into opposing directions (g. 3.16(j)). At this point, we have a correlation-based
2-D motion detector model (Zanker & Zeil, 2005), with nonlinear preprocessing.
For each directional channel, a 60 60 dierence of Gaussians (DoG) spatial lter is
applied every 6
th
pixel in the horizontal and vertical dimensions (g. 3.16(k)). This DoG
is given by K

exp(
dr
18
) 0:095 exp(
dr
288
)

, where the factor K normalizes so that the
integral of the central excitatory region is 1. The spatially ltered values are rectied (g.
3.16(l)) and summed. Figure 3.16(m) shows two terms for this summation, but there are
actually 4: up, down, left, and right.
A correction factor is applied to the sums, in an eort to partially mitigate perspective
distortion (g. 3.16(n)). In order to mitigate the greater weight near the simulated retina
edges, we multiply RGC activation by a factor that depends on the horizontal distance
from the center of the projection (g. 3.16(n)). We considered it desirable to somewhat
108
reduce the simulated salamander's tendency to be distracted by visual features far from
the current direction of motion; a salience compution bias in favor of features in the
direction toward which the salamander is currently moving does not seem to us to be a
problem. Thus, the factor is 1 in the nasal portion between the two projection centers,
and less than 1 in the temporal portions { thus, the factor depends on the horizontal
distance from straight ahead. On the temporal side of the two centers, the factor is given
by cos

45
ocenter to point distance
center to edge distance

. Thus, the minimum factor, at the temporal edge of the
RGC grid, is cos(45
o
). This underestimates the distortion; the eld of view of each eye
is more than 45
o
, and the proportional size dierence of features at the center and at the
edge is greater than the correction applied, but further corrections for this phenomenon
would not have aected our results.
This represents the RGC output (g. 3.16(o)), which is transmitted via the optic
nerve to the brain, particularly to the optic tectum (g. 3.16(p)). See section 3.2.2 for
CNS model details.
3.8 Appendix: Brainstem model
The model of sensorimotor integration and control in the central nervous system (CNS)
is shown in gure 3.1. This model receives visual input from the retina (appendix 3.7),
and sends output signals to the spinal CPGs ((Ijspeert, 2001)).
109
Formally, we rst sum each of the columns of the simulated retinal ganglion cell (RGC)
matrix. Then, a low pass lter is applied to this vector.
V
i
=
X
j2retina rows
RGC
i;j
^
V
i
=
ss
neur

OT1

V
i

^
V
i

wheress
neur
is the neural simulation step size, 0.5 milliseconds, and
OT1
is the presumed
time constant of the summing neural element, 7 milliseconds.
We then apply a scale factor and a softmax global nonlinearity. Rather than provide
a presumed neural architecture for the winner-take-all (WTA) network, we model it
with the abstract softmax WTA implementation (Yuille & Geiger, 2003), in which the
activation of the maximum is enhanced with exponential scaling. The neural details are
irrelevant to the focus of this chapter; softmax is computationally convenient. Thus:
S
i
=exp

sf
softmax

(
^
V
i
min(
^
V
i
))
max(
^
V
i
)min(
^
V
i
)
1
!!
wheresf
softmax
= 7 is a unitless scale factor, andS
i
is the retinal salience in the direction,
relative to the head, preferred by theith salience vector unit. I.e.,S
i
is thus the activity
level of the ith salience vector neural element. The sum of S
i
over all i is 1.
The preferred direction of movement is computed as a weighted sum of the salience
vector neural activity levels. The weights represent the strength of the synapses from
the salience-encoding units to the summing neuron. Our weights were determined by
110
0 10 20 30 40 50 60 70 80 90
−100
−50
0
50
100
vector index
degrees
Figure 3.17: The salience direction vector. This shows the preferred direction, in degrees from
directly forward in head-centered coordinates, of each neuron in the salience vector. A dotted
straight line demonstrates the deviation of the perspective projection from the linear preferred
direction scheme that projection onto a spherical retina would provide.
111
the perspective projection (see gure 3.17). We initialize this constant weight vector
by rst computing the weight of each column in the photoreceptor array. There are 4
symmetrical portions of the two adjacent non-overlapping retinas. Directions to the left
are represented by negative angles, while directions to the right are encoded positively.
The centers of the retinas are given byFOV , where FOV is the OpenGL
r
eld of
view (OpenGL Architecture Review Board, 2000), 57:8
o
. Since the photoreceptor width
of each retina, 300, is divisible by 4, no column actually occupies the center. For each
column, the angular distance from its projection center is given by:

i
= arcsin

i 0:5
half retina width
sinFOV

;
where i2 (1:::half retina width), and half retina width = 150. The preferred direc-
tion of each photoreceptor column is given by
=FOV
i
;
and is a 600 element vector. The preferred direction of the salience vector neurons
is extracted from this, leaving room at the edges for the convolutions and correlation
operations of the retina model:

i
= (6i +radius
DoG
+ 2):
112
The parentheses enclose the vector array index; radius
DoG
= 30, half the width of
the retinal DoG classical receptive eld lter; and i2 (1::: 90). The resulting vector is
plotted as a continuous line in gure 3.17.
The salience-averaged direction is computed as the dot product of the salience vector
and . Thus, the 2-dimensional retinal images have been reduced to a scalar value
for motor control. The simulation program computes this dot product at every neural
simulation step. For locomotion control, the use of this direction may depend on the
timing of the undulatory locomotion cycle.
In chapter 2 (Begley & Arbib, 2007), we found that object motion detection is very
sensitive to background motion motion eects. When the salamander's locomotion in-
volves oscillatory head movement, background motion will be minimized when the head
movement reverses direction. This suggests that important retinal information is avail-
able at these times, after a neural processing delay. We simulate an idealized model of the
tecta's self-motion estimation. It has been suggested that the pretectum and tegmentum
provide self-motion information in their inputs to the tectum (Manteuel & Naujoks-
Manteuel, 1990; Woesler & Roth, 2003). We model this as a perfect estimate of head
orientation. This model knows, with no delay, the head orientation at every simulation
step. The times in the undulatory locomotion cycle when the rotational component of
head movement changes direction are considered times of relative retinal image stability,
with minimal artifacts of self-motion.
113
In order to investigate the eects of sensorimotor neural precision on steering CPG-
based locomotion, we simulated two dierent models of updating the desired direction of
locomotion a xed delay after the reversal of the rotational component of head movement.
In one model, the desired heading is updated to the visually computed value at an instant
32 milliseconds after the reversal. In the other model, the desired heading is under visual
control for a 16 millisecond duration beginning 24 millseconds after the reversal.
We compare these two models with a model of continual visual control of the CPG
drive signals. In this model, the drive signals are never directly coupled with the salaman-
der's actual movement. In what follows, we refer to these models as the instantaneous
model, the duration model, and the continual model.
We model the push-pull hypothesis of tectal modulation of of recticulospinal neurons
as
push
0
r;l
=
ss
neur

push pull

dirpush
0
r;l

push
0
r;l
=max(min(push
0
r;l
; 0:5); 0:0):
wheredir is the desired turn direction in degrees divided by 180
o
, giving a neural activa-
tion level, which is constrained to the range (0::: 0:5).
This value,push
0
r;l
, is used to control the neck muscles, i.e., those moving the head at
the joint to the rst spinal segment.
114
Higher activation levels produce swimming (Cabelguen et al., 2003). In these simula-
tions, the following enhancement of the descending tonic input to the spinal CPGs was
used for the aquatic gait:
if swimming then push
r;l
=max(min(3push
0
r;l
; 1); 0:0)
else stepping then push
r;l
=push
0
r;l
:
Thus, the activation level for the terrestrial gait is constrained to be less than 0.5, while
that for the aquatic gait can reach as high as 1.0.
3
These push/pull signals from the tectum are used to control locomotion via tonic
input to spinal units. The tonic input (from the reticulospinal population) is given by:
tonic
r;l
=K
neck;body
(1 +push
r;l
push
l;r
)
where pull
r;l
= push
l;r
, i.e., rightward push activation equals leftward pull activation,
and vise versa. K
body
= 1:5 andK
neck
= 0:5 are scale factors for tonic input to the body
CPGs and to directly controlled neck muscles. push
r;l
= push
0
r;l
for the neck muscles.
See Ijspeert (2001) for a detailed description of how tonic input from the brainstem to
spinal central pattern generators (CPGs) is used to control locomotion.
3
This is consistent with Cabelguen et al. (2003), where electostimulation of the mesencephalic loco-
motor region (MLR) with 0.5{3.5 A produced a stepping gait, while 1.8{5.5 A produced swimming.
115
3.9 Appendix: Animated graphical simulation system
The animation simulation system was originally developed by Ijspeert for his work on the
spinal CPG simulation(Ijspeert, 2001; Ijspeert & Arbib, 2000a; Ijspeert, 2000). The basic
elements of the system are: a salamander with realistic swimming and trotting gaits, and
a target that the salamander pusues.
In the current work, the target is a black truncated cone which simulates prey. The
cone segment is 15 millimeters long along the axis, with disks at the ends of radii 8 and
3 millimeters { the cone's greatest dimensions are slightly greater than the radius of the
salamander's head. The axis is parallel to the ground, and perpendicular to the line
connecting the target and the salamanders head at the start of each simulation. The
target moves back and forth along the axis line at 2 Hz. It moves at 22 millimeters per
second.
In Schuelert & Dicke (2005), salamander tectal neurons were found most responsive to
black geometric shapes. These neurons were also highly responsive to cricket dummies. In
Sch ulert & Dicke (2002), salamanders were found to be behaviorally responsive to these
stimuli, as determined by the salamanders' orienting movements. Our target is intended
to be a 3-dimensional analog of their large rectangle, deformed to be somewhat closer to
the cricket shape.
The salamander pursues this prey on a checkerboard, with light and dark beige squares
with 100 millimeters sides. which provides background texture. This is not a simulation
of the salamander in a natural environment, but the articial visual environment may not
116
be much more impoverished than some controlled laboratory experimental environments.
Color of the salamander's body was chosen to minimize contrast, in the achromatic visual
system, with the background, but with good visibility in the color displays.
The photoreceptors are simulated with two OpenGL
r
perspective projections (OpenGL
Architecture Review Board, 2000). To simplify the alignment, the origins of the two pro-
jections are at the center of salamander's head. The projections are at angles of 57:8
o
from straight ahead, with a nominal eld of view angle of 74:75
o
. Each retina has a
photoreceptor layer width of 300 and a height of 144 pixels. This generates two adja-
cent, non-overlapping images, which are combined into a single photoreceptor layer. For
all subsequent operations, the retina is modelled as a single, cyclopean unit. The three
channels (Red, Green, and Blue) are averaged; all subsequent operations are achromatic.
The visual elds of salamanders' eyes may be very wide { up to nearly 180
o
(Roth,
1987) per eye. In this model, the nominal width is 115:6
o
. This is a wide angle, with con-
siderable distortion associated with the perspective projection onto a
at retina { a live
salamander has the advantage of a curved retina, and some accomodation through lens
focus. Perspective distortion may be viewed as a nonlinear spatial lter { an additional
nonlinearity in a retinal model that already contains nonlinear transformations at several
steps (x3.7 and gure 3.16). The distortion is particularly evident where the visual elds
of the two eyes meet { shapes are continuous across this threshold, but there is obvious
distortion from two dierent projections on either side of the threshold. In order to miti-
gate distortion characteristic of this projection onto a planar \retina", a correction factor
117
was applied at the nal retinal ganglion cell (RGC) output (g. 3.16(n)). Furthermore,
we adapted sigmoidal nonlinearity (g. 3.16(e)) parameters to the visual properties of
the articial environment of our simulation model.
The user controls several features of the simulation by means of either a pop-up
menu or single-key keyboard commands. One of the most important controls is the
water level: the salamander swims or trots based on the water depth. Highly simplied
visual properties of the water are simulated by using OpenGL
r
color blending. Optical
properties such as refraction are not simulated { in the simulations reported in this
chapter, either all objects in the simulation are completely out of the water, or else all
are completely submerged.
The simulation display consists of several windows (gure 3.3). One is an aerial view of
the salamander world (g. 3.3a). Another (g. 3.3b) is the view through the salamander's
eyes { This window's pixel map is the photoreceptor layer (g. 3.16a) , as explained
above. Another window (g. 3.3c) displays the achromatic activity of the spatially and
temporally preltered image (g. 3.16f). Yet another window (g. 3.3d) displays the
sum of the motion-sensitivity correlations (g. 3.16j). The directions are color coded:
the blue channel at each pixel is given by the sum of the horizontal activations, while the
red channel is the sum of the vertical channels. Still another window (g. 3.3e) displays
RGC activity (g. 3.16o). Finally, the direction command vector (i.e., the 2-D salience
vector) is shown in a window as a bar graph (g. 3.3f).
118
The system is developed and runs on Intel-based PCs running the Linux operating
system. It uses OpenGL
r
graphics, including the GLUT user interface utility library.
The physics models were hand coded in C, based on Ekeberg (1993).
119
Chapter 4
Modeling Salamander Numerical Preference
4.0 Chapter abstract
Uller et al. (2003) reported that salamanders tend to prefer the greater number of visually
attractive stimuli when that number is less than 4. We propose that this preference may
emerge when behavior is guided by a simple visual salience computation. We test this
proposal with a neuromechanical model of visual control of salamander locomotion. Our
results suggest that salamanders are only indirectly sensitive to the cardinality of visual
stimuli. Our model is sensitive to apparent stimulus surface area, and to stimulus motion
{ we abolish numerical discrimination with simple methods of equalizing stimuli in these
measures. We further nd that our model's numerical discrimination varies according to
the spatial distribution of the stimulus clusters.
120
4.1 Introduction
Uller et al. (2003) report on experiments in which salamanders were shown to prefer, under
the experiment conditions, the larger of two quantities of attractive visual stimuli when
the numbers of the stimulus items are less than 4. These results have been interpreted
as indicating that salamanders prefer the greater of two small integral numbers of prey
items (Ansari, 2008).
In these experiments, a salamander was held in a small chamber at the center of the
long side of a 227 72:5 millimeter chamber. On opposite sides (from the salamander's
viewpoint) of the larger chamber, two 45 mm. long by 5 mm. diameter tubes containing
live fruit
ies were placed, so that the salamander could see the length of both tubes.
With this setup, we infer that the salamander could see both tubes simultaneously from
the holding chamber. After a waiting period, the door from the small chamber to the
larger chamber was opened. The salamander's choice between the two groups of
ies was
recorded as the rst tube that the salamander touched with its snout.
The methods of Uller et al. (2003) were inspired by 2-alternative forced choice experi-
ments on primates. The experimental apparatus was designed so that, in reaching a tube
of
ies, the salamander had made a clear choice of one tube, and a clear rejection of the
other. When walking in the chosen direction, the \rejected" tube of
ies will be in the
far periphery of the salamander's eld of view, or perhaps not visible to the salamander
at all. The experimental apparatus allows the experimenter a clear interpretation of the
salamander's behavior as a preference for the group of
ies enclosed in the tube rst
121
touched by the salamander's snout. However, since the conguration of the stimuli, the
salamanders' initial distance to the
ies, and their orientation with respect to the
ies,
did not change throughout the experiments, the results do not rule out mechanisms of
visual behavior that could produce dierent results with conguration changes.
Agrillo et al. (2007) found comparable numerical discrimination in the behavioral
preferences of sh for shoals of conspecics. Agrillo et al. (2008) further investigated
the quantity discrimination ability of sh by controlling for continuous quantities that
might aect the apparent preference, specically, the speed and the size of the stimuli.
These experiments exploited, and depended upon, the preference of the subjects for larger
groups of conspecics. The continuous quantities investigated were motion and size. The
subjects' preference for the more numerous of two small shoals of sh vanished when the
experimenters equalized the apparent surface area of the projection of the shoals as seen
by the test sh.
The results were not as conclusive when the experimenters controlled for the eects
of the motion of the two shoals. Water temperature was used to change the motion of the
stimulus sh. The motion of the two stimulus shoals in each experiment was equalized,
so that number of grid line crossings was the same for both the larger and smaller group
in each trial. In a 4 vs. 8 comparison, the behavioral preferrence vanished with motion
equalization, while for 2 vs, 3, the preference for the larger group remained.
These sh experiments attempted to predict and interpret behavior in terms of a
simple model: the preference of the subject sh for larger groups of conspecics. However,
122
conspecic preferences could be rather complex. Even if simple quantity preferences are
fundamental motivations for these animals, these preference may be subject to modulation
by a variety of factors that aect conspecic relations.
Nevertheless, the methodology suggests an analogy with salamander feeding behavior.
In addition to their preference for the more numerous of two small groups of prey animals,
their preferences among prey-like visual stimuli have been reported to be sensitive to
stimulus size and motion (Sch ulert & Dicke, 2002; Schuelert & Dicke, 2005). Withing
limits of foodlike object sizes, salamanders have shown a preference for larger visual
stimuli. The preference for larger stimuli is limited; a very large apparent stimulus size
elicits predator-avoidance behavior in frogs and toads (Roche King & Comer, 1996; Liaw
& Arbib, 1993; Cobas & Arbib, 1992). Uller et al. (2003) suggested experiments to
test salamanders' preference for stimulus speed by implementing something like their 2-
alternative choice protocol on a computer display. Thus, we have the elements to check
for possible confounding of salamanders' apparent quantity preferences by systematically
altering stimulus size and speed, in a manner analogous to Agrillo et al. (2008).
Salamanders' discrimination between small numerosities inspires a search for possible
neural mechanisms that may explain the ability. Such a mechanism must explain not
only the successful discriminations, but also the apparent failures to discriminate. Fur-
thermore, such a mechanism must operate within the constraints imposed by the limited
neural resoures of the salamander brain (Roth et al., 1995).
123
Dehaene & Changeux (1993) modeled nonlinquistic numeric processing in human in-
fants. The numeric discrimination ability of the model was based on one-to-one mapping
between groups. The training phase of the model used elementary addition and sub-
traction. We are unaware of evidence that would support the existence of such neural
processes in amphibians.
Domijan (2004) reproduced certain features of human rapid numerical estimation with
a model of circuitry proposed to exist in cerebral cortex. It may be that salamander nu-
merical sensitivity arises from similar circuitry. However, the salamander brain has very
few neurons to apply to this task (Roth et al., 1995). We propose an alternative expla-
nation: that salamanders' apparent numerical sensitivity emerges from the elementary
properties of the early visual system.
In chapter 2 (Begley & Arbib, 2007) we demonstrated the possibility of remarkable
sensitivity of certain salamander retinal ganglion cells (RGCs) to small object movements
at times of low retinal slip in the oscillatory head/eye movement cycle of undulatory loco-
motion. In this model, a linear/nonlinear outer retina model was input to a correlation-
based model of inner retina motion sensitivity, which was input to a linear/nonlinear
RGC model with classical center/surround opponency. We showed that the outer retina
nonlinearity had a large in
uence on the RGC sensitivity to small object movement at
times of relative image stability (i.e., low retinal slip). These RGCs may constitute the
only cell type that tiles the salamander retina (Segev et al., 2006).
124
The hypothesis that RGCs are very sensitive to small object movement at times of
low retinal slip suggest that it may be useful for the animal's brain to enable certain
visual processing functions at times of very low global image motion, and disable such
functions when the entire retinal image is moving rapidly. Since the salamander does not
use eye movement to compensate for oscillatory head movement during undulatory loco-
motion (Roth, 1987), motion-sensitive RGC signals may be most useful at the extremes
of this head motion, when the head motion changes direction. In chapter 3 we tested
this hypothesis by integrating our retina model with the spinal CPG-based locomotion
model of Ijspeert (2001). The spinal cord was modeled as bilateral interconnected neural
oscillators. The tectum was modeled as the brain region at which the visuomotor integra-
tion occurs (Saitoh et al., 2007). We showed that intermittent visual locomotion control,
enabling locomotion control when the retinal image is relatively stable, and disabling this
control during periods of high retinal slip, produced more reliable visual guidance, and a
more regular gait, compared with continual visual control.
In the present work, we propose that a simple parameter modication of that sim-
ulation model suces to reproduce the results of Uller et al. (2003). This parameter
modication enhances the strength of the RGC receptive eld (RF) inhibitory surround,
relative to the central excitatory receptive eld (ERF). To see how this property could
aect numerical discrimination, consider that the retinal projection of the tubes of
ies
would be within the ERF of a small handful of RGCs (but certainly more than one
RCG). If surround inhibition is weak, then adding
ies to these tubes would tend to
125
increase RGC activity. This RGC activity could be used to discriminate the numerosities
of rather large groups of
ies. However, if surround inhibition is strong, additional
ies
will increase the activities of the RGCs having ERFs covering the
ies' retinal images,
but each
y's retinal image will also inhibit the activity of other RGCs. If the inhibitory
surround is strong enough and spatially extensive enough, the addition of
ies beyond
a small number will not increase the total RGC activity { in this case, this activity can
be used to discriminate, on the average, only small numerosities. This is the hypothesis
tested in the present work. If the results of Uller et al. (2003) can be reproduced by our
model, then the hypothesis oers a parsimonious explanation of those results, using a
simple model of visual processing in the salamander brain. That model has already been
investigated in the context of locomotion steering. No qualitative change to the model is
proposed, only a simple parameter adjustment of the shape of a visual RF.
Inx4.2, we brie
y describe the model of chapter 3 (to which the reader is referred
for further details). Inx4.3, we describe how we have adapted this model to investigate
salamander numerical preferences. Inx4.4, we show how the model reproduced the re-
sults of Uller et al. (2003); we also show how some of the results of Agrillo et al. (2008)
apply to our model; and we show that the results are sensitive to the shape of the retinal
ganglion cell (RGC) classical receptive eld. Finally, inx4.5, we conclude that salaman-
ders probably lack specic neural computations and representations related numerical
cognition.
126
4.2 Model description
This section summarizes and provides motivation for our model of salamander visuomotor
control. See chapter 3 for a more thorough discussion.
Figure 3.1 shows our central nervous system (CNS) model. Our salamander simula-
tions were developed in the course of modeling visual control of undulatory CPG-based
locomotion. Visual target selection is based on salience, a combination of features of
the scene that cause neural activity in topographically organized, visually-driven neurons
{ tectal neurons in this model. Winner-take-all enhancement of the ring rate of neu-
rons associated with the most salient location results in locomotion steering toward the
stimulus that caused the neural activity. This steering is relative to a head/eye centered
reference frame; there is no neural representation of an overall direction of locomotion.
Furthermore, visual control of locomotion is active only at an instant after the reversal of
the head motion associated with undulatory locomotion; vision is suppressed at times of
high self-motion induced retinal image slip { this image slip can produce erratic steering,
and can cause relatively erratic gaits. Chapter 3 describes and motivates this mechanism
more fully. For our purpose here, we are interested in the property that the model steers
toward visually salient locations.
Much of the visual salience computation is performed by our model of motion-sensitive
retinal ganglion cells (RGCs). The model is fully described in chapter 2 (Begley & Arbib,
2007), and is shown in gure 3.16. Appendix 3.7 gives a brief formal description of the
model, including the parameters used in the particular implementation of the current
127
work. These parameters adapt the system to visual of the articial world of our graphical
animation simulations.
A key parameter dierence between the current retina model implementation and
that of chapter 2 (Begley & Arbib, 2007) is the dierence-of-Gaussians spatial lter, g.
3.16(i). We have found that varying surround inhibition can in
uence the experiment
results. Stronger surround than that used in chapters 2 and 3 produces results like those
of the behavioral experiments of Uller et al. (2003). This is discussed further in subsequent
sections.
A heuristic demonstration of the dynamics of visual control of locomotion is shown in
gure 4.1. The direction-weighted salience vector sum is scaled in approximate degrees.
This is treated in the model as an estimate of the bearing to the most visually salient
object, in this case, the salient group of
ies. The short vertical spikes indicate the time
of the reversal of the rotational component of head movement. After a xed delay (32
milliseconds), the preferred direction of movement is updated according to the direction-
weighted salience sum { these times are indicated in gure 4.1 with long vertical spikes.
In this run, the salamander went to the right from its starting position, to the larger
group of
ies. Locomotion starts at 400 milliseconds, before the time period covered by
gure 4.1 (seex4.3, following, for simulation protocol details).
The two large negative dips in the salience sum indicate the brief dominance of the
salience computation by the smaller group of
ies to the left. Since the motion direction
update times (the long vertical spikes) do not coincide with the brief salience of the
128
600 650 700 750 800 850 900 950 1000 1050
−80
−60
−40
−20
0
20
40
60
80
simulation time (milliseconds)
direction weighted salience (degrees)
Figure 4.1: The weighted sum, in degrees, of the salience vector, the basis of direction of motion
commands to the CPGs. This heuristic quantity shows the time course of the motion direction
command. The short spike from zero indicates the reversal of the rotational component of head
movement. The long spike from the bottom of the graph indicates the time that the CPG control
signals are updated, a xed delay after the reversal. In this run, the salamander goes to the right,
toward the greater number of
ies. Motion begins at 400 ms., before the period covered by this
gure. The large negative salience dips before 700 ms indicate brief salience of the smaller group
of
ies to the left. After this, the the rightward group of
ies is salient. The relatively stable
salience sums re
ect the bearing to the
ies relative to the salamander's head at time of relative
retinal image stability, the \xations" indicated by the short vertical spikes.
129
a)
b)
c)
d)
e)
f)
Figure 4.2: Snapshot of the real-time simulation display showing the transition of salience from
the leftward group of
ies to the rightward. Due to neural delays, this snapshot is somewhat
earlier than the early large dip in the direction-weighted salience sum of gure 4.1. a) Top view
of the salamander in its environment; b) Photoreceptor input to the retina model (g. 3.16a)
{ the simulation display shows full RGB color, but the model sums the channels to yield an
achromatic visual system; c) Retina prelter (g. 3.16e) { the background is ltered out, so the
most active units correspond to the moving, high-contrast
ies { their projection is very small; d)
Motion sensitive retinal units { in the actual simulation display, horizontal activity is shown in
red, vertical in blue. Again, the most active units are in very small regions corresponding to the
retinal projections of the
ies. e) Retinal ganglion cell (RGC) activity (g. 3.16o)i { the bright
blob at the position of the 3-
y group on the right indicates that the salience is transitioning
to this group; f) Bar graph of one-dimensional salience map activity { there are spikes at the
positions of both virtual
y tubes.
130
leftward group of
ies, the salamander does not, in this run, ever alter its course. This
situation is shown in gure 4.2, which introduces the real-time simulation simulation
display. Window f) shows the salience vector as a histogram. The salience is shown in
transition from the leftward group of 2
ies to the rightward group of 3.
After 700 milliseconds, the salience computation in gure 4.1 is dominated by the
larger group of
ies. The relatively stable values re
ect the bearing to the
ies, relative
to the salamander's head, at the time of relative retinal image stability { the \visual
xation" times indicated by the short vertical spikes of gure 4.1. The relative stability
of this salience is largely due to modeling neural elements as low pass lters (gure 3.1).
The simulated retinal image shows motion artifacts due to the salamander's locomotion-
induced head movement, but this sporadic neural activity has little eect on the ltered
salience.
This situation is shown in gure 4.3. The less numerous group of
ies is outside of
the eld of view, so will not aect the salience computation as long as the salamander
continues its trajectory toward the more numerous group. Figure 4.3 also shows the
eect of neural delays. In this gure, the image projected onto the photoreceptor layer
(top right) has, due to locomotion-induced eye movement, moved subtantially rightward,
compared with the delayed activity of the relatively stable later visual layers.
For a more detailed discussion of the model's dynamics, please see chapter 3.
131
Figure 4.3: Snapshot of the real-time simulation display showing the salamander approaching
the virtual tube with the greater number of
ies. This snapshot near a long vertical spike in gure
4.1 { the preferred direction of movement has just been updated to the direction-weighted sum
of the salience vector at a time of relatively high retinal image quality. The
ys' image position
oset between the photoreceptor layer (right top) and the prelter (bipolar cell) layer (right 2
nd
from top) is due to the eect of neural delays coupled with undulatory locomotion which moves
the head, and therefore the eyes, and thus the photoreceptor image.
132
4.3 Methods
The initial conditions of the simulation experiments are demonstrated in gure 4.4. The
visual stimuli of Uller et al. (2003) consisted of 2 narrow tubes of fruit
ies, one to the
salamander's left and the other to the right. We simulate these stimuli as 4 spheres mov-
ing in 1 dimension as if constrained by a tube perpendicular to the line of sight from the
salamander's initial position. The spheres' 4 millimeter diameter gives an angular dimen-
sion of less than 1
o
from the salamander's initial position. Because our 250 mm. long
simulated salamander (Ijspeert, 2001) is larger than the salamanders in the experiments
of Uller et al. (2003), we place the stimuli at a greater distance from the salamander's
starting position { the tubes are 237.5 millimeters from the starting position of the sala-
mander's head, at angles of 57:43
o
from straight ahead. The virtual tubes constraining
the
ies movement are nominally 45 millimeters long, giving an initial apparent angle of
nearly 11
o
. The salamander has a clear choice: to turn left or right. Early in the run,
both virtual tubes are in the salamander's view. When the salamander turns toward
one tube, the other leaves the eld of view { after this happens, the salamander is very
unlikely to change its choice of tube. Occasionally, the salamander oscillates between the
two tubes for some time, resulting in straight ahead movement as the left and right turns
cancel each other. Eventually, the salamander always approaches close enough to a tube
to record a choice and end the simulation run.
133
Figure 4.4: Snapshot of the simulation display just after the start of a demonstration run.
This represents the situation at simulation time of 79 milliseconds. Unlike actual simulation
experiments, the
ies in this demonstration did not move. The left virtual tube has 6
ies,
the right 4. The
ies are evenly spaced throughout the entire length of the \tube", showing
the full extent within which the
ies' motion is constrained. During actual experiement runs,
the
ies are distributed at random within their respective tubes, they move at a constant 10
millimeters/second, and they reverse direction randomly.
134
The stimulus spheres (\
ies") nominally move at 10 millimeters per second
1
. Their
initial positions within their virtual tubes, and their initial directions of movement, are se-
lected at random. They reverse direction during the simulation when they reach the ends
of their tubes. They also reverse direction at random, with direction changes simulated
as Poisson events with a mean of 2 changes per second.
The salamander's direction of motion is determined by the salience map activity at
an instant in time. For these experimental simulation runs, the salamander is motionless
for the rst 400 milliseconds. At 300 milliseconds, the salience vector is rst translated
into movement direction-commanding neural signals to the spinal CPGs. This is eective
when locomotion commences after a 99 millisecond delay. Subsequently, the direction is
updated at times of likely retinal image stability during locomotion, with neural delays
as explained in section 4.2.
The simulation stops when the salamander's head comes within 80 centimeters of
a virtual tube. This is because we do not simulate visual behavior directed at nearby
foodlike objects. When the salamander's eyes are near a
y, the extent of the projection
of the
y onto the retina is greater than the extent of an RGC ERF. Strong inhibition
reduces the retinal activity induced by that
y; more distant
ies will then elicit greater
RGC activity. The simulated salamander is likely to be distracted and change its target
preference. Because we do not model the visual processing that guides orienting toward
1
This speed is the slower of the two articial stimuli speeds suggested in Uller et al. (2003) to test
the possibility that salamanders select the group of
ies showing greater movement. The faster speed (50
mm/s) seemed implausible for
ies constrained to a short, narrow tube.
135
nearby objects, strong surround inhibition results in a preference for more distant
ies,
which may lead to instability near a target.
4.4 Results
A number of experiments were carried out. In each experiment, groups of 11 runs were
generated by successively incrementing by 1 from an initial random number generator
seed. For each experiment, a cumulative binomial
2
was computed to test the hypothesis
that the simulated salamander does not prefer one group over the other, i.e., that the
probability of selecting each group is 0.5. We reject the null hypothesis when P is less
than 0.05
3
The experiment results are summarized in tables 4.1{4.4.
Numbers More Less P Prefer
of
ies more?
3 & 2 29 4 5:46 10
6
YES
6 & 4 20 13 0.148 no
4 & 3 12 10 0.416 no
Table 4.1: Key results of Uller et al. (2003) reproduced in simulation. The strong surround
inhibition DoG was used in the RGC classical RF model, the length of the
y tube was
45 mm, the radius of the spheres used to simulate
ies was 2 mm, and the
ies moved at
a constant speed of 10mm=s. P is the cumulative binomial probability of the observed
number or less of \LESS" results, assuming equal probabilities of both outcomes. We
reject the hypothesis of equal probabilities when P <, where = 0:05.
2
In the binomial test, we compute the probability of the given number of \less" selections, plus the
probability of all lower numbers of these selections, given the null hypothesis of equal probability that the
salamander selects each virtual tube. This cumulative probability is called P.
3
As customary in biological research, Uller et al. (2003) used a threshold of 0.05. Caution in interpreting
the results of such experiments can be recommended. If we allow the possibility that salamanders might
prefer less, as well as more, then note that, in 15 experiments in which the null hypothesis (no preference)
is in fact true, the probability of incorrectly rejecting the null hypothesis at least once is 0.79.
136
One condition varied in the experiments was the number of
ies on each side. Uller
et al. (2003) demonstrated that their salamanders showed a statistical preference for 3

ies over 2. No preference was observed in a choice between 6 and 4
ies; likewise,
no preference was observed in a choice between 4 and 3
ies. Our basic experiment
reproduced these results. See table 4.1.
ERF size Numbers More Less P Prefer
of
ies more?
nominal 3 & 2 28 5 3:31 10
5
YES
nominal 4 & 3 26 7 0.000659 YES
nominal 6 & 4 29 4 5:46 10
6
YES
wide 3 & 2 29 4 5:46 10
6
YES
wide 4 & 3 26 7 0.000659 YES
wide 6 & 4 31 2 7 10
8
YES
Table 4.2: Weak surround inhibition improves model's numeric discrimination. Both
ERF radii produced the same basic result: 6
ies was preferred over 4, in contrast to the
results of Uller et al. (2003). Cf. table 4.1. The length of the
y tube was 45 mm, the
radius of the spheres used to simulate
ies was 2 mm, and the
ies moved at a constant
speed of 10mm=s. See table 4.1 for statistic explanation.
Another condition varied in the experiments was the relative strength of simulated
RGC surround inhibition compared with the strength of the excitatory center. Compared
with the dierence of Gaussians (DoG) spatial lters used in chapters 2 (Begley & Ar-
bib, 2007) and 3, a relatively strong inhibitory surround was required to reproduce the
salamander numerical preference pattern of Uller et al. (2003). Additionally, experiments
were run with two dierent \weak surround" DoGs { see gure 4.5. In the rst, the
diameter of the excitatory receptive eld (ERF), measured at the zero crossing, was the
same as the ERF diameter of the strong surround DoG { we call this the nominal ERF
137
size. Maintaining the ERF diameter while weakening the surround reduces the excitatory
strength in the region of overlap of the ERFs of adjacent RGCs. In order to eliminate
possible eects of gaps thus induced in the RGC tiling, we tested a second weak surround
DoG, one with a wider ERF, and somewhat greater overlap of adjacent RGC ERFs than
with the strong surround DoG.
The DoG formulas are:
strong surround exp(
d
r
18:0
) 0:25 exp(
d
r
288:0
)
weak surround nominal ERF radius exp(
d
r
8:0
) 0:04 exp(
d
r
288:0
)
weak surround wide ERF exp(
d
r
33:62
) 0:158 exp(
d
r
288:0
);
where d
r
is the distance, in pixels (i.e., photoreceptor receptive eld width units), from
the 2-D point to the receptive eld center.
With both weak surround DoGs, the result was improved numerical discrimination,
beyond salamanders' experimentally demonstrated ability. Thus, the model's apparent
integer discrimination ability is sensitive to fundamental low level vision parameters,
specically, the shape of the classical receptive eld. See table 4.2.
A harmless anomaly was occasionally observed during simulations using the strong
inhibition DoG. Nonintuitive RGC activity sometimes occurred during times of high
head movement induced background motion. This activity occurred in retinal locations
corresponding to the featureless sky, while retinal locations corresponding to contrastive
visual features, such as the
ies and/or the checkerboard ground pattern, were silent.
138
−30 −20 −10 0 10 20 30
−0.01
0
0.01
0.02
0.03
0.04
0.05
pixel widths
−30 −20 −10 0 10 20 30 40
−0.01
−0.005
0
0.005
0.01
0.015
0.02
0.025
0.03
0.035
pixel widths
−30 −20 −10 0 10 20 30 40
−0.005
0
0.005
0.01
0.015
0.02
0.025
0.03
0.035
0.04
0.045
pixel widths
−30 −20 −10 0 10 20 30 40
−2
0
2
4
6
8
10
12
14
x 10
−3
pixel widths
Figure 4.5: We have run simulations with three dierent dierence of Gaussian (DoG) spatial
RGC lters modeling the classical receptive eld (RF): one with strong surround inhibition (bot-
tom left), and two with relative weak inhibition: bottom center, a narrow excitatory RF (ERF)
with diameter (measured from zero crossings) near that of the strong inhibition case, and a wide
ERF (bottom right). Top, the three DoGs are displayed together. The y-axis shows the unitless
weight of connection at each distance from the center. Each lter is normalized so that the total
ERF weight is 1. The total inhibitory strength of the surround in each of the weak inhibition
cases is 1:45 the total center excitatory strength, while in the strong inhibition case, the total
surround weight is 7:72 the total center weight. The bottom graphs show the RF overlaps of
diagonally adjacent RGCs, showing how we model RGC tiling of the retina. The wide ERF shows
the greatest overlap, while the narrow ERF with weak surround inhibition shows the least overlap.
139
This was harmless because it did not occur at times of visual updates to the commanded
direction of motion { non-intuitive visual system activity was never observed at update
times. Motion-sensitive RGCs were most active at locations corresponding to
ies at
such update times. This anomaly has never been observed with relatively weak RGC
surround. We think that this may be a numerical artifact of the convolution computation.
We emphasize that the results were not aected.
Stimuli Stimuli More Less P Prefer
radii (mm) speed more?
2
q
2
3
& 2 10 & 10 8 14 0.143 no
2 & 2
q
3
2
10 & 10 10 12 0.416 no
2 & 2 10 & 15 20 13 0.148 no
Table 4.3: Speed or size equalization abolish model's numeric discrimination. The top
two rows demonstrate the elimination of integer discrimination with equalization of the
apparent projected surface area of the two stimuli groups. The bottom row shows aboli-
tion of integer discrimination with equalization of the sum of speeds of the two groups.
In every case, the task was to distinguish the group of 3 on one side from the group of
2 on the other. The strong surround inhibition DoG was used, and the length of the
y
tube was 45 mm. See table 4.1 for statistic explanation.
We next varied continous features of the simulated
ies, applying the basic ideas of
Agrillo et al. (2008) to our salamander model. We equalized the nominal apparent area
of the projection of the
ies onto the salamander retina, i.e., we varied the size of the
simulated
y spheres. This was done so that the sum of the areas of the circles of sphere
radii was the same on both sides. In order to check for possible eects of smaller or larger
stimuli, we tested both conditions: in one case, the larger number of
ies got a smaller
radius (compared with the nominal 2 mm), while in the other case, the smaller number
140
of
ies got a larger radius. In both cases, the model's preference for the more numerous
group of
ies vanished. See the top two rows of table 4.3.
Next, we increased the speed of the
ies in the less numerous tube group so that
the sum of the speeds of both groups was equal. Again, the model's preference for the
more numerous group of
ies vanished. ( See the bottom row of table 4.3.) This is
not necessarily expected, since the model's visual motion sensitivity is highly nonlinear
(chapter 2, Begley & Arbib (2007)).
Numbers Tube More Less P Prefer
of
ies length more?
3 & 2 22.5 26 7 0.000659 YES
6 & 4 22.5 24 9 0.00677 YES
4 & 3 22.5 22 11 0.0401 YES
3 & 2 90 22 11 0.0401 YES
4 & 3 90 20 13 0.148 no
6 & 4 90 23 10 0.0175 YES
Table 4.4: Changing tube length changes model's numeric discrimination. Results of
simulations with
y tube length half (top) and double (bottom) the nominal length.
Both cases demonstrate discrimination with quantities (6 vs. 4) that salamanders did
not distinguish in Uller et al. (2003). Other conditions are like those of table 4.1, in
which the limitations of Uller's salamanders were reproduced. See table 4.1 for statistic
explanation.
Finally, we changed the length of the tubes containing the
ies. The purpose of this
manipulation was to test the model's sensitivity to the visual conguration of the stimuli.
A set of runs was executed with tubes half the nominal length, while another set used
tubes double their nominal 45 mm. length. In both cases, numeric discrimination was
demonstrated that did not exist (table 4.4) in the runs with the nominal length tubes (cf.
table 4.1).`
141
4.5 Discussion
We were able to simulate the salmander's preference for 3 items over 2; we were also
able to simulate a lack of discrimination between 4 and 3 items and between 6 and 4.
We did this with a model that has no representation, neither explicitly nor implicitly, of
integer numbers. The discrimination was sensitive to the shape of the classical receptive
eld, a low-level vision property. The apparent discrimination between 2 and 3 item sets
vanished when we equalized the apparent areas of the two sets, and when we equalized
the sum of the speeds of the stimulus objects in each set. Finally, we showed that our
model's numerical discrimination is sensitive to the spatial conguration of the stimulus
sets; changing the length of the tube containing the
ies could improve the numerical
discrimination ability.
Ansari (2008) included, by citation, the salamander among the animals that can
discriminate the \total number of items in a set". Our modeling suggests a dierent
interpretation for the behavioral results that seemed to indicate that salamanders have
this ability. The apparent numerical discrimination may be an artifact of the low-level
visual processing, combined with accidental features of the experimental conguration.
Our results suggests that the results of Uller et al. (2003) may arise from intrinsic
properties of the visual system, rather than as a schema of preferences for specic num-
bers of prey items. The salamander's visual system may be more tuned for successfully
snapping at prey in the terminal phase of the hunt than for foraging { the foraging neu-
ral system may operate within very tight constraints, with numerical preferences evident
142
in carefully control laboratory experiments as emergent properties rather than primary
preferences.
Agrillo et al. (2008) recently found that sh are sensitive to analog quantities, and
that this sensitivity confounds any simple preference the sh may show for the larger
of small integers. Rather than simply preferring the school with the larger of two small
cardinalities, these female mosquitosh preferred the school of sh with a larger apparent
surface area on a 2-dimensional projection toward the subject. Our simulation experi-
ments reveal analogous behavior in our model. Equalizing the total surface area of the two
groups of
ies abolished any apparent preference between these groups in the salamander.
Visual preferences in salamanders are strongly in
uenced by simulus size (Sch ulert
& Dicke, 2002; Schuelert & Dicke, 2005), a property shared with our simulation model
under the conditions of chapter 3, in particular with the comparatively weak RF surround
used in that work. Under the strong surround condition in the current work, our model
does not adequately model such size preference. Although we showed size sensitivity for
the simulated
ies at the distance at which the salamander made the decision to turn
toward one side or the other, as the simulated salamander approached closer to the
ies,
it became distracted by
ies in the visual eld that presented a smaller retinal image. To
overcome this defect, the model would need a terminal guidance mechanism, including
discrimination of prey at distances close enough to distinguish detailed stimulus features.
We believe that it is likely that salamanders possess such a neural mechanism. Visual
control of prey approach has been the subject of past modeling eorts (Eurich et al.,
143
1995; Cobas & Arbib, 1992), as has subtle prey discrimination (Wang & Arbib, 1990).
Our current model is adequate for stimuli which project to a retinal area less than that
of an RGC ERF.
Compared with their results for size preference, the results of Agrillo et al. (2008) con-
cerning the interaction of speed preference for conspecics among mosquitosh, together
with numeric preference, produced less conclusive results. Our results, by contrast, were
clear. Equalizing the sum of the speeds of the simulated
ies abolished the apparent nu-
merical preference. This eect from a simple linear speed combination is not necessarily
expected. The model's motion sensitivity is highly nonlinear (chapter 2, Begley & Ar-
bib (2007)). The highly distorting perspective projection used to simulate photoreceptor
input introduces further complexity.
The interpretation of Uller et al. (2003) may be considered simpler than that of
Agrillo et al. (2008) in the sense that we might expect salamanders' preferrences for

ies to be less complicated than shes' interaction with conspecics. Our experimental
methods simplify still further. For example, our simulated
ies all moved at the same
speed. Uller et al. (2003) mentioned the possibility of using computer displays of simplied
stimuli to investigate quantity discrimination and its relationship with motion preference.
Simplied visual stimuli abstracted from natural stimuli have long been used to study
salamander visually guided behavior (Roth, 1987). Continued development and use of
such methods could lead to a greater understanding of the nature of visual preferences,
including quantity discrimination.
144
Figure 4.2 suggests a diculty the salamander brain would encounter in determining
the number of
ies in a closely spaced group. While the group of 2
ies on the left
is represented by 2 separate blobs of RGC activity (g. 4.2e), the group of 3
ies on
the right is represented by what could be a single blob. The counting neural network
of Domijan (2004) merged connected \objects", but did not consider the possibility of
multiple objects within a single input neuron's RF. This can be considered a fundamental
limitation of such a system. Yet, our results indicate that a primitive visual system can
chose between groups of 3 and 2 objects, although in many cases that visual system would
not distinguish individual objects, as in g. 4.2e.
Schuelert & Dicke (2005) studied the responses of salamander tectal neurons to food-
like visual stimuli. These neurons had a central ERF as well as a surround of large spatial
extent; the surround was inhibitory in most, but not all, neurons. The the central ERF
of the tectal neurons studied had a diameter of between 6
o
and 35
o
, with even larger
ERFs in a small number of neurons. The average ERF diameter was in the low 20
o
s.
An average neuron's ERF is very nearly the diameter of the length of the project of the

y tube when the salamander is in the starting chamber in Uller et al. (2003)
4
. An
alternative to our hypothesis thus might be that these neurons control the direction of
locomotion, with limitations on the numerical discrimination resulting from saturation.
On the other hand, if 6
o
diameter ERF tectal neurons with large, strong inhibitory
surrounds provide the immediate visual input to the neural structure determining the
4
Our
y tubes were further away to compensate for our simulated salmander's long body, so their
projections could t easily within the average tectal visual neuron's ERF.
145
direction of motion, these small ERF tectal neurons could take the place of RGCs in our
model. Such a model is similar enough to ours that our results extend to that model; i.e.,
a strong inhibitory surround will limit the numerical discrimination ability for clusters of
objects covered by a small handful of ERFs. That is, the central excitation induced by a
stimulus may be roughly balanced by the peripheral inhibition it induces, so that adding
or removing stimuli may have little eect on the total induced neural activity.
Our modeling and simulation suggest that the salamander brain does not count items.
We would not expect to nd a salamander neuron sensitive to the number of items in its
receptive eld, independent of other characteristics of those items. The discovery of such
a neuron would be a challenge to the ndings of this chapter.
146
Chapter 5
Salamander Graphical Simulation Software
5.1 User Guide
The salamander simulation system is run from a Linux command line. While the program
is running, pressing the left mouse button within any of the program windows will cause
a pop-up menu to appear. A command may be selected by moving the mouse to the
selection and releasing the mouse button. Alternatively, commands may be entered from
the keyboard; each command corresponds to one or more single characters. Keyboard
entry is case sensitive. We warn the user that many of the commands on this menu were
not used during the work reported in this dissertation. Some of the commands will not
work, and may have seemingly perverse eects.
To run the program from a Linux command line, enter
$./salamander optional-arguments
where $ is the prompt. No arguments were used for the chapter 3 version of this code. For
the chapter 4 version, an optional single unsigned integer less than 2
32
seeds the random
147
number generator { if no number is given, a default constant is used. The program results
are repeatable for any given seed. Other arguments may be given. If the arguments can
be parsed as GLUT parameters (Kilgard, 1998), they will be used by the OpenGL GUI
system; we have not used this capability.
The program will run to completion unless the user command 'Quit' is entered, either
by pressing the 'q' or Esc key, or by selecting 'Quit' from the pop-up menu. The version
of the program used in chapter 3 will run until a simulation time limit; the version of
chapter 4 will run until either the salamander get close enought to a virtual tube, or a
time limit, which ever occurs rst. In the work done for chapter 4, the time limit never
occurred { the salamander always \chose" a virtual tube.
Besides the Quit command, 'q', other useful commands include 'G', to take a graphical
snapshot (a JPEG image { the menu incorrectly says 'gif'), 'z' to zoom out on the overview
window, and 'a' to zoom in on the overview window. The user may wish to raise the
water level by pressing '4', or lower the water level with '3'. Other commands that appear
on the pop-up menu may or (more likely) may not work { user beware! In particular,
it is not recommended to use the space bar during the simulation, since this invokes the
unsupported \Toggle Pause" command, which may decouple the visual system simulation
from the CPG-based locomotion simulation.
The user is warned to not hide the \photoreceptor view" window. Visual system
input is taken directly from the graphics buers for this window. Anything covering this
148
window may be used as the photoreceptor input to the retina model instead of the desired
input in the (possibly partially) hidden photoreceptor layer window.
5.2 Results Data Files
Program image data are found in les salam video.mpg, for the movie created by
the program, and salamander2Dsnapshotn.jpg, where n is a decimal integer. If
salam video.mpg exists when the program starts, it will be overwritten. If any snapshot
les exist when the 'G' command is entered, the program will generate a new lename
using the rst available integer.
Numerical program results are stored in text data les in subdirectory DATA. The
les are:
sim env water level, target position, etc.
sim neur neural activity for each segmental oscillator neuron
sim pos position of the center of each segment of the salamander's body
sim speed speed of each segment of the salamander's body
sim t time at each simulation step
sim vis visual system data (see below)
sim vstat retinal model statistics { not used in this thesis
149
The data presented in chapters 3 and 4 are from les sim pos, sim t, and sim vis.
The only segmental position data used were for the head, the rst segment. There are 4
elements for each segment. The rst 3 are position coordinates, the 4
th
is the horizontal
orientation. Note that the y position is in screen coordinates, so must be negated to
reproduce the screen position on a Matlab
r
plot.
24 visual system data elements are stored in sim vis at each simulation step. Here we
describe some of the more important, including those used in chapters 3 and 4. For this
description, we use the convention sim vis(n), where n runs from 1 to 24. sim vis(16) is
0 except at steps where the head orientation movement changes direction, the rotation
reversals. sim vis(15) is +1 when the commanded direction of motion is updated accord-
ing to visual \salience"; it is -1 when the allocentric desired movement direction is held
constant (and the head-centered direction is therefore dead-reckonned).
sim vis(18) and sim vis(19) are the left and right CPG descending tonic inputs com-
puted by the visual system, the "push" signals in the push-pull model; sim vis(22) and
sim vis(23) are the left and right push signals computed by the \ideal" model; and
sim vis(20) and sim vis(21) are the left and right push tonic activations actually used
for CPG control during the simulation. sim vis(17) is the direction-weighted sum of the
retinal salience vector. sim vis(2) is the desired turn direction in head-centered coordi-
nates.
150
5.3 Code Overview
The salamander graphical simulation system was written in C and runs under Linux.
The code uses OpenGL
r
(Shreiner et al., 2000), GLUT (Kilgard, 1998), OpenCV (Intel
Corporation, 2000), mpeg, and FFTW (Frigo & Johnson, 2005) application program
interface (API) libraries. The OpenCV library is used to output static screen snapshots;
mpeg is used to output movies { the simulation does not depend on these features.
FFTW is used for convolution; these could be rather easily implemented without FFTW.
On the other hand, OpenGL and GLUT are essential. The salamander world, and visual
input to the neural simulation, depend on OpenGL. The simulation program is organised
as a GLUT event-driven system.
The simulation C source code is organised into a few les with names \salam *.c".
Each \*.c" le has a corresponding \*.h" header le dening externally visible procedures
and structures. The source code les are:
salam main.c top level calls, initialization, and data storage
salam graphics.c interface to OpenGL and GLUT, window management
salam graphics draw.c renders the simulation world, determining positions of moving
objects
salam vision.c visual system simulation
salam cpg.c locomotion neural control simulation
151
salam mec.c mechanical simulation of salamander locomotion; much of this code was
automatically generated by MathEngine's FastDynamics
salam video.c ad hoc utilities to generate video output via mpeg API
util.c Numerical Recipes implementations (Press et al., 1988)
Our explanation of the code will use the ad hoc pseudocode convention
\(le name root).program entity", where le name root is the le name without the \.c"
and/or \.h" le type sux, and program entity is a function, variable, or preprocessor
constant or macro. For example, (salam main).idle() refers to the function \idle" in
source le salam main.c, dened in header le salam main.h.
The salamander simulation is a GLUT event-driven program (Kilgard, 1998). The
main simulation loop is in function (salam main).idle(), which is called during glut-
MainLoop(). This is established with the glutIdleFunc call in (salam graphics).visible().
In the implementation used for the work reported in this thesis, (salam main).idle() is
always enabled as the idle function. It could be disabled for more robust handling of
occluded screen visibility { in the current implementation, every salamander simulation
program window must always be visible to ensure correct simulation peformance.
5.4 Initialization
This section describes the processing executed before the glutMainLoop() call in
(salam main).main(). These include initializations in salam mec.c and salam cpg.c.
152
(salam cpg).neur set param() reads le CPG DATA/CPG.txt, which contains the CPG
parameters established by the evolutionary algorithm approach in citetijsp1.
glutInit() is a GLUT library function which can parse the command line. Arguments
may be passed to GLUT in this way, as described in Kilgard (1998). Such arguments
were not used in the work described in this dissertation. The only salamander program
command line arguments were random number generator seeds for the version of the
program described in chapter 4, in which the
ies were assigned to tubes randomly, then
initially position randomly within their tubes, and then changed directions randomly
during the simulation.
The glutInit() call in (salam main).main() is immediately followed by a
(salam graphics).initGraphics() call. The function initializes the graphical user inter-
face (GUI) windows. It also initializes the visual system with a call to
(salam vision).initRetina 1(), and initializes video output with a call to
(salam graphics).initVideo(). (salam vision).initRetina 1() initializes the data struc-
tures described in detail in chapter 3. Its name is somewhat deceptive, since it initializes
the entire visual system model, not just the retina model.
(salam graphics).initVideo() initializes video le (*.mpg) output.
The animated graphical simulation program's six windows are initialized with a block
of code, mostly GLUT library function calls (Kilgard, 1998), repeated (with obviously
necessary changes) for each window. Note in particular the calls to glutDisplayFunc().
153
The function passed as the argument to glutDisplayFunc() for the \photoreceptor view"
window, window 2, drives the simulation, as explained in the next section.
5.5 Simulation Loop
The top level simulation loop processing is found in function (salam main).idle(). The
mechanical and CPG simulations are explicit in this function with calls to
(salam mec).mec sim() and (salam cpg).neur sim(). The mechanical simulation has a
shorter integration step size; there are 5 mechanical integration steps for every neural
integration step.
The visual system simulation is implicit in the call to
(salam graphics).my glutPostRedisplay() at the end of (salam main).idle()
1
. We rec-
ommend 1 for the value of preprocessor constant
(salam main).FACTOR NSTEPS GRAPH { the visual system simulation is assumed
to use the same integration step size as the CPG simulation.
(salam graphics).my glutPostRedisplay() calls the GLUT library function glut-
PostRedisplay() (Kilgard, 1998) for every window. This GLUT function calls the function
(salam graphics).drawWindown, where n2 1::: 6, corresponding to the six windows;
these calls are established during initialization, in the calls to the GLUT function glut-
DisplayFunc() in (salam graphics).initGraphics().
1
The one near the beginning of the function is invoked during the deprecated pause processing
154
(salam graphics).drawWindow2() is the top level visual system simulation function.
The simulated photoreceptor input to the two eyes results from the processing in the two
calls to (salam graphics).oneEyeView(). The rest of the visual system simulation occurs
in the call to (salam vision).compute visual system activity().
155
Bibliography
Agrillo C, Dadda M, Serena G, Bisazza A. 2009. Use of number by sh. PLoS ONE
4(3):e4786.
Agrillo C, Dadda M, Serena G, Bisazza A. 2008. Do sh count? Spontaneous discrimi-
nation of quantity in female mosquitosh. Animal Cognition 11(3):495{503.
Agrillo C, Dadda M, Bisazza A. 2007. Quantity discrimination in female mosquitosh.
Animal Cognition 10(1):63{70.
Amari S, Arbib MA. 1977. Competition and cooperation in neural nets. In Metzler J, ed.,
Systems Neuroscience. Academic Press.
Amthor FR, Keyser KT, Dmitrieva NA. 2002. Eects of the destruction of starburst-
cholinergic amacrine cells by the toxin AF64A on rabbit retinal directional selectivity.
Visual Neuroscience 19:495{509.
Ansari D. 2008. Eects of development and enculturation on number representation in
the brain. Nature Reviews Neuroscience 9(4):278{391.
Arbib MA, House DH. 1987. Depth and detours: an essay on visually guided behavior.
In Arbib MA, Hanson AR, eds., Vision, Brain, and Cooperative Computation. MIT
Press. Cambridge MA.
Armstrong-Gold CE, Rieke F. 2003. Bandpass ltering at the rod to second-order
cell synapse in salamander (Ambystoma tigrinum) retina. Journal of Neuroscience
23(9):3796{3806.
Azizi E, Landberg T. 2002. Eects of metamorphosis on the aquatic escape response of the
two-lined salamander (Eurycea bislineata). Journal of Experimental Biology 205:841{
849.
Baccus SA. 2007. Timing and computation in inner retinal circuitry. Annual Review of
Physiology 69:271{290.
Baccus SA, Olveczky BP, Manu M, Meister M. 2008. A retinal circuit that computes
object motion. Journal of Neuroscience 28(27):6807{6817.
156
Balasubramanian V, Berry MJ 2nd. 2002. A test of metabolically ecient coding in the
retina. Network: Computation in Neural Systems 13(4):531{552.
Balboa RM, Grzywacz NM. 2000. The role of early retinal lateral inhibition: more than
maximizing luminance information. Visual Neuroscience. 17:77{89.
Begg RK, Palaniswami M, Owen B. 2005. Support vector machines for automated gait
classication. IEEE Transactions on Biomedical Engineering 52(5):828{838.
Begley JR, Arbib MA. 2007. Salamander locomotion-induced head movement and retinal
motion sensitivity in a correlation-based motion detector model. Network: Computa-
tion in Neural Systems 18(2):101{128.
Berry MJ, Warland DK, Meister M. 1997. The structure and precision of retinal spike
trains. Proceedings of the National Academy of Sciences of the United States of America
94(10):5411{5416.
Borst A. 2007. Correlation versus gradient type motion detectors: the pros and cons.
Philosophical Transactions of the Royal Society B Biological Sciences 362(1479):369{
374.
Borst A, Egelhaaf M. 1989. Principles of visual motion detection. Trends in Neurosciences
12:297{306.
Burkhardt DA, Fahey PK, Sikora M. 1998. Responses of ganglion cells to contrast steps
in the light-adapted retina of the tiger salamander. Visual Neuroscience 15(2):219{229.
Cabelguen J-M, Bourcier-Lucas C, Dubuc R. 2003. Bimodal locomotion elicited by electri-
cal stimulation of the midbrain in the salamander Notophthalmus viridescens. Journal
of Neuroscience 23(6):2434{2439.
Cervantes-Perez F, Lara R, Arbib M. 1985. A neural model of interactions subserving
prey-predator discrimination and size preference in anuran amphibia. Journal of The-
oretical Biology 113(1):117{152.
Cherno EA, Stocum DL, Nye HL, Cameron JA. 2003. Urodele spinal cord regeneration
and related processes. Developmental Dynamics 226(2):295{307.
Cli D. 2003. Neuroethology, computational. In Arbib MA, ed., The Handbook of Brain
Theory and Neural Networks. The MIT Press. Cambridge, Massachusetts; London,
England.
Cli D, Husbands P, Harvey I. 1993. Explorations in evolutionary robotics. Adaptive
Behavior 2:73{110.
Cli D, Noble J. 1997. Knowledge-based vision and simple visual machines. Philosophical
Transactions of the Royal Society of London B 352(1358):1165{1175.
157
Cliord CWG, Ibbotson MR. 2003. Fundamental mechanisms of visual motion detection:
models, cells, and functions. Progress in Neurobiology 68:409{437.
Cobas A, Arbib M. 1992. Prey-catching and predator-avoidance in frog and toad: dening
the schemas. Journal of Theoretical Biology 157(3):271{304.
Coleman PA, Miller RF. 1989. Measurement of passive membrane parameters with whole-
cell recording from neurons in the intact amphibian retina. Journal of Neurophysiology
61:218{230.
D'Aout K, Aerts P. 1997. Kinematics and eciency of steady swimming in adult axolotls
(Ambystoma mexicanum). Journal of Experimental Biology 200:1863{1871.
Dehaene S, Changeux JP. 1993. Development of elementary numerical abilities: a neu-
ronal model. Journal of Cognitive Neuroscience 5(4):390{407.
Delbr uck T. 1993. Silicon retina with correlation-based, velocity-tuned pixels. IEEE
Transactions on Neural Networks 4:529{541.
Delvolve I, Bem T, Cabelguen JM. 1997. Epaxial and limb muscle activity during swim-
ming and terrestrial stepping in the adult newt, Pleurodeles waltl. Journal of Neuro-
physiology 78:638{650.
Denton TA, Diamond GA. 1991. Can the analytic techniques of nonlinear dynamics dis-
tinguish periodic, random and chaotic signals? Computers in Biology and Medicine
21:243{263.
Dicke U. 1999. Morphology, axonal projection pattern, and response types of tectal neu-
rons in plethodontid salamanders. I: Tracer study of projection neurons and their path-
ways. Journal of Comparative Neurology 404:473{488.
Didday RL. 1976. A model of visuomotor mechanisms in the frog optic tectum. Mathe-
matical Biosciences 30:169{180.
Didday RL, Arbib MA. 1975. Eye movements and visual perception: 'two visual systems'
model. International Journal of Man-Machine Studies 7:547{569.
Diamond MR, Ross J, Morrone MC. 2000. Extraretinal control of saccadic suppression.
Journal of Neuroscience 20(9):3449{3455.
Domijan D. 2004. A neural model of quantity discrimination. Neuroreport 15(13):2077{
2081.
Dowling JE. 1987. The retina : an approachable part of the brain. Belknap Press of
Harvard University Press. Cambridge MA.
Dror RO, O'Carroll DC, Laughlin SB. 2001. Accuracy of velocity estimation by Reichardt
correlators. Journal of the Optical Society of America A 18(2):241{252.
158
Duellman WE, Trueb L. 1986. Biology of Amphibians. McGraw Hill.
Egelhaaf M, Reichardt W. 1987. Dynamic response properties of movement detectors:
theoretical analysis and electrophysiological investigation in the visual system of the

y. Biological Cybernetics 56:69{87.
Ekeberg

O. 1993. A combined neuronal and mechanical model of sh swimming. Biological
Cybernetics 69(5-6):363{374.
Euler T, Detwiler PB, Denk W. 2002. Nature 418:845{852.
Eurich C, Roth G, Schwegler H, Wiggers W. 1995. Simulander: a neural network model
for the orientation movement of salamanders. Journal of Comparative Physiology A
176:379{389.
Eurich CW, Schwegler H. 1997. Coarse coding: calculation of the resolution achieved by
a population of large receptive eld neurons. Biological Cybernetics 76(5):357{363.
Eurich CW, Schwegler H, Woesler R. 1997. Coarse coding: applications to the visual
system of salamanders. Biological Cybernetics 77:41{47.
Fagerstedt P, Orlovsky GN, Deliagina TG, Grillner S, Ulln F. 2001. Lateral turns in the
Lamprey. II. Activity of reticulospinal neurons during the generation of ctive turns.
Journal of Neurophysiology 86(5):2246{2256.
Fahey PK, Burkhardt DA. 2003. Center-surround organization in bipolar cells: symmetry
for opposing contrasts. Visual Neuroscience 20:1{10.
Fried SI, Munch TA, Werblin FS. 2002. Mechanisms and circuitry underlying directional
selectivity in the retina. Nature 420:411{414.
Frigo M, Johnson SG. 2005. The design and implementation of FFTW3. Proceedings of
the IEEE 93(2):216{231
Frolich LM, Biewener AA. 1992. Kinematic and electromyographic analysis of the func-
tional role of the body axis during terrestrial and aquatic locomotion in the salamander
Ambystoma tigrinum. Journal of Experimental Biology 162:107{130.
Geisler WS. 1999. Motion streaks provide a spatial code for motion direction. Nature
400(6739):65{69.
Gillis GB. Anguilliform locomotion in an elongate salamander (Siren intermedia): Eects
of speed on axial undulatory movements. Journal of Experimental Biology 200:767{784.
Goldberg ME, Wurtz RH. 1972. Activity of superior colliculus in behaving monkey: I.
Visual receptive eld of single neurons. Journal of Neurophysiology 35(4):542{559.
159
Grillner S, Wall en P, Saitoh K, Kozlov A, Robertson B. 2008. Neural bases of goal-directed
locomotion in vertebrates{an overview. Brain Research Reviews 57(1):2{12.
Grzywacz NM, Amthor FR, Merwine DK. 1998. Necessity of acetylcholine for retinal
directionally selective responses to drifting gratings in rabbit. Journal of Physiology
512:575{581.
Grzywacz NM, Koch C. 1987. Functional properties of models for direction selectivity in
the retina. Synapse 1:417{434.
Hildreth EC, Koch C. 1987. The analysis of visual motion: from computational theory
to neuronal mechanisms. Annual Review of Neuroscience 10:477{533.
Hildreth EC, Royden C. 1998. Computing observer motion from optic
ow. In Watan-
abe T, ed., High Level Motion Processing: Computational, Neurobiological, and Psy-
chophysical Perspectives. The MIT Press. Cambridge, MA.
Ibbotson MR, Cliord CWG. 2001. Characterising temporal delay lters in biological
motion detectors. Vision Research 41:2311{2323.
Ijspeert AJ. 2008. Central pattern generators for locomotion control in animals and
robots: a review. Neural Networks 21(4):642{653.
Ijspeert AJ. 2001. A connectionist central pattern generator for the aquatic and terrestrial
gaits of a simulated salamander. Biological Cybernetics 84:331{348.
Ijspeert AJ. 2000. A 3-D biomechanical model of the salamander. In Heudin JC, ed.,
Proceedings of the Second International Conference on Virtual Worlds, pp. 225{234.
Springer Verlag.
Ijspeert AJ, Arbib M. 2000. Visual tracking in simulated salamander locomotion. In Meyer
JA, Berthoz A, Floreano D, Roitblat H, and Wilson SW, eds., Proceedings of the Sixth
International Conference of The Society for Adaptive Behavior (SAB2000), pp. 88{97.
MIT Press, Cambridge MA.
Ijspeert AJ, Arbib M. 2000. Locomotion and visually-guided behavior in salamander: a
neuromechanical study. In McKee GT, Schenker PS, eds., Proceedings of Sensor Fusion
and Decentralized Control in Robotics Systems III, (SFCRSIII), volume 4196 of SPIE
Proceedings. SPIE.
Ijspeert AJ, Crespi A, Cabelguen JM. 2005. Simulation and robotics studies of salamander
locomotion: applying neurobiological principles to the control of locomotion in robots.
Neuroinformatics 3(3):171{195.
Ijspeert AJ, Crespi A, Ryczko D, Cabelguen JM. 2007. From swimming to walking with
a salamander robot driven by a spinal cord model. Science 315(5817):1416{1420.
160
Ijspeert AJ, Kodjabachian J. 1999. Evolution and development of a central pattern gen-
erator for the swimming of a lamprey. Articial Life 5(3):247{269.
Intel Corporation. 2000. Open Source Computer Vision Library Reference Manual, Ver-
sion OO1. Available at http://developer.intel.com.
Kagan I, Shik ML. 2004. How the mesencephalic locomotor region recruits hindbrain
neurons. Progress in Brain Research, vol. 143, chapter 22, pp. 219{228.
Kilgard MJ. 1988. OpenGL Programming for the X Window System. Addison-Wesley.
Reading, MA; Harlow, England; Menlo Park, CA; Berkeley, CA. Don Mills, Ontario.
Sydney; Bonn; Amsterdam; Tokyo; Mexico City.
Koch C, Ullman S. 1985. Shifts in selective visual attention: towards the underlying
neural circuitry. Human Neurobiology 4(4):219{227.
Kozlov AK, Lansner A, Grillner S, Kotaleski JH. 2007. A hemicord locomotor network
of excitatory interneurons: a simulation study. Biological Cybernetics 96(2):229{243.
Kozlov AK, Ulln F, Fagerstedt P, Aurell E, Lansner A, Grillner S. 2002. Mechanisms for
lateral turns in lamprey in response to descending unilateral commands: a modeling
study. Biological Cybernetics 86(1):1{14.
Lamb M. 1997. Modeling behavior-based depth vision in frog and salamander. Ph.D.
thesis. University of Southern California.
Lee S, Zhou ZJ. 2006. The synaptic mechanism of direction selectivity in distal processes
of starburst amacrine cells. Neuron 51:787{799.
Liaw J-S, Arbib MA. 1993. Neural mechanisms underlying direction-selective avoidance
behavior. Adaptive Behavior 1:227{261.
Lock A, Collett T. 1979. A toad's devious approach to its prey: a study of some complex
uses of depth vision. Journal of Comparative Physiology A 131:179{189.
Manteuel G. 1989. Compensation of visual background motion in salamanders. In Ewert
J-P, Arbib MA, eds., Visuomotor Coordination: Amphibians, Comparisons, Models,
and Robots. Plenum Press, New York and London.
Manteuel G, Naujoks-Manteuel C. 1990. Anatomical connections and electrophysio-
logical properties of toral and dorsal tegmental neurons in the terrestrial urodele Sala-
mandra salamandra. Journal f ur Hirnforschung 31(1):65{76.
Manteuel G, Plasa L, Sommer TJ, Wess O. 1977. Involuntary eye movements in sala-
manders. Naturwissenschaften 64:533{534.
Manteuel G, Roth G. 1993. A model of the saccadic sensorimotor system of salamanders.
Biological Cybernetics 68:431{440.
161
Mar n O, Gonz alez A, Smeets WJ. 1997. Basal ganglia organization in amphibians: ef-
ferent connections of the striatum and the nucleus accumbens. Journal of Comparative
Neurology 380(1):23{50.
Nakamura Y, Suzuki T, Koinuma M. 1997. Nonlinear behavior and control of a non-
holonomic free-joint manipulator. IEEE Transactions on Robotics and Automation
13:853{862.
Okamoto M, Ohsawa H, Hayashi T, Owaribe K, Tsonis PA. 2007. Regeneration of retino-
tectal projections after optic tectum removal in adult newts. Molecular Vision 13:2112{
2118.

Olveczky BP, Baccus SA, Meister M. 2003. Segregation of object and background motion
in the retina. Nature 423:401{408.
O'Malley DM, Sandell JH, Masland RH. 1992. Co-release of acetylcholine and GABA by
the starburst amacrine cells. Journal of Neuroscience 12(4):1394{1408.
OpenGL Architecture Review Board, Dave Schreiner, ed. 2000. OpenGL
r
Reference Man-
ual, Third Edition. Addison-Wesley. Boston, San Francisco, New York, Toronto, Mon-
treal, London, Munich, Paris, Madrid, Capetown, Sydney, Tokyo, Singapore, Mexico
City.
Phillips JB. 1986. Two magnetoreception pathways in a migratory salamander. Science
233(4765):765{767.
Phillips JB, Deutschlander ME, Freake MJ, Borland SC. 2001. The role of extraocu-
lar photoreceptors in newt magnetic compass orientation: parallels between light-
dependent magnetoreception and polarized light detection in vertebrates. Journal of
Experimental Biology 204:2543{2552.
Press WH, Flannery BP, Teukolsky SA, Vetterling WT. 1988. Numerical Recipes in C.
Cambridge University Press. Cambridge, New York, Port Chester, Melbourne, Sydney.
Reichardt W. 1961. Autocorrelation, a principle for the the evaluation of sensory informa-
tion by the central nervous system. In Rosenblith WA, ed., Sensory Communication.
MIT Press, Cambridge MA, and John Wiley & Sons, Inc., New York, London.
Reichardt W, Egelhaaf M, Guo A. 1989. Processing of gure and background motion in
the visual system of the
y. Biological Cybernetics 61:327{345.
Roche King J, Comer CM. 1996. Visually elicited turn behavior in Rana pipiens: compara-
tive organisation and neural control of escape and prey capture. Journal of Comparative
Physiology A 178:293{305.
Roth G. 1987. Visual Behavior in Salamanders. Springer-Verlag. Berlin, Heidelberg, New
York, London, Paris, Tokyo.
162
Roth G, Blanke J, Ohle M. 1995. Brain size and morphology in miniaturized plethodontid
salamanders. Brain, Behavior, and Evolution 45:84{95.
Roth G, Dicke U, Grunwald W. 1999. Morphology, axonal projection pattern, and re-
sponse types of tectal neurons in plethodontid salamanders. II: Intracellular recording
and labeling experiments. Journal of Comparative Neurology 404:489{504.
Roth G, Dicke U, Wiggers W. 1998. Vision. In Heathwole, ed., Amphibian Biology, Vol.
3. Sensory Perception. Surrey, Beatty & Sons, Chipping Norton.
Roth G, Nishikawa KC, Naujoks-Manteuel C, Schmidt A, Wake DB. 1993. Paedomor-
phosis and simplication in the nervous system of salamanders. Brain, Behavior, and
Evolution 42:137{170.
Roth G, Nishikawa KC, Wake DB. 1997. Genome size, secondary simplication, and the
evolution of the brain in salamanders. Brain, Behavior, and Evolution 50:50{59.
Roth G, Rottlu B, Linke R. 1988. Miniaturization, genome size and the origin of func-
tional constraints in the visual system of salamanders. Naturwissenschaften 75(6):297{
304.
Roth G, Wake DB. 2001. Evolution and devolution: the case of bolitoglossine salaman-
ders. In Roth G, Wullimann MF, eds., Brain Evolution and Cognition. Wiley Spectrum.
New York.
Ruderman DL. 1994. The statistics of natural images. Network: Computation in Neural
Systems 5:517{548.
Ruderman DL, Bialek W. 1994. Statistics of natural images: scaling in the woods. Physical
Review Letters 73:814{817.
Saitoh K, M enard A, Grillner S. 2007. Tectal control of locomotion, steering, and eye
movements in lamprey. Journal of Neurophysiology 97(4):3093{3108.
Sarpeshkar R, Kramer J, Indiveri G, Koch C. 1996. Analog VLSI architectures for motion
processing: from fundamental limits to system applications. Proceedings of the IEEE
84:969{987.
Schlerf JE, Spencer RM, Zelaznik HN, Ivry RB. 2007. Timing of rhythmic movements in
patients with cerebellar degeneration. Cerebellum 6(3):221{231.
Schuelert N, Dicke U. 2005. Dynamic response properties of visual neurons and context-
dependent surround eects on receptive elds in the tectum of the salamander
Plethodon shermani. Neuroscience 134:617{632.
Sch ulert N, Dicke U. 2002. The eect of stimulus features on the visual orienting behaviour
of the salamander Plethodon jordani. Journal of Experimental Biology 205:241{251.
163
Segev R, Puchalla J, Berry MJ 2nd. 2006. Functional organization of ganglion cells in the
salamander retina. Journal of Neurophysiology 95(4):2277{2292.
Shen WM, Krivokon M, Chiu H, Everist J, Rubenstein M, Venkatesh J. 2006. Multimode
locomotion for recongurable robots. Autonomous Robots 20(2):165{177.
Schreiner D, ed., OpenGL Architecture Review Board. 2000. OpenGL
r
Reference Man-
ual, Third Edition. Addison-Wesley. Boston, San Francisco, New York, Toronto, Mon-
treal, London, Munich, Paris, Madrid, Capetown, Sydney, Tokyo, Singapore, Mexico
City.
Spencer RM, Ivry RB, Zelaznik HN. 2005. Role of the cerebellum in movements: control
of timing or movement transitions? Experimental Brain Research 161(3):383{396.
Srinivasan MV. 1990. Generalized gradient schemes for the measurement of two-
dimensional image motion. Biological Cybernetics 63(6):421{431.
Stuphorn V, Bauswein E, Homann KP. 2000. Neurons in the primate superior colliculus
coding for arm movements in gaze-related coordinates. Journal of Neurophysiology
83(3):1283{1299.
Teeters JL, Arbib MA. 1991. A model of anuran retina relating interneurons to ganglion
cell responses. Biological Cybernetics 64(3):197{207.
Teeters JL, Arbib MA, Corbacho F, Lee HB. 1993. Quantitative modeling of responses of
anuran retina: stimulus shape and size dependency. Vision Research 33(16):2361{2379.
Teeters J, Jacobs A, Werblin F. 1997. How neural interactions form neural responses in
the salamander retina. Journal of Computational Neuroscience 4(1):5{27.
Uller C, Jaeger R, Guidry G, Martin C. 2003. Salamanders (Plethodon cinereus) go for
more: rudiments of number in an amphibian. Animal Cognition 6(2):105{112.
van Hateren JH, van der Schaaf A. 1998. Independent component lters of natural images
compared with simple cells in primary visual cortex. Proceedings of the Royal Society
of London B 265:359{366.
V gh J, B anv olgyi T, Wilhelm M. 2000. Amacrine cells of the anuran retina: mor-
phology, chemical neuroanatomy, and physiology. Microscopy Research and Technique
50(5):373{383.
Wall en P, Ekeberg O, Lansner A, Brodin L, Tr av en H, Grillner S. 1992. A computer-based
model for realistic simulations of neural networks. II. The segmental network generating
locomotor rhythmicity in the lamprey. Journal of Neurophysiology 68(6):1939{1950.
Wang DL, Arbib MA. 1992. Modeling the dishabituation hierarchy: the role of the pri-
mordial hippocampus. Biological Cybernetics 67:535{544.
164
Wang DL, Arbib MA. 1990. How does the toad's visual system discriminate dierent
worm-like stimuli? Biological Cybernetics 64:251{261.
Warland D, Reinagel P, Meister M. 1997. Decoding visual information from a population
of retinal ganglion cells. Journal of Neurophysiology 78:2336{2350.
Werner C, Himstedt W. 1985. Mechanism of head orientation during prey capture in sala-
mander (Salamandra salamandra L.). Zoologische Jahrb ucher-Abteilung f ur Allgemeine
Zoologie und Physiologie der Tiere 89(3):359{368.
Westho G, Roth G. 2002. Morphology and projection pattern of medial and dorsal pallial
neurons in the frog Discoglossus pictus and the salamander Plethodon jordani. Journal
of Comparative Neurology 445, 97-121.
v. Wietersheim A, Ewert J-P. 1978. Neurons of the toad's (Bufo bufo L.) visual system
sensitive to moving congurational stimuli: a statistical analysis. Journal of Compar-
ative Physiology 126:35{42.
Wisse M, Schwab AL, van der Linde RQ, van der Helm FCT. 2005. How to keep from
falling forward: elementary swing leg action for passive dynamic walkers. IEEE Trans-
actions on Robotics 21:393{401.
Woesler R. 2001. Object segmentation model: analytical results and biological implica-
tions. Biological Cybernetics 85:203{210.
Woesler R, Roth G. 2003. Visuomotor coordination in salamander. In Arbib MA, ed.,
The Handbook of Brain Theory and Neural Networks. The MIT Press. Cambridge,
Massachusetts; London, England.
Yoshida K, Watanabe D, Ishikane H, Tachibana M, Pastan I, Nakanishi S. 2001. A key role
of starburst amacrine cells in originating retinal directional selectivity and optokinetic
eye movement. Neuron 30:771{780.
Yuille AL, Geiger D. 2003. Winner-take-all networks. In Arbib MA, ed., The Handbook
of Brain Theory and Neural Networks. The MIT Press. Cambridge, Massachusetts;
London, England.
Zanker JM. 2004. Looking at op art from a computational viewpoint. Spatial Vision
17:75{94.
Zanker JM. 1996. On the elementary mechanism underlying secondary motion processing.
Philosophical Transactions of the Royal Society of London B 351(1348):1725{1736.
Zanker JM, Srinivasan MV, Egelhaaf M. 1999. Speed tuning in elementary motion detec-
tors of the correlation type. Biological Cybernetics 80:109{116.
Zanker JM, Zeil J. 2005. Movement-induced motion signal distributions in outdoor scenes.
Network: Computation in Neural Systems 16(4):357{376.
165
Zeil J, Zanker JM. 1997. A glimpse into crabworld. Vision Research 37:3417{3426.
Zhang J, Wu SM. 2001. Immunocytochemical analysis of cholinergic amacrine cells in the
tiger salamander retina. Neuroreport 12:1371{1375.
Zhang J, Yang Z, Wu SM. 2004. Immuocytochemical analysis of spatial organization of
photoreceptors and amacrine and ganglion cells in the tiger salamander retina. Visual
Neuroscience 101:163{169.
166

Abstract (if available)

Abstract This thesis examines the behavioral consequences of motion sensitivity of salamanders' visual neurons. The analysis is centered on a correlation-based motion detector model of motion-sensitive retinal ganglion cells (RGCs). This model is then integrated with Ijspeert's model (Ijspeert, 2001) of spinal central pattern generators (CPGs) for salamander walking and swimming. The integration in the salamander brain is studied by means of a model of a sensorimotor pathway centering on the tectum. Vision is used to control locomotion, while locomotion strongly affects vision.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Computational modeling and utilization of attention, surprise and attention gating

PDF

Computational modeling and utilization of attention, surprise and attention gating [slides]

Asset Metadata

Creator Begley, Jeffrey Robert (author)

Core Title Modeling the integration of salamander vision and behavior

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Computer Science

Publication Date 05/06/2009

Defense Date 02/17/2009

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag brain modeling,CPG-based locomotion,motion sensitivity,neural locomotion steering,numerical discrimination,numerical preference,OAI-PMH Harvest,salmander vision

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Arbib, Michael A. (committee chair), Itti, Laurent (committee member), Mel, Bartlett W. (committee member)

Creator Email jbegley@usc.edu,jeff.begley@ngc.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-m2199

Unique identifier UC149244

Identifier etd-Begley-2784 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-242256 (legacy record id),usctheses-m2199 (legacy record id)

Legacy Identifier etd-Begley-2784.pdf

Dmrecord 242256

Document Type Dissertation

Rights Begley, Jeffrey Robert

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Repository Name Libraries, University of Southern California

Repository Location Los Angeles, California

Repository Email cisadmin@lib.usc.edu