00001.tif

PDF

Download

Open document

Flip pages

Contact Us

Copy asset link

Request this asset

Transcript (if available)

Content A NEURAL NETWORK AND SCHEMATIC .MODELING OF ANURAN VISUOMOTOR. COORDINATION IN DETOUR BEHAVIOR by Hyun Bong Lee A DlssertatiomPresented'tohhe FACULTY OF THE GRADUATE SCHOOL UNTVERSITY’ OF'SOUTHERN' CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (Computer Science) May 1994 Copyright 1994 Hyun Bong Lee U M I Number: DP22886 All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if material had to be removed, a note will indicate the deletion. Dissertation Publishing UMI DP22886 Published by ProQuest LLC (2014). Copyright in the Dissertation held by the Author. Microform Edition © ProQuest LLC. All rights reserved. This work is protected against unauthorized copying under Title 17, United States Code ProQuest LLC. 789 East Eisenhower Parkway P.O. Box 1346 Ann Arbor, Ml 48106-1346 UNIVERSITY OF SOUTHERN CALIFORNIA THE GRADUATE SCHOOL UNIVERSITY PARK LOS ANGELES, CALIFORNIA 90007 This dissertation, written by under the direction of h.is... Dissertation Committee, and approved by all its members, has been presented to and accepted by The Graduate School, in partial fulfillment of re quirements for the degree of P h .p . cps l A i 2 > DOCTOR OF PHILOSOPHY Dean of Graduate Studies Date ... DISSERTATION COMMITTEE Chairperson To the memory of my father, Lee, Mun Jun iii Acknowledgments I deeply thank my advisor Prof. Arbib for all his directions in all the stages of my work. I was very fortunate to have an advisor who was always understanding, patient, and never parsimonious in encouraging a fledgling researcher. I will be ever in debt for his academic and financial support that made this study possible. My understanding of the research topic is deepened by discussion with Prof. A. Herrera and experiments conducted in his lab, and I am grateful for his kindness. I thank Prof. I. Biederman in helping me to expand my knowledge in the psychology of vision. I can recall many thoughtful discussions with Dr. Lucia Simo throughout the years, and I thank for her advice and encouragement that made my study much more fruitful. I am truly fortunate to have known all the wonderful colleagues at the Brain Simulation Lab. I can just say that one can hardly hope to have a better colleague and friend than Jim liaw or Fernando Corbacho. Thank you Alfredo for your work on the Neural Simulation Language with which I wrote my simulation programs. I thank Erwin King, Jean Marc Fellous, Nicolas Schweighofer, Deliang Wang, Peter Dominey, Amanda Bischoff, Andy Fagg, Bruce Hoff, Jozsef Fiser, and Toshio Uchiyama for all the invigorating discussions and friendship that made this neural engineering community such a wonderful place to work in. I will always cherish the support and friendship you gave. Paulina, thank you for your wonderful support with many administrative tasks. I have the deepest gratitude to my family, especially my mother, for their love, support, and encouragement. Lastly, words can hardly describe my love and gratitude to my wife, Mi Jung, for her support, understanding, and sacrifice that made all this possible. iv Table of Contents Acknowledgments iii List of Figures vi List of Tables ix A bstract x 1. INTRODUCTION.....................................................................................................1 1.1. Anuran Detour Behavior................................................................................1 1.2. Overview of Anuran Anatomy.................................................................... 4 1.3. Anuran Detour Behavior and Robot Path Navigation............................... 8 1.4. Problem Statement........................................................................................ 9 1.5. Contributions of this Research....................................................................10 1.6. Organization of the Dissertation.................................................................12 2. LITERATURE REV IEW ...................................................................................... 15 2.1. Lesion Studies.................... 16 2.2. Anuran Retina Models.................................................................................19 2.3. Tectal Column Model of Prey Catching Behavior...................................24 2.4. Anuran Depth Perception Model............. 27 2.5. Anuran Detour Models....................................................................... 30 2.6. Models of Anuran Sensorimotor Transformation.....................................34 2.7. Robot Navigation.........................................................................................38 2.8. Discussion.................................................................................................... 39 3. ANURAN RETINA M O D EL............................................................................... 43 3.1. Introduction.................................................................................................. 44 3.2. R l.2 Model.................... 50 3.3. R3 M odel..................................................................................................... 59 3.4. R4 M odel..................................................................................................... 70 3.5. Discussion.................................................................................................... 76 4. ANURAN STATIONARY OBJECT PERCEPTION M O DEL..................... 85 4.1. Introduction.................................................................................................. 86 4.2. Single Cell Models of Pretectal Neurons.................................................. 91 4.3. Model of Pretectal SO Representation....................................................101 4.4. Anuran Depth Perception of Stationary Objects....................................109 4.5. Memorized Representation of Stationary Objects..................................116 4.6. Overview of Anuran SO Representation.................................................129 4.7. Comparison of Anuran and Mammalian SO Perception....................... 132 V 5. ANURAN DETOUR M ODEL.............................................................................139 5.1. Introduction.................................................................................................. 140 5.2. Lesion Studies..............................................................................................144 5.3. Anuran Detour Model............................................................................... .148 5.4. Performance of the Detour Model.............................................................169 5.5. Discussion.................................................................................................... 174 6. ANURAN DETOUR BEHAVIOR AND LEARNING................................... 177 6.1. Anuran Detour Behavior and Learning.................................................... 178 6.2. Learning W hat?...........................................................................................181 6.3. Anuran Perceptual Learning Model...........................................................185 6.4. Future Research.......................................................................................... 206 7. CONCLUSION......................................................................................................208 References 212 vi List of Figures Figure 1.1. Anuran detour behavior......................................................................... 2 Figure 1.2. Collett’s detour experiment...................................................................3 Figure 1.3. Schematic illustration of anuran anatomy............................................5 Figure 2.1. Summary diagram of vertebrate retinal organization....................... 20 Figure 2.2. Structure of T&A m odel..................................................................... 25 Figure 2.3. Neurons and synaptology of the tectal column model..................... 26 Figure 2.4. Time course of cue-interaction model................................................29 Figure 2.5. Arbib & House vector field model of detour behavior.................... 31 Figure 2.6. Lara et al.’s detour model.................................................................... 33 Figure 2.7. Positional and motor heading m aps................................................... 36 Figure 2.8. Liaw & Arbib’s motor integration..................................................... 37 Figure 3.1. T & A’s R 1 cell model response...........................................................51 Figure 3.2. T&A’s R2 cell model response...........................................................52 Figure 3.3. Schematic structure of R1.2 model.................................................... 53 Figure 3.4. R1.2 model behaving as R1.................................................................55 Figure 3.5. R1.2 model behaving as R2.................................................................56 Figure 3.6. R1.2 model behaving as intermediate of R1 and R2........................ 58 Figure 3.7. R1.2 model with different On channel...............................................59 Figure 3.8. Respiratory eye movement..................................................................62 Figure 3.9. R3 response with Teeters et al. parameters....................................... 63 Figure 3.10. R3 response with reduced IRF weight.............................................65 Figure 3.11. R3 response with T&A masks.......................................................... 66 Figure 3.12. Resp-R3 response with ATD=0....................................................... 67 vii Figure 3.13. Resp-R3 response with ATD=0.35.................................................68 Figure 3.14. Resp-R3 response with ATD=1...................................................... 69 Figure 3.15. T&A’s R4 model performance..................... 71 Figure 3.16. Comparisons between T&A and revised R4 models.................... 74 Figure 3.17. Schematic stmcture of R4 m odel............... 75 Figure 3.18. Final R4 model response..................................................................77 Figure 4.1. Structure of the pretectal cell m odel............................................. ...94 Figure 4.2. Simulation of ThlO-1 model..............................................................98 Figure 4.3. Simulation of ThlO-2 model..............................................................99 Figure 4.4. Simulation of ThlO-3 model............................................................100 Figure 4.5. Simulation of ThlO-1 patterns..........................................................107 Figure 4.6. Thl0-3’s receptive field to R1.2..................................................... 109 Figure 4.7. Simulation of ThlO-3 patterns..........................................................110 Figure 4.8. Revised cue-interaction model for double barriers........................117 Figure 4.9. Comparison of Ewert’s and N&N’s pretectum organization 123 Figure 4.10. Traces of striatal memory model................................................... 126 Figure 4.11. Thl0-1 pattern by horizontal body movement.............................131 Figure 5.1. Anuran detour behavior for double barriers................................... 141 Figure 5.2. Anuran detour behavior for different gap alignments................... 143 Figure 5.3. Overall schema model of prey capture............................................153 Figure 5.4. A schema model of anuran detour behavior................................... 156 Figure 5.5. Performance of detour model for single barriers...................... 170 Figure 5.6. Performance of detour model for double barriers..................... 172 Figure 5.7. Performance of detour model for gap alignments.....................173 Figure 6.1. An ethogram of anuran detour learning..........................................180 viii Figure 6.2. Vertical cross-section of striatal barrier patterns............................. 186 Figure 6.3. Transformation of barrier pattern based on eq-1............................. 192 Figure 6.4. Eq-1 based transformation of barrier pattern with strong E ....193 Figure 6.5. Eq-1 based transformation of barrier pattern with average E .194 Figure 6.6. Transformation of barrier pattern based on eq-2............................. 196 Figure 6.7. Eq-2 based transformation of barrier pattern with strong E....197 Figure 6.8. Eq-2 based transformation of barrier pattern with average E .198 Figure 6.9. Transformation of barrier pattern based on eq-3.............................202 Figure 6.10. Eq-3 based transformation of barrier pattern with small E ......... 204 Figure 6.11. Eq-3 based transformation of barrier pattern with strong 1......... 205 ix List of Tables Table 3.1. Equations for T&A model..........................................................45 Table 3.2. Equations for R1.2 model....... 54 Table 3.3. Equations for Teeters et al. R3 m odel.......................................60 Table 3.4. Revised R4 model................................................................................. 71 Table 4.1. Equations for pretectal cell m odels...........................................96 Table 4.2. Equations for the revised cue-interaction model....................115 X Abstract A perceptual mobile agent, be it an animal or a robot, that tries to reach a goal location in a cluttered environment needs to perceive, localize, then devise a navigational strategy taking into account objects interposed between itself and the goal. Mobile robots that navigate around obstacles have been built, and in the animal kingdom, animals such as anurans (frogs and toads) show detour behavior when confronted with stationary objects on their way to a prey or when escaping from a threat. To model the complex behavior of detour around a stationary barrier on the way to prey requires an understanding of anuran prey and barrier recognition, depth perception, and appropriate motor pattern generation mechanisms. This thesis traces and models such processes building upon Rana computatrix (Arbib, 1989), an evolving computer model of anuran visuomotor coordination. With a retina model serving as visual front end, we model neural networks believed to be involved in anuran stationary object perception. We then investigate how stationary objects may be represented in two dimensional neural maps and how they may be interpreted in light of the behavioral data on the animals for different stationary objects. Memory is believed to play a role in the representation, modulation, interpretation, and initiation of an action for stationary objects. We model anuran short term spatial memory and present hypotheses on its function in overall animal visuomotor coordination. The thesis discusses some parallels and comparisons between mammalian and anuran stationary object perception. xi The stationary barrier and prey signals have to be integrated to generate appropriate motor commands for detour behavior. We present a schema-based model of the detour behavior that builds on Cobas and Arbib’s (1992) prey-capture model. The detour model is constructed as an assemblage of schemas that is responsible for prey capture, barrier detection, depth estimation, and integration of prey-barrier information. The final motor output is determined by cooperation and competition between the schemas. By adopting an evolutionary strategy of building the detour model on the Cobas & Arbib model, we show how a more complex model can be constructed by inclusion of cooperation, competition, and modulation built on a simple model. The thesis concludes with a chapter that deals with the learning aspect that anurans appear to incorporate during detours, and relates it to artificial potential field based robot navigation. First, new data from our lab that implicates learning are examined. Then, based on the animal learning metaphor, we present a model of the potential field evolution whose shape dynamically changed due to the reflection of the agent’s (robot or animal) ongoing experience with the corresponding objects or places. CHAPTER 1 INTRODUCTION 1.1. Anuran Detour Behavior It is well known that frogs and toads are very responsive to moving objects. They orient and snap at a small moving object such as a fly, worm or even a small wiggling piece of cardboard, but run away from a large moving object. But also, Collett (1982) and Ingle (1983) have observed that a frog or a toad's approach to prey or escape from a threat are also determined by the presence of stationary objects (henceforth called SO). The motivation for this work is shown in Fig. 1.1 in which we see snapshots from a “movie” (Ingle, 1976) of a toad observing a worm through a semi-transparent paling fence. Instead of launching directly at its prey as would occur if no barrier is present, the toad reacts appropriately, detouring around the barrier (as we see in progress in the fourth frame) to get its prey. However, if no worm is present, the animal does not move. Hence, it is the worm that triggers the animal’s response, but when the barrier is present the animal’s trajectory to the worm changes in a way that reflects the relative spatial configuration of the worm and the barrier. 2 OI2D % i .*'• J'/y arc m v* t: P Figure 1.1: When a toad sees a worm behind a barrier, it often detours rather than approaching the worm directly (Ingle, 1976) Fig. 1.2 shows the results of several of Collett's experiments where he presented toads with various prey-with-barrier configurations. When prey is at 12cm behind the barrier, the toad shows 75% detours and 25% direct approaches as illustrated in Fig. 1.2a. As shown in Fig. 1.2b, if a barrier with a passable gap is introduced, the animal usually aims to the gap (90%). When a double barrier, constructed from the single fences of Fig. 1.2a and Fig. 1.2b, was presented as in Fig. 1.2c, the toad showed a response that was nearly an average of the two responses, with detours on about 50% of the trials and the other 50% directed to the gap in the front fence. As the two constituent fences are brought closer together, the toad detours around the barrier more often. When the gap in the front fence was filled as in Fig. 1.2d, most of the toad’s approaches were detours (94%). Collett’s data further show that toads possess the ability to visually detect sophisticated spatial arrangements of two fences, gaps, and a prey. 3 ' Z'' Figure 1.2. Approaches to prey with single and double barriers interposed. In each case, the “hammer” indicates the toad's starting position and orientation, the rectangle of “squiggles” shows the position of the worm, the filled circles show the fence, and the arrows summarize the directions of the approaches on repeated trials. The results are given in terms of the percentage of approaches made in a particular direction, (a) Single fence with prey 12cm behind, (b) Single fence with 6cm gap and with prey 22cm behind, (c) Double fence composed from single fences of a and b. (d) Double fence with no gap. (e) Approaches to prey behind the “cage.” Cage is formed from double fence with palings connecting the ends. (Collett, 1982). 4 1.2. Overview of Anuran Anatomy We summarize anuran brain regions that are involved in such visually directed behaviors as detour behavior, and the information flow between them, in Fig. 1.3. Light patterns impinging on the two eyes are first preprocessed at retina level. Optic fibers that convey retina signals to higher visual centers in anuran brain project to contra-lateral optic tectum (anuran midbrain region homologous to mammalian superior colliculus) and to other areas in the thalamic midbrain such as pretectum. Because anurans do not have visual cortex, the retino-thalamic pathway does not continue on to the visual cortex. In mammals, superior colliculus is implicated in the novelty detection that mediates attention mechanisms such as saccadic eye movement, while general higher level visual processing is attributed to the Retina->LGN-»Visual CortexInfero-Tem poral/Posterior-Parietal pathways. Anuran tectum, together with signal from the (caudal thalamic) pretectum, plays a similar role to superior colliculus in that it is involved in the recognition and initiation of an action for moving stimuli such as prey or predator that require an immediate attention. A tectal cell called T5-2 neuron shows worm sensitivity and is suggested to act as an integrative sensory filter by appropriately combining the worm-discriminatory signals from the other feature-sensitive tectal and pretectal cells in the form of an interacting network (Ewert, 1987). Neural network models of prey (Cervantes-Perez et al., 1985) and predator (Liaw & Arbib, 1993) recognition have been presented. Unlike the perception of a moving object like prey or predator where cooperation of tectum and pretectum is needed, the pretectum appears to be the only 5 Barrier-Detour Barrier S i l M S S i i - f t * " * * 8 > Retina ■ R1: Scacic edge R2: moving convex R3: moving edge R4: local dark blot Striatum Memory of stationary object )rized ier Pretectum Center for SO perception and localization ThlO: excited by stationary object Tectum Center for moving object perception and localization Isthmic Region Tegmentum Center for sensory integration and coding transformation Left Turn Motor heading map Right Turn Motor Heading Map Medulla/ Spinal Cord Direction of actior is represented Figure 1.3. The overall schematic illustration of the anuran brain regions and pathways involved in visually directed behaviors. Visual signals from retina project to optic tectum and pretectum. Tectum, together with signal from the pretectum, detects and initiates an action for moving stimuli such as prey or predator. Pretectum is responsible for the perception of stationary objects like barriers. Sensory information from the tectum and pretectum projects to tegmentum and medulla for data integration and generation of motor signals. For more details, see the text. 6 main region involved in anuran SO perception since tectum-lesioned frogs do not seem to have difficulty perceiving SOs (Ingle, 1983). The pretectum receives both direct and indirect visual information from retina and via tectum respectively. The retinal projection to pretectum is topographical and bilateral (Griisser & Griisser- Cornehls, 1976). Ewert (1971), Ingle (1976), and Brown & Marks (1977) have found neurons in pretectum that are sensitive to SOs, but because the stimuli they used were relatively simple (black squares or disks with varying sizes), it is difficult to extrapolate anuran SO perception to biologically significant objects from the physiological data. However, anuran physiology, anatomy, and behavioral studies suggest that anuran SO perception and the number of biologically significant SO categories may be quite limited compared to those of mammals. The path taken in this paper for the modeling of anuran SO perception is (1) a bottom-up approach focusing on how an SO may be portrayed in the pretectum while being constrained by the retinal signal and physiological data of pretectal single cell recordings, but which is also guided by (2) top-down clues of how the neuronal SO picture “should” be based on our interpretation of anuran behavioral data in response to different SOs. With the recognition of different visual constituents of the environment, the animal should integrate the information to transform the results of sensory perception into appropriate motor commands. In a series of experiments, Ingle (1983, 1991a, 1991b) observed that anuran SO (including the barrier) signals from pretectum pass through different pathways than tectal efferents which convey prey or predator signals. The tectal efferents transmitting prey-information cross at the ansulate commissure in the ventral tegmentum and project to the contralateral 7 medulla. Severing the ansulate commissure abolishes prey turning but results in no deficit in predator avoidance, implying that the predator signal is conveyed by different pathways. It is very important to observe the flow direction of these efferent signals as exemplified by the following example: A prey in the right (monocular) visual field is projected to the left tectum via retino-tectal projection. The retinotopically coded prey signal in tectum efferents to the premotor area (tegmentum/medulla). This prey-conveying tectobulbar pathway crosses to the other side (right-side) at the ansulate commissure region. Grobstein and his colleagues (Grobstein, 1988; Masino & Grobstein, 1989b), however, claim that the tectobulbar pathway crossing at the ansulate commissure is not the main channel transmitting the prey signal but rather it is the tectum- tegmentum-medulla pathway which is primarily responsible for transmitting the prey signal. At the level of tegmentum, Grobstein (1988) observed that the prey- signal coming from tectum shows population encoding, meaning that the existence and the intensity of the signal on one side makes the animal turn ipsilaterally proportional to the signal intensity. Thus, somehow, the retinotopically coded tectal signal should be transformed to the population coded intensity signal before it reaches the premotor area. Grobstein and his colleagues also observed other characteristics of sensorimotor transformation, such as lateralization and parcellation of the tectal signal. The pretectal efferent pathway conveying the barrier signal is divided into two sub-pathways. One group projects ipsilaterally to the medulla while the other group crosses to the other side projecting to the contralateral medulla. Ingle (1983) showed that by transecting the crossing of the efferent pretectal fibers at the border 8 of tegmentum and medulla, frogs exhibited a profound and lasting deficit when confronted with barriers. The split does not affect normal prey orienting nor the threat escape. This experiment suggests that prey and barrier signals do not interact or get integrated before the medulla. One of the aims of this thesis is to model anuran sensorimotor transformation processes that integrate tectal and pretectal signals and generate motor commands, taking Grobstein’s data into consideration. 1.3. Anuran Detour Behavior and Robot Path Navigation Anuran detour behavior offers a valuable biological metaphor for the field of robot path navigation, while at the same time, robotic methods provide animal study with computational tools helping the experimentalists to better formulate animal behavior in analytical terms so that they may better identify variables or a drive behind the behaviors they are observing. Thus, study of such animal behavior as anuran detour modeling and research in robot path navigation can benefit one another, fostering greater advancements in both fields of study. As will be examined in detail in Chapters 2 and 5, anuran detour behavior suggests that the animal is acting reactively to a limited subset of obstacles (paling fences) at a given time rather than basing its path selection upon all the available data it can gather. The animal’s reactive behavioral pattern does not seem to be due to a problem in data acquisition, but in the way it employs the data to determine an action. In this respect, the animal behavior closely resembles a mobile robot that determines its path to a goal following a force field created by an artificial potential field (Khatib, 1986; Arkin & Murphy, 1990). 9 While the potential field method offers convenient computational tools to describe some of the anuran behavior, anuran behavior also provides some interesting ideas for robot path navigation. From our observation of the animals, we see clues that the animals learn to interpret an object (barrier gap) differently as their experience with the object accumulates. That is, a naive frog tends to aim at the barrier to approach a prey behind the barrier at first, but as its negative experience with the barrier grows, the animal shifts its strategy from direct approach to detour. We believe this learning aspect can be utilized by the artificial potential field method in the form of adaptive modification of the potential field. Much like the frogs that learn to detour around a barrier through its negative experience of not being able to pass through the it, a potential field based robot can raise (lower) or expand (shrink) the potential field associated with an object/place based on its unfavorable (favorable) experience with the object/place during its attempt to reach a goal. 1.4. Problem Statement How do frogs and toads-without visual cortex and having retinas oriented to perceiving movement-gated stimuli-perceive stationary objects? What is the nature of prey and barrier signals and how does the animal use this information to choose where to detour? If a target is selected, how does the animal generate appropriate motor commands? In this thesis we trace and model the information processes required when an anuran exhibits detour behavior in its stationary surround. More specifically, we 10 seek to unravel the mechanisms behind stationary object recognition, depth perception, integration of motor signals, and the mechanism that transforms the result of sensory perception into appropriate motor commands. Ethological data offer high level constraints on how our final model should behave. On the other hand, neurophysiological data of the relevant cells provide the front-end with the lowest level constraints. Between these two constraints, there are lots of gaps that need to be filled, providing the topics this thesis tries to shed some light on. We also investigate the significance of learning component in anuran detour behavior and how this may tie in with robot path navigation. 1.5. Contributions of this Research This dissertation makes a number of contributions to the study of anuran visuomotor processes at the neural and schematic level, and the related issues of robotic navigation. 1) An anuran retina model is developed from a populational point of view. Rather than questing for the neuronal model of a given type, more emphasis is laid on the variation of response properties in a population of the same class to find new capabilities of the type. A populational approach to grouping R1 and R2 class neurons as a single class is proposed. The model simulates the gradual response transition between the R1 and R2 class neurons. A new capability for R3 class moving-edge detection neurons is investigated, linking it with capability of detecting static-edges through the edge’s retinal movement made possible by the animal’s 11 own movements. A new formulation of R4 class neurons is proposed. The model can account for some responses of the biological cells that were not reproducible with earlier models. It also raises some interesting hypotheses that need to be tested. 2) A neuronal model of anuran stationary object recognition is presented. Based on the aforementioned retina model, an internal representation of the stationary object is portrayed on a population of pretectal neurons. Pretectal neurons thought to be involved in stationary object recognition are modeled taking into account neurophysiological data and the effect pattern interpretation has on the animal’s behavior. A depth perception model is presented that utilizes disparity and accommodation cues from the internal representation of the stationary objects. A neuronal model of anuran spatial short term memory is proposed in the form of a recurrent shunting feedback loop, and its significance in overall anuran SO perception is discussed. 3) Schema-based models of anuran sensorimotor transformation process, data integration, and detour process are presented. Data integration is posited to take place after the sensorimotor transformation process. That is, the data integration takes the form of motor integration rather than sensor fusion. Besides conforming to anatomical data, the integration model raises some interesting predictions, which are elaborated upon in Chapter 5. The detour model is constructed as an assemblage of perception, sensorimotor transformation, and data integration schemas whose end- goal is to direct the animal appropriately to a prey in a given surrounding. Information flow of the model is basically parallel and feed-forward from the 12 perception to the data integration module, at which final motor action is determined through cooperation and competition. 4) New data on anuran detour behavior is presented. We show some evidence that suggests that some form of learning is an integral component of anuran detour behavior. We believe the animal is learning to characterize a visual object differently through experience. We then assimilate the animal learning experience to mobile robots to propose an adaptive modulation scheme of the artificial potential field for robotic path navigation. We show how the shunting self excitatory loop can modify the potential field, reflecting an agent’s learning experience. 1.6. Organization of the Dissertation Chapter 2 surveys experimental data and models that are relevant to the work presented in this thesis. We review anuran lesion/section data, models of anuran retina, prey/predator recognition, detour behavior, and robotic path navigation. The chapter concludes with a critique of the models and a statement of the directions this dissertation will take. Chapter 3 deals with the retina models. We present a population model for R1 and R2, investigate new characteristic responses of R3 to stationary objects, and propose a new model for R4 type retinal ganglion cells. In addition to being as capable of reproducing many of the characteristic responses as earlier models, by adoption of population contiguity within and across the traditional cell classification boundaries the proposed models offer a richer repertoire of responses to diverse stimuli such as 13 stationary objects. The retina models thus lay the ground for the anuran SO perception model that uses the retina signal as its input. Chapter 4 proposes a neuronal model of anuran pretectal stationary object perception based on the retinal signal furnished by the retinal models in Chapter 3. This chapter analyzes physiological and anatomical data related to anuran stationary object perception to construct single cell models of the pretectal cells. Then, through comparison of portrayals of different SOs painted on the two-dimensional layer of pretectal cells with an animal’s behaviors to the same SO, a pattern recognition strategy for the animal is proposed. We posit that depth estimation of an SO forms an integral component of SO recognition. To this end, we present an enhanced version of an earlier anuran SO depth estimation model. The chapter concludes with an SO memory model with an account of its significance in the overall anuran SO perception process. Chapter 5 presents a schema-based model of how frogs/toads select their route to a prey depending on their visual perception of a 3-D world and transform the result of visual perception into appropriate motor commands. We examine how the visual information encoded in retinal coordinates in tectum and pretectum is transformed to body-centered (population coded) intermediate representations and gets integrated in the premotor area (tegmentum or medulla) to generate coordinated motor patterns. Based on anatomical and ethological data, we present a model of the sensorimotor transformation network underlying anuran detour behavior. 14 Chapter 6 discusses new data on anuran detour behavior from our lab. A learning experience during the detour behavior is identified and modeled. We then show how this learning can be adapted to robotic path navigation in the form of adaptive modification of an artificial potential field. Chapter 7 provides a concluding discussion, and offers directions for future research. 15 CHAPTER 2 LITERATURE REVIEW This chapter summarizes some o f the experimental studies and models relevant to the models presented in this thesis, and presents the relationship between the thesis and previous work. Lesion studies offer important clues as to the functional role the lesioned site may have assumed. Lesion studies coupled with behavioral experiments are valuable tools in constructing models o f animal behavior at the level o f functional units called schemas. Once a schema network is formalized, then neural network implementation o f a schema may be posited based on anatomical/physiological data about the region. After the brief review o f the lesion/behavioral studies, we examine past modeling work. In the second section, previous models o f anuran retina and tectum are reviewed. These neural network models take into account data concerning the physiology and connection patterns o f the cells comprising the biological counterparts. In the fourth and fifth section, an anuran depth perception model and detour models are reviewed. These schematic models are constrained by the data o f nuclear level anatomy and the functional role these nuclei may assume as suggested by lesion studies. We then look into sensorimotor transformation whereby sensory information is translated into appropriate motor signals fo r anuran approach and avoidance behavior. In the last section, we review some computational models o f navigation. 16 2.1. Lesion Studies 2.1.1. Tectum Lesion After ablation of the entire tectum, both visual prey-catching and predator-avoidance behaviors fail to occur (Ewert, 1968; Ingle, 1970), although some particular avoidance patterns, such as ducking or sidestepping can still be observed when the pretectal surface is stimulated (Ingle, 1983). The lesion studies suggest that the tectum is the primary region for the detection and initiation of action with regard to visual objects like prey and predators. 2.1.2. Pretectum Lesion Ablation of pretectum results in a “disinhibition” of prey catching behavior in which the animal snaps at objects much larger than prey that normally would have elicited avoidance behavior (Ewert, 1968). Also, the atectal animal was unable to avoid predators or collision with a static obstacle like paling fence barrier. The lesion experiment demonstrates that the region is involved in prey-predator discrimination through its inhibitory action on the tectum, and also is involved in stationary object perception. 2.1.3. Striatum Lesion Ingle and Hoff (1990) showed that frogs possess an ability to remember the location of a static barrier. After a frog {R.pipiens) is shown a vertically striped barrier for 17 about 5 seconds and the barrier is suddenly removed, when the animal is presented with a threat stimulus, the animal avoids jumping toward the former barrier location even though the location may be the optimal direction for escape. They observed that the memory lasts about 1 minute. They further demonstrated that the memory depends on the striatum because striatum lesion abolished the memory of the static barrier, resulting in jumps to the former barrier location. 2.1.4. Unilateral White Tract Hemisection Just Caudal to Tectum After a lesion of unilateral white tract at the ventro-medial midbrain just caudal to the tectum, Grobstein and his colleagues (Grobstein & Masino, 1986; Grobstein, 1988) observed entire ipsilateral hemifield deficit (not visual scotoma). That is, with left tract section, frogs respond to prey in the affected left hemifield but the movements are always forwardly directed regardless of (horizontal) eccentricity. Movements do vary with stimulus elevation and distance in the affected ipsilateral hemifield. However, the variation with stimulus distance in the affected hemifield is different from that in the opposite unaffected hemifield: In normal conditions, a frog snaps when a prey is nearby, but reorients and hops toward a prey that is further away. The transition from snap to hop occurs at a distance of one body length for a lateral prey and two body lengths for a frontal prey (Grobstein et al., 1985). However, the lesioned frogs show the transition boundary at two body lengths to the stimulus located anywhere in the affected hemifield (Kostyk & Grobstein, 1987a), the same as if the stimulus had been located in front, which is also the same direction to which the animals always advance. It is obvious that the frogs do not 18 experience a general motor deficit because the hemisected animals show spontaneous turns in both hemisphere. Based on the data, Grobstein suggested that at the level of the lesion site (tegmentum), the tectal signal conveying the prey location shows lateralization and parcellation traits: The horizontal eccentricity signal conveying the prey location of one visual hemifield runs through the ipsilateral body side but not the contralateral side (lateralization); Different components of the prey location (eccentricity, elevation, and distance) are handled through different pathways that can be disturbed independently (parcellation). The prey location is represented relative to the animal’s body (body-centered) which is different from its retinotopic representation in the tectum. 2.1.5. Bilateral Ventral Lesion at the Midbrain/Medulla Junction Grobstein & Staradub (1989) studied the effect of bilateral ventral lesion at the midbrain/medulla junction. Large ventral lesions yielded a slight forward movement with no variation with eccentricity or distance of the prey. With smaller ventral lesions, frogs responded to prey with a turn that was appropriate, but with an overshoot for distance that increased with the size of the lesion, suggesting that the signal in the pathway is population (frequency/intensity) coded transmitting the “closeness” information. With this information, Grobstein concluded that the tectal signal conveying the prey location at the caudal midbrain level shows lateralization, parcellation, and populational encoding in a body-centered coordinate system. Grobstein also reports 19 that somatosensory and tectal prey information are integrated at the level of midbrain tegmentum. 2.1.6. Localized Lesions of the Tegmentum Nuclei When the anteroventral nucleus in the tegmentum was lesioned, Ingle (1991a) observed that contraversive jump in frogs was abolished and all the jumps were directed toward the incoming stimulus from all directions. On the other hand, after ablation of the posteroventral nucleus in the tegmentum, the frogs jumped to the contralateral visual field when looming objects were presented from all directions. This behavior contrasts to the “cut-back” ipsilateral jump the normal frogs exhibit when the looming stimulus is not on a collision course but crosses the view field. This observation further reinforces the idea that tegmentum plays an important role in relaying, modulating, and integrating the tectal signal conveying stimulus information. 2.2. Anuran Retina Models 2.2.1. Experimental Data The vertebrate retina is one of the best understood neural systems. A schematic connection diagram showing all six major cell types (photoreceptor, horizontal, bipolar, amacrine, interplexiform, and ganglion) is shown in Fig. 2.1. Synaptic interactions occur primarily in two laminae: Photoreceptor, bipolar, horizontal, and 20 interplexiform cells interact in the outer plexiform layer (OPL); bipolar, amacrine, interplexiform, and ganglion cells in the inner plexiform layer (IPL). o \° o lo 58, FB Figure 2.1. Summary diagram of vertebrate retinal organization. RT~Receptor Terminals. H_Horizontal Cells. IB~Invaginating Bipolar Cells. FB -Flat Bipolar Cells. A-Amacrine Cells. G l, G2, G3“Ganglion Cells. IP-Interplexiform Cells. (Dowling, 1987) 21 Photoreceptors are divided into rods and cones, and they act as transducers, converting absorbed light into changes in hyperpolarizing membrane potential. Bipolar cells are feed-forward cells that transmit receptor signals from OPL directly or indirectly to the dendrites of the ganglion cells in IPL. Horizontal cells connect almost exclusively within the OPL with their large laterally spread processes. Their main task is to collect input widely from receptors, and provide their outputs to receptors with feedback and to bipolar dendrites. Amacrine cells connect exclusively within the IPL. Amacrine processes collect input from bipolar axons and other amacrine processes and provide output to ganglion cells, other amacrines, and bipolar cells. Interplexiform cells receive synapses on its IPL processes and provide synaptic outputs in both the IPL and OPL. While its connection pattern makes the interplexiform cell well suited to transmit information centrifugally from IPL to OPL, its functional significance is not well understood. Ganglion cells are the retinal output whose axons form the optic tract projecting to higher visual centers in the brain. Traditionally, the retina ganglion cells in frogs and toads have been classified into four (R1-R4) different types (Lettvin et al., 1961; Gaze & Keating, 1970; Griisser & Griisser-Comehls, 1976). Here we briefly summarize basic properties of the ganglion cells and will present more details as modeling progresses in Chapter 3: Class R1 neurons: With an ERF of 2-4° oval shape, these cells respond to any boundary between two shades of gray in their receptive fields. The response is most prominent to small (< 2°) moving contrast stimuli that lie within the ERF. The cell continues to fire even when the object is stationary within the ERF. The 22 cell does not respond to general illumination changes. The response due to the IRF is such that it does not totally suppress the ERF response. Due to the characteristics above, these neurons could play a role in detecting stationary objects. While frog (Rana) and tree frog (Hyla) are known to possess this type of cell, sufficient evidence is lacking that toads (Bufo) have this.1 Class R2 neurons: With an ERF of 2.5-5° and a large IRF (20-45°) surrounding the ERF, this cell is strongly activated by a small contrast object moving into the ERF. When a stimulus is stopped within the ERF, many R2 cells continue to discharge for several seconds or even to a minute. Turning off the general illumination immediately interrupts this after-discharge, which does not reappear, unlike R ls, when the stimulus is reilluminated. Similarly to R3 neurons, R2s adapt to repetitive stimulus movement within the ERF. While most R3 neurons continue to respond at a lower activity level, the neuronal adaptation of R2 leads to a complete cessation of the activation. Class R3 neurons: With an ERF of about 8°, these cells can be simply described as moving contrast detectors (Maturana et al., 1960) or On-Off neurons (Hartline, 1938) because they show transient responses to changes in the general illumination. These cells are prominently activated when a stimulus of the size of their ERFs crosses the ERF, but also show a response if a large contrast boundary crosses both their ERF and IRF. The response strength is similar when a dark 1 However, Guiloff (1980) reports that she found R1 type neurons from toad's (Bufo Spinulosus) optic nerve and from the tectum. 23 object boundary traverses the ERF against a light background or when a bright object boundary crosses the ERF against a dark background. Class R4 Neurons: These are dimming detectors. With a large ERF of 12-16°, these cells are excited when the general illumination on their ERF is decreased. Most of these cells show spontaneous discharge during the Off of general illumination (Fite & Scalia, 1976). This cell fires when a dark object is introduced to its ERF and continues to discharge when the object is stopped in the ERF. It appears that the cell's response is independent of the shape of the object. 2.2.2. M odels Anuran retina models have been presented by Schipperheyn (1965), Eckmiller & Griisser (1972), Moreno-Diaz, Rubio & Rubio (1980), Lee (1986), an der Heiden & Roth (1987), Teeters & Arbib (1991), and Teeters et al. (1993). Even though the models became increasingly realistic in time with incorporation of more biological data, still many details, such as the feed-back between the receptors and horizontals and the mechanism behind the function of amacrine cells, need to be understood. Nevertheless, an anuran retinal model such as Teeters & Arbib (henceforth called T&A) was successful in replicating many of the qualitative responses of anuran ganglion cells, and its refinement (Teeters et al., 1993) was able to account for the quantitative responses anuran ganglion cells show for stimuli of different sizes and shapes. Because of T&A’s comparative faithfulness to biological data and its success in replicating characteristic anuran retinal cell responses, it is used as the base for the retina models developed in Chapter 3. 24 The structure of the T&A model is shown in Fig. 2.2. The top part shows the layers of cells which feed all the ganglion cells, while the bottom part shows the specific inputs for each of four (R1-R4) ganglion cell types. Each single cell in these diagrams represents a layer of cells in the formal model. Since the retina model proposed in this thesis derive many elements from T&A, a detailed account of T&A will be presented in Chapter 3. 2.3. Tectal Column Model of Prey Catching Behavior Anuran prey orienting behavior depends on the form, size, and contrast of the prey stimulus (Ewert, 1987 for a review). Anuran tectum, together with signal from the (caudal) thalamic pretectum, plays a key role in the recognition and initiation of an action against a prey or predator. Several models of prey recognition have been proposed by researchers like Ewert & von Seelen (1974), Lara et al. (1982), and Cervantes-Perez et al. (1985). Fig. 2.3 illustrates the structure of the Lara et al.’s model. Each tectal column comprises one pyramidal cell (PY) as sole output cell, one large pear-shaped cell (LP), one small pear-shaped cell (SP), and one stellate interneuron (SN). All cells Figure 2.2. Top (a): Overview of model structure. Cell types are: R -R ecep to rs; HC-Horizontal cells, DBC & HBC_Depolarizing and Hyperpolarizing bipolar cells; PBD & PBH -positive part of bipolar cell potentials; ATD & ATH~transient amacrine cells from DBC and HBC channel; OPL~outer plexiform layer. IPL“inner plexiform layer. Bottom (b): Ganglion cells R1 (Class 1) through R4 (Class 4). The receptive field for ganglion cells type RO through R3 is composed of a small excitatory receptive field (ERF) and an overlapping larger inhibitory receptive field (IRF). The ERF and IRF in the R4 model are the same size. Input to both ERF and IRF are from bipolar and amacrine cells (PBD, PBH, ATH, ATD). Spatial connections and other details of the algorithms are not shown here but are given in the text. (Teeters and Arbib, 1991) 25 I R ectification DBC High-pass . filte r HBC PBD / a t B n /A T H N PBH on-S \ on-T J \0 ff-T j off-S T (^""G a n g lio n Cells OPL Processing IPL Processing a. Overview of Model Structure (b) Class 1 PBD PBH on-S off-S / A T ^ / a t h N V o n-T^ ^ o ff-T / ? *w-J iSt ** (d) Class 3 '/ATD' AT IT o ff-T o n -T (delaw delaw b. Overview of Ganglion Cell Models 26 are modeled as excitatory except the stellates. The retinal input to the model is a lumped “foodness” measure, and activates the column through glomeruli with the dendrites of the LP cell. LP axons return to the glomerus, providing a feedback loop. A branch of LP axons also goes to the inhibitory SN cell. There is thus competition between “runaway positive feedback” and the stellate inhibition. PY is excited by both SP and LP, and requires sufficient activation from both of them to trigger a motor response. r G lom erulus SN Stellate k neuron r Small V . pear cell { V/ O utput f LP Large pear cell A PY Pyram idal cell Tectal colum n Figure 2.3. Neurons and synaptology of the tectal column model. The numbers at the left indicate the different tectal layers. Each column is constituted by one glomerulus (GL), one large pear-shaped cell (LP), one small pear-shaped cell (SP), one stellate neuron (SN), and one pyramidal cell (PY). For more details, see the text. (Lara et al., 1982) 27 Cervantes-Perez et al. extended Lara et al’s work by using an 8 x 8 array of tectal columns, receiving inhibitory modulation from a pretectum modeled by an array of Th3 cells receiving R3 and R4 input with synaptic connections tuned to make it act like an “antiworm” filter. With appropriate setting of the various masks and parameter values, the model exhibits qualitatively a good match to the neural data. Cervantes-Perez & Arbib (1990) presented an analytical model of the tectal column model to better understand the parameter dependency of the tectal column and to construct a more detailed model that would enable for their predictions. 2.4. Anuran Depth Perception Model By use of prisms and lenses, Collett (1977) showed that toads use both binocular and monocular cues to determine depth. Collett found that while binocular toads utilize binocular cues predominantly (94%) when estimating the depth of prey in the binocular field, nevertheless their depth estimation does include a small (6%) effect from monocular accommodation. Deprived of binocular cues, monocular toads still retain an ability to judge distance through accommodation cues. Thus, data exist about frog/toad’s depth perception for prey. However, very little is known about how the animal might estimate the depths of SOs. House (1989) offered a depth model, the Cue Interaction model, that is based on cooperative computation between two schemas, monocular layer (M) and stereoscopic layer (S), where each schema builds its own depth map of the visual field but they cooperate with each other to compensate for the deficiencies of each. The model takes into account physiology and anatomy specific to frogs/toads which 28 says that the model: i) should not depend on eye vergence, ii) must account for the use of an active process like accommodation, iii) must not depend on the tectal map, and iv) must be able to operate on sparse images, because the visual maps cannot be assumed to represent local illumination level since the visual input from the retina is feature encoded. The accom m odation driven M field receives inform ation about accommodation (the sharper the image at a particular depth in a given direction, the greater the activity of the neuron corresponding to that location), while the stereoscopic input driven S field bases its depth calculation to the Dev-Marr (Dev, 1975; Marr and Poggio, 1976) type of cooperation-competition algorithm. The initial state of the accommodation field is blurred, representing the lack of fine tuning offered by accommodation. Targets are better tuned in the stereopsis field, but they include ghost images in addition to the correct images. However, as the process goes on, due to the intercoupling between the M and S fields, a point in the M field will excite the corresponding point in S and vice versa. As a result, ghost targets are suppressed while accommodation is sharpened. The model demonstrated its ability to estimate the depth of a single object (prey or predator) and a single paling fence. It also predicts that concave lenses will have a consistent effect, with stronger lenses causing a fence to appear closer to the animal. Figure 2.4. Time course of the cue interaction model from its initially inert state (a) to a satisfactory depth segmentation (f). All figures are in the retinal angle verses disparity coordinate system. The two-dimensional grids show the level of excitation of the various fields, and the line graphs under the grids indicate the intensity and localization on the retina angle axis of excitation in the inhibitory pools. Depth segmentation in the initial state of accommodation field, M layer in (b), is blurred representing the lack of fine tuning offered by accommodation cue. Depth segmentation is sharper in the stereopsis field, S layer in (b), but includes ghost images in addition to the correct images. As the process goes on, with the cooperation between the M and S field, depths of the paling fence barrier and a pair of worms become better segmented. (House, 1989) Pr *y b a r r ie r p re y s \r \ 30 Prisms, on the other hand, should have a more variable effect, even causing the fence to be fragmented, so that the animal may occasionally behave as if there were gaps in the fence. Fig. 2.4 shows the model’s simulation of a paling fence barrier and a pair of worms. 2.5. Anuran Detour Models Arbib & House (1987) presented a model of anuran detour behavior based on the potential field method. Here, a prey sets up a radially symmetric attractant field whose strength decays gradually with distance from itself. A barrier sets up a repellent field whose effect is more localized than that of prey. The barrier field is not radially symmetric but has a lateral component that is stronger but decays more rapidly with distance than does its rostro-caudal component. Associated with a toad itself is a radially diverging field that represents the animal’s “urge” to move. When prey, barrier, and toad are present simultaneously, the interaction of the associated fields determines the animal’ s path. The model is computationally adequate in that it provides a parallel computation scheme for converting the perception of prey and barriers into the parameters that characterize an appropriate trajectory. The model was able to explain anuran detour behavior when the barrier configuration was simple as in Fig. 1.2a and Fig. 1.2b. Fig. 2.5 shows a path chosen by Arbib & House’s model. 31 \ * r , , * . . . . \ r * , * ~ . ' *-j-« 7 ) Figure 2.5. The potential fields depicted here represents an exploratory attempt at defining a set of primitive fields which will interact in interpreting a com plex scene. Each field provides, for each position in the ground plane, a vector showing the direction and “strength” o f a movement the animal might make were it situated at that position, (a) A single prey object sets up a radially symmetric attractant field whose strength decays gradually with distance from the prey, (b) A single barrier object sets up a repellent field whose effect is more localized to its point of origin than is that of the prey field. The barrier field is not radially symmetric but has a lateral component that is stronger but decays more rapidly with distance than does its opposing component, (c) The model also contains a representation of the animal itself. This representation is radially symmetric but diverges from its point of origin. (Arbib & House, 1987) On the other hand, as will be examined in detail in Chapter 5, Arbib & House’s model does not accord well with the biological lesion data, and also has problems in accounting for anuran detour behaviors if the barrier configuration is more complex as when the barrier is composed of two paling fences separated in depth. 32 Lara et al. (1984) provided a schema model for a wide range of toad behavior which shows how the toad’s response to prey is modified by intervening chasms as well as barriers. The model postulates perceptual schemas for gaps which provide a target for detour behavior, rather than having this behavior driven by inhibition/repulsion derived from perceptual schemas for barriers. Ballistic movement is obtained in their model by a sequence of several schema activations. Fig. 2.6 shows their schematic model and simulation results. Figure 2.6. Top: Prey Acquisition Master Schema. If a prey is in the visual field of the animal, the orient motor schema, MS, is activated. Then three perceptual schemas, PS, start to compete to control the motor response: The schema that senses prey with no obstacle (FP), the schema that senses prey behind a barrier, and the schema that senses prey behind a chasm. If the prey- no obstacle S is selected, then depending on the prey’s distance (d) the approach or snapping S will be activated. If the prey distance is small (d-), then the fix, snap and swallow motor schemas (shown as a hexagon) are activated. If the prey distance is long, then the approach MS is activated and control is returned to FP-PS until it reaches the goal state. In case the prey- barrier S is activated, then depending on the prey-barrier distance (dpo) and height of the barrier (h) three schemas compete to take control of the motor response. If dpo is short (dpo-), the approach-barrier S is activated and control is sent to the prey-with-no-obstacle PS to activate the snapping schema to reach the goals state; if dpo is long and h is small (h-), then the jump barrier schema is activated and control is returned to the prey PS to reinitiate the process. If dpo is long (dpo+) and the height is big (h+), then the detour schema is activated This schema first searches for the activation of a gap PS. If there is not a gap, then control is sent to either of the other schemas. If a gap is present, then the orient and approach gap MS are activated and control is returned to the initial PS. In case the prey-chasm PS is activated, then depending on the width (w) and depth (d) of the gap, three schemas can be activated. If the chasm is shallow (d-), then the cross chasm S is activated; if the chasm is deep but short (d+, w-), the leap S is activated; and finally if the chasm is deep and long (d+, w+), the go-away schema is activated which is a failure exit of the prey-acquisition S. Bottom: Computer simulation of toad’s behavior in the presence of barriers and the change in the trajectory depending on the gap’s depth. In (a) the toad always prefers the closer gap but if the farther gap is deeper (b), the choice is for the second gap. (Lara et al., 1984) 33 9 dpo* . h+ dpo- A p p ro K h ' p s > Gao Gap Approach I < * > I • 4 53 A t « 3 33 A o ' A 34 2.6. Models of Anuran Sensorimotor Transformation Our knowledge of the sensorimotor system, its connectivity and the functions of its components, is still very limited. The situation makes the effort of modeling more important in providing insights for studying and understanding this complicated system. To this end, Cobas and Arbib (1992) developed a schema model for prey capture and predator avoidance showing how the toad's brain may construct behavior via a process of competition and cooperation where two or more motor schemas may be active simultaneously and cooperate to yield a complex motor pattern. In general, an animal exhibiting visually directed behavior should detect, localize, and label stimuli in the 3-D world, decide on and convey appropriate motor action signals based on perception, and the conveyed motor signal should be integrated with other signals such as posture information, presumably in a central and abstract representation, to generate appropriate coordinated action. Based on lesion studies in frogs, Grobstein and his colleagues (Kostyk & Grobstein, 1987a-c; Grobstein, 1988) observed that the tectal signal conveying the prey location at the caudal midbrain level shows lateralization and parcellation traits and is population (frequency) coded: The horizontal eccentricity signal conveying the prey location of one visual hemifield runs through the ipsilateral body side but not the contralateral side (lateralization). Different components of prey location (eccentricity, elevation, and depth) are handled through different pathways that can be disturbed independently (parcellation). The prey location is represented relative to the animal’s body (body-centered) which is different from its retinotopic representation 35 in the tectum. Grobstein reports that somatosensory and tectal prey information are integrated at the level of midbrain tegmentum. Cobas and Arbib distinguished two alternative hypotheses in relation to Grobstein’s data: i) The Positional Heading Map hypothesis: Both prey and predator recognition systems have the same projection to the heading map, which codes the position of the object relative to the toad's body, so that the required motor behavior arises from a different connection to the motor schemas depending on which system is active at the moment, ii) The Motor Heading Map hypothesis: Each system has a separate projection pathway that converges in a different way onto the heading map which codes the direction of the required motor response, so that the map has a single connection pattern to those motor schemas common to both systems. Because in the prey approach, the heading of the prey and the approach direction are identical both heading maps can account for the animal behavior. However, in the predator avoidance behavior where the predator heading and the avoidance direction are different (Ingle, 1983), the Motor Heading Map is better suited to explain the animal behavior. Fig. 2.7 illustrates the schematic description of the Positional and Motor Heading Map hypotheses. Figure 2.7. (a) Layout of a Positional Heading Map used for both prey acquisition and predator avoidance. The connections corresponding to the ipsilateral binocular field would terminate in the same loci as the homologous ones in the other tectum. The map codes the position of the stimulus in the world, no matter whether it be prey or predator. The divergence between both systems occurs between the map and the Orient motor schemas, (b) Layout o f a Motor Heading Map applicable for both prey acquisition and predator avoidance. Only the connections associated with the contralateral visual field in the binocular regions of each tectal hemisphere are represented. Those of the contralateral tectum would terminate in symmetrical locations. The connections corresponding to the ipsilateral binocular field would terminate in the same loci as the homologous ones in the other tectum. The map codes the direction of the appropriate turn for prey acquisition or predator avoidance. The divergence between both systems occurs between tectum and the map, and so its connections with the Orient motor schemas can be the same for both systems. (Cobas & Arbib, 1992) 36 p o s it i o n a l , h e a d i n g m a p FOR A PPROA CH AND AVOIDANCE BEHAVIORS 90® right 'ighr tectu m S p o tio ily -co d ed pothw cys F re q u e n c y -c o d e d p ath w a y s Prey A cquisition S y stem P re d a to r A voidance S y stem L E F T FRONT POSITION AL HEA DING M A P RIGHT p ro jec tio n w e ig h t/d e n s ity * popuiorion co d e O R IEN T L E F T O R IE N T RIGHT m otor schem os le ft tu rn m u sc le s right tu rn m u sc le s 9 0 ° left 90® right 0 ° front MOTOR HEADING M A P FOR A PPR O A C H AND AVOIDANCE B E H A V IO R S left tectu m right tectum s Y ( y ' /• / / ' / \ ' ' / V ' i / / / A s s / * \ i / / / / i 1 x / / / \ \ \ O R IE N T RIGHT L E F T FRO NT Rig h t S p a tto ily -c o d e d pathw ays F re q u e n c y -c o d e c c c rh w a y s P rey A cquisition System (tu rn toword s tim u lu s) P redator A voidance S y ste m (tu rn aw o y fro m stim u lu s) MOTOR HEADING MAP p ro je c tio n w e ig h t/d e n s ity = s p o p u la tio n co d e m o to r s c h e m a s left tu rn m u sc le s right tu rn m u scles 37 Liaw and Arbib (1993) presented a neural network model of how various regions of the brain cooperate to produce motor patterns for avoidance behavior. In favor of the Motor Heading Map hypothesis, Liaw and Arbib proposed a basis for integrating signals from multiple sensory circuits in the following way (Fig. 2.8): Signals from different sources converge onto the map and interact or compete with one another, and an appropriate heading direction can be obtained through proper coordination of their interaction. In looming avoidance, the T3 looming detecting neurons which encode the stimulus position project to the ipsilateral map to indicate the escape direction to the opposite side of the visual field from which a looming stimulus is approaching on a collision course; whereas the T2 neurons detecting a crossing stimulus project excitatory signals to the contralateral map and inhibitory T2 Neuron left . right -j T3 Neuron left right k k left right Motor Heading Map Cut-Back ---------- Collision Figure 2.8. Integration of multiple signals through the gating of the tectal projection onto the Motor Heading Map. Only half the projection are shown. The T3 neurons project ipsilaterally to the heading map to indicate the escape direction for a looming object on a colliding trajectory. When the stimulus is crossing the visual field, the T2 signal blocks the T3 signal while exciting the contralateral heading map, thus resulting in a cutback ipsilateral jump. (Liaw & Arbib, 1993) 38 signals to the ipsilateral map to direct a cutback maneuver. When the looming stimulus is on a collision course (and hence there is no T2 modulation), the ipsilateral heading map will have higher activity and suppress the contralateral field. As a result, the frog will jump to the contralateral field. However, when the stimulus is crossing the visual field, the T2 signal inhibits the T3 signal while exciting the contralateral field heading map, thus resulting in a cutback jump. 2.7. Robot Navigation In the mobile robotics circle, the potential field method for path planning has gained popularity in part due to its simplicity and elegance. In this approach, obstacles exert repulsive forces onto the robot, while the target applies an attractive force to the robot. The resultant sum of all forces determines the subsequent direction and speed of travel. Arkin (1989) and Arkin & Murphy (1990) presented robot path navigation method based on the potential field approach. Their mobile robot, AuRA, uses schema-based architecture which is concurrently instantiable. Interaction of motor schemas and perceptual schemas determines the path. The individual schemas bear much similarity to the Arbib & House model. Warren (1990) used similar technique for coordinating the paths of multiple robots in the presence of obstacles. Tilove (1990) offers an overview of the potential field method, and Koren & Borenstein (1991) present inherent problems/shortcomings associated with the method. 39 2.8. Discussion Because the retina is among the best known systems in the anuran brain, it is also the most biologically faithful model. On the other hand, as was stated briefly earlier, much still needs to be done and we are finding new capabilities and ways to enhance the retina model through “evolutionary” updating as presented in Teeters et al. (1993). Traditionally, anuran retina research and modeling have emphasized retina cell properties when viewing moving stimuli. However, because the perception of a barrier is a very important perceptual requirement in detour behavior, Chapter 3 of this thesis presents retina models that will serve as inputs to the SO detection system. The retina models are not specifically constructed to better respond to SOs, but are more general models that not only match the earlier models’ capabilities but also can simulate some biological neuron’s responses which earlier models could not. The neuronal prey recognition model of Lara et al. and Cervantes-Perez et al. are capable of simulating the characteristic responses of anuran prey recognition. However, the structural notion of the tectal column is not rooted in firm ground, and we believe, for the time being, that modeling of prey-predator recognition should better be approached from a mostly functional aspect. Yet, notwithstanding the fact that the models did not use a realistic retina model for the input, the models do simulate prey recognition and so Cervantes-Perez et al.’s model is taken to be the prey recognition model of this thesis. 40 House’s schematic Cue Interaction model of anuran depth perception offers interesting predictions while also suffering from a shaky ground. The primary uncertainty of the model stems from the fact that most experimental data about anuran depth perception came from depth perception of prey. Collett’s (1977) finding that toads use binocular cues was derived from animals approaching prey, and Gaillard’s (1985) data that show there are tectal cells sensitive to disparity only apply to the depth perception of a prey. On the other hand, the hypothesis that pretectal SO depth perception is utilizing both accommodation and binocular disparity cues in a cooperative process raises some interesting prospects of finding disparity and accommodation sensitive cells in pretectum. And also, coupling between them posits a corresponding neural link in a form of excitatory feedback loop. One of the problems with House’s model was its inability to resolve the double barriers of Collett’s experiment. So, Chapter 4 presents a revised depth perception model based on cooperating Cue Interaction process that is also capable of estimating the double barrier, and this model is assumed to provide SO depth information. Two detour models (Lara et al. and Arbib & House) are basically a schematic account of anuran detour behaviors at motor/navigation level. As such, they do not account for the neural sensory processes nor provide a schematic description of how the sensory information may be transformed into coordinated motor patterns constrained by available biological data. Now with better understanding of the anuran sensorimotor transformation process w.r.t. a prey through the works of Ingle, Ewert, and Grobstein, we are at a stage where we can better formulate the model in neural terms. 41 Cobas and Arbib’s work is a step in the direction of providing a schematic account of anuran prey catching behavior that conforms to the nuclear level lesion studies. While data exist on the general information pathways and processing styles of the prey-related sensorimotor transformation, comparatively little is known about the SO-related pathways and its sensorimotor processes beyond the pretectum. One of the aims in this thesis is to present a schematic account of SO-related information processes after the pretectum, and posit how this information is integrated with other information to generate appropriate motor actions. Here, we follow the Motor Heading Map hypothesis of Cobas and Arbib in that not only do we posit that the tectal flow conveying the prey information encodes the prey motor heading, but also that the pretectal signal transmitting a barrier information encodes the motor heading of some selected target rather than the position of the barrier itself. The sensorimotor integration model proposed by Liaw and Arbib assumes a parallel, spatial topographic projection from tectum to the motor heading map, where multiple target motor heading signals can be received simultaneously and interact or compete in place. This does not accord well with Grobstein’s observation that the tectal signal coming into the tegmentum, where the heading map is assumed to reside, is population encoded. Unless the animal employs some form of temporal multiplexing, the parallel transfer of multiple target headings is infeasible. These are some of the issues that will be examined in Chapter 5. This thesis posits a scheme that allows tectal and pretectal signals conveying the prey and barrier target information to be received and integrated serially in the heading map. The serial processing does not incur excessive processing time because the computation 42 intensive sensory processes have already been finished in parallel, and candidate targets have been greatly reduced in number. 43 CHAPTER 3 ANURAN RETINA MODEL Any biologically significant neural model o f visual object recognition should be based on a correspondingly faithful retina model. This consideration is more important in the modeling/study o f the anuran SO perception process that is o f interest here because anuran retina is better adapted to perceiving a moving object. Data on anuran retina cell responses, and modeling studies w.r.t. stationary objects are comparatively scarce. In this chapter, we present new formulations o f anuran retina ganglion cells based on a populational approach to anuran cell classes. We test the models ’ responses to static stimuli as well as to better known characteristic stimuli such as moving stimuli and light changes. The proposed retina models furnish visual signals to pretectal SO perception mechanism presented in Chapter 4. The new retina model lies in the evolutionary path o f Teeters et al. (1993) which itself is a further advancement o f the Lee (1986), an der Heiden & Roth (1987), and Teeters & Arbib (1991) models. 44 3.1. Introduction 3.1.1. The Teeters & Arbib M odel The starting point of the retina model proposed in this chapter is the anuran retina model of Teeters and Arbib (1991) [henceforth referred to as T&A]. While full details of the T&A model are given in the aforementioned paper, we summarize the model here because the proposed model description is based on the conventions used by T&A. The structure of T&A’s retina model was shown in Fig. 2.2, and the summary of the equations for the model is given in Table 3.1. Receptors (R) convert light energy into neural potentials. The hyperpolarizing response to light is modeled by setting the receptor potential to the inverse of light intensity (i) which ranges from 0 (dark) to 1 (light). Adaptation and other complexities are not included in the model. Horizontal cells (H) form the surround receptive field of both bipolar cell types. They are modeled so that they are only sensitive to the background illumination of the surround (HO in Table 3.1) and are spatially invariant (uniform potential model) through the infinite spread of the activation within the cells. This simple interpretation of horizontal cell function ignores the effect of presentation of a local stimulus and uses them to bias the bipolar cells so they operate in their region of maximal sensitivity. Bipolar cells (HBC, DBC) are computed as a difference between receptor and horizontal cell activity. Hyperpolarizing bipolar cells (HBC) hyperpolarize in 45 Algorithms for T&A m odel base (receptors through amacrine cells) (a) Receptor R = 1 - i dH (b) Horizontal = ^ ! HO = 0 ambient light, 1 am bient dark; Tj^ = 0.1 O ff channel On channel (c) Bipolars HBC = R - H PBH = max(HBC, 0) DBC = H - R PBD = max(DBC, 0) (d) Amacrines -c a d(™ — -HBC-HBX: ! xa = 03 ATH, = max(HBC-HBX, ATH,.! • e 'A t/ta ) d(DBX) xa = DBC - DBX a d t ATD, = max(DBC-DBX, ATD,.! • e‘At/Ta) _____________________ Algorithms for T&A model of ganglion cells__________________ RO Cells R O = kOATD - k l * ((3 • ATH) + ATD) with kO = m ask(4,1.8, 1), k l = mask(15.5, 3.7, 0.8)__________________________________ R1 Cells R1 = kO * (PBD+PBH+ATD+ATH) - kl * (ATD+ATH) with kO = mask(3, 23,1), k l = maskU93, 4.6,3)___________________________________ R2 Cells The model uses two temporary variables (tc and g). ”tc” is the total transient input to the cell, "g" is a gate which is set to 1 if the net transient excitation is larger than the inhibition. tc = kO»ATH - kl * (ATH+ATD) g = pos(tc) where pos(x) = 1 if x > 0, 0 otherwise R2 = g • ((kO*PBH) + tc) with kO = mask(4, 2.4, 1), k l = mask(19.5, 4.6, 3)___________________________________ R3 Cells a = p • ATD + ATH R3 = kOa - (kl*a)delayed with p = 0.4, kO = mask(8, 2.4,1), k l = mask(19.5,4.6,1.4) while delayed ~ signal s delayed by 40 milliseconds. R4 Cells R4 = kO * (ATH - x • ATD) with x = 1, kO = mask(15.5, 3.5,1).________________________________________________ Table 3.1. 46 response to light, depolarizing bipolar cells (DBC) depolarize in response to light. PBH and PBD are the positive components of the HBC and DBC responses. Transient Amacrine Cells (ATH, ATD) convert the sustained bipolar outputs into transient signals. The transient amacrines are modeled as pseudo-differentiators which operate by subtracting the leaky-integrated bipolar potential from the sustained bipolar potential, and then amplifying the difference if it is above threshold. Bipolars and amacrines are modeled to have one-to-one connections from the preceding layers based on the following assumptions: (i) horizontal cells in this model have a uniform potential which in effect makes the spatial connection properties mostly irrelevant, and (ii) dendritic tree diameters of the Bipolars and Amacrines are smaller than those of the ganglion cells. Thus, the retina model input to the ganglion cells (receptors through bipolar and amacrine) ignores optics, different receptor types, light adaptation, and distinctions between other subtypes of horizontal, bipolar, and amacrine cells. Nonetheless, it accounts for a wide range of ganglion cell responses. Ganglion cells (R0-R4) receive input from bipolar and amacrine cells. Unlike the bipolar and amacrine cells which have one-to-one connections to their preceding layers, each ganglion cell input is composed of a central ERF (Excitatory Receptive Field), and a wider IRF (Inhibitory Receptive Field). The spatial properties of the ERF and IRF are specified as two dimensional Gaussians. The notation "mask(dia, sig, wgt)" in Table 3.1 denotes a 2-dimensional Gaussian with standard deviation sig (in visual degrees) which is truncated with diameter dia (so that the Gaussian values are replaced by 0 for points more distant than dia/2, also in visual degrees, from the 47 center), and which is normalized so that the sum of all elements is equal to wgt (for a more detailed description, see the “Methods” section). The outputs of the receptors, bipolars, horizontals and amacrine cells in our “leaky integrator” model are their graded membrane potentials whereas the output of the ganglion cells is their “firing rate”. The firing rate of a cell depends only on the membrane potential of each cell (each cell is modeled as a single compartment), which follows the differential equation * m M = _ m(() + s at where m(t) denotes the membrane potential of that cell at time t; rm is the time constant for the rate of change of this potential and Sm(t) is the total input the cell receives from other cells. For ganglion cells, a threshold function a (a(x) = x if x > 0, 0 otherwise) then converts m(t) into a firing rate M(t) = a(m(t)). To specify the overall structure of our model, we must specify how each term Sm(t) incorporates the input from all the other cells to which the given cell is connected. The notation B = W * A denotes a 2-D spatial convolution operation where the array of activity B is obtained by connecting array A to array B according to the connection weights in the mask W. Thus if layer m receives its input from layers of neurons whose firing rates are given by the arrays A and B, then the model represents the total input to m by a sum of the form W a*A + Wg*B. 48 3.1.2. Methods Stimulus Representation While the single cone receptors in the retina have a density of about 5 to 30 cells per visual degree depending on the location (Carey, 1975), simulation tests have shown that a density of only 1 cell per visual degree allows sufficient accuracy for modeling responses to the stimuli we mostly consider here. But when a stimulus is very small (< 1°) or moves very slowly (< l°/sec), cell density of 5 cells/0 is used to capture more detailed responses (section 3.3). When the stimulus edge partially covers a receptor, we set the receptor inputs to values proportional to the area covered by the actual (analytical/continuous) stimulus. The error from the edge effect is then about 4% relative to the analytical solution (Teeters, 1989). Gaussian Mask A 2-dimensional Gaussian mask “mask(dia, sigx, sigy, wgt, ort)” denotes a mask: .2 Y wgt -exp f 2 •* | y ysigx2 sigy2 j sigx • s ig y ^ ln rotated by “ort” in clockwise direction. When the mask has three arguments, mask(dia, sig, wgt), as the masks used in T&A’s model, it denotes a symmetrical mask with no rotation where sigx = sigy =sig, and ort = 0. Calculation of ERF Size The ERF size of an anuran ganglion cell is calculated from single cell recordings by measuring the duration of the cell response to a stimulus moving at a given velocity: ERF-size = Velocity (°/sec) • Response-duration (sec). The size calculated depends 49 on various characteristics of the stimulus, among them the velocity of the stimulus and the method used (Ewert, Krug & Schonitz, 1979; Garcia & Gaillard, 1989; Gaillard, personal communication, 1991). Thus, the ERF and IRF masks used in the simulation should be regarded as reflecting an anatom y (dendritic field) of a ganglion cell receptive fields, but cells and the models can also show dynamic “effective E R F ” sizes depending on stimulus characteristics such as shape, size, contrast, and velocity. In the simulation (Section 3.3), however, because an instantaneous effective ERF size of a cell is depicted in a spatial firing pattern on a layer of cells, the effective ERF size is measured from the size of the spatial representation. So, the effective ERF size is calculated as the extent of the spatial response curve above threshold. If the response decays in an exponential manner and is not actively abolished, the response extent will be excessively large. For that reason the threshold used is not zero but a small positive number (0.1). Computer Simulation Steps In computer simulation, we update the state of the network every Dt ms, proceeding through every cell type to compute new values of membrane potentials, and then forming new values of the firing rates in the case of ganglion cells. Step 1. Updating the Membrane Potentials: The differential equation Tm dm(t)/dt = - m(t) + Sm(t) for the membrane potential is simulated using the time step Dt to compute the new value m(t+Dt). Our NSL simulation system (Weitzenfeld, 1991) allows one to write the model without reference to any numerical method, al lowing the user to choose different numerical methods (e.g., the Euler method, 50 trading off accuracy for speed) on different occasions without re-specifying the model. Step 2. Updating the Firing Rates: For the ganglion cells, we convert m(t) into the firing rate M(t) = a(m(t)). (cr(x) = x if x > 0, 0 otherwise). For other cells, M(t) = m(t). 3.2. R1.2 Model R1 class neurons were called “sustained edge detectors” by Maturana et al. (1960) since they show a prolonged response to a stationary edge. When a short moving edge enters and stops within the ERF, the response is initially strong then decays to a maintained rate. If the general illumination is turned off, the activation ceases, but it reappears usually at a lower firing rate after some delay when the light is turned on again. The response to a white spot moved on a black background is of similar strength to that elicited by a black spot moved on a white background. These cells have an inhibitory surround because the response to a stimulus movement is reduced for large objects. The cells do not show On, Off responses in that they do not respond to general illumination changes if there is no contrast present. In T&A’s R1 model, both transient (amacrine) and sustained (bipolar) On and Off channels act as input to the ERF to account for the cell’s initially high then decayed sustained response to stimuli of both contrast signs. Since the stimulus movement in the IRF region is required for the inhibition, transient ATH and ATD provide IRF input. T&A’s R1 model, shown schematically in Fig. 2.2b, is 51 implemented as in Table 3.1 and the performance of the model for various stimuli is shown in Fig. 3.1. Model Data (a) « * - Diffuse light < j. 1 on off { ( No response (b) <<2- Shifted square Q 1 j -------------1 ------1 ------1 ------1 — j ^ ^ _ .l.aci S _' (C) 4 42- Tran. ° + n i i i j u i . i . .« ...u <11 , ty k iiu dark o- --------1 --------,------- ,----- — i — | BG off BG on 'j ^ (d) <•«- Stopping bar slops ^ R esponse inoreases 0- Figure 3.1. Class-1 (R l) cell model response, (a) Model response to diffuse light flashes with no contrast present. Like the data (Maturana et al.) the model shows no response, (b) Response to a 3° black square which is shifted into the ERF center. Like the data (Griisser and Griisser-Cornehls, 1976) the model response is initially strong then decays to a maintained level, (c) Response to a stationary 3° square during the transient darkening of general illumination. Like the data (Keating and Gaze, 1970) the model response is inhibited, but returns after a short delay, (d) Response to moving 2 X 20° bar which stops in receptive field center. As occurs in some class-1 cells (Teeters, 1989), response increases after movement stops. R2 cells were named “convex edge detectors” by M aturana et al. (1960) because they respond best to small dark contrasts. Like R l cells, they have a sustained response at a reduced rate to a moving object that stopped within the ERF, are inhibited by movement in their IRF, and do not respond to general light changes. Unlike R l cells, R2 cells respond better to dark on white contrasts, do not respond to moving edges that extend far into the IRF suggesting R 2’s IRF exerts greater inhibition than R l’s, and the sustained response to a stopped stimulus in the ERF is halted by a transient turning Off of the general illumination (erasability). T&A’s R2 model, shown schematically in Fig. 2.2c is implemented as given in Table 3.1 and the performance of the model is shown in Fig. 3.2. 52 Model______________________ Data ( a ) 1 Diffuse light, u t t , Ut(| moving & stationary bar, white on black stimulus no responsa , no response ( b ) 0.43 ■ Moving square A 1 \ »■*! 0*1 1 " 1 J * , P T " — j— -j | ( c ) 0.3 7 - Square 1 f — ■ ■ ■ ■ ----------------m u fja a a m v n tm itt i i n . ^ OH (d ) 0 .37 - Transient darkening — S --------1 --------1 --------1 " ~ .....i' ........i | A °.n off on ------*-r ------- i— — r - * r i | Figure 3.2. Class-2 (R2) model performance, (a) Like R2 cells, the model gives little or no response to diffuse light changes, a 2 X 10° vertical bar, or a small white square stimulus on a black background all whether moving or stationary, (b) - (d) Response to a 1° black square. (b) The square moves through the receptive field (5.125°/sec). (c) The square moves into and stops at the ERF center, (d) Same as c) except that the general illumination is transiently turned off which terminates the response (erasability), (Teeters and Arbib, 1991) The above is the traditional view of anuran ganglion cell classification where R l and R2 neurons are classified as distinct retinal ganglion cell classes. But their responses also show so much similarity that these neurons may be categorized as one broad cell population as Keating & Gaze (1970) suggested. Keating & Gaze based the populational view of R l and R2 on the observation that the basic receptive field organizations of R l and R2 are similar (also refer to the almost identical receptive field properties of T&A’s Rl and R2 in Table 3.1), and other supposedly differentiating features like erasability and longevity of the duration of the response to a stationary edge are only a quantitative difference rather than ideal criteria on which to differentiate two functionally distinct cell classes. They observed that erasability varies among the units in the class and that it is related to the strength of IRF inhibition: The stronger the inhibition, the greater the 53 erasability. W hile the relationship accords with the tendency that the R2 type typically shows greater IRF inhibition than the R l class, nevertheless, the difference of IRF inhibition between R l and R2 should be viewed from a populational perspective. So, this explains why erasability may not be an ideal criterion. Likewise, the duration of response to a static edge is not a good differentiating feature because the difference between static edge responses of R l and R2 is not clear and, at best, only shows a quantitative difference. Recent observation by Gaillard (personal communication) supports this populational view in that some R2 cells do respond to the reilluminated SO, after the transitory darkness, with long duration. With these, we also believe that R l and R2 should be grouped into one population with R l and R2 representing the two extremes of the continuum, and show that this formulation is possible through the simulation. This single population concept of R l and R2 will be adopted in this dissertation and will be denoted as R1.2. The schematic illustration of the R1.2 model is shown in Fig. 3.3 and the algorithm is presented in Table 3.2. Class 1.2 P B H PB D off-S on-S A T . A T D off-T on-T Figure 3.3. Schematic structure of R1.2 model. 54 Model for R1.2: The model uses two temporary variables (tc and g). “tc” is the total transient input to the cell, “g” is an erasability modulation function which takes a value 1 if the tc is greater than “thp” (tc> thp), a value in a range [0, 1] if the tc lies between “thn” and “thp” (thn^tc <thp), and 0 if the tc is less than thn (tc< thn). tc = kO*ATH - k l * (ATH+ATD) R1.2 = g • ( kO* (PBH + d- (ATD+PBD)) - k l * (h- PBH) + tc) with kO = mask(4, 2.4, 1), k l = mask(19.5, 4.6, 3) - R1.2 behaves as “pure” R l of T&A if thp is a sufficiently small negative value such as thp = -10, d=l, & h=0. - R1.2 behaves as “pure” R2 of T&A if thn = thp = 0, d=0, & h=0. * The erasability modulation function g enables us to tune the erasability while the parameters d and h let us tune the cell’s sensitivity to depolarizing channels (such as white on dark contrast) and static dark edges respectively. Table 3.2. Fig. 3.4a shows R1.2 behaving as pure R l when the stimulus is a 3 x3° black square on white background (BAV) moving at vel=10°/sec, stopped at t= l.l sec, and transient darkness occurring at t=3.5 sec and ending at t=4.5 sec. For this, R 1.2 is set with g=l for all tc, d= l, and h=0. As we can see, R1.2 exhibits high initial response, then decreased but sustained response when the stimulus is stopped. When the light is turned off, the sustained response ceases, then reappear with latency of 0.5 second at t=5 sec when the light is turned back on at t=4.5 sec. When 55 the stimulus is a 2x 10° vertical bar moving at vel=10°/sec, R1.2 in pure R l mode does not fire while the stimulus is moving, then begins to show sustained firing once the stimulus is stopped at t= l.l sec (Fig. 3.4b). These responses accord well with the experimental data shown in Fig. 3.1. a 5 S h i f t e d / s t o p p e d s q u a r e & T r a n , d a r k 4 0 ) 4 - > < T J O c c 3 c • H O U * * H 1 0 0 2 3 1 4 5 T im e ( s e c ) b 5 Stopped, b a r " — — 4 a ) 4 J (0 , P C 3 1 0 0 1 2 3 4 5 T i m e (sec) Figure 3.4. Performance of R1.2 model behaving as R l. (a) Response to a 3° black square which is shifted into and stopped at the ERF center. Like the data (Griisser and Griisser- Cornehls, 1976) the model response is initially strong then decays to a maintained level. When the general illumination is transiently turned off during the interval 3.5 - 4.5 sec, the model response ceased as in the data, but returns after a short delay (0.5 sec) after the light is turned back on. (b) Response to moving 2 X 10° vertical bar which stops in receptive field center. A s occurs in some class-1 cells, response increases after movement stops. 56 On the contrary, R1.2 behaving as pure R2 as shown in Fig. 3.5a exhibits the erasability from the transient darkness in that activation to a stopped 2 x 2 ° B/W square does not reappear even when the light is turned back on at t=4.5 sec after the transitory darkness. Fig. 3.5b shows the above R1.2’s response to the 2x2° black square continuously moving at vel=5.5°/sec. These responses of R1.2 in pure R2 mode accord well with the experimental data shown in Fig. 3.2. a 4 i i i i i " S h i f t e d / s t o p p e d ‘s q u a r e ,& T r a n , d a r k " ------ < D 4J < T 3 -3 05 3 G -H 2 M ♦H D m 1 / 0 ( b / 1 i i \ i i 5 1 2 3 4 5 T im e (s e c ) 0 4 1 i 1 .......................... 1 "M oving s q u a r e " ------ Firing R a te to C O . - i 0 i i i 5 1 2 3 4 . T im e (se c ) Figure 3.5. Performance o f R1.2 model behaving as R2. (a) Like the data, when a 2° black square is moved (10°/sec) into and stopped at the ERF center, the model response is initially strong then decays to a maintained level. Then, when the general illumination is turned off during the interval 3.5 - 4.5 sec, the model shows erasability like the data by not showing a renewed response even when the light is turned back on at t=4.5 sec. (b) R1.2 response to the 2° black square moving at 5.5°/sec, which accords well with the data (Fig. 3.2b). 57 Because the channels in the retina model do not contain a memory of past events, the erasability property of R2 that involves a memory of past transient darkening requires an addition of an “erasability modulation function (g)”, which is an extended formulation of an “erasability gate” used in the T&A. The function g relies on total transient input “tc” (Table 3.2) and lower (thn) and upper (thp) threshold values: g is set upon detection of movement in the ERF (tc> thp), and is reset when strong global transient changes, such as the transient darkening, occur, which is captured by the IRF (tc< thn). With this formulation, the lower the thn and thp values are, the less likely the g will reset to a tc (i.e., less likelihood of erasability). Or, given the thn and thp values, the stronger the IRF, the more likely the g will reset (more erasability), which accords with the observation that R2s have stronger IRF inhibition than R ls. Thus, with thp sufficiently small value like -10, the R1.2 model shows R l characteristics (no erasability), while setting thn=thp=0 produces R2 characteristics (erasability). Also, by constructing the g as a saturation function, 0 when tc < thn = - 0.1 5 • tc + 0.5 when -0.1 = thn < tc < thp =0.1 1 when tc > thp = 0.1 with 2 x2° black square moving at vel=10°/sec, stopped at t= l.l sec, and transient darkness between 3.5<t<4.5, R1.2 shows the response shown at Fig. 3.6. We can see that this cell reflects both R l and R2 characteristics in that while SO response reappears after the transitional darkness as in R l, the activation level is lower and its response latency is longer at about 1.5 sec. 58 5 "R2 e r a s a b i l i t y 4 a ) 3 0 3 T im e (s e c ) 0 1 2 5 4 Figure 3.6. Performance o f R1.2 model behaving as an intermediate o f R l and R2 in the context of erasability. By constructing the erasability modulation function “g” as a saturation function (see the text for equations), the R1.2 exhibits characteristics of both R l and R2 in that while the erasability is not complete as in the R2, the maintained activity level for the static object is lower and the response latency after the reillumination is longer at about 1.5 sec compared to 0.5 sec of R l. The parameter “d” is a depolarizing channel gate that controls the On channel (PBD and ATD) contribution to ERF. When 1, the On channel is fully gated to ERF, which results in an R1.2 equally sensitive to W /B stimuli as it is to B/W stimuli as R l. Fig. 3.7 shows the R1.2, with the g as in Fig. 3.6, response to a W/B square moving at vel=10°/sec when d=0 (a) and d=0.5 (b). We can see that the response increased as d is increased from 0 to 0.5. The R l model of T&A does,not fire in response to a stationary edge per se, but to presence of a region of different luminance relative to the general illumination within its ERF. R l’s sensitivity to a static edge calls for a receptive field capable of detecting a contrast edge “painted” on the bipolar layer. Even though a movement of a stimulus appears to be necessary in order to induce inhibition from the IRF, R l’s special sensitivity to a static edge can be best explained by inclusion of PBH to IRF, thus forming a static edge-sensitive DOG mask. Thus, R1.2 is modeled to have PBH to IRF controlled by the parameter h. In Teeters et al. (1993), the inclusion of 59 PBH to IRF made the R2 model better match the quantitative data. In the same vein, it is reasonable to expect R ls to have PBH based inhibition, but with lesser effect (< 0.3) following the general tendency that R ls have weaker IRF inhibition than R2s. a 5 M ov ing w h i t e s q u a r e 4 a > 4 - J o4 3 c n c • h o u * Cu 1 0 0 1 2 3 4 5 T im e (se c ) b 5 "Moving w h i t e s q u a r e (d= 0 .5 ) 4 C n c • H Cu 1 0 0 1 2 3 4 5 T im e (se c ) Figure 3.7. Performance o f R1.2 model behaving as an intermediate of R l and R2 in the context of sensitivity to white on black background stimulus, (a) R1.2 model response to a 2° white square moving at 10°/sec with d=0. (b) R1.2 response to the same stimulus o f a) but with d=0.5. The response increased as d is increased from 0 to 0.5. 3.3. R3 Model R3 cells were called “changing contrast detectors” by Maturana et al. (1960) since they respond to an increase or decrease in general light with brief but prominent On 60 and Off activations. Moving dark objects generate a response at both leading and trailing edge. Traditionally, the anuran R3 class is mainly associated with the capability of detecting moving edges. As such, these neurons were thought to be involved in the recognition of moving prey or predator type stimuli but not with static objects. On the other hand, researchers like Griisser & Griisser-Comehls (1976: Table 7) report that the R3 neurons are activated when an animal is viewing a static edge, often accompanied with an eye or body movement. Here, we concentrate on the R3 cell’s responses to static objects that experience retinal movements due to the animal’s own movement. The starting point of the proposed R3 model is the R3 model of Teeters et al. (1993) shown in the right side of Table 3.3. The model is basically similar to T&A’s R3 model, and thus capable of exhibiting all the characteristic responses T&A show. Because the proposed R3 model only involves changes in the parameters of T&A, the model is capable of providing all the characteristic responses exhibited by T&A’s model. T&A Model for R3: Teeters et al. R3 model: R3 = k0*a - (kl*a) d elayed R3 = k0*a - (kl*a) delayed with a = p • ATD + ATH, with p = 0.4 with a = p • ATD + ATH, with p = 0.4 and (s) dela yed = signal s delayed 40 ms. and (s) dela yed - signal s delayed 40 ms. kO = mask(8, 2.4, 1) kO = mask(8, 2, 1.15) k l = mask(19.5, 4.6, 1.4) k l = mask(19.5, 10, 2.38) Table 3.3. 61 Schipperheyn (1963, 1965) claims that R3 responses can be elicited by a respiration induced, predominantly vertical, eye oscillatory movement of about 0.5°/s. The existence of the respiratory R3 response is not widely accepted, even though such a response is possible in theory because the displacement and the velocity of the respiration-induced retinal image shift are above the minima needed for R3 excitation and because the neuronal adaptation of R3s is incomplete.1 Ingle (1976) found a retinal input in the pretectum of the frog, which he called R6, interspersed among the usual R3s. These cells with small receptive field sizes (3-6°) discharged for 10 seconds or more when a dark edge was moved into the field and was oscillated at 0.5°/s, similarly to the oscillations produced by the respiratory eye movement. We hypothesize that Ingle's R6 is the same as Schipperheyn's respiratory R3, and so model these cells (Resp-R3) by using the R3 model with a resolution of 5 cells/0 — our base cell density of 1 cell/0 might not be enough to capture the details since that the oscillation amplitude is very small (0.5°). To stimulate its model retina, the oscillatory movement is simulated as follows. y(t) is the upper edge location of the oscillating stimulus. The oscillation can be described as repeating a movement that starts from an initial point, y(0), to the point “y(0) + amp” and back again to the starting point in “period” seconds: y(t + dt) = y(t) + v(t) • dt +1 / 2 • a(t) • dt2,with J Acc When y(0) < y(t) < y(0) + amp / 2 1 - Acc Otherwise. 1 Garcia and Gaillard (1989) report that neuronal adaptation of a frog (Rana Esculenta) R3 is almost nil. 62 2 where acceleration magnitude A cc= 2 • (a m p /2 ) /( p e r io d /4 ) , time step dt = 40ms, oscillation amplitude amp = 0.5°, and period is 1 second.2 Here the acceleration is kept constant until the stimulus reaches the “y(0) + amp/2” conversion point, the midpoint of oscillation, where the acceleration sign is reversed. We believe the simulated oscillation (Fig. 8a) is a good approximation of real movement (Fig. 8b). a 2 R e s p - o s c i . 5 1 > 0 . 5 0 10 8 9 6 7 5 2 3 4 0 1 T im e ( s e c ) b ................ - . - . - . - . - . - . - • • ■ - ■ • . Y ..... . ............................ .y. y'.'....................W Figure 3.8. (a) The time course of a stimulus' up-side edge location induced by a simulated respiratory eye movement. We assumed that the eye oscillation of 0.5°/sec induced an equal amount of retinal image oscillation', (b) Schipperheyn's data on respiratory eye movement: Time course o f respiratory eye movements in unrestrained frog. The slight movements (frequency 1.3/sec) have a maximum amplitude of 28 minutes of arc. The slow movements at intervals of 15-20 seconds are caused by periodic emptying of the lungs. (Note that the respiratory eye movement is the train of small sinusoidal curves. The slow rhythm is ignored because the response from the movement is negligible compared to that from the respiratory eye movement due to its much lower average speed) Upward deviation o f the optical axis is recorded downward. Time marking 1/sec. Temperature 25.6° C. (Schipperheyn, 1963). The larger but slower rhythm caused by periodic emptying of the lungs produces an average speed that is about 25% of the respiratory eye movement. 2 The oscillation may also be formulated as y(t) = y(0) + amp/2 + amp/2-sin [(27t/period) t - 7t/(2-period)]. This gives results similar to those from the one used in the model. 63 Because of the slower speed, the slow rhythm produces responses that is about 20- 30% of those caused by respiratory eye movement, and so its significance is not incorporated in the model. Because an oscillating edge undergoes changes in its instantaneous velocity during the respiratory cycle, the response amplitude and the ERF characteristics of an R3 ganglion cell that depend on the stimulus velocity also show variation during the cycle. So, first we compared the responses of R3 to leading edge of a 2 x2° dark object moving upon a bright background at constant velocity of l°/s, an average 50.0 D S to f c u 0.0 ERF-size (5 cells/deg) 50.0 0 ) < 3 to .5 s, o.o ERF-size (5 cells/deg) Figure 3.9. In the R3 figures, Fig. 3.9 - 3.14, the density of cells in the simulation is 5 cells/0. Thus 5 vertical bars along the horizontal axis amounts to 1 visual degree in figures Fig. 3.9a to Fig. 3.11b. (a) R3 response to 2 X 2 ° stimulus moving at 10°/sec. The R3 here is the Teeters et al. model - ERF = (8, 2, 1.15), IRF = (19.5, 10, 2.38). We see that the effective ERF size o f the R3 is about 8°. (b) R3 response when the stimulus moves at l°/sec. All other parameter settings are identical to those used in Fig. 3.9a. We can see that the effective ERF size is reduced to about 4° and that the peak excitation is also reduced to about 1/4 of that of Fig. 3.9a. C^/::.3-9^^^$:5/++/579B 64 speed during the respiratory cycle, to those of moving at the reference velocity (10°/s). With Teeters et al. parameters, the simulation revealed that the effective ERF of the R3 was about 8° at vel=10°/s and about 4.0° at vel=l°/s (Fig. 9a and 9b, respectively). The peak excitation of the stimulus at l°/sec is also reduced to about 1/4 of that at 10°/sec. The spatial profile of ERF and IRF masks also influences the R3's ERF size as the velocity changes. If the IRF weight in the IRF mask is reduced, the resulting effective ERF size will be more dependent upon the length of the spatial amacrine firing pattern3 (length of exponentially decreasing tail) because the IRF signal, which in part is proportional to the spatial extent of the amacrine firing pattern within the IRF region, is reduced. When we decreased the weight of the IRF to that of the ERF, the effective ERF (12.5°) was reduced to about 1/3 (4.2°) when the velocity was decreased from 10°/s (Fig. 10a) to l°/s (Fig. 10b). The peak amplitude is also reduced to about 1/4 of that at the 10°/sec. The standard deviation of the masks influences the ERF size as well. When tested with T&A's base parameters (ERF = (8, 2.4, 1), IRF= (20, 4.6, 1.4)), the effective ERF size at vel = l°/s (3.2°; Fig. lib ) is about 65% of that at the 10°/sec (5°; Fig. 11a). We observed that it is the general tendency of ERF size to decrease 3 The model uses a high pass filter to represent the amacrine cells as they convert sustained bipolar signals into transients. The resulting amacrine cell layer forms an exponentially decaying surface starting from the edges of the moving stimulus: the ATH layer forms such a surface starting from the leading edge of a dark moving surface, and the ATD layer from the trailing edge. The length of amacrine firing profile correlates with the stimulus velocity. If the shapes of the stimulus classes are restricted to rectangles and if an amacrine cell has maximum instantaneous firing rate o f 1, overall activity of ATH on its layer is: ATHsum= h - r e - x/vTdx J x=Q where x is the distance between the amacrines corresponding to the leading edge and the position of amacrines the stimulus has passed over. 65 a so-D m i n i n r r m 11nitr r n n 111111r u n it i 1111rn111111111111rTii n TTrn i r frrirrT TrfTn C 4 to .5 * C E 0 . 0 LL m t f t l l i r n u i U I , l LU li.U l.U i I IlLli Lilli III 1U 111 1 LU.tJJJL,£il Lit lU lJ U U i l i t U L liU .Lli.lLU ERF-size (5 celis/deg) b 5 0 , 0 n n t i i i 'iii i n 1 1 1 1 1 1 1 1 1 1 1 1 i n n 1 1 1 1 1 1 1 m i m i i i n n u n m i n i n n i n 'i i i m u r r i 'i i m 0 £ 60 c C £ o. o lllLilLu llilJ 1111111 i 1 1 i 1111111. 11MIL . jjJ4-nTl I i i 1 HI 1 m-i-li i 11111 1 i M11111 1 i 11111 1 Hill ERF-size (5 cells/deg) Figure 3.10. (a) Here we changed the IRF weight from 2.38 to 1.15. Other parameters of ERF and IRF masks are identical to those used in Fig. 3.9a - ERF = (8, 2, 1.15), IRF = (19.5, 10, 1.15). The reduction of IRF-induced inhibition to overall R3 response is manifested in both an increase in peak excitation (15% increase compared to Fig. 3.9a) and in the ERF size (12.5° compared to 8°). Stimulus vel = 10°/sec. (b) When the stimulus velocity is reduced to l°/sec, both the ERF size (about 4.2°) and the peak excitation level (1/4 o f Fig. 3.10a) are reduced. However, comparing Fig. 3.9a,b and Fig. 3.10a,b, we can see that the reduction of IRF weight results in a more drastic decrease of the effective ERF size as the velocity decreases. Whereas Fig. 3.9b showed a 50% decrease in the effective ERF size, this figure shows more than a 65% decrease. with lower velocity. This becomes more prominent if the ERF mask's standard deviation is decreased. This is due to the fact that if the ERF mask has a sharp peak, then as the spatial amacrine firing profile shrinks with a velocity decrease, R3's ERF size is more affected by the shrinkage because the mask gathers less input from its peripheral regions when the mask is being convolved off-center to the amacrine firing pattern. 66 a 50.0 < a OS t s o c 0.0 ERF-size (5 cells/ deg) so.o n OS t > 0 c u, 0 - 0 U l l U U LIU 1,1,1, l LIX I.m i l. L m i.U l l.U 11 I I 1 11 LRT1 111 M IIT K J m m i 11 111 1 H L L L U U .I1I 111 II I I ERF-size (5 cells/deg) Figure 3.11. (a) The RF parameters used here are the original masks o f T&A - ERF = (8, 2.4, 1.0), IRF = (19.5, 4.6, 1.4). Stimulus vel = 10°/sec. The differences from the mask used in Fig. 3.9a are that the IRF mask here has a smaller standard deviation and a lesser weight and that the standard deviation o f the ERF mask is a little increased. We can see that R3 exhibits an ERF size o f about 5°. (b) Compared to Fig. 3.11a, when the stimulus velocity is reduced to l°/sec, ERF size is reduced to about 3.2°. This reduction (35%) in effective ERF size is somewhat less than those experienced by R3s in Fig. 3.9b and Fig. 3.10b. From the above constant velocity simulations we have seen that an R3 could respond to a small object moving at respiratory speed, albeit with lesser intensity and smaller ERF size. Teeters et al.’s R3 is then tested on the upper edge of the simulated oscillating stimulus of a 10x2° dark object without taking the cell's neuronal adaptation into account. (The temporal response of the R3 neuron was sampled from the cell that is located at the conversion point) W ith the ATD channel contribution p set to zero, the peak excitation occurs when the edge passes the conversion point, where the velocity is highest, and continues to decrease until 67 the initiation of a new oscillation period where it repeats an identical oscillatory response (Fig. 12a). The peak excitation is about 1/5 of that when the stimulus moves at the continuous velocity of 10°/s (Fig. 9a). Effective ERF size also varied during the oscillation cycles (Fig. 12b). It was largest (4°) at the conversion points in the advancing phases and smallest (1.4°) at oscillatory ends. The average ERF size was 3°. a 50 ATD=0 C n c ■ H U • H ^ 10 0 1 2 3 4 5 6 7 8 9 10 T im e (se c ) b 15 'ATD=0 • H o 2 1 3 6 5 8 9 10 4 7 T im e (s e c ) Figure 3.12. (a) The Teeters et al. R3 response to a small (2°) dark edge oscillating at 0.5°/sec with the ATD component (p) set to zero. (The rest of the R3 simulations on the oscillating edge are performed with the masks - ERF = (8, 2, 1.15), IRF = (19.5, 10, 2.38)) The temporal Resp-R3 response is sampled at the conversion point. Minima occur at oscillation ends, “y(0)” and “y(0)+am p”, and the maxima occur at the conversion points during the advancing phases o f the oscillation, (b) The effective ERF size o f the R3 during the respiratory eye movement. The ERF size of the R3 varies between 1.4° and 4° during the oscillatory cycles. The average ERF size is 2.96°. 68 When the ATD channel contribution is set to 0.35, the R3 becomes a little sensitive during the retreating phase of the edge. As expected, R3 response shows low/smooth humps around the conversion points during the retreating phases of the edge (Fig. 13a). The effective ERF size is also affected by the input from the ATD channel (Fig. 13b). a 50 ATD-0.3 5 " < U 4 0 u cd * 30 C P •5 20 U * » 4 10 2 3 0 5 4 6 1 7 8 9 10 T im e ( s e c ) b 15 ATD-0.35 2 3 6 0 4 5 7 8 9 10 1 T im e ( s e c ) Figure 3.13. (a) The Resp-R3 response when the ATD channel contribution is set to 0.35. Due to the ATD contribution, Resp-R3 is made a little sensitive to the retreating edge. We can see the effect of this through the low/smooth humps around the conversion points during the retreating phases of the edge, (b) The effective ERF size of the R3 during the respiratory eye movement with the ATD component set to 0.35. The ERF size of the R3 varies between 3.2° and 4.2° and the average ERF size is 3.87°. By setting the ATD channel contribution to 1 and thus making the R3 fully sensitive to the advancing brightness during the retreating phase of the dark edge, the R3 response shows two cycles of oscillation during one period of respiratory oscillation (Fig. 14a). One peak occurs when the edge passes the conversion point 69 during the advancing phase (contribution of ATH channel) and the other happens when it passes the point during the retreating phase (contribution of ATD channel). The effective ERF size varies between 3.8° and 4.6° and the average size is 4.29° (Fig. 14b). Compared to the response when the ATD channel was zero (Fig. 12a), the peak amplitude is a little increased (24%) but the average intensity is doubled. a 50 " ATD— l .0 < d 4 0 o j < U « 30 O ' c • h 20 u ♦H 10 o 9 10 2 3 6 8 1 5 7 4 T im e (s e c ) b 15 ’ATD=1.0 10 e 8 9 2 5 7 0 3 1 4 T im e (s e c ) Figure 3.14. (a) The Resp-R3 response when the ATD channel contribution is set to 1. Two peaks o f activation for one cycle of stimulus oscillation is observable. Compared to Fig. 3.12a, the most prominent change lies in the increase (103%) of the average Resp-R3 response level, (b) The effective ERF size of the R3 during respiratory eye movement with the ATD component set to 1.0. The ERF size varies between 3.8° and 4.6° and the average size is 4.29°. 70 In conclusion, as observed in the simulations, the respiratory R3 response can occur and its effective ERF size is significantly (25 - 60%) smaller than normal R3s responding to faster moving objects. Further, specific tuning such as the reduction of IRF inhibition made R3s more sensitive to the respiratory movement and fit the ERF size of Ingle's R6 when an edge was oscillated at 0.5°/s. Such findings support the hypothesis that Ingle's R6 may in fact be one subclass among the spectrum of v R3s. 3.4. R4 Model R4 model of T&A is capable of simulating; i) response to a decrease of light in ERF, ii) absence of response to an increase of light in ERF, iii) delayed response in some cells to increase in light outside the ERF - Surround On (Keating & Gaze, 1970), and iv) diffuse-On response caused by turning on the light after the lights-off. The model only considered transient data of the cell and so it was modeled with the responses caused by ERF stimulation resulting from transient Off (ATH) and On (ATD) channels providing excitation and inhibition to the cell respectively. The model is shown schematically in Fig. 2.2 and is implemented as in the left side of Table 3.4. 71 T&A Model for R4: Revised R4 model: R4 = kO * (ATH - x • ATD) R4 = kO * (w- PBH + x • ATH - y- ATD + (z • ATD )d ela ye d O.Ssec) with x = 1 with w = 1, x = 1, y= l, and z = 0.5. kO = mask(15.5, 3.5, 1.0) kO = mask(15.5, 3.5, 1.0) Table 3.4. Simulation of T&A R4 model and corresponding experimental data are shown in Fig. 3.15. The model matches the data qualitatively in that there is a strong Off Model Data (a) Dif 4 3 9 ' 0 - ‘ use light change* fast amacrines ^ t T . ■ , J 1 off k , . Ti 1 2 sec (b) Dif 4 . 3 9 ' 0 * 'use light change* normal amacrines 1 k on \ off off . X X L ? M l l l i i l l l i 1 (C) Dif 4 . 3 9 ' 0 - : use light change. On inhibition reduced! V off / V on ✓ i ___ Same a s above (d) Stir 4 . 3 9 ' o - round On 1 1 sec on W . T_ _ iLll.tl 1 1 r (e) Mo 4 . 3 9 " 0 - ving 2x2 square 1 ---------------------- , „ t Little if any response (0 Mo 4 . 3 9 * ving 2x10 andworm 1 -------------------- — . tl M l I r ti i ur.... ... 1 ---- ‘ -- 1 ------ ■ 1 ‘ 1 1 ‘ Figure 3.15. Class-4 (R4) cell model performance, (a) Amacrine cell time constant of 80ms causes a delayed On response, (b) With a time constant of 300ms, the delayed On response is eliminated. These amacrines are used for (a) - (f). (c) Reducing the On inhibition to 10% by setting “x” to 0.1 results in a delayed On response with the 300ms amacrine. (d) The model responds to surround On which is simulated by making the horizontal cells hyperpolarize by 25%; data is from Keating and Gaze (1970). (e)-(f) Response to a black 2° square compared to 2X 10° antiworm. The model response to the larger stimulus is much stronger, as is found in typical R4 cells. (Teeters and Arbib, 1991) 72 response, and in two instances, a weak delayed On response. By incorporating a non-linear “squash” function in the calculation of R4’s average response, Teeters et al. (1993) further showed that the R4 model is capable of replicating the biological R4’s stimulus shape and size dependency. However, T&A’s R4 model was not so successful in reproducing the durations of the responses. For example, the model Off responses in Fig. 3.15b, c had duration of about 1 second while the data shown lasts over 4 seconds. An adjustment of time constants enables the match to experimental data but then this makes it impossible to match the short duration of On response. Also, exclusive reliance on transient amacrine channels for the input suggests that T&A’s model may have difficulty providing continuous response to a large dark object moved into the ERF and stopped there unlike typical R4s (Lettvin et al., 1960; Grtisser & Griisser-Comehls, 1976). In a first attempt to remedy T&A’s deficiencies, R4’s prolonged response to a stationary large dark object and general darkness is made possible by incorporation of the “PBH” channel to the cell; R4 = kO * (w- PBH + x • ATH - y- ATD), with w = 1, x = l ,y = 1. So, R4s in the new model fire briskly if the dark object is moving but also show prolonged response, with a lesser frequency, if the object becomes stationary or general illumination is decreased. In order to let PBH cells to be responsive to diffuse light Off, it is necessary to set HO in the horizontal cell model to a lesser value than receptor’s depolarizing level; that is, if the receptor is depolarized to 0.8 during darkness, HO should have a value less than 0.8. Similarly, it is necessary to set the HO level greater than the hyperpolarizing level of receptors during the 73 general light On in order to have On channels responsive to a lighter object than background lighting. This implies that while the assumption that the horizontal cell’s main function is to bias the bipolar cells so they operate in their region of maximal sensitivity may generally be valid, nevertheless, the horizontal cell membrane potential’s dynamic range should be contained within that of the receptors. First, to test T&A R4’s inability to provide the sustained response to a large dark object, a 20x20° black square is moved into the ERF and stopped there at t=2 sec. As we can see from Fig. 3.16a, the response decays to zero in about one second. But with the new model, R4 continues to fire when the stimulus is stopped as shown in Fig. 3.16b. Fig. 3.16c shows the diffuse light change test where the light is turned off at t= l second. Contrary to T&A’s model, the new R4 model provides continued response while the light is Off. This model offers similar responses to surround-On, moving square, and antiworm stimuli as those of T&A’s model. Despite the fact that the current model offers additional performance over T&A’s model in the above respects, it lost the ability to simulate the delayed, brief, but prominent diffuse-On response. In T&A’s model, this response is achieved through the reduction of ATD channel inhibition by setting x=0.1. The diffuse-On response and surround-On response stem from the horizontal cell’s (H) interaction Figure 3.16. (a) When a 20X 20° black square is moved into and stopped at the ERF at t=2 sec., T&A’s R4 stops responding in about 1 second. This response does not accord well with Griisser and Griisser-Cornehls’ (1976) report that the R4s continue to fire to a static large dark object within the ERF. (b) With the inclusion of PBH channel and when the stimulus is the same as in (a), the R4 initially fires briskly when the stimulus enters the ERF but also continues to fire to the stopped stimulus, (c) With the inclusion of PBH channel and setting the H0=0.5 during the lights-Off, the R4 model provides continued response during the lights-Off, which accords well with the Griisser and Griisser-Cornehls’ (1976) and Fig. 3.15b data. 74 a 5 "T&A's SO Resp 4 a > u a s 0 0 5 3 O ' c -H p M * ♦ H 6 m 1 0 0 1 2 3 4 5 T im e ( s e c ) b 5 SO_Resp 4 0 ) - p a s 0 0 5 3 c • H 2 ■ H C m i o 0 1 2 3 4 5 T im e (s e c ) c 5 O f f _ R e s p 4 t u jj < 0 , a 3 c - H 2 * H u , 1 0 0 2 3 5 1 4 T im e (s e c ) 75 with amacrine precursor (HBX, see Table 3.1). Here, HO level and the difference between the H and HBX’s time constants play an important role. While setting HO to 0.75 is a reasonable way to simulate the effect of the surround-On, the basis for achieving the diffuse-On response by setting the y to 0.1 is not clear. Indeed, the source of the delayed diffuse-On response comes from post-On ATH rebound, which really is an artifact that stems from the H-HBX temporal dynamics dependent on the specific choice of corresponding time constants. Because the ATH rebound is very weak in intensity, T&A had to set the ATD inhibition to a very small value so as not to suppress the signal completely. The reason for the inability to respond to diffuse light-On in our model is due to the setting of the HO value (0.5) to less than the depolarizing level of receptors (1.0) during the light off. If we overlook neuronal level analysis of the probable cause of these responses but concentrate on the higher level of channels, the delayed diffuse-On responses can be ascribed to the delayed reception of On channel input at the ganglion cell level. Due to the nature of brief but intense response, input for the On response may come from the delayed excitatory ATD channel. This led us to incorporate the delayed (0.5 sec) excitatory ATD channel to the final R4 model whose schematic diagram is shown in Fig. 3.17 and described at the right side of Table 3.4. Class 4 ATD on-T P B H A T I ATD off-T on-T off-S Figure 3.17. Schematic structure of R4 model. See the text for details. 76 Fig. 3.18a shows the R4 model’s response to diffuse light changes. When the light is turned off at t= l sec, the model shows strong initial Off activation followed by the weaker sustained response to darkness. When the light is turned back on at t=3.5 sec, the model responds with a brief but prominent On response after the latency of 0.5 second. Fig. 3.18b shows the R4’s response to 20x20° object stopped at t=2 sec, and the Fig. 3.18c shows the surround-On response at t=1.5 sec. Figs. 3.18d-f show the R4’s response to 2x2° square, 10x2° horizontal bar (worm like), and 2 x 10° vertical bar (antiworm-like) stimuli moving at vel=10°/sec. These responses accord well with the experimental results. 3.5. Discussion While all vertebrate retinas are organized according to a common basic structure, some significant differences exist between anuran and most mammalian retinas, especially with regard to the synaptic contacts in the inner plexiform layer (IPL) (Dubin, 1970; Dowling, 1976; Dowling, 1987). In the IPL of primates and cats, about 80-90% of the bipolar cell outputs contact ganglion cell dendrites directly. In frogs, however, most of the bipolar cell outputs do not synapse onto ganglion cells directly but contact amacrine cell processes, and it is these amacrine cell processes the ganglion cell inputs mostly consist of. A comparison of the ratio of conventional Figure 3.18. (a) R4 model’s response to diffuse light changes. Light is turned off at t=l sec and turned back on at t=3.5 sec. Note that the temporal firing pattern closely resembles the data shown at the right side of Fig. 3 .15b. (b) R4 model response to a 20 X 20° stopped black square, (c) R4 model surround On response at t=1.5 sec, which is simulated by making the horizontal cells hyperpolarize by 25%. (d) - (f) The model R4 responses to 2 X 2 ° black square, 10X 2° worm, and 2 X 10° antiworm stimuli all moving at 10°/sec. The model shows stronger response to larger and antiworm like stimuli. 77 a 5 D i f f u s e _ l i g h t _ c h a n g e • H 1 0 2 3 0 1 4 5 T im e ( s e c ) b 5 20x20 s t o p p e d o b j e c t 4 4) 4 - > < 0 3 t n 1 0 0 1 2 3 5 4 T i m e ( s e c ) c 5 S u r r o u n d On 4 3 1 0 T i m e ( s e c ) Firing Rats Firing Rat© ^ Firing R a ts 78 Moving 2x2 s q u a r e ” 2 3 T im e ( s e c ) "Moving 10x2 worm” 2 3 T i m e ( s e c ) 5 'Moving 2x10 a n t i w o r m 4 3 1 0 0 1 2 3 4 5 T im e ( s e c ) 79 (amacrine) to ribbon (bipolar) synapses in the IPLs of a number of species made by Dubin (1970) shows that in the central part of human retina the ratio is about 1.7:1 whereas frog retina shows the ratio about five times (9.6:1) that of central human retina. These data suggest that the amacrine cells are likely to have a much more important role in the processing of visual information in the frogs. There exists a strong correlation between the synaptic organization of the retina and its physiological responses. With much more numerous amacrine cells, all the frog’s ganglion cells seem to be influenced by the transient amacrine cell response to a certain degree. For example, all anuran ganglion cells show increased response with stimulus velocity (Griisser & Griisser-Cornehls, 1976). The anuran ganglion cell models proposed in this thesis all include amacrine inputs. On the other hand, cats possess a much greater ratio of bipolar cell synapses in the IPL and their influence is revealed in the fact that the sustained, contrast sensitive (X) cells in the cat retina that receive 70% of the bipolar outputs comprise about 55% of cell population and that they predominantly reflect bipolar cell characteristics. Similar analysis is possible with regard to the P cells in the monkey retina. X (P) cells appear to linearly summate luminance signals throughout their receptive field (RF), while Y cells (M cells in monkey), which respond to inputs in a transient fashion, exhibit a more complicated nonlinear spatial summation of luminance signals throughout their RF. The RF pattern of the retina models presented in this thesis was also used by several earlier models which attempt to explain mammalian retinal ganglion cell properties. For example, the DOG center-surround structure was used to account for cat ganglion cell receptive fields by Rodieck and Stone (1965), and Enroth-Cugell 80 and Robson (1966). Young (1987) proposed a more elaborate center-surround structure for the primate ganglion cell receptive field in the form of “difference-of- offset-Gaussians (DOOG)”. Thus, so far as the receptive fields are concerned, there are not much differences between the anuran and mammalian retinas/modeling. Rather, we believe the difference lies in the inputs to the ganglion cells. Each X (P) and Y (M) cell population is divided into on-center (ON), and off- center (OFF) cells having RFs consisting of a central region and a surrounding annulus of opposite polarity: increasing luminance (darkening) in the RF center (surround) causes increased activation for ON cells, but decreased activation for OFF cells. Conversely, darkening (illumination) in the RF center (surround) causes increased activation for OFF cells, but decreased activation for ON cells. Thus, a X or Y cell in cats has a specific luminance contrast specificity. On the other hand, R1.2 anuran ganglion cells show equal responses to stimuli of both contrast signs that fall in the ERF. To account for this, R1.2 model is constructed to have both excitatory On and Off channel inputs to the ERF. However, in the same region of X or Y cells, if one channel is excitatory the other channel should then be inhibitory. Also, whereas an appropriate luminance signal falling in the RF surround, like darkness for ON cells or luminance for OFF cells, in cats can increase the cell activity, the surround regions of anurans only contribute inhibitory effect (IRF). The inclusion of both On and Off channels to R1.2 ERF and the simple inhibitory surround IRF make R1.2s less sensitive to a contrast stimulus than the cat X or Y cells. It is not an easy task to compare anuran and mammalian retinal ganglion cells. Because anuran R1.2 cells show sensitivity to static stimuli, we may associate them 81 with cat’s X and Y cells. On the other hand, Y cells’ more transient response, larger RF, larger conduction velocity, and projection to the superior colliculus show that they also have anuran R3 or R4 characteristics of larger RFs, greater conduction velocity through their myelienated fibers, and more transient properties. Also, W cells in cats that show On-Off responses and direction sensitivity to local edges resemble anuran R3 cells. As stated above, an understanding of anuran retinal ganglion cells should start from the observation that transient amacrine inputs have much more influence over the anuran cells than the mammalian cells. With the greater amacrine influence, all the anuran ganglion cells are biased to perceive moving objects while the sensitivity to static stimuli is of secondary importance: whereas each X and Y cells are specialized to perceive one contrast sign stimuli through simpler construction and less preprocessing, even the primary static object- sensitive anuran R1.2s show sensitivity to moving stimuli as well as the response to static stimuli of both contrast signs. This multiple capability makes R1.2s compromise their ability to perceive a simple stimulus like a static stimulus of single contrast sign. This non-ideal property of anuran retinal processing for the static stimuli is considered when designing the anuran static object perception mechanism which is the topic of Chapter 4. We conclude this chapter with some explicit predictions based on the simulation results of the anuran ganglion cells. 3.5.1. Horizontal Cell Horizontal cells average receptor signals over a broad region and feed-back the spatially averaged signal onto receptors under its influence. The issue here is how widely spread the horizontal network’s lateral effect would be. T&A simplify the 82 horizontal cell model by assuming that its potential is only dependent on the background illumination and is spatially invariant through the infinite spread of the potential. This has proved to be adequate for the ganglion cell responses considered in their paper. T&A’s model assumes that the horizontal cell’s main function is to bias the bipolar cells so they operate in their range of maximal sensitivity. Taking this notion to an extreme, the horizontal cell potential is not computed from receptor inputs but takes value 1 during ambient darkness and 0 when background illumination is bright. However, simulation of R4 class neurons necessitated horizontal cells having dynamic range of their potentials within that of receptors [0 - 1] to let R4s exhibit continued response during the light-Off. That is, the horizontal cell potential follows but does not reach the receptor’s potential. The physiological basis for the horizontal’s potential trailing of the receptors may be due to its synaptic characteristics. The input from each cone receptor is delivered to a horizontal cell over a high-resistance receptor-to-horizontal segment (spine neck) which connects to the low-resistance horizontal cell dendritic network (Sterling, 1990). Consequently, the receptor signal suffers considerable voltage drop when the signal is injected to the horizontal cells, after which further signal attenuation is much less due to the low-resistance connection of the horizontal dendritic network. Prediction 1: The large voltage drop across the receptor-to-horizontal connection is the neurophysiological basis for the horizontal cell’s potential trailing of the receptor’s potential proposed in the thesis. 83 Prediction 2: W ith selective disruption of photoreceptors’ neuro-transm itter responsible for the depolarization of the postsynaptic horizontal cells, R4’s sustained response to the general darkness will increase. 3.5.2. R1.2 Cell We showed that R1 and R2 class cells can be modeled as a single population where each represents an extreme of the continuum. We believe the issue of erasability should be dealt with in the context of measure of a adaptability (sensitivity) and the speed of recovery from it. The formulation of the erasability modulation function is the first step in that direction in that degree of erasability is determined by a cell’s sensitivity to a transient darkness that is measured in graded rather than all-or- nothing fashion. Prediction 3: Among the (traditional) R2s, those showing lesser adaptation characteristic and faster recovery from it, greater dark edge sensitivity, and lesser IRF inhibition are less likely to show erasability. 3.5.3. R3 Cell Schipperheyn (1963) and Griisser & Griisser-Cornehls (1976) suggested that respiratory R3 cells may play an important role in the detection of stationary objects. Even though the peak response to the oscillating edge is small, the respiratory R3 response could furnish excellent information on the boundaries of stationary objects because of its small ERF size and the continuous response. We have also seen that 84 by increasing the ATD channel contribution, the respiratory R3 response is made stronger and steadier. It is interesting to see that the R3 response of Hyla, one of the anurans with the highest sensitivity to stationary objects, appears to rely more on the trailing edge response (ATD contribution) than other anurans like Rana and Bufo (Griisser-Cornehls, 1988). When simulating the Resp-R3 responses, we ignored one important aspect of ganglion cell characteristics. Even though “Neuronal Adaptation” is not complete in the R3s, oscillatory movement is the perfect stimulus movement that can cause the adaptation. Prediction 4: Cells that may be maximally sensitive to Resp-R3 movement should be the ones that show least neuronal adaptation. Also, cells that show relatively small ERF size, low IRF inhibition and higher response to the stimulus moving at higher speeds could be more responsive to the respiratory induced movement of stimulus. 3.5.4. R4 Cell Due to the nature of brief but prominent firing profile of the delayed diffuse-On response of R4 class neurons, we believe the response is due to direct input from the transient ATD channel. Inclusion of PBH channel to the ERF accounts for the SO and sustained Off responses. And, incorporation of the delayed ATD channel made possible the reproduction of the delayed diffuse-On response. As a result, the R4 model was able to better match the experimental data. 85 CHAPTER 4 ANURAN STATIONARY OBJECT PERCEPTION MODEL Anurans (frogs and toads) are able to perceive stationary objects, as shown in their adaptive behavior when stationary objects are present in the surround. Yet, anuran retina is oriented to perceiving movement-gated stimuli and these animals did not develop visual cortex. Rana computatrix (Arbib, 1989), an evolving computer model o f anuran visuomotor coordination, analyzes schema and neural network models o f complex behaviors such as detour around a stationary barrier to capture a prey or escape an enemy. The retina model o f Chapter 3 serving as a front-end, we model pretectal neurons believed to be involved in anuran stationary object perception. We then investigate how stationary objects are represented in a two dimensional neural map and may be interpreted in view o f the behavioral data on animals with different stationary objects. Memory is believed to play a role in the representation, modulation, interpretation, and subsequent action against stationary objects. The chapter concludes with a section discussing some parallels and comparisons between mammalian and anuran SO perception. 86 4.1. Introduction Even though we know that anurans perceive different SOs thanks to behavioral studies done by Ingle (1977, 1983) and Collett (1982), the underlying mechanism for anuran SO perception is not well understood. From Ingle’s lesion studies, and Ewert (1971) and Brown and Marks’ (1977) physiological experiments we know that pretectum is the main locus for anuran SO perception with certain cells showing responses to simple SOs. The aim of this chapter is to build a plausible neural model of anuran stationary object perception based on the data above. The SO related behavioral data offer us top-down clues as to how the neuronal SO pictures “should” be based on our interpretation of anuran behavioral data in response to different SOs. On the other hand, neurophysiological data of retinal and pretectal cells provide the front-end with the lowest form of constraints for building the neuronal SO pictures. The retina models presented in Chapter 3 furnish “raw” visual signals to anuran SO perception process occurring in pretectum. 4.1.1. Behavioral D ata o f Anuran Stationary Object Perception Ingle (1977, 1983) has shown that when a frog is placed in a white walled experimental arena having a real aperture, an open window through which white vertical stripes placed 8cm behind are visible, and a false aperture, a window with white vertical stripes of equal angular width affixed, the animal typically chooses the correct, open, aperture during the escape. Tectal ablation did not affect the performance but frogs that had undergone pretectum lesion were not able to 87 recognize the correct escape window and actually showed a tendency to avoid the aperture and collide with the wall during the escape. Ingle also tested the behavior of normal frogs to various objects like homogeneous black patches, and striped patterns/barriers with various luminosities, sizes, orientations, and relative distances w.r.t. the background (1971, 1983). When a wide (145°) vertically striped barrier (vertical paling fence) is placed in front of a prey, frogs usually detour around the barrier (78%), but the frogs usually collided (84%) with the same sized horizontally striped barrier. However, when a narrow (90°) horizontally striped barrier is introduced rostrally, the animals showed more detour behavior (56%). When a vertically striped pattern of 30x30° is affixed on the white background wall, frogs usually avoided the pattern (90%) during the escape. However, against the same sized horizontally striped pattern affixed to the wall, frogs actually jumped to it in 59% of the trials (chance = 18%). Moreover, to a solid black patch of equal size, the frogs actually jumped more to it (71%). Ingle further showed that when a black panel subtending the same retinal angle is placed in front of the wall, the frogs avoided the panel. Ingle’s aperture experiment shows that the tectum or any information coming via tectum is not necessary for stationary object processing. It also suggests that the pretectum-lesioned animal’s failure is likely due to its inability to distinguish the correct aperture from the wrong one for the lack of the depth segmentation of the stripes and the wall. These data suggest that the pretectum may be sufficient for the depth estimation of SOs and that the animal not only relies on form but also depth information to classify an SO. Response of a frog to the different patterns, stripes and solid patches gives us an idea of how the animal may interpret these objects and 88 also an underlying mechanism. The observation that the frogs reliably detour around a vertically striped barrier of any width but collide with the wide horizontally striped barrier suggests that the animal’s SO perception is specifically sensitive to the vertical orientation of the pattern but unable to segment the wide horizontally striped barrier from the background. Since the narrower horizontally striped barrier is more reliably detected, not only the orientation of the pattern but also the overall size of the pattern may be a factor in the figure-ground segmentation process. That is, the smaller an SO is, the more likely will it be segmented as a figure. We believe the animal’s detour around the vertically striped pattern is due to its innate nature of avoiding plants and stems that constitute much of the natural SOs. It is understandable why the frogs avoid vertical stripes during the escape for it may be viewed as an obstacle that hinders a maximum escape jump. Then why do frogs approach the affixed horizontal stripe pattern and the black patch? Ingle suggests that it may be the adaptation of “hide in a hole” behavior. A hole as perceived by the frogs should then be a dark patch that does not protrude from the background. The observation that the horizontal stripe pattern is perceived as a “weaker” version of a hole in that it also attracts the animal during escape but less often than the black patch suggests that the horizontal pattern and the black patch must be viewed similarly by the frogs. This view is also supported by the observation that the small horizontal stripe barrier and the solid black panel when they are brought in front of the background wall are perceived as a barrier, most likely due to a depth difference, and the frogs show detour behavior. The fact that average luminosity of horizontal and vertical stripe pattern is about equal suggests that the discriminating feature should then be the stripe orientation. These 89 observations lead into the issue of pattern formation: i) What would the retinal pattern look like?, ii) How would the SO pretectal pattern look like based on the retinal pattern, and iii) What visual processing would lead to the pretectal pattern that offers useful visual features for pattern interpretation? In summary, analyses of Ingle’s behavioral data suggest, i) pretectum is the main locus for anuran SO perception and it may not need any other inputs than the retina, ii) frogs have a special orientation sensitivity to vertical pattern which suggest underlying filters (cells) with vertically oriented specificity, iii) frogs use size as a cue for figure-ground segmentation, and iv) frogs use pattern (edge) orientation and depth information for further segmentation and identification of the object. 4.1.2. Anuran Pretectal Cells Ewert (1971) found in toad’s pretectum, near the ventral part of the pet (posterocentral thalamic nucleus), units (Th 10-class) that give continued discharge in the presence of a large dark stationary object even when the object was revealed by turning on the room lights without prior motion. As Ewert suggested, we believe the ThlO cells are the major component in anuran SO perception. Class ThlO neurons: With ERF of about 30-90°, these neurons exhibit prolonged discharges to a large contrast stimulus that is stationary in their ERFs. Four main subclasses of neurons are described: 90 Class ThlO (l) neurons are activated by a dark contrast object moving into the ERF and continue to respond even if the object stops there. When the lights are turned off, the discharge returns to its spontaneous level, and when lights are turned on again then the neurons discharge above the spontaneous level again. These neurons also respond to sudden changes of diffuse light with brief On - Off bursts. Class Thl0(2) neurons give a prolonged discharge to the room lights being turned off but otherwise behave similarly to class ThlO(l). Class Thl0(3) neurons do not respond to a changes of the general illumination. They are specifically activated by stationary visual objects and if an object is moving within the ERF, the neuronal response is inhibited. Class Thl0(4) characteristics are similar to those of Th 10(1) except that its ERF includes a binocular sector of about 90° in the frontal visual field. Ingle (1976) also found SO sensitive neurons in atectal frogs similar to ThlOs in toads. These neurons are located when the microelectrode penetrates, ventro- medially, below a depth of 0.5m m from the denuded pretectal surface, and only within that region where retinal units were recorded (Ingle, 1983). This physiological data supports the lesion/behavioral observation that the tectum is not involved in SO perception. Thus we have evidence that there are neurons in anuran pretectum that are sensitive to stationary objects. While it is known that some differences exist between frog and toad cells of the same region (Griisser & Griisser-Comehls, 1976), their overall similarities in the qualitative characteristics may be sufficient for us to apply data from frogs and toads interchangeably. Thus in the remainder of this 91 chapter, behavioral data from frogs will apply to the toads and neurophysiological data from toads will apply to the frogs. 4.2. Single Cell Models of Pretectal Neurons 4.2.1. Inputs to Pretectal Cells Retinal projections to the pretectal area terminate in the nearly cell-free neuropil near the lateral edge of the diencephalon, while the more medial pretectal cell groups send dendrites laterally to contact these retinal fibers (Ingle, 1983; Lazar, 1989). The projection to pretectum is topographical and bilateral1. However, among the retinal ganglion cells, only R3 and R4 cells are positively identified to project to pretectum (Ewert, 1971; Grusser & Grusser-Cornehls, 1976; Ingle, 1983). The data are somewhat disturbing because there is no concrete evidence that R l, sensitive to stationary edges, projects to the pretectum. Grusser & Griisser-Comehls (1976) report that when a large black stationary figure (10x30° bar) resembling a horizontal tree trunk in a natural environment was presented to frogs or toads, R l (in frog), R3 and R4 neurons were activated during detour behavior. Interestingly, R l and R3 activation were accompanied by active eye, head or body movement. Pretectal afferents, besides retina, come mainly from the tectum, anterior thalamus, and striatum (Fite & Scalia, 1976; Scalia, 1976; Wilczynski & Northcutt, 1983a). We have seen earlier that the tectum is not involved in SO recognition. 1 Compared to retino-tectal projection, the retino-pretectal projection forms a retinal representation that is to some extent a mirror image of the projection diagram in the ipsilateral tectum and in a coarser grain (Grusser & Grusser-Cornehls, 1976). 92 Striatum does not receive direct retinal inputs so its function as the primary visual source for SO perception is dubious. Anterior thalamus does receive retinal inputs and is known to possess a rudimentary SO perception faculty (Ingle, 1983). However, it unlikely that the pretectum uses the indirect anterior-thalamic information as raw material for its SO processing. So, the ThlO class neurons’ primary visual input most likely comes directly from retina. Because the more elaborate visual preprocessing in anuran retina is developed to offer extra sensitivity and specificity to m ovem ent-gated stimuli, the sophistication does not offer any beneficial effect for the frog’s perception of SOs. In fact, frog’s more specialized retinal information for moving stimuli could be detrimental to developing the more elaborate visual capabilities to SOs because the general information adequate for SO perception may have already been lost at the retina level (Lennie et al., 1990). So, the challenge is to determine how frogs and toads perceive SOs with retinal signals that are inherently better equipped to furnish later visual processes with information about movement-gated features of dynamic objects than of SOs. There are also some clues that ThlO neurons may receive afferents from other pretectal cells as well: While all ThlO class neurons are responsive to an SO, their firings also depend on general illumination (ThlO (l) & T hl0(2)) and also on cutaneous stimuli. Since there are pretectal neurons that are sensitive to general illumination (Th7 class) and to cutaneous stimuli (Th2, Th5(3), and Th7), it is possible that these ThlO neurons have inputs from the above pretectal neurons. It should be noted that, in accordance with the hypothesis of independent SO processing in the pretectum, the pretectal cells that may furnish afferents to the ThlO 93 cells should derive their own inputs directly from the retina or other relevant sensory organs. 4.2.2. M odels o f Pretectal Cells In this section, based on Ewert’s (1971) data, we present models of anuran pretectal cells believed to be involved in anuran stationary object perception. Among the four subtypes of SO sensitive ThlO cells, we present models of ThlO (l), Thl0(2), and Thl0(3) cells. The binocular Thl0(4) cell model is not attempted because explicit physiological recordings of the cell are not available. But from Ewert’s report that says Thl0(4) characteristics are similar to T hlO (l)’s, we believe Thl0(4) model could be constructed similarly to Th 10(1)’s. We first attempted to model the ThlO class neurons based solely on direct retinal inputs. However, simulation showed it to be difficult to replicate the experimental data with direct retinal inputs alone. Thus, we posit that the ThlO class neurons receive further inputs from other pretectal cells specialized to movement detection and general illumination changes such as Th4 and Th7 neurons. However, modeling of Thl0(3) neuron was still problematic based on known retinal (R3 and R4) and intra-pretectal inputs. Thl0(3) responses are best accountable with Thl0(3) model having inputs from SO sensitive units, R1.2s, which are not found in pretectum. Although simulation of ThlO (l) based SO representation in the next section shows that population of T hl0(l)s suffices to encode appropriate features for the pattern recognition of SOs, we develop a hypothesis of T hl0(3)s’ role in the 94 overall anuran SO perception strategy based on the assum ption that Thl0(3)s receive R1.2 inputs. Because we are interested in the visual information processing of anurans, the cutaneous stimuli related responses of the Thl0-3s are not dealt with in this thesis. As stated earlier, pretectal cell models presented here rely on the retina models in Chapter 3 to furnish input visual signals. The structures and the equations of the pretectal neuron models are illustrated in Fig. 4.1 and Table 4.1 respectively. a) T h4 neurons: These are called movement-sensitive units with large visual fields (Ewert, 1971). With relatively large ERFs of 90-180°, these neurons fire to any object movement within the ERF. These units seldom show On or O ff responses to diffuse light changes. Th3 neurons show similar responses as Th4 neurons except that these cells have smaller ERFs of about 30° and show Off activity (Ewert, 1971). ThlO-l ThlO-3 Excitatory Connection: Inhibitory Connection: Figure 4.1. The structure of the pretectal cell model. See the text for details. 95 Because of this functional similarity, the Th4 model proposed can easily be modified to a Th3 model by changing the parameters. Th4s are modeled to receive R3 (retinal movement detector) inputs through the DOG receptive field of inner ERF region of 90° and IRF region beyond. Detection of a movement within the ERF is based on the linear summation of those signals that are above a certain threshold. Th4 is simply modeled as an on-off neuron that fires at a constant value when the summation is above the specified threshold. When there is sufficient firing across the IRF region as the illumination changes, this inhibitory effect suppresses Th4’s On or Off responses during the diffuse light changes. The Th3 neuron can be constructed from the Th4 model by reducing the ERF size and lessening or abolishing the IRF influence. Because the Th4 cells do not affect pretectal SO pattern formation, for the simplicity of simulation, we modeled single Th4 cell to represent a population of Th4 cells. b) Th7 neurons: These are called darkness-detecting units (Ewert, 1971). Among the three subclasses, we modeled Th7(l) neurons which respond to any reduction of the general illumination with a prolonged tonic discharge sometimes lasting 5 minutes or longer. They are modeled to receive R4 (retinal dimness detector) inputs through the receptive field which is modeled to be the entire view field of the animal. The cell is made to generate prolonged tonic discharge so long as the R4 summation across its receptive field remains above the specified threshold. Because Th7 cells do not take part in pretectal SO pattern formation, we modeled a single Th7 cell that represents a population of Th7 cells. 96 Th4 Cells: fl if kO * R3th - kl * R3th > 0 Th4 = \ 1 [0 otherwise where R3th = R3 - (Threshold=1.3) with kO = mask(90, 5.0, 1), k l = mask(180, 6.3, 3) * For 77*3: kO = mask(30, 5.0, 1), k l = mask(90, 5.0, 0.5) Th7(l) Cells: 77*7(1) = 0 i f £ R4 < Threshold 77*7(1) = 1 i f R4 > Threshold & 77*7(1) = 0 _ d(T hl(l)) n r Thi 1--------- = -77*7(1) i f T hl(l) > 0 dt with Threshold = 25000, rThl =100 ThlO-l Cells: Thl0-1 = kO* R3 + kl* R4 - 10- Th7(l) with kO = {mask(4, 1.5, 4.0) - mask(8, 5.0, 2.0)}, k l = mask(45, 10, 10) ThlO-2 Cells: ThlO-2 = kO* R3 + kl* R4 with kO = fmask(4, 1.5, 4.0) - mask(8, 5.0, 2.0)}, k l = mask(45, 10, 10) ThlO-3 Cells: Thl0-3 = kO * R1.2 + kl* R4 - 10- Th4 with kO = {mask(9, 1.0, 2.0, 10.0) - mask(9, 2.0, 2.0, 9.0)}, k l = mask(45, 10, 2.5)____________________________ Table 4.1. Equations for pretectal cell models 97 c) ThlO-1 neurons: These units are modeled to receive excitatory R3 and R4 inputs and the inhibitory input from T h7(l) neurons. A detailed account on the construction of the model is presented in the next section. Fig. 4.2 shows that the ThlO-1 model simulation matches well with the experimental data. d) ThlO-2 neurons: These units are modeled identically to ThlO-1 except that they do not receive T h7(l) input. Fig. 4.3 shows that the ThlO-2 model simulation matches well with the experimental data. e) ThlO-3 neurons: Because the neuron does not exhibit On-Off responses to a diffuse light change, it either may not receive R3 inputs which show On-Off responses or receives R3s but with On-Off responses somehow suppressed. Assuming the Thl0-3s receive R3 inputs, the best way to account for the absence of On-Off responses would be to model the Thl0-3s to receive inhibitory input from cells such as Th3 and Th5(l) that show Off responses. In fact, inhibition from Th3 or Th5(l) can also account for T hl0-3’s inactivity when moving objects appear in the ERF because Th3 and Th5(l) respond to moving objects. On the other hand, the above modeling of ThlO-3 does not account for the neuron’s lack of response to On activity and the large receptive field size because the Th3 and T h5(l)’s receptive field size is smaller (15-50°) than T hl0-3’s (30-90°). With the above problems, the ThlO-3 is modeled to receive R1.2 and R4 inputs that do not show On-Off responses. Because neither R ls nor R2s have been found in the pretectum yet, it is our assumption that these cells project to pretectum and contact ThlO-3 neurons. Thl0-3s stop firing when moving objects appear in the 98 5 1 1 1 l l 1 1 1 1 ' I .. r— ■.i .i “ i— 4.5 13.data" --- " d ) 4 r a 3.5 - V 3 - - <?2.5 - - •H 2 - _ ■ H 1 .5 / I - h. x / \ - 0.5 n t i l l V j 1 1 ! 1 I i \ i i t “ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Time(sec) "R4.data 0 2 3 4 1 5 6 8 7 9 10 11 12 13 14 15 16 17 Time(sec) 2 Th7.data 1.5 1 0.5 0 0 2 6 1 3 4 5 8 9 10 11 12 13 14 15 16 17 7 Time(sec) 5 4.5 4 3.5 3 2.5 2 1. 5 1 0.5 0 'ThlO-1 .data 0 1 2 3 4 5 6 8 9 10 11 12 13 14 15 16 17 7 Time(sec) off on so Figure 4.2. Ewert’s (1971) data for stationary-object sensitive ThlO-1 pretectal cell (bottom) and the simulation of the ThlO-1 model (top four). Firing-rate Firing-rate Firing-rate 99 5 R3.data 4 3 2 1 0 2 0 4 3 10 6 12 14 16 Time(sec) 5 R4.data" 4 3 2 1 0 0 2 4 6 8 10 12 14 1 6 Time(sec) 5 "ThlO-2.data 4 3 2 1 0 0 2 4 6 8 10 12 14 16 Time {sec) off________ on Figure 4.3. Ewert’s (1971) data for stationary-object sensitive ThlO-2 pretectal cell (bottom) and the simulation of the ThlO-2 model (top three). 100 !U J-> r t J M I CP 5 4.5 4 3.5 3 5 2 1.5 1 0 . 5 0 "Rl.2 data 0 2 1 3 5 6 7 8 4 9 10 11 12 13 14 Time(sec) 5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 R4.data 0 2 3 6 7 8 9 10 11 12 13 14 1 4 5 Time (sec) 2 Th4.data 1.5 1 0.5 0 3 8 2 5 6 7 9 10 11 12 13 14 0 4 1 Time(sec) 5 4.5 a 4 S 3-5 r 3 ?2.5 ■ 5 2 ■ h 1.5 ^ 1 0.5 0 ThlO-3.data 0 2 6 8 9 10 11 12 13 14 1 3 4 5 7 Time(sec) so mo H 1 sec so mo so Figure 4.4. Ewert’s (1971) data for stationary-object sensitive Thl0-3 pretectal cell (bottom) and the simulation of the Thl0-3 model (top four). 101 ERF. Thus, these neurons are modeled to receive inhibitory input from the Th4 neuron that is sensitive to moving objects with large ERF. Fig. 4.4 shows that the ThlO-3 model simulation matches well with the experimental data. 4.3. Model of Pretectal SO Representation As with other animals, anurans evolved with abilities that are crucial in the survival of species in their ecological environment. As such, anurans developed a visual system that is specifically adapted to perceiving the moving objects that are of greatest concern to them. In this context, it is possible the anuran visual perception of SOs is less developed than the perception of moving objects since SOs are of secondary concern to the animal. A static visual environment of anurans includes objects like leaves, branches, plants, stems, and stones. Even though there are enough form/configural differences in these objects so as to be appropriate inputs for the frogs to develop an adequate SO recognition system, to date the measure of their capability is not clear because the experiments were done only with highly regular SOs. But still, earlier behavioral studies do show us how the anuran SO perception mechanism may work. Earlier we saw that analysis of Ingle’s data suggests that frogs may view a dark horizontally striped barrier and the same sized dark solid patch similarly, whereas a vertically striped barrier (similar to Collett's paling fence barrier) is interpreted differently. This suggests that the neural representation of the horizontally striped barrier and solid dark patch should be similar while that of the vertically striped barrier should be different. It is also certain from observing anuran behavior that the 102 animal perceives the location of each paling comprising the vertical paling fence quite accurately because these animals seldom try to go over the palings but instead aim at inter-paling gaps during their approach to a prey behind the barrier. This suggests that the representation of the paling fence barrier should encode the specific locations of each paling and not just the boundary palings or the barrier as a single integrated object. So, while the animal’s SO perception faculty is generally limited compared to mammals, nevertheless it also clearly indicates that not all SOs look the same to the anurans. It is now known that the anuran recognition system for moving objects is not based on dedicated specialized detectors at the retina level such as the “bug detector” of Barlow (1953). Rather, the recognition of a prey or a predator occurs at the level of tectum and pretectum through interactions and integration of “general purpose” neurons showing certain preferences to specific features (for a review, see Ewert 1987). Likewise, it would be all the more surprising if anurans developed even a few dedicated detectors for the specific SOs among the more diverse but less important repertory. Thus, we believe that SO recognition should be based on a neuronal representation of the SO on a population of “general purpose” feature sensitive neurons. What is lacking here is that, to date, we do not know whether or how SO sensitive cells, ThlOs, may respond to different spatial objects. It is known that high-level mammals’ complex SO perception is based on multitudes of detectors (filters) specifically sensitive to different orientation, spatial frequency, and direction of stationary and moving patterns (De Valois et al., 1982a, 1982b; Wilson et al., 1990). The aim of this section is to develop a plausible model of the anuran SO representation that accounts for the animal’s behavioral data against the 103 SOs (Ingle, 1983) based on populations of the (physiological) cell models described in section 4.2. 4.3.1. SO pattern form ation A two dimensional layer of pretectal ThlO neurons will be the substrate of SO recognition in our model. Since an SO forms a 2-D pattern on the pretectal surface due to the topographical retina-> pretectal projection, features for SO recognition should be based on 2-D firing pattern on the ThlO layer. Even though it is not modeled here, since one subclass, Thl0(4), has a binocular ERF anurans could have a stereoscopic cue to estimate the depth, which is one important cue for SO recognition. The pretectal ThlO based SO recognition model should satisfy the following criteria: i) A simulated ThlO cell should replicate the response of a real cell, including the ERF size and the temporal response to different stimuli, and ii) An SO pattern on the ThlO surface should offer neuronal cues such as edges and boundaries for identification. We have seen earlier that temporal firing patterns of the ThlO models match the experimental data quite well. This section presents a neural process of pattern formation of SOs in pretectum. 4.3.2. ThlO-1 Pattern An understanding of SO pattern formation on the ThlO-1 layer requires knowledge of the distribution of retinal projections to pretectum and the dendritic processes of 104 ^ ThlO-1 cells. Lacking both data, we modeled T hlO -l’s R4 receptive field to be large (45°) to account for ThlO’s large receptive field and its increasing response to large dark objects. Convergence of R3 explains ThlO’s On-Off response to light j changes. Inhibition from Th7(l) suppresses activation during the Off period. Due 1 ; to the R 4’s large ERF and aforementioned broad receptive field characteristic of ThlO-1 to R4s, its boundary sensitivity is very poor. R3 is sensitive to moving edges and has better resolution, but how could a stationary object elicit a retinal image movement to generate R3 activation? There was a suggestion earlier that activation of some retinal ganglion cells responding to SOs was accompanied by active eye, head, and/or body movements. Indeed, anurans seem to generate retinal responses on retinal surfaces that consist of movement-gated neurons such as R3 and R4 through their diverse self induced motions. Pigarev et al. (1971) report that R3 neurons are activated by each active head and body movement. Grusser and Grusser-Cornehls (1976) elicited R2 and R3 i responses from incompletely immobilized frogs and toads during eye-retraction/lid- closure. Burghagen and Ewert (1983) observed that stationary prey can elicit prey- catching behavior from toads if the moving retinal image is induced by the toad's walking or passive rotation. Tsai and Ewert (1988) clearly demonstrate that so far I | as retinal neurons are concerned they do not discriminate whether the movement is I caused by the object’s movement or by self-induced motion. In fact, it seems that i j anurans lack an anticipatory mechanism, such as the reafference signal (Berkinblit et i al., 1986), to distinguish the reason for the retinal image movement, and so the | animals ignore a prey which moves with a structured stationary surround when they j move their head (which makes the retinal image of the surround move) (Ewert, 105 1987). So the issue here is how much retinal movement (minimum displacement and velocity) is needed to activate the corresponding retinal ganglion cells. As we saw in Chapter 3, respiration causes periodic and predominantly vertical eye m ovem ent of about 0.5° every 0.63 seconds (Schipperheyn, 1963). Schipperheyn proposed that the respiration-induced eye movement produces a moving retinal image and thus can elicit activation of movement specific ganglion cells such as R3s. Grusser and Grusser-Cornehls (1976) report that they observed that the most sensitive R3 neurons were activated when a stationary contrast object smaller than the size of the ERF is placed in their ERFs and when small shifts in the retinal image occur during respiratory eye movement. However, Pigarev et al., and Burghagen & Ewert failed to see respiration-induced R3 firings. The above conflicting data led us to see if respiration induced R3 excitation is possible through retinal modeling (Chapter 3.3; Teeters et al., 1993). As we saw in Chapter 3, R3 activation from respiratory eye oscillation is possible and the responses closely resemble the “R6 ” neurons, cells with small receptive fields (3-6°) that discharge for 10 seconds or more to a dark edge oscillating at 0.5°/sec, that Ingle (1976) characterized in the pretectal neuropil. Thus, the R3s in the ThlO-1 model are capable of furnishing the boundary information of an SO due to the respiration induced retinal SO movement. Because the respiration comprises mostly vertical oscillations, respiration induced R3 activation (Resp-R3 response) should signal horizontal edges. In order to preserve an edge location, T hlO -l’s R3 receptive field is modeled to be much more focused than that of R4. Simulation shows that the R3 receptive field should be either very narrow (less than 3°) but could be somewhat broader (up 106 to about 10°) if the receptive field is modeled as a center-excitatory, surround- inhibitory Difference-Of-Gaussian (DOG) mask. The ThlO-1 temporal response (Fig. 4.2) and pattern (Fig. 4.5) shown are based on the DOG mask. Fig. 4.5 shows the pretectal image of a vertically striped barrier, a solid dark patch, and a horizontally striped barrier. We can see that the solid dark object (Fig. 4.5b) and horizontally striped barrier (Fig. 4.5c) form similar patterns across the ThlO-1 surface in that they both do not present conspicuous features along the horizontal axis except that the latter shows blobs of activity at the lateral edges of the stripes whereas the solid object does not. This activity decreases as the R3 input weight to the ThlO-1 is decreased. Also, tighter spacing of the horizontal bars makes the two objects more similar by comparative decrease in the lateral edge response due to the greater lateral inhibition induced by the internal bars, and the rise of the overall potential in the horizontally striped barrier (Fig. 4.5d). On the other hand, the image of the vertically striped barrier (Fig. 4.5a) clearly delineates the horizontal eccentricities of vertical stripes along the lower and upper boundaries. Based on the ThlO-1 patterns, we may simply regard the neuron as a 2-D location (horizontal and vertical eccentricity) detector of the vertical bars. 4.3.3. ThlO-2 Pattern ThlO-2 patterns of SOs are identical to those of ThlO-1 patterns in the photopic range. 107 Figure 4.5. Stationary objects (left) and corresponding pretectal ThlO-1 images (right), (a) Vertically striped barrier, (b) Dark solid patch, (c) Horizontally striped barrier, (d) Dense horizontally striped barrier. 108 4.3.4. ThlO-3 Pattern Lacking data on the anatomy of Th 10-3 neurons, we modeled T hl0-3’s R4 receptive field to be large (45°) to account for T hl0-3’s large receptive field and its increasing response to large dark objects. R1.2 afferents encode the location of edges and boundaries of an SO. Because the parameter setting for R1.2 does not affect the ThlO-3 results, R1.2 for ThlO-3 is simply modeled to be pure R l. Inhibition from Th4 suppresses the activation during object movement. We posit that T hl0-3’s R1.2 receptive field is orientation sensitive to the vertically oriented patterns to explain the animal’s behavior to different orientation of the SOs. The T hl0-3’s receptive field to the R1.2s is modeled as center-excitatory, surround-inhibitory non- concentric DOG mask (Fig. 4.6). With this receptive field, the ThlO-3 is basically a vertical orientation sensitive filter with a spatial frequency correlated with the shape/size of the receptive field. If we regard R1.2 as the least specialized retinal ganglion cell class in the sense that it responds to more stimuli of different characteristics than other types, then with the assumption that ThlO-3 receives R1.2 it is possible to view the neuron as a late development among the subclasses of the ThlO family that is better equipped to perform more elaborate form perception with its general R1.2 signal. As such, similar to mammalian visual cortical cells, Thl0-3s may possess elementary orientation sensitivity to a pattern the animal is most interested in. 109 Figure 4.6. The T hl0-3’s receptive field pattern to R1.2 afferents. Fig. 4.7 shows the pretectal ThlO-3 image of a vertically striped barrier, a solid dark patch, and a horizontally striped barrier. The neural patterns of the solid dark object (Fig. 4.7b) and the horizontally striped barrier (Fig. 4.7c) are similar in that they both do not show conspicuous features (excitation) along the horizontal boundaries and that they both exhibit strong excitation along the two lateral boundaries. As the spacing between the horizontal bars in the horizontally striped barrier becomes closer (Fig. 4.7d), the more its neural representation resembles the solid dark object. On the other hand, the neural pattern of the vertically striped barrier (Fig. 4.7a) clearly shows the individual vertical stripes. 4.4. Anuran Depth Perception of Stationary Objects We stated that depth information is an important feature in the classification of SOs. How then do anurans estimate depth? In this section, we will present an anuran depth perception model of stationary objects such as a paling fence barrier. 110 Figure 4.7. Stationary objects (left) and corresponding pretectal ThlO-3 images (right), (a) Vertically striped barrier, (b) Dark solid patch, (c) Horizontally striped barrier, (d) Dense horizontally striped barrier. I l l Collett (1977) showed that frogs and toads use both binocular and monocular cues to determine depth. Collett found that while binocular toads utilize binocular cues predominantly (94%) when estimating the depth of prey in the binocular field, nevertheless their depth estimation does include a small (6 %) effect from monocular accommodation. Deprived of binocular cues, monocular toads still retain an ability to judge distance through accommodation cues. Thus, data exist regarding frog/toad’s depth perception against prey. However, very little is known about how the animal might estimate the depths of SOs. House (1989) offered a cooperative model, the cue-interaction model, of SO depth perception in which the lens accommodation cue is used to help disambiguate the correspondence problem of stereopsis. The model is constructed based on the physiology and anatomy specific to frogs/toads. House insists that any anuran depth perception model i) should not depend on eye vergence, ii) must account for use of an active process like accommodation, iii) must not depend on the tectal map, and iv) must be able to operate on sparse images, because the visual maps cannot be assumed to represent local illumination level since the visual input from the retina is feature encoded. House assumed that binocular disparity cues are processed using a Dev-Marr (Dev, 1975; Marr and Poggio, 1976) type cooperation/competition algorithm. The model demonstrated its ability to estimate the depth of a single object (prey or predator) and a paling fence. It also predicts that concave lenses will have a consistent effect, with stronger lenses causing a fence to appear closer to the animal. Prisms, on the other hand, should have a more variable effect, even causing the 112 fence to be fragmented, so that the animal may occasionally behave as if there were gaps in the fence. One of the problems with House's model was that it was not able to resolve the double barrier used in some of Collett's experiments. To address the deficiency of House’s model, we looked into Prazdny's (1985) stereo correspondence algorithm. Prazdny formed his algorithm based on a coherence principle: “the world is not made of points chaotically varying in depth but of (not necessarily opaque) objects each occupying a well defined 3D volume.” The principle is different from the continuity constraints widely used in Dev-Marr type stereopsis algorithms in that it recognizes that for transparent surfaces, where proximal points on the (retinal) projection surface may arise from widely separated 3D objects, image proximity does not necessarily imply disparity continuity. Thus, by relaxing the continuity constraint of the typical cooperation/competition stereo algorithm, the model is better equipped to handle quasi-transparent objects like bushes, and double paling fences. The coherence principle requires that neighboring disparities of elements corresponding to the same object be similar, suggesting that the principal correspondence mechanism should be facilitation due to disparity similarity. The disparity similarity function that Prazdny proposed is based on three requirements — i) The disparity similarity function should be inversely proportional to the difference of disparities of interacting points, ii) More distant points should exert less influence while nearby matches should have more disambiguating power, & iii) The more distant the two interacting points are the less seriously should their disparity difference be considered — is the familiar Gaussian distribution function 113 k -4l2 ,„2 |, _,2 where Ws(i,j) expresses the amount of support disparity di at a retinal point i receives from disparity dj at another point j, \i-j\ is the distance between the two retinal locations, and c is a scaling constant. By replacing the Dev-Marr algorithm in House's model with Prazdny's (1985) stereo correspondence algorithm, we were able to overcome the double-fence problem. Table 4.2 illustrates the algorithm for the revised cue-interaction model. Here, M is the two dimensional excitatory field receiving monocular depth cues from lens accommodation layer (A), while S is the two dimensional excitatory field receiving binocular depth cues from disparity matching layer (D). The two layers interact to reinforce similar depth estimates by means of cross coupling pathways. The spatial coordinates of layers M and S are retinal angle q and disparity d. U and V are one dimensional inhibitory layers associated with M and S. Internal potentials in M and S are converted to firing rates by a saturation-threshold function f. Each point in layer M (S) receives excitatory stimulation from the A (D) through the gain Ka (Kd), and from neighboring points through the spread function wm (ws). Excitatory input to U (V) at retinal position q comes from through the gain Ku (Kv) and is the integral along the disparity coordinate of all excitation at position q in the M (S). In turn, layer M (S) receives an inhibitory signal from the inhibitory pool U (V) through the gain Km (Ks). Internal potentials in the inhibitory layers U and V are converted to firing rates by a ramp-threshold function g. Excitatory cross coupling between the M and S layers is represented by additional terms providing Ws(iJ) = c\i~J iW2n 114 stimulation from each point in layer S to the spatially corresponding point of layer M through gain Ksm, and from M to S through gain Kms. The rate of change of potential in layers M, U, S, and V are governed by the time constants ^ " m > Tj» &nd Tv. Input features to the revised cue-interaction model come from the ThlO pattern of a SO. The cue-interaction model samples locations of local maxima points on the ThlO surface along the horizontal axis, which are the horizontal eccentricity coordinates of the constituent palings forming the barrier. This information is used to set up the D field which is an input to the stereopsis component of the cue- interaction model. The accommodative input layer, A, is approximated as a Gaussian distribution along the disparity coordinate centered at the correct depth. Decreasing depth accuracy of farther objects are simulated by associating them with Gaussian distribution with larger standard deviation values. But in principle, the accommodative cue could be derived directly from the measurement of an SO ’s ThlO firing pattern. A correct focus will give a more sharply defined edges than a defocused state. Even though it is not shown in this thesis, we have preliminary data that show a simulated blurry defocused stimulus eliciting a retinal firing pattern that is weaker and spatially broader than that of correctly focused stimulus. We believe this is due to the low contrast gradient of defocused edges related to the contrast sensitivity function of retina ganglion cells (Griisser & Grusser-Cornehls, 1976). With ThlO firing pattern that is dependent on retinal firing profile, by varying the accommodative state and measuring the ThlO firings, the animal could obtain an accommodative cue which is the other component of the cue-interaction model. An animal cannot focus on two barriers simultaneously and it has to remember 115 TmM(q,d,t) = -M (q ,d ,t) + $ wm(q - £)f[M(C,d,t)]d£ + Ksm f[S(q,d,t)] - Km g[U(q,t)\ + Ka A(q,d,t) TuU(q,t)= - U(q,t) + Kujf[M (q, rsS(q,d,t)= - S(q,d,t) + J J ws(q - £,d - rj)f[S(£, ri,t)]d£dri + ? n Km s f[M(q,d,t)] - Ks g[V(q,t)] + Kd D(q,d, t) T vy(<?,0= “ + i5 T v J f[S(q, ri,t)]drj in2 1 2 2 w9((7, 7 ) - . — -e 2c ^ : Weight Function based on Prazdny. q<7|V27F A: Two dimensional accommodative depth inference system. D: Two dimensional binocular disparity depth inference system. M : Two dimensional Monocularly driven excitatory field. S: Two dimensional Stereoptically driven excitatory field. U: One dimensional inhibitory pool for M. V: One dimensional inhibitory pool for S. f, g: Threshold functions. t m,ru,rs,Tv: Time constants. Table 4.2. Equations for the revised Cue-interaction model 116 accommodative states of corresponding stimuli as it scans and compares the their firing profiles. This consideration suggests that the animal relies on memory of accommodation states during the depth estimation of SOs. No data exist as to where and what cells may be involved in the memory of accommodative states. But in light of Ingle’s (1991b) finding that frogs possess short term spatial memory of SOs in striatum, topic of next section, it is possible that the locus of accommodative depth memory is in the striatum as well. Fig. 4.8a shows a double barrier as an input stimulus, and Fig. 4.8b shows the frog's estimation of the barrier location after the cue-interaction process. Fig. 4.8c and Fig. 4.8d show the model’s performance when the inter-barrier distance of the double barrier becomes small. As the inter-barrier distance becomes smaller, the model faces the more difficult task of segmenting the two fences separated in depth, resulting in a fused fence-posts as we see from Fig. 4.8d. Figures 4.8b and 4.8d are generated from the retina_angle - disparity map where cue-interaction takes place. The simulation shows that the revised cue-interaction model can account for anuran depth perception of complex SOs like double paling fences, and thus the animal’s detour behaviors that require accurate depth perception of double barrier. 4.5. Memorized Representation of Stationary Objects We have seen that a reasonable model can be constructed for the frog’s substrate mechanism of SO perception based on retinal R3 and R4 signals in the Thl0-1(2), and R1.2 and R4 in the case of ThlO-3 together with appropriate intra-pretectal inputs. But would frogs have a sustained picture of SOs in light of the fact that all 117 Figure 4.8. Results with revised “cue-interaction” model for a double barrier, (a) A double barrier is given as input stimulus, (b) Frog’s estimation of the depth o f the double barrier, (c) A double barrier with closer inter-barrier distance, (d) Frog’s estimation of the depth o f the double barrier of (c). Estimation of the barrier location in (b) and (d) is depicted by the firing rate of cells corresponding to the location. 118 the anuran retinal ganglion cells show adaptation to a repetitive stimulus to some degree and that even the static edge detector R1 has finite duration of response (usually about one minute)? Even though Resp-R3 signal for the ThlO-1 representation may be capable of providing a weak but continuous signal due to R3’s least adaptation to a repetitive stimulus among the retinal ganglion cells, still the question remains how effective this signal is. R3 activation to an SO is stronger when the stimulus undergoes large retinal shift during the animal's active head and body movements (and short duration after the cessation of body movement due to after-discharge). Because experimental data suggestive of the role of these self induced retinal activation patterns are lacking, we posit their functions based on our qualitative interpretation of the simulation results and the significance these may have in overall anuran SO perception mechanisms. Ingle (1991b) showed that frogs (R. pipiens) possess short term spatial memory of a stationary object such as a paling fence barrier and it seems to be compensated during the frog's passive rotation. The frogs remember the location of a recently seen obstacle (vertically striped barrier) up to a minute after its removal, since they avoid escape to that obstacle direction when frightened, even when that direction offers the optimal escape route. Ingle also observed through lesion studies that this short term memory depends on striatum2 . Why do frogs avoid a barrier that is no longer there? Since it was also shown that striatum lesioned animals retain an ability to avoid a visible barrier, we are led to believe that pretectum suffices for the detection of visible barriers. So, if the 2 Sole afferent to striatum comes from anterodorsal thalamus. Ingle (1991b) suggests pretectum projects to anterodorsal thalamus via short axons. Anyway, the striato-pretectal-tectal loop is a well established concept (Fite & Scalia, 1976; Ewert, 1987; Matsumoto et al., 1991). 119 animal has the ability to take an action based on a currently visible stationary surround, why should the animal avoid the place where the barrier was? Is it, as Ingle suggests, to speed access to a potential escape route in the case of a sudden predation attempt? At the least, it is clear that the animal’s behavior is determined by both its current perception of SOs and the memory of the past location of SOs. Extrapolating this observation, we posit that the anuran internal model of the stationary world consists of integrated view of both the limited short duration “real tim e” image and the “memorized” component of the stationary world. The memorized component may not only encode the location of the SOs but also has precedence over the pretectal representation such that its signal can override the pretectal signal. Thus, the information from the memory works as a higher level signal that modulates the immediate sensory based information, which can be useful in certain circumstances. So we believe that the anuran spatial short term memory serves a much more important function than a mere repository of SOs but is a key component in the overall anuran SO visuomotor system. No data exist about the mechanism of the anuran striatal spatial memory. The way it is constructed also has a direct relevance as to how the memory is represented. Perhaps, we should begin with the issue of pattem-vs.-iconic memory. Does the anuran striatal memory hold a pattern of an SO similar to the one in the pretectal layer? Then, what does it entail? Or, does the striatum hold either a location of an SO or a target location of the animal’s movement as a response to the SO? If so, we need to ask where and how does the striatum-derived “where” information of an SO get integrated with its corresponding encoding of “what” type 120 of an SO, assuming the striatum can hold the location of general SOs and not just a specific object like obstacles. A visual input to striatum should come via other visual centers that receive retinal inputs because the striatum does not have direct retinal afferents (Neary & Northcutt, 1983). Striatal afferents originating in the thalamus, where the pretectal region belongs, come from the ipsilateral Lateral nucleus, anterior division (La) and Central thalamic nucleus (C) which receive tectal and toral input respectively and also from the ipsilateral posterior tuberculum (TP) (Wilczynski & Northcutt, 1983a). Striatal efferents to the thalamic region project to the ipsilateral caudal pole of the central thalamic nucleus (C), the posterior region of the Lateral nucleus, posterodorsal division (Lpd), the Lateral nucleus, posteroventral division (Lpv), and the posterior tuberculum (TP) (Wilczynski & Northcutt, 1983b). Spatial short term memory has been studied in the monkey brain in relation to memory-based saccadic eye movement and delayed matching/response tasks. Segraves and Goldberg (1987) observed that sustained tonic activity in frontal eye field codes the location of a remembered target in the delayed saccade task (Gnadt & Anderson, 1988). Fuster and Alexander (1973) showed that in a delayed response task, mediodorsal thalamus and prefrontal cortex are the substrates of the memory in the form of providing sustained activity during the delay, and that cooling the prefrontal cortex abolished both the sustained activity in mediodorsal thalamus and the animal’s ability to perform the task. With this information, Dominey and Arbib (1992) proposed a spatial short term memory based on a topographic reciprocal connection between the frontal eye field and mediodorsal thalamus forming a reverberating excitatory loop. The 121 memory loop is implemented with all excitatory one-to-one links, where an appropriate choice of time constants or some form of active regulatory control of the activity levels may have been needed to limit the runaway recurrent excitation. Based on Hikosaka et al.’s (1989) data linking caudate in striatum to the memory saccade, Dominey and Arbib proposed that the memory signal in frontal eye field projects to the caudate. Zipser (1991) and Zipser et al. (1993) proposed short term memory based on recurrent network having hidden units that are trained by “backpropagation through time”. The models are inspired by and constructed to account for the neurons involved in the memory saccade and the delayed matching task. The model showed its ability to match single-unit temporal activity patterns exhibited by many cortical neurons involved in the short term memory, and predicted that the neurons cannot maintain the activity level indefinitely but relaxes to one of several stable fixed attractor states. But, the notion that the model needs to train the hidden units through long series of supervised learning process in order to memorize a single neuron’s temporal firing pattern does not accord well with our view of short term memory where the memory is initialized rapidly and whose stable state is always the forgetfulness. Grossberg and his colleagues (Grossberg, 1976; Grossberg & Kuperstein, 1989) also presented a short term spatial memory model based on a reverberatory loop mechanism, and analyzed various aspects of the model. The model is simple, having just excitatory and shunting inhibitory connective patterns, yet self-regulates the activity pattern across the neurons and has the capacity to control the eventual trajectory of the dynamics through the tuning of a parameter. 122 If the striatal short term memory is implemented as a reverberatory loop, there should exist a corresponding anatomical loop. And so, we postulated two candidate loops and examined simulation results in order to deepen our understanding of the mechanism. We first looked into the possibility of a closed loop between the pretectum and striatum. Researchers like Lara et al. (1984) and Lazar (1989) report a loop between the pretectum and striatum. However, Neary & Northcutt (N&N for short), based on their classification of pretectal nuclei3, did not see clear evidence of any nuclei in the pretectal region projecting to the striatum either directly or indirectly through diencephalic regions. The problem of establishing the pretectum striatum loop is in establishing an ascending projection from the pretectum to the striatum. In Fig. 10 of Ewert’s (1971) paper, he gives the typical locations of the posterior thalamic (pretectal) units. Most neurons he observed were found in the posterocentral (pc) and the ventral thalamic nucleus (vt) in the median part of the pretectum. Fig. 4.9 shows the comparison of Ewert’s and N&N’s pretectal nuclear organization. If we compare Fig. 10c (toad) of Ewert and Fig. 5A (frog) of N&N, we can see that Ewert's pretectal regions correspond very well to the P, C, and TP regions of N&N in spite of the possible organizational difference due to species difference; dorso-medial subregion of pc corresponds to the P of N&N, ventro lateral subregion of the pc looks like caudal region of the C, and vt corresponds to the TP of N&N.4 It appears Ewert found “small field units” (Th3) in the P, and the 3 The pretectum, according to N&N, contains three nuclei: nucleus lentiformis mesencephali (NLM), nucleus o f the posterior commissure (NPC), and pretectal gray (PtG). This classification differs from the Ewert’s (1971). 4 Note that these regions were not classified as pretectal nuclei according to N&N. 123 m o t tec pet pet lot Lpd CO PTN TeO Lpv TP DH NPv Figure 4.9. Comparison o f Ewert’s (top) & N& N’s (bottom) pretectal nuclear organization. Ewert's pretectal regions correspond very well to the P, C, and TP regions o f N&N; dorso- medial subregion of pc to the P of N&N, ventro-lateral subregion of the pc to the C, and vt to the TP of N&N. See the text for details. 124 Th5a, Th5b, and “stationary-sensitive” (ThlOb) units in the C. And in the TP region, he found Th8b and Th8c type neurons. The pc region where Ewert found ThlOs is the caudal region of N&N’s C which is located at the pretectal level and is observed by N&N to project to the striatum while also receiving the afferents from the striatum. N&N also have seen that a closed loop exists between TP (Ewert’s vt) and the striatum. So it seems there exist at least two monosynaptic closed loops between the two pretectal nuclei, where Ingle observed caudal thalamic cells, and the striatum; i) Posterior (pretectal) central thalamic nucleus striatum, and ii) TP Striatum loop. Simulation of the pretectum striatum loop based memory model raises some interesting insights and requirements of the model. We first looked at the form in which the striatal memory might be stored. Is it in the form of a spatial pattern related to the pretectal pattern or is it simply a certain motor target coordinate (motor heading), say where to jump, either spatially or intensity coded? If the memory is coded spatially, either in pattern or motor coordinates, the relevant striatum cells should be topographically distributed and the fact the striatal firing profile should be preserved during the memory period requires that the striatum should receive topographic information. Unless some complex topographic encoding and decoding operations are performed at striatum and pretectum, the simplest way to convey the topographic information in the pretectum is to assume that the pathways connecting the pretectum and striatum are topographic. Thus in this case, the firing profile in both the striatum and pretectum during the memory period should be closely related. This raises no problem if the striatum holds the pattern of the SOs assuming the ThlOs are the pretectal cells involved in the memory. On the other hand, if the 125 striatum holds the target location, then it implies that the pretectum during the memory period also holds the target location and this is not compatible with the ThlO cells. With the assumption that the striatum holds the spatial pattern of the SOs, we tested some aspects of the striatal memory model implemented in the topographic pretectal ThlO < -»striatal loop. The initial input to the memory comes from the pretectum. The loop dynamics is modeled as follows : where “Input” is the direct input from the retina and thalamic cells (e.g., Input = kO*Rl + kl*R 4 - 10-Th4, for ThlO-3 neuron). For the reciprocal connection required for the memory loop, we chose simple shunting inhibitory masks with a slight weight. Shunting inhibitory terms, f*St and f *Th, control duration of the memory: greater the/*Sr and f*Th (i.e., stronger the f is), shorter the duration. Fig. 4.10 shows the simulation results of the above striatum memory model. The shunting inhibitory m ask / is set so that the striatum retains vestiges of the input SO pattern after a minute. Note that even a little Resp-R3 activation at the vertical paling locations of a ThlO-1, -2 patterns could prolong the edge specificity of the original pattern by furnishing the maintaining signal that counters the leakage if the object is visible. d(Th\0) Th^St A ■ ThlO + max( Input, RFSt^ Th * St) St R^St^Th ~ St + f * S t Th w i t h / = mask(7, 3, 0.003) Th + f * T h 126 Figure 4.10. Traces of striatal memory model with input from T hl0-3 pattern, (a) Initial pattern at t=0. (b) Memorized pattern at t=30sec. (c) Memorized pattern at t=60sec. 127 W ith the above mechanism, any transient external suppression of firing at either pretectum or striatum results in a loss of memory. Because a transient darkening of general illumination and a detection of movement suppress the ThlO- 1/-2 and -3 respectively, these events terminate the memory in the model. Movement detection can arise due to a moving object or an animal’s own movement in a structured surround. Detection of a moving object typically entails an animal’s reactive movement such as an approach or evasion. And because these movements alter an egocentric representation of the location of an SO, there is no need to maintain the pattern in its original locus. Gating of the SO spatial pattern to the memory network may occur after the cessation of the movement. To account for the remapping capability — an ability to correctly reorient itself to the remembered target after its own movement taking into account its relative position to the target after the movement — the animal is known to possess (Lock & Collett, 1979), the suppression due to the movement detection should be local and locus sensitive. There are no data regarding these aspects of the memory, thus it is a prediction of the model that the temporary darkening or an animal’s movement will erase the anuran short term spatial memory (global loss from the darkening but a local loss for the object movement). It is assumed that the latency of the suppression of the memory is long enough to let the animal utilize the memory before it is erased as shown in the utilization of the spatial memory of an obstacle during an evasive behavior (Ingle, 1991b). If the animal retains the memory after the temporary darkening or movement, we posit that the animal is storing a motor heading. If the motor-heading memory is stored in the pretectum < -» striatum loop, pretectal cells involved in the loop may be a 128 part of a “command releasing cell” (Ewert, 1987) group that may receive and process ThlO afferents and are more directly involved in the initiation of motor action. If the motor heading is spatially coded, the pretectal cells of the loop should be topographically distributed. But if the heading is intensity coded as a summation of activity in a population of neurons, both the pretectal cells and striatal cells in a loop need not be topographically distributed. However this model raises the puzzling question of how the animal retains the initial heading coordinate if the signal intensity itself circulating the loop changes (weakens) during the duration of memory. A substrate of the striatal memory may consist only of the striatal cells. The information stored may be a spatial pattern of a SO or a motor heading either in spatial or intensity encoding. If the memory is implemented as a reverberatory' loop within the striatum, after an input pattern is delivered from the pretectum, the following network could store the SO pattern as in Fig. 4.10 above. d(St) St tS! — — = - A ■ St + ---- —------ * St with/ = mask(7, 3, 0.006) dt St + f * St It is not our purpose to investigate all the possibilities of the striatal short term memory. It is possible that the striatum memory is not based on a reverberatory loop at all but a totally different mechanism such as synaptic changes occurring in the striatal cells. Also, lack of data precludes us to posit an elaborate reverberatory- loop based short term memory. Our aim is to suggest a simple alternative that explains the memory function in the context of the overall anuran SO perception, and its implications. 129 4.6. Overview of Anuran SO Representation The anuran SO acquisition and representation scheme may be composed of several cooperative mechanisms that compensate each other to provide a more sensitive and stable internal model of the static surround: Because anuran pretectum has to work with an inherently less SO-informative retinal signal, it may utilize those signals that are not specifically sensitive to SOs but can be useful under certain circumstances. For instance, R3 input to ThlO-1 and ThlO-2 offers very good edge information when the SO image on the retina shifts due to the animal’s head/body movement. The resulting fast retinal movement of the SOs could be viewed as an active scanning process where the animal quickly builds up or resensitizes a gross picture of the static surround. When the movement ceases, brief afterdischarge and small respiratory-R3 based ThlO response may provide a static picture of the SO as well as the maintenance signal. Spatial localization of the SOs may occur at this stage. ThlO-3 could be viewed as a more advanced detector of SOs because it only discharges to SOs and its presumed more sensitive and stable SO input from the R1.2s. Whereas the ThlO-1 and -2 may be a quick, gross level detector of SOs for a short period, dependent on efficacy of Resp-R3 response, during and after the rapid body movement, the ThlO-3 may be a more enduring SO detector that is also more capable of detecting the fine features of the object due to its R1.2 input. The hypothesis that the ThlO-1, -2 may serve a fast, gross level SO detector is aided by a simulation result showing that the SO patterns due to the animal’s horizontal head/body swaying closely resemble the corresponding ThlO-3 patterns (Fig. 4.11). 130 In summary, we believe that a frog/toad relies on multiple pretectal SO maps each specialized in perceiving different features and having different significance; ThlO-1, -2 are low spatial frequency maps where the patterns can be rapidly painted through numerous R3 and R4 inputs and a body movement that may have been triggered by the animal’s own need to reaquire/update the static world. The ThlO-3 map, on the other hand, may be regarded as a more feature sensitive, higher spatial frequency map where finer details, especially vertical orientation, of SOs are portrayed. Because the number of R 1.2 inputs to the pretectum may be relatively far smaller than the R3 and R4s, it is possible that the T hl0-3’s SO response from a populational point of view is weaker than the combined ThlO-1 and -2. In this respect, ThlO-3 map’s function is not in providing SO information in quick notice, but rather offering finer information through a cumulative process that may even require attention shifting. The significance of the memory based SO representation could be that it lets the animal retain the SO representation longer (more than a minute) than with the pretectum representation alone. Perhaps, another role for the memory is to modulate the direct sensory based signal in the pretectum in order to make a more adaptive decision based on the animal’s experience (memory). It is not certain whether the striatum-based spatial memory can hold the location of other types of stimuli such as an aperture or a (dark) hole that has an attractive effect. Matsumoto et al. (1991) observed that pretectal nuclei receive a relatively greater percent of inhibitory input from the ventral striatum with latencies between 2 and 3 ms. Patton & Grobstein (1986) and Finkenstadt (1989) observed that frogs/toads show no response to prey when the ventral striatum is destroyed. 131 Figure 4.11. ThlO-1 patterns induced by animal’s horizontal body movement. It is assumed that the horizontal body movement causes retinal object movement o f 10°/sec. (a) Vertically striped barrier, (b) Dark solid patch, (c) Horizontally striped barrier. Note that these ThlO-1 pretectal patterns closely resemble corresponding ThlO-3 patterns. 132 Ingle's striatal lesion seems to be more extensive than Matsumoto et al.'s, so direct comparison may not be feasible. However, the new evidence linking the striatum to the spatial STM suggests that more investigation is needed about the role of striatum in the striato-pretecto-teetal loop. It is also not clear if the striatal memory holds the depth of the SO as well. If it does, it could be a good candidate to store the accommodation-based depth map of section 4.4. If the striatum is capable of remembering 3-D location of an arbitrary SO it can be viewed as a memory-gated “where” system. 4.7. Comparison of Anuran and Mammalian SO Perception Perhaps we should start with the question whether frogs and toads have any innately important SO classes in their visual repertory. There is evidence that water (pond) and the (dark) hole may be important SOs to the frogs. It is believed that pond recognition may be mediated by specific retinal ganglion cells (RO: Muntz, 1962) in the anterior thalamic region (Ingle, 1983). Note that these objects are inherently devoid of significant pattern and what we are interested here is the animal’s vision to the SOs of different pattern. Since the retinal processes of anurans and mammalians such as cats and m onkeys are significantly different, a comparison of overall SO functional capabilities between the two animals has to start from the retina. To begin with, while cats and monkeys have area centralis or fovea where the density of the retinal cells are much more greater than the peripheral region, frogs don’t have them. This information alone shows that while the cats and monkeys have a visual space (foveal 133 region) of greater significance enabling them with a finer level visual processing, frogs and toads may view their visual space with more homogenous interest. Photoreceptor density sets an upper limit to the spatial frequencies that can be represented faithfully in the visual system. The sampling theorem tells that the highest spatial frequency that can be represented unambiguously is f y d ' w^ere ^ 1 S the distance between (cone) receptors. The sampling aperture or effective receptive field of an individual photoreceptor sets the upper limit to the spatial frequency that can be resolved by a single photoreceptor. If the receptive field is large this limit may be lower than the sampling limit imposed by the receptor mosaic, but the evidence from monkey study (Miller & Bernard, 19B3) suggests that the receptive field of receptors does not limit the performance of the visual system. But ultimately, it is the ganglion cell mosaic and receptive field that determine the highest spatial frequency an organism can detect. Cat and monkey’s retinal ganglion cells can be roughly divided into two classes; X cells in cat (P in monkey) have smaller receptive fields than Y cells (M cells in monkey), and are much more numerous. X (P) cells are responsible for high visual acuity perception, while Y (M) cells with larger receptive field are inherently more sensitive, better equipped to handle lower contrast objects, fast moving objects, and convey information that needs immediate attention through its faster transmission speed (Lennie et al., 1990). In terms of receptive field size and sensitivity to static stimuli, anuran R1.2s more resemble mammalian X (P) cells. But unlike the X or P cells, anuran R1 and R2 do not comprise the majority of retinal ganglion cells. This with the fact that the overall retinal ganglion cell density of the anurans are much lower (tens of cells/ 0 vs. tens of thousands/ 0 in mammals) 134 than the mammals tells us that the anuran retinal ganglion cells transmit much lower resolution images to the higher visual center than the mammals. Horn & Pasternak’s (1988) finding that the behaving cat resolves between 4-5 cycles/ 0 suggests that the upper limit of cat’s acute spatial vision must depend on X cells because they support up to 6-7 cycles/ 0 but not the Y cells whose limit is about 2 cycles/0. Likewise, it is believed that the mosaic of P cells is responsible for the visual acuity of monkeys. From Carey (1975), the average density of each anuran ganglion cell type is about 1-3 cells/0, and this puts the animal’s visual acuity limit in the neighborhood of 1 cycle/0. An interesting observation follows when we relate anuran visual acuity with the paling fence used in the detour experiment. When the animal sees the barrier comprised of palings 0.25 cm in width from 20 cm away, the palings near in front of the animal make about 0.7°. Paling posts flanked by the gaps do not present well controlled sinusoidal contrast pattern as used in Horn & Pasternak’s experiments. But a rough estimate suggests that so long as the inter paling gaps make more than 0.3°, the animal may not have much difficulty perceiving the paling patterns. On the other hand, because the contrast pattern between the palings and gaps is not ideal and considering the fact that the quality of the anuran ocular media is poor (Krueger & Moser, 1972), the anuran visual acuity for the barrier pattern may be lower. If the mechanism beyond the retina had limited capacity for transmitting or analyzing the retinal information, it would make sense to minimize the information overload as early as possible. One way to do this is to capture and send information that is only of importance to the animal. By introducing selectivity early, and thereby creating distinct classes of retinal ganglion cells that sample the stimulus 135 selectively, an animal reduces the demands on central visual mechanism. For an animal having simple visual requirement — like a frog finding a limited range or prey, avoiding a predator, and perceiving simple SOs — and limited central visual processing, it makes sense for it to have a more preprocessing done from the very early on at the level of retina. Mammalian visual representation is a parallel system based on multiple visual maps, and the neurons in the visual cortex are tuned for specific spatial frequency and orientation and are grouped into modular columnar organization: below each l x l mm^ of cortical surface in monkeys, or 2 x 2 mm^ in humans, are to be found one left and one right eye ocular dominance column. Within each ocular dominance column are found orientation columns representing the full range of orientation plus set of spatial frequency columns with a range of peak frequencies. Each cortical module, therefore, contains a complete set of machinery to encode the orientation, spatial frequency, spatial position, depth, color, and motion of the stimulus falling within a small spatial region of the retina. Based on the anatomical studies of Szekely and Lazar (1976), Lara et al. (1982) proposed similar columnar structure in the anuran optic tectum. However, even if the anuran optic tectum is organized modularly in columnar fashion, it is unlikely that each column packs as much feature sensitive dimensions like the monkey column and certainly it is not involved in elaborate form perception. It is because we believe the tectum is more involved in the perception of movement- gated objects, and doubt that the anurans have multiple spatial frequency and orientation sensitive neurons or maps needed for intricate form perception. 136 Much less is known about the structural organization of the pretectum which in fact is involved in the anuran SO perception. But considering the fact that the pretectum occupies smaller real estate than the optic tectum, it is hard to imagine the pretectum as a general visual processor has more detailed processing capability than the tectum, and certainly the pretectum is not a specialized visual processor for just SOs. Would frogs and toads have in pretectum a master detection cell for a specific SO similar to the T5-2 worm detector in tectum? We do not think there is such thing as a master barrier detector. The ThlO cells we posit and modeled are a substrate to which a neuronal picture of an SO is to be portrayed; an active firing region encodes the presence of an SO in the vicinity, and especially the local maxima encode the location of higher spatial frequency regions. In terms of its function as a feature extractor, ThlOs can be regarded as rudimentary low resolution SO (wide field R4 contribution) detectors with sensitivity to higher spatial frequency (contribution of R1.2 and R3) embedded within them. In a sense, they may be regarded as integrated detectors for both low and high spatial frequency features. In terms of orientation specificity within the ThlO class, ThlO-1 and -2 classes are general edges detectors whereas the ThlO-3 class are better tuned for the vertical edges. Thus, we do not think there exist many visual maps with multiple spatial frequencies and orientation specificities in anuran pretectum as we find in cats and monkeys. Frogs’ perception of a barrier should be built ground up from the population of ThlO cells. The population of ThlO cells should resolve individual paling posts comprising the barrier. The impression we get from Collett’s experiment, where the 137 toads detour around the barrier edge, that the animal perceives the barrier as a whole single object could be misleading. Rather, we believe the animals’ detour behavior is the result of target selection for their visuo-motor responses. In fact, we believe the toad’s ability to detour around the paling fence barrier is the result of learning experience. As will be further discussed in Chapter 7, our pilot experiment shows that a frog (R. pipiens) has to go through a long series of failed attempts at passing through the gap before it shows the detour behavior. Because the animal seldom attempts to go over a paling directly, it could be that the animal has the perception of the individual palings. It is not that the experienced frog perceive the barrier differently, but that its interpretation of the same pattern is changed due to its experience. With the perception of the paling posts, an experienced frog could detect a paling fence as follows: i) See if the pretectal image has overall activity above the spontaneous level, ii) See if there is a succession of local maxima along the horizontal axis within the region of supra-spontaneous level, iii) Use the feature from (ii) to segment and to estimate the depth of the barrier relative to the background. Detection of a passable gap could be as follows: a potential gap would be detected due to its discontinuity in barrier (discontinuity could take place along horizontal or depth axis). Once the discontinuity is detected, the animal could measure the width of the region to estimate the probability of it being a gap. Issues on the validity of the model structure and parameter setting: We have demonstrated that the ThlO models that are constrained by the single cell data and provided with the Resp-R3 response for the case of ThlO-1, yield neural 138 patterns that account for the behavioral data. Because the experimental method cannot show how the SOs are painted in the pretectum, the computer simulation used here can be the best alternative for the purpose. We stress that not only the basic structure but also the parameters involved in the model can greatly influence the outcome of the simulation. Because the front-end retina model is derived from Teeters et al (1993), we are generally confident about the validity of the retina model as used in the current context. So, the focus here is the validity of the organization and parameters associated with the ThlO models and how the changes will affect the SO patterns. We acknowledge that what we presented is not the only way to model the ThlO cells. However, the basic structure (connection) of the model we offer is rather conservative and straightforward. On the other hand, the parameter values could be subject to change. For instance, if the contrast between the peaks and valleys of the ThlO patterns is too extreme, a remedy could be found by changing the corresponding parameters. 139 ! CHAPTER 5 ANURAN DETOUR MODEL Having studied anuran perceptual mechanisms involved in detour behavior in the earlier chapters, in this chapter we investigate how sensory information is integrated with motor mechanisms to deliver an appropriate action. Here, it is o f central concern to understand how different perceptual regions in the brain transform the sensory information into motor signals and how and at which level along the sensory-motor axis the data integration occurs in order to produce appropriate motor actions. This chapter begins with an exposition o f anuran detour behavior, follow ed by further discussion o f the anuran sensorimotor transformation mechanism briefly presented in Chapter 1. Based on the known anuran prey-related sensorimotor transformation data, we propose a barrier-related sensorimotor transformation mechanism. The chapter concludes with a schema-theoretic model o f the integration that builds on the Lara et al. (1984) and Cobas and Arbib (1992) model o f prey-capture. 140 5.1. Introduction i In Chapter 1.1 we showed some of the anuran detour behaviors when a barrier was interposed between the animal and prey. Here we review the animal behavior when | the barrier configuration is more complex than those in Chapter 1.1. i j Fig. 5.1 illustrates several of Collett's (1982) experiments where he presented i | toads with double barriers with various frontal gap positions and separations ] between a worm and a rear barrier. Figures 5.1a-c (left side) show tests in which the gap in the front fence was offset from the frog-worm midline, so that the toad had to | view the worm through both fences. As the worm is brought from in front of the ' rear fence (Fig. 5.1a) to the fence rear, and the separation between them increased, approaches aimed at the gap drop from 67% (Fig. 5.1a) to 36% (Fig. 5.1c). On the ! other hand, when the gap in the front fence lay in line with the worm (Fig. 5.1a-c: j right side), approaches to the gap are more frequent and the response drops from 97% to 64% as the worm is moved from in front to the rear of the fence. These experiments suggest that the animal has an appreciation of the separation between ; the worm and the rear fence and its behavior is determined by the separation and the angle between the prey and the gap. To test whether the toads notice gaps in the rear fence and how they affect the behavior, Collett tested the animals with double barriers having gaps in both front : and rear fences. Fig. 5.2 shows a toad facing a barrier with symmetrically placed I offset gaps in the front fence and various offset gaps in the rear fence. Fig. 5.2a j shows that the toad tends to approach the gap in the front fence on the same side as ! I the gap in the rear fence. When two gaps are placed in the rear fence (Fig. 5.2b), 1 141 lu*l It/* I Figure 5.1. Approaches to double fences with separations between prey and rear fence. Left: gaps in front fence are to the side o f toad’s midline and prey. Right: toad views prey through gap in front fence, (a) Prey 2 cm in front of rear fence, (b) Prey 2 cm behind rear fence, (c) Prey 12 cm behind rear fence. As prey is moved further behind fence, fewer approaches are aimed at gap, from which it is concluded that toads appreciate, to some degree, the distance between rear fence and prey, (from Collett, 1982) 142 where on one side the front and rear gaps are in line with the toad’s viewing angle, but on the other side, the rear gap is seen through the palings, approaches are strongly biased to the side on which gaps in the two fences are aligned. To test the effect of gap alignment on other behavior determining features, a double barrier with two frontal gaps to one side of the frog-worm midline was introduced. When the rear fence has no gap, approaches were almost always directed towards the nearer of two gaps (Fig. 5.2c). But by inserting a gap in rear fence in line with the more peripheral of the two gaps (Fig. 5.2d), approach to the nearer gap is greatly reduced. This experiment suggests that proximity and gap alignments are important cues for target selection and the interaction between them determines the final behavior. From the above examples and the ones from Chapter 1.1, toads obviously are bad at formulating a global path plan based on integration of the spatial information that they have. In the most simplistic terms, the toad behaviors seem to be based on the average response to a sum of “unit” responses. Each unit response is based on a “u n it” rule which is applicable only to a specific prey-barrier configuration consisting of prey and an individual fence but not to the integrative features of a double fence. In a way, each unit rule is like a reflexive stimulus-reaction paradigm. For instance, the response of a frog in Fig. 1.1c is the average of unit responses in Fig. 1.1a and Fig. 1.1b. The unit rules may be summarized as follows: i) Aim for a gap in the barrier (gap in this case also includes the two barrier ends), ii) If there is a choice, select the most salient, and iii) If there is no suitable gap, aim for the prey. The saliency of a gap depends on its location w.r.t. the toad -» prey axis, width, and j depth. A gap in the frontal fence may have more influence on the animal than that J in the rear fence. In a sense, the rear barrier exerts merely a modulatory effect on I 143 ! 6% '2% 20% 10 cm 2% 55%\ M5% Figure 5.2. (a) - (b): Approaches to double barriers with symmetrically placed offset gaps in front fence and various offset gaps in rear fence, (a) Single offset gap in rear fence. Prey in line with long axis o f toad, 12 cm behind rear fence. Toads tend to approach gap in front fence on same side as gap in rear fence, indicating that they see the rear gap. (b) Two gaps in rear fence. On one side the toad views gap in rear fence through the gap in front fence and, on the other side, the rear gap is seen through the palings. Prey 12 cm behind rear fence. Approaches are strongly biased to side on which gaps in two fences are aligned, (c) - (d): Approaches to double barrier with two gaps in front fence, on same side o f midline. Prey positioned in line with toad’s long axis, 12 cm behind rear fence, (c) No gap in rear fence, (d) Gap in rear fence viewed through more peripheral of two gaps in front fence. In (c), toads approach the nearer gap on almost all trials, indicating that the proximity o f the gap to their long axis and/or prey is a powerful influence. In (d), rear gap has considerably reduced this bias, (from Collett, 1982) 144 probable targets determined from the frontal fence alone. The rule i) requires the animal to possess a capability to infer the existence of a gap from the neural representation of a barrier. The rules ii) and iii) suggest that the animal may possess and utilize a “Maximum Selector” mechanism such as presented by Amari and Arbib (1977). Here, a prey behind the barrier should have less input activation than the gaps whose activation values positively correlate with their saliency. Also, a possible source and mechanism behind the probabilistic nature of the animal behavior need to be addressed. 5.2. Lesion Studies 5.2.1. Tectum Lesion After ablation of the entire tectum, both visual prey-catching and predator-avoidance fail to occur (Ewert, 1968; Ingle, 1970), although some particular avoidance patterns, such as ducking or sidestepping can still be observed when the pretectal surface is stimulated (Ingle, 1983). The lesion studies suggest that the tectum is the primary region for the detection and initiation of action with regard to visual objects like prey and predators. i ! 1 ! i j ; 5.2.2. Pretectum Lesion j i ■ Ablation of pretectum results in a “disinhibition” of prey catching behavior in which ' the animal snaps at objects much larger than prey that normally would have elicited j I avoidance behavior (Ewert, 1968). Also, the pretectum-lesioned animal was unable ! i i 145 i ; to avoid predators or collision with a static obstacle like a paling fence barrier. The lesion experiment demonstrates that the region is involved in prey-predator | discrimination through its inhibitory action on the tectum, and the stationary object j ! perception. ! i i | 5.2.3. Striatum Lesion Ingle and Hoff (1990) showed that frogs possess an ability to remember the location of a static barrier. After a frog (R. pipiens) is shown a vertically striped barrier for about 5 seconds after which the barrier is suddenly removed, when the animal is presented with a threat stimulus it avoids jumping toward the former (remembered) barrier location even though the location may be the optimal direction for escape. They observed that the memory lasts about 1 minute. They further demonstrated i that striatum lesion abolished the memory of the static barrier, disinhibiting jumps to the former barrier location. j 5.2.4. Unilateral white tract hemisection ju st caudal to Tectum i After a lesion of unilateral white tract at ventro-medial midbrain just caudal to I tectum, Grobstein and his colleagues (Grobstein & Masino, 1986; Grobstein, 1988) ! observed an entire ipsilateral hemifield deficit (not visual scotoma). That is, with left tract section, frogs respond to prey in the affected left hemifield but the [ movements are always forwardly directed regardless of (horizontal) eccentricity. I Movements do vary with stimulus elevation and distance in the affected ipsilateral j hemifield. However, the variation with stimulus distance in the affected hemifield is 146 different from that in the opposite unaffected hemifield: In normal conditions, a frog snaps when prey is nearby, but reorients and hops toward a prey that is further away. The transition from snap to hop occurs at a distance of one body length for a lateral prey and two body lengths for a frontal prey (Grobstein et al., 1985). However, the lesioned frogs show the transition boundary at two body lengths to the stimulus located anywhere in the affected hemifield (Kostyk & Grobstein, 1987a), the same as if the stimulus had been located in front, which is also the direction to which the animals always advance. It is obvious that the frogs do not experience general motor deficit because the hemisected animals show spontaneous turns in both hemispheres. Based on the data, Grobstein suggested that at the level of the lesion site (tegmentum), the tectal signal conveying the prey location shows lateralization and parcellation: The horizontal eccentricity signal conveying the prey location of one visual hemifield runs through the ipsilateral body side but not the contralateral side (lateralization); Different components of the prey location (eccentricity, elevation, and distance) are handled through different pathways that can be disturbed independently (parcellation). The prey location is represented relative to the animal’s body (body-centered) which is different from its retinotopic representation in the tectum. 5.2.5. Bilateral ventral lesion at the midbrain/medulla junction Grobstein & Staradub (1989) studied the effect of bilateral ventral lesion at the m idbrain/m edulla junction. Large ventral lesions yielded a slight forward 147 movement with no variation with eccentricity or distance of the prey. With smaller ventral lesions, frogs responded to prey with a turn that was appropriate, but with an overshoot for distance that increased with the size of the lesion, suggesting that the signal in the pathway is population (frequency/intensity) coded, transmitting “closeness” information. With this information, Grobstein concluded that the tectal signal conveying the prey location at the caudal midbrain level shows lateralization, parcellation, and populational encoding in a body-centered coordinate system. Grobstein also reports that somatosensory and tectal prey information are integrated at the level of midbrain tegmentum. 5.2.6. Localized lesions o f the tegmentum nuclei W hen the anteroventral nucleus in the tegmentum was lesioned, Ingle (1991a) observed that contraversive jumps in frogs were abolished and all the jumps were directed toward the incoming stimulus from all directions. On the other hand, after the ablation of posteroventral nucleus in the tegmentum, the frogs jumped to the contralateral visual field when looming objects were presented from all directions. This behavior contrasts to the “cut-back” ipsilateral jump the normal frogs exhibit when the looming stimulus is not in a collision course but crosses the view field. These observations further reinforce the idea that the tegmentum plays an important role in relaying, modulating, and integrating the tectal signal conveying the stimulus information. 148 5.2.7. Pretectal efferent pathway lesion Pretectal efferent pathways conveying the barrier signal are divided into two sub pathways (Ingle, 1983). One group projects ipsilaterally to the medulla while the other group crosses to the other side projecting to the contralateral medulla. Ingle showed that after transection at the crossing of the efferent pretectal fibers at the border of tegmentum and medulla, frogs exhibited a profound and lasting deficit when confronted with barriers. The split does not affect normal prey orienting nor the threat escape. This observation suggests that prey and barrier signals do not interact and get integrated before the premotor area in medulla. 5.3. Anuran Detour Model Our work on modeling visuomotor coordination has proceeded at two levels: i) A model of behavior is phrased in terms of interacting functional units called schemas (Arbib, 1989, 1992), and is tested by data on the behavior of normal and lesioned animals, ii) We also posit neural networks to implement schemas, these models being motivated and tested by neuroanatomy and neurophysiology. In the earlier chapters, we identified schemas involved in anuran detour behavior and presented neural network models for SO recognition and depth perception. In this section, we integrate barrier perception and depth-estimation models with the schema model of prey capture (Cobas & Arbib, 1992) to explain the behavior of normal and lesioned animals engaged in detour behavior. 149 5.3.1. Schema m odel o f Prey Capture The prey capture model of Cobas and Arbib (1992) is illustrated in Fig. 5.3. First, the Prey-Selector schema, located in tectum, detects and selects a unique prey and returns the retinotopic eccentricity (x) and elevation (y) of the prey. Prey Depth- Translator then calculates the depth (d) to the prey through x values furnished bilaterally by Prey-Selector. Prey Heading-Translator uses x, y, and d values to obtain the body-centered angle to the selected prey from the animal. The Depth- Map schema provides the transition from a spatially-coded representation to frequency coded information and furnishes a “closeness” ( 1/d) signal for output. Similarly, Heading-Map, using the prey heading signal as input, calculates the target location of the required motor heading and outputs it in population coded form. Note that the Heading-Map signal codes the target location of the required motor response and not the location of the prey per se, the M otor Heading Map hypothesis.1 While full details of the Cobas and Arbib model are given in the aforementioned paper, we adapt and summarize the model here for completeness. PreySelector: Input: retinotopic tectal pattern of moving_objects Select a prey_object among moving objects based on its shape and size Output: retinotopic eccentricity (x) and elevation (y) of the prey_object_______ 1 See Chapter 2.6 for more information on the Heading Map hypothesis. 150 Selection of the target prey is based on its shape and size. Once a target is chosen, the PreySelector schema triggers a cascade of events by activating the PreyDepthTransIator and PreyHeadingTranslator. PreyDepthTranslator: Input: bilateral retinotopic eccentricity (xL , xR ) of the prey_object estimate prey_distance based on retinotopic disparity (xL - xR ) Ouput: frequency coded2 closeness ( 1/prey_distance) signal_________________ The PreyHeadingTranslator computes the angle to the prey from the toad and puts the value in its output port. A positive value represents a left heading, while a negative value represents a right heading. PreyHeadingTranslator: Input: prey_object eccentricity (x) from PreySelector estimate heading by mapping x to egocentric coordinate system output: frequency coded heading signal___________________________________ Orient, Snap, and Approach motor schemas activate corresponding pools of motor neurons in the spinal cord that actually carry out the action. The more strongly Orient is activated by larger heading value, the more strongly will it stimulate motoneurons controlling the turn muscles. The activity of Snap is an increasing function w.r.t. the closeness of the prey from Depth-M ap and a 2 For simplicity, the Depth Map and the Heading Map that transform the retinotopic coordinates to frequency coded signal are merged to the Depth Translator and the Heading Translator respectively. 151 decreasing function w.r.t. the animal’s heading toward the prey as calculated by the Orient schema. In a similar way, the Approach schema determines the distance for the animal to advance as a function of the closeness to the prey and the activation level of the Orient schema. Several of these motor schemas may be active at the same time. In this way, the final motor behavior will result from the combined activity of these motor schemas. The model was successful in simulating the experimental data on anuran prey catching behavior with and without brain lesions. Orient: defaults: threshold = 0 , activationjevel = 0 , heading = 0 , defend_command = 0 , new_heading = 0 Input: heading from PreyHeadingTranslator if heading < 0 then left_tum = FALSE else left_tum = TRUE increment activationjevel by Iheadingl decrement activationjevel by 10 0 0 *defend_command3 if activationjevel > threshold then: if left J u m = FALSE then newJieading = -activationjevel if left J u m = TRUE then new Jieading = activationjevel Output: new jieading ________________________________________________ In the Orient schema, orient jo m m a n d is equivalent to the activation level, which corresponds to Inewjieadingl. The Orient schema receives a heading estimate directly from the PreyHeadingTranslator schema (0 heading is straight ahead). If the final activation is greater than or equal to the schema threshold, then 3 The defend_command initiates a variety of defensive motor responses such as sidestepping, ducking, and poison gland secretion. This motor schema is not incorporated in this thesis. 152 the orient command is enabled, the toad’s new heading is made equal to the prey angle and that heading is returned in the output port. Snap: defaults: threshold = 1, activationjevel = 0 , heading = 0 , closeness = 0 , orient_command = 0 , snapping_command = 0 Input: closeness from PreyDepthTranslator, orient_command from Orient activationjevel = 2 *closeness - 0.1 *orient_command if activationjevel > threshold then: snapping_command = activationjevel Output: new jieading___________________________________________________ Approach: defaults: threshold = 1, activationjevel = 0 , heading = 0 , moving_distance = 0 , default_distance = 4, closeness = 0, orient_command = 0 Input: closeness from PreyDepthTranslator, orient_command from Orient if closeness & orient_command = 0 then closeness = l/default_distance activationjevel = 10*closeness - 0.1 *orient_command if activationjevel > threshold then: moving_distance = min ( 1/closeness, toad_max_move) Output: moving_distance_______________________________________________ The Approach schema is activated by a stimulus that is reasonably close, whereas the Snap schema is stimulated by a stimulus that is even closer. Note that activation levels of the Approach and Snap schemas are determined by closeness and the activation level of Orient schema. 153 S C H E M A M ODEL FOR P R E Y C A P T U R E LEFT RIGHT IM A G E R array of objects IM AG ER P R E Y SELECTO R P R E Y SELECTO R * .y - x,y... PR E Y \ H E A D IN G ' TR A N S L A TO R / PREY j H E A D IN G TRANSLATO R D E P T H T R A N S L A TO R D E P TH T R A N S L A T O R RELAY RELAY D E P TH M A P D E P T H M A P HEA D IN G M A P H E A D IN G M A P \ / d \/d A P P R O A C H S N A P O R IEN T O R IE N T S N A P A PPRO ACH A dvance Snapping m uscles m uscles Spatially coded p ath w ays: • Frequency coded p a th w a y s : P rim in g ' • Left turn muscles Activation Right turn m uscles Inhibition Snapping m uscles A dvance m uscles x, y - retinotopic coordinates d - d istan ce I / d - closeness h - heading rh = rig h t-s id e heading Ih = le ft — side heading Figure 5.3. Overall schema model of prey capture. For further details, see the text, (from Cobas & Arbib, 1992) 154 5.3.2. D etour M odel An integration of barrier and prey signals has to take place in order to determine detour or direct approach to prey depending on prey-barrier configuration. A question is then where and how the integration takes place. Does the integration take place at tectum via topographical projection from pretectum to tectum? Ingle’s (1983) lesion data showing that an atectal frog avoids barriers while escaping noxious cutaneous stimulation suggest that the pretectal barrier signal is not disturbed by the tectal lesion, and so tectum is not likely the place of prey-barrier integration. Could pretectum be the integration site? A lesion of pretecto-tectum connections could shed some light: If pretectum is the integration site, the lesioned animal should exhibit a more direct approach to prey than a normal animal in the same situation. Currently we favor a premotor (tegmentum or medulla) integration hypothesis following Grobstein’s data (1988). However, to develop the integration model we need to know more about the nature of pretectal outflow. For instance, even though the posterior thalamic nucleus (P) in the pretectum projects to the ipsilateral anterodorsal and anteroventral tegmental nuclei (Neary and Northcutt, 1983), its significance and function are not clear. We do not know whether the pretectal outflow arriving at tegmentum is spatially coded or population coded. On the other hand, Ingle’s (1983) lesion experiment on the pretectum — >medulla descending pathway cogently tells us that the pathway is involved in the transmission of the barrier (possibly SO in general) signal to the medulla. Ewert et al. (1990) also saw some clue of tectal and pretectal signals converging at the medulla. They further 155 observed that the visual receptive fields of medullary neurons responsive to moving stimuli are larger than those of comparable tectal neurons and are not topographically mapped in an obvious fashion. With these, we posit that the pretectal signal, including the barrier information, has a populational code and that the pretectal (barrier) and tectal (prey) signals are integrated at the medulla in population coded form. The resultant motor heading signal after the data integration will leave the medulla population coded to the spinal cord. We also posit that the pretectal signal is parcellated and lateralized. Fig. 5.4 illustrates a schematic overview of the complete detour model of unit response for one half of the visual field. The dotted portion shows Cobas & Arbib’s prey capture model and the schemas in the non-dotted region are responsible for recognition/localization of a barrier and integration to the prey capture model. Here, we present a schematic explanation of the barrier recognition/localization networks, and how these are integrated into the prey capture model to constitute a detour model. We provide a brief exposition of the algorithms for the schemas that have not been accounted for earlier. Barrier Recognizer: The Barrier-Recognizer schema is implemented in a neural network as described in Chapter 4.3. It detects palings comprising the barrier through the detection of local maxima on the pretectal ThlO population. From the simulation of the barrier pattern in Chapter 4.3 and the observation—to be discussed in Chapter 6—that the animals avoid going over the palings but aim at the gaps even during their direct approaches, it is quite certain that the animals have a representation of each palings comprising the barrier, at least in the pretectum. 156 Barrier Recognizer Barrier-Heading Translator Barrier/Gaj Depth Translator G aps-H eading Translator s - ■ -« N r Prey| H eading Serial Gaps leading Translator. . . ■ | Prey Depth ■ I Translator Prey Barrier Distance -q a Translator ..... I j j lllh . H eading — — W ' M ap ‘ ' i. Prey Gaps H eading Selector .j. Depth ■ r . Map. I , Prey<-^ Gaps Depth - 4 Selector Approach ■ ■ • ■ SnapO- Orient. Advkrice ’ ‘ ' Snapping " ’ ' Turning . Muscle. ! .Muscle, Muscle Spatially coded pathways: x, y: retinotopic coordinates — •A c tiv a tio n d: distance, 1/d : closeness Frequency coded pathw ays (motor): _ ^eac^ * nS Activation Frequency coded pathw ays (perceptual) : Inhibition Prim ing/G ating: Figure 5.4. A schematic model o f anuran detour behavior for one half o f the visual field. Schemas in the dotted region correspond to the Prey-Capture model of Cobas & Arbib. 157 However, it is different matter what signal will eventually go out from the pretectum. The schema provides x and y retinotopic coordinates of each paling to Barrier/Gap-Depth Translator and Barrier-Heading Translator, furnishing the raw information these schemas work on. The Barrier-Recognizer resides in pretectum and is assumed to become activated, at least the pattern formation part, regardless of detection of prey. Depth estimation and gap detection need further processing with inputs (features) furnished by the Barrier-Recognizer. Barrier/Gap Depth Translator: The Barrier/Gap-Depth Translator schema is implemented as mentioned in Chapter 4.4. Using x and y retinotopic coordinates of the palings received from the Barrier-Recognizer, it computes closeness to the paling posts. Barrier/Gap depth is considered to be an average depth of local palings. An activation of the Prey-Selector is necessary for the activation of Barrier/Gap-Depth translator: Activation of Barrier-Recognizer alone does not activate the schema. This relationship between the Prey-Selector and the Barrier/Gap-Depth Translator schema is depicted by a priming signal going from the former to the latter. The schema efferents its frequency coded closeness (c) signal to Gaps-Heading Translator and Prey <->Barrier-Distance Translator.4 Barrier-Heading Translator: Using x retinotopic coordinates of the palings received from the Barrier-Recognizer, Barrier-Heading Translator schema computes 4 Similar to Cobas and Arbib, where they merged the Depth Map and the Heading Map that transform the retinotopic coordinates to frequency coded signal to the Depth Translator and the PreyHeading Translator, the outputs o f the Barrier/Gap-Depth Translator and Barrier-Heading Translator are already frequency coded. 158 egocentric headings to the paling posts. A positive value represents a left heading, while a negative value represents a right heading. As for the Barrier/Gap-Depth Translator, an activation of the Prey-Selector is necessary for its activation. The schema parallely efferents heading (h) signal to Gaps-Heading Translator. Barrier-Heading Translator: Input: retinotopic paling-posts (x’s) from the Barrier-Recognizer for each paling posts: estimate heading by mapping x to egocentric coordinate system Output: headings______________________________________________________ Gaps-Heading Translator: Based on h and c signals from the Barrier-Heading Translator and Barrier/Gap-Depth Translator respectively, the Gaps-Heading Translator detects the passable gaps and computes the body-centered angles to them. It is assumed that the animal relies on the knowledge of its body width to find internal (passable) gaps. When the Gaps-Heading Translator only selects and transmits two barrier ends for the gap, it is signaling that the barrier has no internal gaps. The schema efferents frequency coded heading signal of the gaps to Prey <->Gaps-Heading Selector. The Gaps-Heading Translator relies on the accurate detection and localization of palings from the Barrier-Heading Translator and Barrier/Gap-Depth Translator for the detection and localization of the gaps. Thus, if any of the above information is inaccurate, reliable gap detection and localization will be in jeopardy. This uncertainty in turn could result in the variability in animal behavior as manifested by 159 its probabilistic detour behavior (Fig. 5.1 & 5.2). As stated earlier, we believe that anuran neural representation of barriers is reliable and accurate enough to provide cues for the detection and 2-D localization of the palings comprising the barriers. However, it is unclear as to how accurately the animals may estimate a barrier depth. In fact, simulation of the revised cue-interaction model of depth perception in Chapter 4.4 suggests that the anuran depth estim ation of barriers can be probabilistically interpreted in the sense that firing rates of cells occupying the locations signify relative confidence of the animal’s depth estimation of the palings at those locations. Then, it is possible that the uncertainty aspect of the depth estimation may be the cause for probabilistic detour behavior. On the other hand, Collett (1982) thinks that toads possess a visual capability that includes reliable depth estimation of a double barrier. This leads to speculation that the source of the probability lies after the depth perception. We believe that the uncertainty of anuran depth estimation and the hypothetical later source may both be responsible for the animal’s probabilistic behavior. But, based on Collett’s observation and the results of the revised cue-interaction model that show quite reliable depth estimation of barriers, we believe the later cause for the probability is more effectual than the vision contribution, at least in most experimental settings where the barriers are placed within 30 cm from the animal. 160 Gaps-Heading Translator: default: gap_headings = (left barrier end, right barrier end) Input: h’s from Barrier-Heading Translator, c’s from Barrier/Gap-Depth Translator convert c’s to distances: di = 1/ci, i= l, ••• n. (n: the number of palings) for i = 1 to n-1 get two consecutive palings di and di+i calculate the gap width by: Output: intemal_gap_headings, two lateral barrier ends____________________ Prey <->Barrier Distance Translator: Using c values of the barrier and the prey from the Barrier/Gap Depth Translator and the Prey Depth map respectively, the Prey Barrier Distance Translator computes the distance between the prey and the barrier. It also determines if the prey is behind or in front of the barrier: When the prey lies behind the barrier, the greater the distance between the prey and the barrier, the stronger is the activity level of its output. However, the activity level does not increase beyond a saturation value (0.4) and also will be set to 0 when the prey is in front of or at the same depth relative to the barrier. The activity level is set to operate from 0 to 0.4 in order to modulate the probability measure of target selection as we will see in the construction of Selectors below. When there is no barrier or the barrier does not pose as an obstacle, the activity level is 0 by default. The schema acts like a controller in the sense that its output level serves as a criterion to gap_width = + d?+ 1 - 2didi+ 1cos(|hi - h 1 + 1 |) if gap_width > frog width return internal gap headings = 161 determine whether a gap-depth from the Barrier/Gap Depth Translator or a prey- depth from the Prey-Depth map will be selected in the Prey 4-^ Gaps Depth Selector. The output level is also used to determine whether a gap-heading value from the Gaps-Heading Translator or a prey-heading value from the Prey-Heading Map will be selected in the Prey<-»Gaps Heading Selector. As prey <-»barrier distance increases, the chances that the gap-depth and the gap-heading selected over the prey in the corresponding selectors increase. The output of the schema also inhibits the Snap schema: The larger the prey < -> Barrier distance, the less likely is the snap to occur. Prey Barrier Distance Translator: default: prey <->barrier distance = 0 Input: prey-depth ( d p r e y = l / c p r e y ) from Prey-Depth Map, barrier-depth (dbarriei^l/Cbarrier) from Barrier/Gap-Depth Translator if prey-depth < barrier-depth p rey -b arrier distance = 0 else prey 4->barrier distance = min [0.01*(prey-depth - barrier-depth), saturation = 0.4] Output: prey ^b arrier distance__________________________________________ Prey < -> Gaps Heading Selector: Using heading values of the prey and the gaps from the Prey-Heading Map and the Gaps-Heading Translator respectively, the Prey Gaps Heading Selector selects a target influenced by the controlling signal from the Prey Barrier Distance Translator. The Heading Selector has three input ports, one for gap-heading, one for prey-heading, and the other for the 162 Prey < -> Barrier Distance signal. These ports can be active simultaneously. The Heading Selector includes a Maximum-Selector network where the probability of one heading being selected over the other depends on the activity level of a potential target’s saliency input. The Maximum-Selector network picks the largest, in a parallel distributed fashion, among the given set of inputs, and was used by Didday (1976) to model prey selection by the frog. Compared with the Didday’s model where the activity level of a prey was associated with its “foodness” value, the input strengths of probable targets, the gaps and the prey, to the Heading Selector reflect saliency from prey-barrier-gap relationship such as prey-barrier distance and prey- gap angle. The activity level of the gap-heading input signal positively correlates with the controlling signal level from the Prey ^B arrier Distance Translator. So, as the prey < h> barrier distance increases, the activity level of the gap-heading increases, which makes the gap-heading signal have increased probability to win over the prey heading. If the gap heading is selected, the Heading Selector gates the gap-heading value to the Orient schema to innervate the motor neuron pools to make the animal turn to the direction of the selected gap. If the prey is in front of or not far behind the barrier, a zero or weak gating signal from the Prey Barrier Distance Translator may not raise the activity level of the gap high enough to have it gated. In this case, the prey heading signal will be selected and the animal turns to the prey. Between the gaps, the Heading Selector prefers internal gaps and selects a gap that has the least angle between the prey and the gap with higher probability. Note that the barrier endings are also considered to be gaps. When there is only one internal gap, then that tends be chosen. When there is no internal gap, the schema 163 tends to select a barrier ending with smaller prey <-»gap angle. The schema outputs the heading value of the selected target to the Orient schema. We assume the Heading Selector is located at the medulla. Earlier, we postulated that the main cause of probabilistic anuran detour behavior may originate at a region caudal to perceptual (pretectal and tectal) region. We posit that the Heading Selector, and thus the medulla is the locus for the probability. The reason for the probabilistic behavior may be due to an inherent “noisy” processing in the Heading-Selector/medulla in the form of error-prone maximum selecting process, and/or the saliency input values themselves may undergo changes after the reception in the Heading-Selector by a telencephalic influence. At present, we are not aware of data that could clarify these issues. For the modeling of target selection, each potential target is associated with an activation value that is dynamically variable as prey-barrier setting changes. The activation values are probabilistic measures which are set based on the saliency functions as described above. The actual behavior in each trial is simply mediated by adding a noise to the activation values. Then, based on the noise-added activation values, the Maximum-Selector chooses a target with the largest value. To set the basis for the probabilistic target selection, a set of activity levels associated with basic detour behaviors are initially assigned based on the observation data. The idea is that the basic detour behaviors serve as seeds, unit responses, from which more complex behaviors are accountable/predictable as prey- barrier settings become more complex. Based on Collett’s data, we chose three unit responses that applies to a single fence: i) When prey is not blocked by a barrier (i.e., prey «->barrier distance signal = 0), the animal approaches the prey with 100% 164 (activity level=l). ii) When prey is behind a barrier with no internal gaps but within the 7 cm (prey <-»barrier distance = 0.07) snapping distance from the barrier (i.e., 0 < prey < -> barrier distance < 0.07), probability of making a direct approach to the prey is greater than the detour, iii) When prey is behind a barrier with internal gap(s), the frog chooses a gap that makes the smallest prey < h< gap angle in 90% (activity level=0.9) and 10% (activity level=0.1) to the next salient gap (see Fig. 1.2b). With the above unit responses, we show how the animal adapts to more complex situations, including the double barrier settings, by modulating the activity levels based on the saliency functions described earlier. In this thesis, for simplicity, the animal is modeled to choose a target from two alternatives (target 1 or target2). In the following description of the Heading Selector, the expression “target 1-heading = prey-heading[0.57] - prey barrier distance” denotes that the probability of the first potential target (targetl) being the prey heading is “0.57 - prey barrier distance”. That is; Prob[targetl-heading = prey-heading] = 0.57 - prey<-»barrier distance. Note that the prey <-»barrier distance ranges between 0 and 0.4, and so the Prob [targetl-heading = prey-heading] ranges between 0.57— animal more likely to choose prey-heading because prey is within the snapping distance behind the barrier— and 0.17, which is the case when prey is far behind the barrier. The above function describes the unit response-ii) and shows the decreased probability of direct approach to the prey as the prey <->barrier distance increases. The constant “0.57” is chosen so that the probability of choosing the prey heading or a gap heading is 50% - 50% at the limit of snapping distance. 165 Prey Gaps Heading Selector: Input: prey-heading from Prey-Heading Map, gap-headings (internal & barrier-end gaps) from Gaps-Heading Translator, p rey -b arrier distance from Prey < -> Barrier Distance Translator if prey barrier distance signal = 0 targetl-heading = prey-heading [1.0] else if no internal-gap // Select among [prey-heading, left barrier end, right barrier end] targetl-heading = prey-heading[0.57] - p rey -b arrier distance target2-heading = barrier-end with smaller prey ^ g a p angle[0.43] + prey barrier distance else // there is intemal-gap // Select among [prey-heading, internal gap-headings, barrier ends] targetl-heading = internal gap with the smallest prey <->gap angle[0.9] - 0.5*prey <-^barrier distance target2-heading = next salient gap: internal or barrier end[0.1] + 0.5*prey barrier distance Output: Maximum-Selectornoisv5 (targetl-heading, target2-heading)__________ Prey Gaps Depth Selector: Using depth values of the prey and the gaps from the Prey-D epth Map and the Barrier/G aps-D epth Translator respectively, the Prey«->Gaps Depth Selector selects a target depth based on the controlling signal from the Prey Barrier Distance Translator. 5 Maximum-SelectornojS y is implemented as follows: i) If the probabilities of the targetl-heading and target2-heading are x and y respectively, partition the range of real number 0 to 1 to [ 0 < — < x] and [x< ••• < 1], ii) Generate a random number between [0, 1]. iii) If the random number falls in the range [()<••• < x ], then let the activation level of the targetl-heading larger than the target2-heading. Otherwise, let the activation level of the target2-heading larger than the targetl-heading. 166 The principle of the Depth Selector is identical to the Heading Selector. In this case, the Depth Selector has to select between prey-depth and gap-depth coming from Prey Depth Translator and Barrier/Gap Depth Translator respectively. Just as for the Heading Selector, the probability that the gap-depth is chosen increases as the prey < -> barrier distance increases. Since both the Heading Selector and the Depth Selector have to agree on their selection, the model forces both schemas to vote for the selection until they agree. We assume the Depth Selector is located at the medulla. Prey ^ G ap s Depth Selector: Input: prey-depth ( d p r e y = l / C p r e y ) from Prey-Depth Map, gap-depth from (dgap=l/cgap) Barrier/Gaps-Depth Translator p rey -b arrier distance from Prey Barrier Distance Translator if prey <->barrier distance signal = 0 targetl-depth = prey-depth[1.0] else if no internal-gap // Select among [prey-depth, barrier-end depths] targetl-depth = prey-depth[0.57] - prey barrier distance target2-depth = barrier-end-depth with smaller prey <^gap angle[0.43] + prey ^barrier distance else // there is intemal-gap // Select among [prey-depth, intemal-gap-depths, barrier-end depths] targetl-depth = intemal-gap-depth with the smallest prey ^ g a p angle[0.9] - 0.5 *prey ^b arrier distance target2-depth = next salient gap-depth: internal or barrier-end[0.1] + 0.5*prey <->barrier distance Output: Maximum-Selectornojsv (targetl-depth, target2-depth)______________ Detour Approaches in double barrier surroundings: 167 Prey <-»Gaps Heading/Depth Selector: Detour in double barrier Input: prey-heading from Prey-Heading Map, prey-depth from Prey-Depth Map frontal_gap-headings from Gaps-Heading Translator frontal_gap-depths from Barrier/Gaps-Depth Translator rear_gap-headings from Gaps-Heading Translator prey <->frontal_barrier distance from Prey Barrier Distance Translator prey <->rear_barrier distance from Prey ^B arrier Distance Translator if rear_barrier has no internal gap targetl double = targetl front - inter-barrier inhibition^ - prey <-»rear_barrier distance target2double = target2front + inter-barrier inhibition + prey «->rear_barrier distance else // rear_barrier has internal gap(s) // a frontal_gap that is closer in line with rear gaps acquires increased probability // to be chosen as a target £|targetlfro n t - rear _ gap J targetldouble = 40 - 40 • —------------------------------- n: number of rear gaps K target2double = 4 0 - 4 0 i|ta rg et2 fro m -re a r_ g a Pi| i=i K targetldouble = ---------- -H^fidouble------------. normalization targetldoubie target2d0uble target2double targetldouble targetldoubie target2d0uble Output: Maximum-Selectornoisv (targetldouble. target2double) 6 Inter-barrier inhibition depicts increasing inhibition the rear barrier exerts as the inter-barrier distance decreases. It is modeled to be inversely proportional, to some extent, to the inter-barrier distance. inter-barrier distance = prey <-»rear_barrier distance — prey <->frontal_barrier distance, inter-barrier inhibition = NSLsaturation (1/inter-barrier distance, 0.2): if 1/inter-barrier distance < 0.2 inter-barrier inhibition = 1/inter-barrier distance else inter-barrier inhibition = 0.2 Refer to Weitzenfeld (1991) for more details. 168 As stated earlier, anuran detour behavior in double barrier settings is influenced by barrier <-»barrier relationship such as gap alignment. Here, we look at how double barrier detours can be modeled from the single barrier model discussed above. It is assumed that the existence of rear barrier in the double barrier configuration only modulates the choice of targets based on the frontal fence alone. Thus, the introduction of a rear barrier does not add new (potential) targets besides the possible targets already determined from the frontal fence. Modifications to Prey Capture Model: The basic organization of the Prey Capture network is unchanged. Modifications to the Depth Map and Heading Map only involve re-routing of their output connections, and the Snap schema has one additional input from Prey < -» Barrier Distance Translator. These modifications do not affect the performance of the Prey Capture model because the changes do not alter the internal information processing of the schemas, and can be made on top of or independent of the original schemas which could be encapsulated to preserve modularity and integrity. The Snap schema is modified to receive an inhibitory signal from the Prey Barrier Distance Translator7. Thus, when a frog approaches a gap, the possibility of it snapping during the way is almost nil due to inhibition from both Orient and Prey Barrier Distance Translator schemas. Note that the pretectal barrier/gap signal is parcellated and lateralized. Both the prey related signal and the barrier/gap related signal from tectum and pretectum 7 The activation level of the Snap schema is changed to; activation_level = 2* closeness - 0.09*orient__command - 0.01 *prey barrier distance 169 remain independent until they are integrated at the Heading and Depth Selectors presumed to be located at medulla. 5.4. Performance of the Detour Model 5.4.1. Normal functions First, we look at simpler cases when only prey or barrier alone is present. When the barrier is absent, the Barrier-Recognizer schema is not activated and so the prey signal is gated to the motor schemas by default. Thus in this case, our model performs just like the Prey Capture model. With barrier alone, even though the Barrier-Recognizer performs paling pattern recognition, because of no priming signal from the Prey-Selector, Barrier/Gap Depth Translator and Barrier-Heading Translator are not activated. Thus, the Depth and Heading Selectors receive neither prey signal nor gap signal. As a result, the animal does not show any movement, which conforms to the behavioral data. Fig. 5.5 shows simulation results of the detour model when the animal is faced with various single barrier configurations. Prey in front of or not obstructed by the barrier: As shown in Fig. 5.5a and 5.5b, a frog in this situation aims directly at the prey with 100% probability. Prey close behind the barrier: In this situation, frogs usually approach prey directly, because the worm is within the snapping distance from the barrier. In our model, the behavior is replicated by a weak controlling signal from the 170 / i 100% b i 100% I ,25% 75% • • J 10 cm 25% ^ 100% e Ji 90% 0% lO O ^K ^ { I 10% 90% ^ 0 0 % 0 Figure 5.5. Examples o f the computer simulation o f the detour m odel when our computational frog is faced with various single barrier configurations. See the text for further details. 171 Prey <-»Barrier Distance Translator making prey signal more gateable to the motor schemas resulting in a direct approach to prey as shown in Fig. 5.5c. Prey far behind the barrier: In this situation, frogs usually detour around the barrier. Due to a high level of controlling signal from Prey Barrier Distance Translator, selected gap (barrier end) signal will be routed to the motor schemas. Thus, our computational frog shows a greater chance of making a detour as shown in Fig. 5.5d. A passable gap within the barrier: A frog/toad in this situation seems to prefer the internal gap over the two lateral barrier endings, even if one of the latter gaps offer a closer path to prey. This situation is handled in our model (Fig. 5.5e) because the Prey <->Gaps Selectors prefer internal gaps that have higher activation values than the barrier endings. Two passable gaps within the barrier: Even though Collett does not present the toad behavior in this configuration, from Fig. 5.2c, we believe Fig. 5.5f is what may happen. The simulated behavior is the result of the computational frog’s preference to a gap that makes least angle with the prey. Fig. 5.6 shows simulated frog’s behavior when it is confronted with various double barriers. When a prey is in front of the barrier, the animal aims at the prey directly (Fig. 5.6a). If the prey is close behind the frontal fence, the animal chooses to go directly at the prey with 40% and detour with 60% probability (Fig. 5.6b). Note that the probability of a direct approach is decreased from 75% in the similar situation of a single barrier (Fig. 5.5c) to 40%. And as the distance between the prey and frontal fence increases, the probability of direct approach (15%) decreases while detour increases (Fig. 5.6c). When the prey is behind the double barrier with no 172 / Ic^l i 100% 60%. 100% ^ 3 • • • § • • • • • • • • .85% 100% 10 cm 32% 68% 4% .48% 25%^ 75% 25% '5% 75% 100% 100% IcH 100% L 25% Figure 5.6. Examples of the computer simulation of the detour model when our computational frog is faced with various double barrier configurations where there is no gap in rear fence, (a) Prey 3 cm in front o f front fence, (b) Prey 2 cm behind front fence, (c) Prey 5 cm in front o f rear fence, (d) - (f): Prey 12 cm behind rear fence, (g) Prey 5 cm behind front fence, (h) Prey 2 cm behind rear fence, (i) Prey 12 cm behind rear fence. Comparing the computational frog’s behaviors with data in Fig. 5.1, we can see that the model captures basic behavioral pattern of living animal. 173 gap, approaches are almost always detours as shown in Fig. 5.6d. But if there is a gap in the frontal fence, more than half the approaches are directed at the gap (Fig. 5.6e & 5.6f). If the prey is located between the frontal and rear fence as in Fig. 5.6g, a greater number of approaches (80%) is directed to the gap. When the prey is placed close behind the rear fence, 74% of the approaches are directed to the gap (Fig. 5.6h), which is greater than in the case of Fig. 5.6e but less than Fig. 5.6g. If there are two gaps in the frontal fence and the prey is behind the rear fence, the animal chooses a gap that makes smaller angle with the prey with greater proportion (Fig. 5.6i). Fig. 5.7 shows the simulated frog’s behavior when it is confronted with various double barriers with gaps in both frontal and rear fence. From the comparisons of Fig. 5.7a with Fig. 5.2a and Fig. 5.7b with Fig. 5.2b, we can see that i b 1 cl Figure 5.7. Examples of the computer simulation of the detour m odel when our computational frog is faced with various double barriers with gaps in both front and rear fence, (a) - (b): Prey 12 cm behind rear fence, (c) Prey 3 cm in front o f rear fence. Comparing the computational frog’s behaviors with data in Fig. 5.2, we can see that the model simulates basic qualitative behavioral aspect o f living animal. 174 the computational frog simulates the qualitative behavior of the animal. If the prey is placed between the frontal and rear fence as in Fig. 5.7c, the animal makes greater proportion of approaches to the aligned gap (61%) compared to the case in Fig. 5.7b. 5.4.2. Lesion effects According to Ingle (1976, 1983), a pretectum lesioned frog seems to be unaware of a large hemifield barrier. This behavior is replicated in our model by unavailability of Barrier-Recognizer in pretectum. Ingle also observed that a frog with “split pons” seems to know whether or not to approach/snap a prey behind the barrier. But it will always collide with the barrier if the critical barrier edge is in monocular view field and located on the same side of prey. This behavior can be replicated by severing the Prey <->Gaps Heading Selector link coming from the contralateral side which is responsible for the gap in the ipsilateral monocular view field: Due to presumed lateralization of the pretectal signal, information of the gap in the ipsilateral view field comes solely from the contralateral pretectal lobe. Splitting the pons severs this pretectal decussation but preserves tectal decussation responsible for the lateralization of tectal signal. [ i I | 5.5. Discussion ; While we tried to construct the model reflecting as much known data on real animals as possible, much of the formulation is based on hypotheses that need to be verified by experiments. For instance, the model is based on the hypothesis that the prey and barrier signals are integrated in frequency domain at the medulla. From this we can 175 draw some strong implications on anuran sensori-motor transformation process. First, by the hypothesis that the heading values (prey and gap headings) and depth values (prey and depth depths) coming into the selectors are in population coded form, they may be gated to the selectors not in a parallel fashion. That is, because a final target is selected at the selector level at medulla considering the prey and (multiple) gap locations, the animal has to receive multiple gap locations if they exist in a barrier. But due to the population coded pathway, only a limited amount of information can be transmitted at a time and this necessitates a quasi-serial transmission scheme. We may assume that prey selection is performed at the tectal level. Second, the selectors (medulla) need some sort of memory mechanism to hold and select the incoming data through Maximum-Selector network. We do not think the quasi-serial transmission incurs too much temporal overhead because computationally intensive recognition task is already performed at the tectum and pretectum level and so only a limited amount of information, prey and gap locations, needs to be conveyed to the selectors. We believe the hypothesized sensorimotor transformation characteristics reflect the behavioral simplicities of the animal: By delaying the data integration as much as possible until the highly processed and abstracted prem otor representation is available, the animal is freed from combinatorial data explosion of sensory integration, but at the same time suffers ; from limited number of alternatives. Given the complexity of the animal and tasks it i has to perform, the sensorimotor process the anurans employ should then be the best compromise. i Collett's data also show that the animal's behavior is stochastic: Even during i the same experimental session with the prey-barrier configuration unchanged, a toad j 176 i 1 sometimes show detour and in another trials a direct approach. Our model can I account for this stochasticity because the selection of prey or gap in the Depth and Heading Selector schemas is modeled as a stochastic process. 177 CHAPTER 6 ANURAN DETOUR BEHAVIOR AND LEARNING In Chapter 5 we saw how anuran detour behavior observed by Collett (1982) can be modeled. In this chapter we discuss new data o f anuran detour behavior from our pilot study. We show some evidence that suggests some form o f learning is an integral component o f anuran detour behavior. We believe the animal is learning to characterize a visual object differently through its learning experience. Consistent with the hypothesis discussed in Chapter 5, it is not that the animal learns to better calculate the metric information but that its interpretation o f the visual features undergoes change through its interaction with the visual stimuli. We adapt the animal learning process to propose an adaptive modulation scheme o f the artificial potential field fo r the robot path navigation method. We show how a shunting self excitatory loop could offer a plausible metaphor capable o f modifying the potential field, reflecting an agent’s learning experience. Although more elaborate verification is needed, a possible future refinement to the detour model would be an incorporation o f the learning aspect. 6.1. Anuran Detour Behavior and Learning 178 Collett's data make one think that a frog/toad is a simple automaton. Recently, our pilot observation1 suggests that it may be more flexible than we thought earlier. We divided naive frogs into two groups (A and B). To group A (seven R. pipiens), a narrow paling fence (10 cm wide) with 2 cm inter-paling gap is initially presented. Frogs started from about 25 cm or 10 cm in front of the barrier and a worm is always put 12 cm behind the barrier. From the first day, all the frogs that started from 25 cm away showed reliable detour behaviors whereas the ones that started from 10 cm always chose direct approaches. It was quite apparent from this observation that the frogs inherently have a detour motor category (schema) among their motor behavioral patterns. Eventually, after 2-4 experimental sessions, even the frogs that started from 10 cm distance showed reliable detours. When the frogs showed mostly detour behaviors no matter where they started from, the barrier was widened to 20 cm. Then, initially, their approaches to a worm were more biased to direct approach than the detours even when the animals started from 25 cm away. With repeated sessions, their approaches gradually shifted to detours. During the transition sessions, before animals showed reliable detour behaviors, we observed interesting behaviors suggestive of learning taking place, which will be discussed later. Finally, when the barrier was widened to 35 cm, some frogs again initially showed direct approach to prey even when started from 25 cm away, but after one or two sessions, they all showed reliable detours. The observation suggests that not 1 We thank two undergraduate students, I. Henry and J. Risher, for their assistance in conducting the experiment. 179 only the fence width but the separation between the animal and the fence is a factor in determining approaches to prey. This conclusion contrasts to Collett’s data where he observed that it is the prey-barrier separation that determines the frog’s approach to prey. Group B frogs (five R. pipiens), when they were introduced to the 35 cm wide barrier from the beginning, always approached the worm directly ignoring the barrier irrespective of prey, barrier, or animal location. Then, they tried to squeeze their way through a gap between the neighboring palings (inter-paling gap). They often tilted their bodies when passing through the gap and were successful in about 50% of the attempts. For another 30%, after the failed squeeze, they sidestep laterally, usually poking their nose to some inter-paling gaps on the way, to a barrier edge where they reorient and approach a prey. In some instances, they backstep about 3-5 cm and then initiate detour behavior. It was as if the animal was trying to re-examine the barrier from a more distant, global perspective. In the remaining 20%, they either moved away from the barrier or remained inactive. When they were frightened, the animals often jumped into the barrier even though it was located in the direction of escape. Similar behaviors were also observed from the Group A frogs during the sessions when they were shifting from the direct approach to detour behaviors. The Group-B animals began to show detour behavior after two week’s experimental session. Then they usually showed detours when the prey-barrier-frog configuration was appropriate. Even when the frogs first made an occasional direct approach, after poking at one or two gaps, they tend to initiate detour behavior. Even 180 when frightened, they seldom jumped into the barrier even though its location was in the typical escape direction. Sidestep Push- j l Stagefl: /(B a c k u ] Nafvle / ...............- frog \ Stage-2: Learning frog Learned Sta^e; Experienced frc Start Figure 6.1. An ethogram of anuran detour learning. Although more experiments are needed, it is quite evident that the two groups’ reactions to the barriers reflect their experience. The Group-A frogs that experienced similar but smaller (easier) barriers immediately or very soon showed detours against 35 cm wide barrier, whereas the Group-B frogs that did not have such experience needed a much longer time with the wide barrier before they showed any detour behavior. Certainly, Group-A’s prior experience with the smaller barriers made the frogs quickly perceive or adapt the interpretation strategy 181 to the large barrier, and this we believe is evidence of learning. Fig. 6.1 illustrates Group-B frogs’ temporal shift in approach strategy to prey. Frog’s prey approach behavior is divided into three stages. Stage-1 depicts a naive frog without any experience with the barrier, and stage-2 describes a frog during a learning stage. After spending some time in stage-1 and -2, frogs begin to show reliable detour behaviors as depicted in the learned stage. 6.2. Learning What? Anurans are known to possess some learning capability as observed in Cott’s (1936) bee or Dean’s (1980) beetle discrimination experiments where the animal learns to distinguish bee from fly or beetles based on its unpleasant experience of snapping at a bee or beetle. On the other hand, researchers like Ingle report that it is impossible to train a frog in such a task as navigation-learning where a specific path is associated with reinforcing stimulus. Even though a new round of experiment in a more controlled setting is needed to verify this, from our observation it looks as if frogs exhibit behavioral patterns, during that transitory period before the animal shows detour behavior, which suggest the animals may be forming a new view or interpretation of the barrier based on their experience with it. We stress here that it is not that the frog is learning to make detour motor behavior but that it is learning to interpret differently the visual barrier stimulus. Because an adaptation of interpretation to a stimulus based on the agent’s experience with the stimulus implies a “learning”, we claim that a frog’s ability to 182 detour around wider barriers reflects a learned adaptation of perceptual interpretation. By dividing the frogs into two groups and presenting them with different barriers, we were able to observe the animals’ perceptual classification scheme of barriers of various width and the learning rate. From Ingle’s (1983) experiment, frogs seem to better classify a barrier as an object as the size becomes smaller. And indeed, from group-A frogs’ immediate detours with small 10 cm barrier, the smaller size of the barrier seems to make the animals better classify it as an obstacle. And as the barrier width became larger, the frogs needed experience before they showed detour behavior. In order to understand why a naive frog behaves as it does to the barrier initially, and then subsequently “learns” to make a different interpretation of the wider barrier, we need to see how the barrier might be perceived by a frog in its natural environment. Perhaps, the closest thing a wide paling fence barrier resembles to a frog should be vertical grass or stems of plants that are all around the animal. Thus, the visual pattern of the barrier resembles one of the most frequently encountered natural objects by a frog. But the key difference lies in that while the grass and stems may be flexible enough so that a frog could simply trample them down, the barriers we present are made of rows of wooden dowels that are not pliable. In this respect the barrier is an unnatural object to frogs. This explains why the inexperienced frogs choose to barge through a gap initially. And the hypothesis that a frog’s detour behavior involves learning implies that the frog’s characterization of the barrier should be changed from the “row of stems” to “row of vertical things through which it can see but not negotiable”. Note that this is a 183 tremendous change in the interpretation of a visual object. In this regard, it may be interesting to test if the characterization of the vertical objects is completely changed from the stems to barriers. If they have, an experienced frog may act differently from a natural frog when it sees a worm amongst the stems. It will also be interesting to test if a frog can learn to differentiate stems from a barrier by, for instance, different luminosity, texture, or color. To test this, we could plant green- colored plastic straws amongst the black dowels and see if the animal can associate the green palings as a passable region and the others as an impassable region. It appears that frogs have an innate ability to perceive individual palings (stems) because even an inexperienced frog’s direct approach is aimed at gaps between palings. Even to the frogs in a natural habitat, poking the head directly at stems may not be a clever tactic. What the frog should learn is that the gap in general is impassable from its experience with some gaps. Thus, the animals should have some faculty to generalize their experience to other gaps. Then we may ask, eventually how the barrier will be perceived by the animal. Will it be perceived as composition of transparent but impassable regions flanked by the palings, or will the barrier be perceived as one entity by an emergent new classification like transparent “obstacle”. Because the transparent obstacle was originally alien to the frogs, couldn’t we say that the animal has an ability to form a new concept if the animal interprets the barrier as such? Ingle did not elaborate on his failed attempt to teach frogs. It was basically a simple detour learning where he tried to train the animal to detour to just one side of the barrier. We believe the failure is due to the frog’s inability to form the concept of “path” with such metrics as distance and efficiency. On the other hand, we 184 believe the frogs have an ability to form a rudimentary perceptual concept. We think a frogs is a relatively simple reactive animal whose course of action is mostly determined by direct sensory interaction. But also these animals may be capable of forming a perceptual concept that is only a small distance beyond the sensorial representation. The concept of path may then be more abstract than a bee-vs.-fly discrimination perceptual schema (concept) or the concept of barrier because it is not directly related to sensorial representation (metrics) but action and localization/estimation of space that are some distance away from the sensorial data. This intuitive deduction conforms to our common sense belief that the more intelligent an animal is, the more easily it can form a complex concept or schema and manipulate it. Because frogs do not have the concept of path, the animal cannot associate a reward with a specific path, and this could explain why the animal chooses an inefficient path or gets trapped in a cage even when appropriate sensory data are acquired as in Collett’s experiment. 6.2.1. Neural Substrate o f Detour Learning W ang and Arbib (1991) proposed that medial pallium (MP) is involved in habituation/dishabituation learning. In order to link MP with learning, there should exist connections between the pretectum and MP or striatum and MP. It appears that evidence of ascending pathway from pretectum to MP is not conclusive and the connection does not exist between the MP and striatum (Neary & Northcutt, 1983). 185 Thus, the possibility of MP as substrate for anuran form interpretive learning is not good. Because we postulate that anuran detour learning is happening at the internal interpretation level of the visual objects, it is thinkable that the learning locus may be closer to the sensory side than the motor side. We have modeled pretectum and striatum as the places for SO recognition and SO memory respectively. Because memory is an essential ingredient for learning, associating the striatum as the locus for both short term spatial memory and modulation center for form interpretation could be an interesting thought. 6.3. Anuran Perceptual Learning Model Here we model anuran barrier learning process and eventual effect it has on detour behavior. Although the learning process of Group-A frogs (narrow -» wide barrier) is interesting, because the data on this learning process is more preliminary than Group-B’s (learning to make detour around a wide barrier; Fig. 6.1), we concentrate on metaphorical model of the Fig. 6.1 learning mechanism with the hope of reflecting some aspect of the biological process. In Chapter 4.7, we modeled the striatal memory with a reverberatory loop that preserves a visual pattern of a barrier. A lateral shunting inhibition is assumed to be the cause of “forgetfulness” by decreasing the firing levels of a barrier. Thus, temporal change (decrease) in the spatial firing pattern of an object is associated with the animal’s temporal change in the interpretation (memory loss) of the object. 186 We posit similar process, taking place in a much longer time scale, may lie behind the animal’s learning to interpret the barrier as an impassable obstacle. Fig. 6.2 shows mid-transversal vertical cross-section of striatal barrier images showing peaks and valleys corresponding to palings and gaps, that closely resemble a 1-dimensional potential field representation of the barriers. Fig. 6.2a shows a paling fence barrier with identical gap-width whereas 2b shows a barrier having different gap widths. First and fourth gaps from left in 2b are narrower than gaps in 2a while other gaps are wider than those of 2a. Even though Fig. 6.2 is a visual b Figure 6.2. Mid-transversal vertical cross-section of striatal barrier images showing peaks and valleys corresponding to palings and gaps that closely resemble 1-dimensional potential field representation o f the barriers, (a) A paling fence barrier with identical gap widths, (b) A barrier with different gap widths. First and fourth gaps from left are smaller, while other gaps are wider than those of (a). 187 representation of barriers, we may also associate the figure as an initial psychological representation of the barrier upon which modification can occur based on the animal’s learning experience. An artificial potential field U is typically defined as the sum of an attractive potential pulling an agent (robot or an animal) toward the goal (prey) and a repulsive potential pushing the agent away from the obstacles (barrier). Based on the artificial potential field metaphor, the attractive potential field Uatt set up by a prey can simply be defined as a parabolic well (Khatib, 1986; Arbib & House, 1987; Latombe, 1991); I W « ) = where £ is a positive scaling factor and pg oai(q) denotes the Euclidean distance jq - Assuming each paling occupies a point location, the repulsive potential function Urep set up by the paling fence barrier can be defined as sum of the repulsive potential fields created by each individual palings; U rep(tf)= £ Urep(4) with k= l U r e o ( < ? ) = 1 — r] 2 2 if — Po> ,PM) Po. 0 if p k(q )> p 0. where 7 7 is a positive scaling factor, p k(q) denotes the distance from q to the k’th paling, and p0 is a positive constant called the distance of influence of the (paling) obstacle. Then overall potential field U(q) is; U (9 ) = Uatt(9) + U rep(?) which creates the force field F(q); 188 F(<?) = -V U (^) that determines the animal’s trajectory by providing the gradient vector at each location q that leads the animal to the promising direction of motion to the goal. A frog’s trajectory, however, does not follow the force field above. It is because in real environment, the animal relies on its long range visual sensor to activate appropriate approach schema rather than “feel” its way to the prey along the trajectory as potential field method dictates. So, we may regard that the animal takes the measurement of the potential field through vision and determines its approach. Also, this navigation method fits well with the schema-based detour model presented in Chapter 5 in that; i) the animal only has to perceive and localize relevant stimuli such as prey and barrier/gap and need not set up the force field at all the locations, and ii) saliency of an object can be represented and modulated by the associated potential field shape and interactions between them. Even though the formulation above can create an appropriate potential field for two or three dimensional space, for simplicity, here we deal with one dimensional case (single barrier) where the shape of the 1-dimensional potential field itself is the cue for determining the animal’s direction of approach. That is, the animal is attracted to region that has lower elevation and avoids the region of higher elevation. There has been controversy about whether a (passable) gap should be formulated as having an attractant effect in addition to the repellent effect of barrier (Lara & Arbib, 1984). Based on the potential field metaphor, it is obvious that the gap should be an attractant object. Fig. 1.2a (in Chapter 1) only elicits 25% direct approach, but when the frontal fence with a gap is introduced as in Fig. 1.2c, the direct approach is increased to 52%. It shows that unless the gap itself is creating an 189 attractant field, the Fig. 1.2c is unexplainable because the addition of the frontal fence can only contribute to the repulsive potential field but not decrease it based on the formulation of Urep. Regard Fig. 6.2 as an initial impression of a barrier to an inexperienced frog. Based on animal’s experience with a gap, the spatial pattern of the barrier should be changed. The general idea is that when an animal is often successful in negotiating a gap of a certain width, potential fields that correspond to gaps of greater or equal width should be lowered. On the other hand, if an animal was not successful with the gap, potential fields of gaps equal or narrower width should be heightened. Grossberg and his colleagues (for review, see Grossberg and Kuperstein, 1989) presented a spatial short term memory (STM) based on recurrent shunting network. Here, we adapt the idea to modify the spatial pattern of a barrier over time, describing an evolution of the barrier characterization by learning experience. Dynamics of the system here is assumed to operate in much slower time scale (order of days) than short term memory (order of seconds or minutes) because the learning we are interested here is acquired in a time scale much slower than STM. 6.3.1. Global Shunting Inhibition Here, evolution of the potential field Fig. 6.2 is governed by an equation of the form dqi dt - A q i + ( B - q i ) i+ r ,k= i— r ^ g ( q k ) kA i (eq-1) where coefficient E\d describes the fall-off with the distance between cells qi and q/o r is the effective range of recurrent excitation, and activity of each qi fluctuates between -D and B as we can see from the equilibrium analysis; 190 n k— i— r_____________k&i Hi ~D if Y i S W » n i+r k=i-r k*i k=i-r k=i-r The eq-1 is a recurrent network having proximal-excitatory (Eiq in the second term of right-hand-side of eq-1), global-inhibitory shunting connection pattern (the third term of right-hand-side of eq-1). The equation above is translated into NSL2 as; with E = mask(2r,sigq,wgtq) Qe = NSLsigmoid{q, 0, 25, 0, 25) < 2 , = NSLsigmoid(q, 0, 25, 0, 1) where mask(2r, sigq, wgtq) denotes a Gaussian mask (see, section 3.1), and Qe and Q i are firing rate of q by thresholding q with sigmoid functions3 / and g . Although the equilibrium equation tells an eventual shape of the pattern, it is extremely difficult to envisage what the stabilized pattern will be. All the variables influence the evolution of the pattern. With A =l, B=25, and D=5, we present an evolution of the barrier pattern through the changes in the E mask shape. The size and strength of E can be thought to reflect the size of the animal. That is, when the animal finds it difficult to negotiate through a gap of a 2 Neural Simulation Language by Weitzenfeld (1991). 3 B = NSLsigmoid(A, k xl, kx2, k yl, ky2): ifA < k x l B = ky 1 else if k xl <A<kx2 B = (ky2-kyl) (72 (3-2 <7) + kyl where <7= (A -kxl)/(kx2-kxl) else if A >kx2 B = ky2 Refer to Weitzenfeld (1991) for more details. ^ - = - A * q + (B-q)[E*Qe]-(q + D)[SUM(Qi)-Q] at 191 certain width because its body size is too big for the gap, the gap’s potential needs to be heightened to reflect the animal’s inability to negotiate through the gap by increasing the r, sigq, and wgtq of the E mask. By enlarging and strengthening the E mask, a difficult-to-negotiate gap gathers more excitation from neighboring fences, and thus raising its potential. On the other hand, if the animal is successful in negotiating a gap, the gap’s potential can be lowered by reducing the mask size and strength. The learning occurs in the process of tuning the E mask that will raise (lower) potentials for impassable (passable) gaps upon the completion of learning. Also, this process implies “generalization” of learning in that the learning from one gap affects other similar gaps because the tuning of the E mask based on a certain gap is applied to all the other gaps. Thus, if an animal was successful (unsuccessful) in negotiating a certain gap, not only that gap-potential but also all the other gap- potentials of gaps of similar width are lowered (raised). Fig. 6.3 depicts evolution of the barrier pattern (Fig. 6.2a) when the animal is able to negotiate a gap with the E=(1.5, 1.0, 1.58). After 40 feedback cycles, the original barrier pattern transformed to Fig. 6.3a, and Fig. 6.3b shows the pattern after 1000 cycles. It appears that the Fig. 6.3b is a stabilized pattern. Here, we use the term “stabilized pattern” rather loosely to describe a pattern that does not change noticeably in 500 cycles. The parameters for E were chosen to simulate the lowering of gap-potentials describing the animals’ favorable experience of passing through the gaps of given width. As we see from Fig. 6.3a, the E lowers the gap potentials initially then stabilizes the pattern to Fig. 6.3b that shows pronounced humps in the locations that correspond to two central palings. 192 a b Figure 6.3. The eq-1 based transformation of the barrier pattern of Fig. 6.2a with A = l, B=25, D=5, and E=mask(1.5, 1.0, 1.58). (a) The barrier pattern after 40 cycles, (b) The barrier pattern after 1000 cycles. To simulate the increase of gap-potentials describing the animals’ inability to pass through the gaps, E is set to E=(7.5, 3.0, 3.16). Fig. 6.4a and 6.4b show initial stage (20 cycles) and stabilized'stage (320 cycles) of evolution of the Fig. 6.2a barrier pattern respectively. We can see that initially E raises the gap-potentials, then stabilizes the pattern into one big hump centered at midpoint of 3 - 4’th palings from left. By setting parameters of E to mid-values used in Fig. 6.3 and 6.4, E=(4.5, 2.0, 2.44), we tested the equation to the Fig. 6.2b barrier pattern having gaps smaller and wider than those of 6.2a. As we see from Fig. 6.5a, first E raises gap potentials of the smaller first and fourth gaps while lowering gap potentials of other wider gaps. 193 But similar to Fig. 6.3b and 6.4b, Fig. 6.2b pattern stabilizes to one big hump centered at midpoint of 4-5’th palings as shown in Fig. 6.5b. Figure 6.4. The eq-1 based transformation o f the barrier pattern of Fig. 6.2a with A = l, B=25, D=5, and E=mask(7.5, 3.0, 3.16). Compared with the E in Fig. 6.3, r, sigq, and wgtq are all increased, (a) The barrier pattern after 20 cycles, (b) The barrier pattern after 330 cycles. With local excitation and global recurrent inhibition, Eq-1 takes a form similar to maximum-selector (Amari & Arbib, 1977). So, after the initial dynamics settled down, the stabilized pattern accentuates a region whose initial pattern showed greater activation. Closer examination of Fig. 6.2a input pattern reveals that the two central (3 - 4 ’th) palings show greater activation than palings lateral to them. With this input, when the local excitation (E) is small (Fig. 6.3), the pattern eventually settled down to two activated regions centered each at two central palings (Fig. 6.3b). In this case, because the two paling heights were about equal they tied and a b B3.:+/+:.+++:59870^:.^B 6901770^620^02095032340^087575 194 the equation ended up choosing both of them. But if E is made larger and more significant, the pattern settles down to one broad hump that encompasses two central palings (Fig. 6.4b). Simply, the local excitatory mask E tunes the sharpness of maximum selector. In the case of Fig. 6.5b, when E is tuned not sharp enough to catch the individual (highest) palings but sharp enough to discriminate higher fourth-fifth paling group from the first-second group, the pattern settled down to one broad hump encompassing the fourth and fifth palings. a b Figure 6.5. The eq-1 based transformation of the barrier pattern of Fig. 6.2b with A = l, B=25, D=5, and E=mask(4.5, 2.0, 2.44). (a) The barrier pattern after 20 cycles. Gaps with smaller width, first and fourth, are heightened while other wider gaps’ potentials are lowered, (b) The barrier pattern after 420 cycles. 195 6.3.2. Local Shunting Inhibition - 1 Here, evolution of the potential field Fig. 6.2 is governed by an equation of the form at i+r. X /(% >£« <q,+D) i+ n k i k= i— r . (eq-2) where coefficient E/a (Iid) describes the fall-off with distance between cells qi and qk in the recurrent excitatory (inhibitory) term, re (r() is the effective range of recurrent excitation (inhibition), and activity of each qi lies between -D and B as we can see from the equilibrium analysis; i+ r. i+ r . k=i-r. k= i— r . i+ r „ i+ r. A + X f^k)Eki+ ^giqkVki k=i-r. k= i— r . i+ r. B if J , f ( q k)Eki » Y ,S (q k)hi k=i-r: k=i-re i+ r, i+ r, -d if k=i-r, k=i-r„ The eq-2 is a recurrent network having center-excitatory, surround-inhibitory shunting connection pattern. The equation above is translated into NSL as; ^ = - A * q + (B -q )[E * Q e] - ( q + D )[I*Q i] at with E = mask(2 re, sigqe, wgtqe ) / = mask(2ri, sigqt, wgtq() Qe = NSLsigmoid(q, 0, 25, 0, 25) Qj = NSLsigmoid(q, 0, 25, 0, 10) With A =l, B=25, D=5, and I=(7.5, 2.5, 10), we present an evolution of the input barrier pattern through the changes in the E mask shape. As in Fig. 6.3, we 196 first tried to lower the gap potentials by picking a small radius (re) and small values for sigqe and wgtqe. With E=(1.5, 2.0, 2.23), Fig. 6.6 shows stabilization of the barrier pattern (Fig. 6.2a) after 1000 cycles. It shows lowered gap potentials and two lowered potential regions right beside the barrier ends. And unlike Fig. 6.3b, it did not stabilize into choosing a small region of maximal inputs but stabilized into an appropriate pattern. In this respect, Fig. 6.6 is a better stable representation of the learning dynamics than those with eq-1. iiii! i i i i Figure 6.6. The eq-2 based transformation of the barrier pattern o f Fig. 6.2a with A = l, B=25, D=5, I=mask(7.5, 2.5, 10) and E=mask(1.5, 2.0, 2.23). The barrier pattern stabilized, after 1000 cycles, into a well formed pattern showing peak potentials corresponding to the paling locations and lowered gap potentials corresponding to the passable gaps. To simulate the increase of gap-potentials depicting the animals’ inability to negotiate given gaps, E is made broader and more influential by setting E=(3.5, 2.0, 3.6). Fig. 6.7a and 6.7b show initial stage (70 cycles) and stabilized state (580 cycles) of evolution of the Fig. 6.2a barrier. Notice that initially E only raises the gap-potentials (Fig. 6.7a), then the pattern repeats itself laterally and stabilizes into a wavy form that spans the entire view-field (Fig. 6.7b). 197 Figure 6.7. The eq-2 based transformation of the barrier pattern of Fig. 6.2a with A = l, B=25, D=5, I=mask(7.5, 2.5, 10), and E=mask(3.5, 2.0, 3.6). Compared to the E of Fig. 6.6, re and w g tq e are made larger, (a) The barrier pattern after 70 cycles showing the rise of gap potentials, (b) The stabilized barrier pattern after 580 cycles showing the wavy form. With E=(2.5, 2.0, 2.82), about mid-point of E ’s in Fig. 6.6 and 6.7, we tested the eq-2 to the Fig. 6.2b multi-gap barrier pattern. Fig. 6.8a shows stabilized pattern of Fig. 6.2b after about 600 cycles with the E. The pattern shows lowered gap potentials of similar depth irrespective of original gap widths in Fig. 6.2b. Also, peaks and valleys are rearranged in a position where the distances between them are about equal. So, it appears that the stabilized p attern show s regularization/homogenization of an original pattern based on “average” feature of it. When E is made a little larger and stronger with E=(3.5, 2.0, 3.46), the Fig. 6.2b 198 Figure 6.8. The eq-2 based transformation of the barrier pattern of Fig. 6.2b with A = l, B=25, D=5, I=mask(7.5, 2.5, 10). (a) The stabilized barrier pattern after 600 cycles with E=mask(2.5, 2.0, 2.82). Notice the equal spacing between the peaks and valleys, (b) The barrier pattern after 30 cycles with E=mask(3.5, 2.0, 3.46). We can see that early in the evolution of Fig. 6.2b, the gap potentials with smaller widths rise while those of larger width are lowered, (c) The stabilized pattern of (b) after 1120 cycles showing the wavy form that spans the whole field as in the Fig. 6.7b. A+4/:.^^^..^//^+:.+8.:B 199 pattern transformed to Fig. 6.8b initially (30 cycles) then stabilized to the pattern Fig. 6.8c. Notice that as E is made larger and stronger, the gap potentials rise. A dramatic example of regularization property of the eq-2 can be found by comparing the Fig. 6.7b and Fig. 6.8c: Even though Fig. 6.8c is based on irregular gap-width Fig. 6.2b pattern, the stabilized pattern looks very similar to Fig. 6.7b that is based on Fig. 6.2a when both patterns are evolved from similar E. With center-excitation and surround-inhibition, Eq-2 connection pattern is very similar to the patterns found in many animal visual centers such as frog retina and visual cortex of monkeys. Used in recurrent network, as we saw above, the connection pattern yields interesting patterns during the transitory period before it stabilized to the repetitive pattern. The regularization property of the network becomes more apparent as the size of E gets closer to the gap width (5-6 units). The regularization property is one aspect of self-organization (Malsburg, 1973; Kohonen, 1982) where non-regular stimulus pattern like Fig. 6.2b arranges itself into a regular form dictated by the eq-2. 6.3.3. Local Shunting Inhibition - 2 Although it is possible to predict and so utilize the evolution of a pattern based on eq-1 or eq-2 in short term, it usually “degenerates” into a maximum selector or a repetitive pattern. Perhaps one way to get around this problem is to assume that the learning process ceases after some time, unlike in our model where it always tries to reach the equilibrium state. Here, we present another attempt to find a right form of equation whose dynamics is always on the path of producing the right pattern we 200 want. Here, evolution of the potential field Fig. 6.2 is governed by an equation of the form dqj_ dt --Aqi+iB-qi) I + r , k= i— r. - iq i + D) i+r, k= i— r . (eq-3) where coefficient E/q (hi) describes the fall-off with the distance between cells qi and qk in the recurrent excitatory (inhibitory) term, re (r,) is the effective range of recurrent excitation (inhibition), and activity of each qi lies between -D and B as we can see from the equilibrium analysis; i+ re i+r, B X f t q t )Eu - D K + D B if J , f ( q t )Ek l » k=i-r, k= i-r, Qi ~ i+r] i+rj ^ - D if >:> Y . f ,lit>Eh K + A+ 'Zf(.qk)E ki- k=i-r, k=i-r. k= i-r. k=i-r. The eq-3 above is translated into NSL as; ^ = - A * q + (B -q )[E * Q e] - ( q + D )[ K -(I* Q i)] at with E = mask(2re, sigqe, wgtqe) I = maskEIri, sigq-, wgtq,) Qe = NSLsigmoid(q, 0, 25, 0, 25) < 2 , = NSLsigmoid(q, 0, 25, 0, 1) The eq-3 has inhibitory term (the third term of right-hand-side) whose strength is inversely proportional to the local summation (H g iq O h i)’ which contrasts to the formulation in eq-2. With this formulation, the smaller the I*Qi the greater must be the inhibitory effect that will lower the gap potentials. So, if the mask I is smaller than the gap width, gap regions will have little I*Qi (large inhibition) that will lower the gap-potentials. As / gets larger to the size of gap width, I*Qi of gap regions will 201 get more strength from its neighboring palings, so the inhibition that has lowering effect to the gap potentials becomes smaller. Because passable gaps should always be wider than impassable gaps, the mask size of I (2re) should be larger than that of1 E. W ith this formulation we have explicit means to control the gap potentials through both local excitatory and inhibitory interactions. With A =l, B -25, D=5, E=(1.5, 1.0, 1.22), and I=(2.5, 2.5, 1.0), we simulated evolution of a barrier pattern with different values of K. With K=3.64, figures 6.9a and 6.9b show the transformed pattern of Fig. 6.2a at early stage (60 cycles) and late stage (500 cycles) of evolution. The pattern Fig. 6.9b changed very little after 1000 cycles, but slowly gap potentials were rising to the direction of peaks. The point here is that, after the initial settling period, the semi-stabilized pattern shows very slow rate of change that is constrained within the barrier pattern. When K is increased to 3.67, the semi-stabilized pattern looked very similar to the Fig. 6.9b but the change was in other direction where peak potentials were coming down to the level of gap potentials. With K=2.0, Fig. 6.2a quickly stabilized to Fig. 6.9c where the barrier is evolved into one single raised potential region. Notice that the pattern does not exhibit maximum selector or self-organizing properties like earlier formulations. When the barrier with irregular gap (Fig. 6.2b) is presented with K=3.0, the pattern evolves to Fig. 6.9d after 300 cycles, which then eventually evolves to a pattern similar to Fig. 6.9c. Fig. 6.9d shows that gaps with smaller Figure 6.9. The eq-3 based transformation of the barrier pattern of Fig. 6.2 with A = l, B=25, D=5, E=mask(1.5, 1.0, 1.22), and I=mask(2.5, 2.5, 1.0). (a) The Fig. 6.2a barrier pattern after 60 cycles with K=3.64. (b) The semi-stabilized Fig. 6.2a barrier pattern after 500 cycles with K=3.64. (c) The stabilized Fig. 6.2a barrier pattern after 520 cycles with K=2.0. (d) The Fig. 6.2b barrier pattern after 300 cycles with K=3.0. At this early stage, gap potentials corresponding to the smaller gaps have already risen while those of the larger gaps show lowered levels. With this setting for K, the pattern eventually stabilizes into a homogeneously raised level similar to the (c). 203 width get filled up before wider gaps are filled. From this experiment we can see that, with given settings for A, B, D, E, and I, a value for K (Ks) may be defined that acts like a saddle point where for a value greater than Ks the pattern evolves into a homogeneously lowered pattern, and if smaller than Ks the pattern stabilizes to a homogeneously raised pattern. And as K departs more from Ks, the faster the pattern approaches a stable form. Fig. 6.10 shows our attempt to elicit a stable pattern that shows well defined potential regions corresponding to palings and gaps. With E=(1.0, 1.0, 1.22), I=(2.5, 2.5, 1.0), and K=3.64, parameter change involving only a slight decrease of E mask size from 1.5 to 1.0 compared with Fig. 6.9a, Fig. 6.10a is the stabilized pattern (500 cycles) of Fig. 6.2a that shows sharply defined potential boundaries between palings and gaps. The pattern showed no noticeable change after 2000 cycles. It appears that the heights of the palings are translated to the potential widths from the observation that the two central paling potentials of Fig. 6.10a are wider than their neighbors. Also, with E fixed, a stabilized pattern changes very little from Fig. 6.10a even though I is varied over wide range. But there is upper limit as to how large or strong I can become. When I is set to I=(6.5, 2.5, 9.0), the gaps fill in and rise beyond B, taking neighboring regions along with them. This is because I*Qi > K and the whole inhibitory term “-(q+D)[K-(I*Qi)]” > 0 and changes into excitatory term. So the eq-3 should satisfy the relation I*Qi < K. The parameter setting also lets Fig. 6.2b evolve into sharply defined potential regions as shown in Fig. 6.10b. 204 Figure 6.10. The eq-3 based transformation of the barrier pattern of Fig. 6.2 with A = l, B=25, D=5, E=mask(1.0, 1.0, 1.22), I=mask(2.5, 2.5, 1.0), and K=3.64. (a) The stabilized Fig. 6.2a barrier pattern after 500 cycles, (b) The stabilized Fig. 6.2b barrier pattern after 400 cycles. Fig. 6.11 shows evolution .of pattern with K=3.64, E=(1.5, 1.0, 1.22), and I=(5.5, 2.5, 3.16). Compared with Fig. 6.9a-b, Fig. 6.11 has I with larger size (2rj) and bigger weight (wgtqj). In Fig. 6.9 we saw that lowering the K results in a rise of gap potentials, and in Fig. 6.11 we can see that enlarging and strengthening the mask I have a similar effect as lowering the K. Fig. 6.11a and 6.1 lb show initial stage (50 cycles) and stabilized state (790 cycles) of evolution of the Fig. 6.2a barrier. Notice that initially I lowers the gap-potentials, then settles them down to a pattern similar to the Fig. 6.9c. Fig. 6.11c shows the evolution of Fig. 6.2b after 410 cycles, B9++^^.5.^$::.82/.+.^^ 205 showing that the gap potentials of smaller gaps have already begun to fill up while the gap potentials of wider gaps still have lowered gap potentials. The pattern is very similar to the Fig. 6.9d. Figure 6.11. The eq-3 based transformation of the barrier pattern of Fig. 6.2 with A = l, B=25, D=5, E=mask(1.5, 1.0, 1.22), I=mask(5.5, 2.5, 3.16), and K=3.64. The experiment is designed to test the effect of changing I mask on the evolution of barrier potential. Compared to the earlier experiments of Fig. 6.9 and -10, r{ and wgtqi are made larger, which should have effect on raising the gap potentials, (a) The Fig. 6.2a barrier pattern after 50 cycles, (b) The stabilized Fig. 6.2a barrier pattern after 790 cycles, (c) The Fig. 6.2b barrier pattern after 410 cycles. At this stage, gap potentials corresponding to the smaller gaps have already risen while those of the larger gaps show lowered levels. The pattern eventually stabilizes into a homogeneously raised level similar to the (b). 1 206 6.4. Future Research The proposed neural models of anuran barrier learning are based on our observation that frogs appear to learn to classify a paling fence barrier through experience, and the hypothesis that the mechanism of a modulatory recurrent network may be involved in the learning. The issue whether indeed learning is a major component in anuran detour behavior needs to be further substantiated through more elaborate experiments. Also, because we tentatively put the learning center at the striatum, an interesting experiment will be observing frogs’ detour performance after striatum lesion. Recurrent shunting dynamics is proposed to be the underlying conception of the anuran perceptual learning model through modulation of potential field pattern of a barrier. The learning models we presented all show capacity to modulate a barrier-potential in a favorable way over a short term, but exhibit long term dynamics that are not so favorable. This raises question as to when to stop the learning process. In real situation, this may simply be handled by the animal by freezing the learning state if it finds the representation favorable and further modification does not offer any better or even a worse model of a real world. In our model the connection patterns are fixed throughout the modulating process. An interesting future research may involve dynamic modification of the connection pattern and strength during the modulation process. Another future research topic would be the modeling of Group-A’s barrier learning. A very simple explanation for this would be to use the generalization aspect of the learning with some twist. If a frog finds it harder to generalize its learning as barrier width becomes wider (i.e., 207 gap count becomes larger), it will take shorter time to learn to detour around a narrow barrier than a wider barrier. Then, it is possible that those frogs that experienced smaller barriers (Group-A) would find it more easier to generalize their learning to a wider barriers than those who did not (Group-B). We believe the learning models we presented earlier can accommodate the above generalization by linking the potential field equations’ learning rate to the width of a barrier. 208 CHAPTER 7 CONCLUSION This chapter summarizes the main contents and results o f this dissertation noting the coherent nature o f developed models from the low level retinal processing to the higher level detour and learning models. ' In this thesis we traced and modeled information processes required when an anuran exhibits detour behavior in its stationary surround. Starting from the lowest retina model to the final detour model, the aim of this dissertation is to offer plausible and coherent models for the successive information processing stages based on our current understanding of substrate’s anatomical, neurophysiological, and ethological data. The progression of the thesis follows the direction of information flow and corresponds to the biological and functional data. W ith available neurophysiological data on anuran retina and pretectum, we were able to build the corresponding neural models constrained by the data, whereas scarcity of relevant physiological data on later processes of sensorimotor transformation and integration did not give us such opportunity. But the schema-based models presented in Chapters 5 and 6 lie in the plausible information processing path of neural models of 209 Chapters 3 and 4, and so it is our hope that they reflect basic information processing style of underlying neural structures. In Chapter 3, we presented a populational approach to grouping R1 and R2 ganglion cells into a single class, R1.2, where each R1 and R2 occupies the two extremes of the population. With the new inputs and the erasability modulation function, the R1.2 model exhibits the gradual response transition between the R1 and R2 class neurons as well as capable of showing the typical R1 and R2 responses. New characteristics for R3 class neuron are investigated based on slight modifications to its receptive field property. Through the measurement of the firing rate and the ERF size, we showed that the R3 is capable of detecting static-edges through the edge’s retinal movement made possible by animal’s own movements. A new formulation of R4 class neuron is proposed. With the inclusion of new input channels and delayed excitation mechanism, the model can account for biological responses— dark SO response, sustained O ff response, delayed diffuse-On response—that earlier models were not capable of. The proposed retina models lay ground for anuran SO perception model that uses the retina signal as its input. With the R1.2 that is sensitive to static edges, R3 that is capable of detecting SOs by self induced retinal movements, and R4 that is responsive to a large dark static object, Chapter 4 investigates how the pretectum may utilize the retinal signals to perceive stationary objects. First, we presented individual pretectal cell models that are thought to be involved in SO perception and showed that the model responses correspond well with the biological data. Based on our hypothesis that anurans recognize SOs from the neuronal patterns portrayed on the population of pretectal cells, we showed how the SO patterns may be depicted on the population 210 of pretectal cell models mentioned above and presented plausible interpretation cues the animal may employ based on our observation of ethological data. The anuran neuronal pattern of SOs supports the view that the animals may recognize the presence of a large dark SO through SO-sensitive neuron’s broadly tuned receptivity and that further detailed analysis may be handled by its sensitivity to vertical edges. A depth perception model that utilizes disparity and accommodation cues showed its ability to correctly segment the depths of double barriers. A neuronal model of anuran spatial short term memory is presented in the form of recurrent shunting feedback loop involving striatum, and its significance in overall anuran SO perception is discussed. The model showed that it can retain an SO pattern up to a minute and its function is thought to be in prolonging the neuronal representation of the SOs. In Chapter 5, we presented schema-based models of anuran sensorimotor transformation, data integration, and detour. Based on Grobstein’s data and building on Cobas and Arbib’s prey capture model, the detour model is constructed as an assemblage of perception, sensorimotor transformation, and data integration models whose end-goal is to direct the animal appropriately to prey in a given surrounding. Information flow of the model is basically parallel and feed-forward from the perception to the data integration module, at which final motor action is determined through cooperation and competition. Pretectal and tectal data integration is posited to take the form of motor integration at the level of medulla rather than sensor fusion. The detour model showed its ability to match the Collett’s ethological data. In Chapter 6, new data on anuran detour behavior from our lab are presented. The data show some evidence that suggests a some form of learning is an integral 211 component of anuran detour behavior. We hypothesized that what animal is learning is to characterize a visual object differently from its learning experience. We then related the animal learning experience to artificial potential field of robotic path navigation method to propose an adaptive modulation scheme for the potential field. We proposed that recurrent shunting connection could be a good substrate metaphor for the anuran learning model due to its ability to change a firing pattern through its iterative dynamics. The learning models we presented all showed capacity to modulate the barrier potentials, but their long term dynamics also showed some undesirable characteristics, raising the question as to when to stop the learning process. 212 References A m ari, S., Arbib, M. A. (1977): C om petition and cooperation in neural nets, (in Systems neuroscience, j. M etzler, Eds.), Academ ic Press, 119-165. an der H eiden, U., Roth, G. (19S7): M athem atical m od el and sim ulation of retina and tectum opticum of low er vertebrates. Acta Biotheoretica, 36,179-212. Arbib, M. A . (1989): The M etaphorical Brain 2: Neural N etworks and Beyond. W iley Interscience. Arbib, M. A. (1992): Schem a theory, (in The Encyclopedia of Artificial intelligence, Shapiro, S. Eds), 2'nd edn. N e w York, NY: W iley Interscience, 1427-1443. A rbib, M. A ., H ou se, D . H. (1987): D epth and Detours: A n essa y on v isu a lly g u id ed behavior, (in Vision, Brain and Cooperative computation, M .A. Arbib & A.R. H anson, Eds.), A Bradford B ook/M IT Press, 129-163. Arkin, R. (1989): M otor Schem a based m obile robot navigation. Int. J. Robotics Res. V ol 8, N o 4, 92-112. Arkin, R., M urphy, R. R. (1990): A utonom ous navigation in a m anufacturing environm ent. IEEE. Trans. Robotics and Automation, V ol 6, N o 4, 445-454. Barlow, H. B. (1953): Sum m ation and inhibition in the frog's retina. Journal of Physiology, 119,69-88. Berkinblit, M. B., Feldm an, A. G., & Fukson, O. J. (1986): A daptabilty o f inn ate m otor pattern and m otor control m echanism s. Behavioral and Brain Sciences, 9, 585-638. Brow n, W. T., Marks, W. B. (1977): U nit R esponses in the Frog's C audal Thalam us. Brain Behav. Evol, 14, 274-297. B urghagen, H ., Ewert, J. -P. (1983): Influence of the background for d iscrim inating object m otion from self-induced m otion in toads Bufo bufo (L.). J. Comp. Physiol, 241-249. Carey, R. G. (1975): A quantitative analysis of the distribution of the retinal elements in frogs and toads w ith special emphasis on the Area Retinalis. M asters T hesis, U n iv ersity o f M assachusetts at Am herst, Am herst M assachusetts. C ervantes-Perez, F., Arbib, M. A. (1990): Stability and param eter d ep en dency analysis of a facilitation tectal colum n (FTC) m odel. /. Math. Biol., 29,1-32. C ervantes-P erez, F., Lara, R., & Arbib, M. A. (1985): A N eural M od el of Interactions Subserving Prey-Predator D iscrim ination and Size Preference in A nuran A m phibia. J. Theor. B iol, 113,117-152. 213 Cobas, A ., Arbib, M. A. (1992): Prey-catching and P redator-avoidance in frog and toad: D efining the schem as. J. Theor. B iol, 157,271-304. Collett, T. (1977): Stereopsis in toads. Nature, 267, 349-351. C ollett, T. (1982): D o toads plan routes? A stu d y of detour behavior of B. viridis. J. Comp. Physiol. A , 146,261-271. Collett, T. (1983): Picking a route; D o toads fo llo w rules or m ake plans? (in Advances in Vertebrate Neuroethology, J.-P. Ewert, R.R.Caprinica and D.J. Ingle, Eds.), 321-330. Cott, H. B. (1936): The effectiveness of protective adaptation in the h ive bee, illustrated by experim ents on the feeding reaction, habit form ation, and m em ory of the com m on toad (Bufo bufo bufo). Proc. Zool. Soc. Lond., 111-133. D ean, J. (1980): Encounters b etw een bom bardier beetles and tw o sp ecies of toads (Bufo am ericanus, B. marinus): Speed of prey-capture d oes not determ ine success. }. Comp. P hysiol, 135,41-50. D ev, P. (1975): Perception of depth surfaces in random -dot stereogram s: a neural m odel. Int. J. Man-Machine Studies, 7, 511-528. D eV alois, R. L., Albrecht, D. G., & Thorell, L. G. (1982a): Spatial frequency selectivity of cells in m acaque visual cortex. Vision Research, 22, 545-559. D eV alois, R. L., Yund, E. W., & H epler, N . (1982b): The orientation and direction selectivity of cells in m acaque visual cortex. Vision Research, 2 2 ,531-544. D id day, R. L., (1976): A m od el of visuom otor m echanism s in the frog optic tectum . M ath. Biosci. 30,169-180. D om in ey, P. F., Arbib, M. A. (1992): A cortico-subcortical m od el for generation of spatially accurate sequential saccades. Cerebral Cortex, M ar/A pr, V 2 ,153-175. D ow lin g, J. E. (1976): P hysiology and M orphology of the Retina, (in Frog Neurobiology, R. Llinas and W. Precht, Eds.), 278-296. D ow lin g, J. E. (1987): The Retina: A n Approachable Part of the Brain. H arvard U niversity Press, C am bridge, M assachusetts. D ubin, M. (1970): The inner plexiform layer of the vertebrate retina: a quantitative and com parative electron m icroscope analysis. J. Comp. Neurol. 140,479-506. Eckm iller, R., G russer, O. -J. (1972): Electronic sim u lation of the v elo city fu n ction of m ovem ent-detecting neurons. Bibliotheca Ophthalmologica, 82, 274-279. Enroth-C ugell, C., R obson, J. G. (1966): The contrast sen sitivity of retinal gan glion cells of the cat. J. Physiol, 187,517-552. 214 Ewert J. -P., Krug, H ., & Schonitz, G. (1979): A ctivity of retinal class R3 ganglion cells in the toad Bufo bufo (L.) in respon se to m o v in g configurational stim uli: Influence of the m ovem en t direction. Journal of Comparative Physiology, 129, 211-215. Ewert, J. -P. (1968): Der Einflufi von Z w ischenhirndefekten auf d ie V isuom otorik im Beute- und Fluchtverhalten der Erdkrote (Bufo bufo L.). Z. vergl. Physiol. 61,41-70. Ewert, J. -P. (1971): Single unit response of the toad's (Bufo americanus) caudal thalam us to v isu al objects. Z. vergl. Physiol, 74,81-102. E w ert, J. -P. (1987): N eu ro eth o lo g y of releasing m echanism s: P rey-catching in toads. Behavioral and Brain Sciences, 10, 337-405. Ew ert, J. -P., Fram ing, E. M ., Schiirg-Pfeiffer, E., & W eerasuriya, A. (1990): R esponse of m edullary neurons to m ovin g visual stim uli in the com m on toad. I. Characterization of m edial reticular neurons by extracellular recording. /. Comp. Physiol. A , 167, 495-508. E w ert, J. -P., Seelen, W. v. (1974): N eu ro b io lo g ie u n d System -T heorie ein es v isu ellen M uster-Erkennungsm echanism us b ei Kroten. Kybemetik, 14,163-183. Finkenstadt, T. (1989): V isual associative learning: searching for behaviorally relevant brain structures in toads, (in Visuomotor Coordination: Amphibians, Comparisons, M odels, and Robots. Ewert, J. -P., & Arbib, M. A. Eds.), Plenum Press, N e w York, London, 799-832. Fite, K. V., Scalia, F. (1976): The amphibian visual system. (Fite, K. Eds.), A cadem ic Press. Fuster, J. M., A lexander, G. E. (1973): Firing changes in cells of the nucleus m edialis dorsalis associated w ith m em ory response behavior. Brain Res., 61, 79-91. G aillard, F. (1985): Binocularly driven neurons in the rostral part of the frog optic tectum . /. Comp. Physiol. A. 157,47-55. Garcia, R., Gaillard, F. (1989): Q uantitative stu d ies on ipsilateral typ e 2 retinotectotectal (IRTT) units in frogs: h om ologies w ith R3 ganglion cells. J. Comp. Physiol. A , 164, 377- 389. G aze, R. M ., K eating, M. J. (1970): R eceptive field properties of single units from the visu al projections to the ipsilateral tectum in the frog. Q uarterly Journal of Experimental Physiology, 55,143-152. G nadt, J. W ., A nderson, R. A. (1988): M em ory related m otor planning activity in posterior pariatal cortex of m acaque. Exp. Brain Res., 70,216-220. G robstein, P. (1988): B etw een the retin otectal projection and d irected m o vem en t: Topography of sensorim otor interface. Brain Behav. E vol, 31, 34-48. G robstein, P., M asino, T. (1986): Sensorim otor circuitry und erlyin g directed m ovem en t in the frog: ev id en ce for an interm ed iate represen tation of sp ace in the tectofu gal pathw ays. Soc. Neurosci. Abstr. 12, 684. 215 G robstein, P., R eyes, A ., Z w an ziger, L., K ostyk, S. (1985): Prey orien tin g in frogs: A ccou ntin g for variations in ou tp ut w ith stim ulus distance. /. Comp. Physiol. A. 156, 775-785. G robstein, P., Staradub, V. (1989): Frog orienting behavior: the d escend ing distance signal. Soc. Neurosci. Abstr. 15,54. G rossberg, S. (1976): A d ap tive pattern classification and universal recoding, II: Feedback, expectation, olfaction, and illusions. Biological Cybernetics, 23,187-202. G rossberg, S., K uperstein, M. (1989): N eural dynam ics of adaptive sensory-m otor control. Pergam on Press, 39-53. G riisser, O.J., G russer-C om ehls, U. (1976): N europh ysiology of the A nuran V isual System , (in Frog Neurobiology, R. Llinas and W. Precht, Eds.), 297-385. G russer-C om ehls, U . (1988): N eu rop h ysiological properties of the retinal g a n glion cell classes of the Cuban treefrog, Hyla septentrionalis. Exp. Brain Res., 73, 39-52. G uiloff, G.D. (1980): Class 1 retinal ganglion cells in the toad, Bufo Spinulosus. Vision Res., 20, 549-551. H artline, H. K. (1938): The respon se of single optic n erve fibers of the vertebrate ey e to illum ination on the retina. American Journal of Physiology, 121,400-415. H ikosaka, O., Sakam oto, M ., U su i, S. (1989): Function p roperties of m o n k ey caudate neurons. III. A ctivities related to expectation of targets and rew ard. /. of Neurophysiol. 61:814-832. H orn, K., Pasternak, T. (1988): Spatial resolu tion of th e cat: V ariation w ith retinal eccentricity. Investigative Ophthalmology & Visual Science, Supplem ent, 29,141. H ouse, D. H. (1989): D epth Perception in frogs and toads. Lecture Notes in Biomathematics, 80, Springer-Verlag. Ingle, D. (1970): V isuom otor functions of the frog optic tectum. Brain Behav. Evol. 3, 57-71. Ingle, D. (1971): D iscrim ination of edge-orientation by frogs. Vision Res. 11. 1365-1367. Ingle, D. (1976): Behavioral correlates of central v isu a l fu n ction in A nurans. (in Frog Neurobiology, R. Llinas and W. Precht, Eds.), 435-451. Ingle, D. (1977): D etection of Stationary Objects by Frogs follow in g O ptic Tectum Ablation. J. Comp. Physiol., 91.1359-1364. Ingle, D. (1983): Brain m echanism s of visual localization by frogs and toads, (in Advances in Vertebrate Neuroethology, J. -P. Ewert, R. R. Caprinica and D. J. Ingle, Eds.), 177-226. 216 Ingle, D. (1991a): Control of frog evasive direction: T riggering and b iasin g system s, (in Visual Structures and Integrated Functions, M .A. Arbib & J.-P. Ewert, Eds.), R esearch notes in N eural C om puting, Springer-Verlag, 181-189. Ingle, D. (1991b): The striatum and short-term spatial m em ory: From frog to m an. (in Visual Structures and Integrated Functions, M .A. Arbib & J.-P. Ewert, Eds.), R esearch notes in N eural C om puting, Springer-Verlag, 273-280. Ingle, D ., H off, K. vS. (1990): V isually elicited ev a siv e behavior in frogs: G ivin g m em ory research an ethological context. Bioscience, 40(4), 284-291. Keating, M. J., Gaze, R. M. (1970): O bservations on the 'surround' properties of the receptive fields of frog retinal ganglion cells. Quarterly Journal of Experimental Physiology, 55,129- 142. K hatib, O. (1986): R eal tim e obstacle avoidan ce for m an ipu lators and m o b ile robots. International Journal of Robotics Research, 5(1), Spring, 90-98. K ohonen, T. (1982): Self-organization of topologically correct feature m aps. Biol. Cyb., 43:59- 69. Koren, Y., Borenstein, J. (1991): Potential Field M ethods and their Inherent Lim itations for M obile R obot N avigation . Proceedings of the 1991 IEEE International Conference on Robotics and Automation, Sacramento, California, 1398-1404. K ostyk, S. K., G robstein, P. (1987a): N euronal organization und erlyin g v isu ally elicited prey orienting in the frog - 1. Effects of various unilateral lesions. Neuroscience, 21, 41- 55. K ostyk, S. K., G robstein, P. (1987b): N euronal organization underlying v isu ally elicited prey orienting in the frog - II. Anatom ical studies on the laterality of central projections. Neuroscience, 21, 57-82. K ostyk, S. K., G robstein, P. (1987c): N euronal organization underlying v isu ally elicited prey orienting in the frog - III. Evidence for the existence of an uncrossed d escend ing tectofugal pathw ay. Neuroscience 21, 83-96. Krueger, H ., M oser, E. A. (1972): The influence of the m odulation transfer function of the dioptric apparatus on the acuity and contrast of the retinal im age in R. Esculanta. Vision Res. 12, 1281-1289. Lara, R., Arbib, M. A., & Cromarty, A. S. (1982): The role of the tectal colum n in facilitation of am phibian prey-catching behavior: A neural m odel. /. Neurosci., 2, 521-530. Lara, R., C arm ona, M ., D aza, F., & C ruz, A. (1984): A G lobal M od el of the N eu ral M echanism s R esponsible for Visuom otor Coordination in Toads. /. theor. B iol, 110, 587- 618. Latom be, J. -C. (1991): Robot m otion planning. K luwer A cadem ic Publishers. 217 Lazar, G. (1989): Cellular Architecture and C onnectivity of the Frog's O ptic Tectum and Pretectum , (in Visuomotor Coordination; Amphibians, Comparisons, M odels and Robots. Ewert, J. -P., and Arbib, M. A., Eds.), Plenum Press, 175-199. Lee, Y. B. (1986): A Neural N et Model of Frog Retina: A Discrete Time-Space Approach. Ph.D. D issertation, D ept of C om puter and Inform ation Science, U niversity of M assachusetts at Am herst. Lennie, P., Trevarthen, C., Van Essen, D ., & W assle, H. (1990): Parallel Processing of V isual Inform ation, (in Visual Perception: The Neurophysiological Foundation, Spillm an, L., & W erner, J. S. Eds.), 103-128. Lettvin, J. Y., M aturana, H. R., Pitts, W. H ., & M cC ulloch, W. S. (1961): W hat the frog's eye tells the frog's brain. Proceedings of the IRE, 47,1940-1951. L iaw , J. -S., Arbib, M. A. (1993): N eu ral M ech anism s U n d erly in g D irection -S elective A voidance Behavior. Adaptive Behavior, V ol 1, N o. 3, 227-261. Lock, A ., Collett, T. (1979): A toad's devious approach to its prey: A study of som e com plex u ses of depth vision. J. Comp. Physiol. A , 131,179-189. M alsburg, Ch. v on der. (1973): Self-organization of orientation sensitive cells in the striate cortex. Kybemetik. 14, 85-100. Marr, D., P oggio, T. (1976): C ooperative com putation of stereo disparity. Science, 194, 283- 287. M asino, T., G robstein, P. (1989a): The organization of d escend ing tectofu gal p ath w ays u n d erlyin g orienting in the frog, Rana pipiens: I. Lateralization, parcellation, and an interm ediate spatial representation. Exp. Brain Res. 75, 227-244. M asino, T., G robstein, P. (1989b): The organization of d escen d in g tectofu gal p ath w ays u nd erlyin g orienting in the frog, Rana pipiens: II. E vidence for the in v o lv em en t of a tecto-tegm ento-spinal pathw ay. Exp. Brain Res. 75,245-264. M atsum oto, N ., S chw ippert, W . W ., Beneke, T. W ., & E w ert, J. -P. (1991): Forebrain- m ediated control of v isu a lly g u id ed prey-catching in toads: investigation of striato- p retectal con n ection s w ith intracellular re co rd in g /la b elin g m eth od s. Behavioural Processes, 2 5 ,27-40. M aturana, H. R., Lettvin, J. Y., M cC ulloch, W . S., & Pitts, W. H. (1960): A n atom y and p h y siology of vision in the frog (Rana Pipiens). Journal of General Physiology, 43 (Suppl.), 129-175. M iller, W. H., Bernard, G. D. (1983): A veraging over the foveal receptor aperture curtails aliasing. Vision Res. 23,1365-1370. 218 M oreno-D iaz, R., Rubio, F., & Rubio, E. (1980): A theoretical proposal to account for visual com putation in a frog's retina. International Journal of Biomedical Computing, 11,415-426. M untz, W . R. A. (1962): M icroelectrode recordings from the d iencephalon of the frog (R. pipiens) and a blue-sensitive system . Journal of Neurophysiology, 2 5 ,699-711. N eary, T. J., N orthcutt, R. G. (1983): N uclear O rganization of the Bullfrog D iencephalon. J, Comp. Neurol. 213, 262-278. Patton, P., G robstein, P. (1986): Possible striatal involvem ent in prey orienting behaviour in the frog. Soc. Neurosci. Abstr., 10: 61. Pigarev, J. N ., Z enkin, G. M ., G irm an, S. B. (1971): The d etectors o f the frog retina. R esponses in unrestrained frogs. Sechenov J. Physiol. USSR 57,1448-1458. Prazdny, K. (1985): D etection of binocular disparities. Bio. Cybernetics. 52, 93-99. Rodieck, R. W., Stone, J. S. (1965): A nalysis of receptive fields of cat retinal ganglion cells. J. Neurophysiol. 28, 965-980. Scalia, F. (1976): The O ptic Pathw ay of the Frog: N uclear O rganization and C onnections, (in Frog Neurobiology, R. Llinas and W. Precht, Eds.), 386-406. Schipperheyn, J. J. (1963): Respiratory eye m ovem ent and perceptions of stationary objects in the frog. Acta. Physiol. Pharmacol. Neerl. 12. 157-159. Schipperheyn, J. J. (1965): Contrast detection in frog’ s retina. Acta. Physiol, pharmacol. Neerl. 13. 231-277. Segraves, M ., G oldberg, M. E. (1987): Functional properties of corticotectal neurons in the m onkey's frontal eye field. J. Neurophysiol., 58,1387-1419. Sterling, P. (1990): Retina, (in The synaptic organization of the brain, G. M. Shepherd Ed.). O xford U niversity Press, 170-213. Szekely, G., Lazar, E. (1976): Cellular and synaptic architecture of the optic tectum , (in Frog Neurobiology, R. Llinas and W. Precht, Eds.). Teeters, J. L. (1989): A Sim ulation System for N eural N etw orks and M odel for the A nuran Retina. Technical Report 89-01. Center for N eural Engineering. U niversity of Southern California. T eeters, J. L., Arbib, M. A. (1991): A m o d el of anuran retina relating interneurons to ganglion cell responses. Biol. Cybern. 64, 197-207. Teeters, J. L., Arbib, M. A ., Corbacho. F., Lee. H . B. (1993): Q uantitative m o d elin g of R esponses of A nuran Retina: Stim ulus shape and size dependency. Vision Research., vol 33, N o 16, 2361-2379. 219 T ilove, R. B. (1990): Local O bstacle A voidance for M obile Robots Based on the M ethod of Artificial Potentials. Proceedings of the 1990 IEEE International Conference on Robotics and Automation, Cincinnati, O hio, 566-571. Tsai, H. J., Ewert, J. -P. (1988): Influence of Stationary and M oving Textured Backgrounds on the R esponse of V isual N eurons in Toads (Bufo bufo L.). Brain Behav. Evol., 32,27-38. W ang, D. L., Arbib, M. A. (1991): H ow s does the toad's visual system discrim inate different w orm -like stimuli? Biological Cybernetics, 64,251-261. W arren, C. W. (1990): M ultiple Robot Path C oordination U sin g A rtificial Potential Fields. Proceedings of the 1990 IEEE International Conference on Robotics and A utom ation, Cincinnati, O hio, 500-505. W eitzen feld , A. (1991): NSL, N eural Sim ulation L anguage, V ersion 2.1, CNE-TR 91-05, Center for N eural Engineering, U niversity of Southern California. W ilczynski, W ., N orthcutt, R. G. (1983a): C onnections of the Bullfrog Striatum: A fferent Organization. J. Comp. Neurol. 214, 321-332. W ilczynski, W ., N orthcutt, R. G. (1983b): C onnections of the B ullfrog Striatum: Efferent Projections. /. Comp. Neurol. 214,333-343. W ilson, H. R., Levi, D ., M affei, L., R ovam o, J., & D eV alois, R. (1990): The Perception of Form: Retina to Striate Cortex, (in Visual Perception: The Neurophysiological Foundation, Spillm ann, L., & W erner, J. S. Eds.), A cadem ic Press, 231-272. Y ou n g, R. A . (1987): The G aussian d eriv a tiv e m o d el for sp a tia l vision : I. R etinal m echanism s. Spatial Vision, Vol. 2, N o. 4, 273-293. Zipser, D. (1991): Recurrent netw ork m od el of the neural m echanism of short-tem m em ory. Neural Comput. 3:179-193. Zipser, D ., K ehoe, B., Littlewort, G., & Fuster, J. (1993): A spiking netw ork m o d el of short term active m em ory. The J. of Neurosci., A ug. 1993 13(8), 3406-3420.

Linked assets