University of Southern California Dissertations and Theses

The Modification Of Partial Reinforcement Effect As A Consequence Of Electrical Stimulation Of The Caudate Nucleus In Cats

(USC Thesis Other)

The Modification Of Partial Reinforcement Effect As A Consequence Of Electrical Stimulation Of The Caudate Nucleus In Cats

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content This dissertation has been microfilmed exactly as received 6 7 - 4 1 0 LAUPRECHT, C arl W alter, 1930- THE MODIFICATION OF PARTIAL REINFORCEMENT E FFEC T AS A CONSEQUENCE OF ELECTRICAL STIMULATION OF THE CAUDATE NUCLEUS IN CATS. U niversity of Southern C alifornia, Ph.D ., 1966 Psychology, experim ental University Microfilms, Inc., Ann Arbor, Michigan THE MODIFICATION OF PARTIAL REINFORCEMENT EFFECT AS A CONSEQUENCE OF ELECTRICAL STIMULATION OF THE CAUDATE NUCLEUS IN CATS by Carl Walter Lauprecht A Dissertation Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (Psychology) August 1966 UNIVERSITY O F S O U T H E R N CALIFORNIA T H E G RADUATE S C H O O L U N IV ER SITY PA R K L O S A N G E L E S , C A L IF O R N IA 9 0 0 0 7 This dissertation, •written by Carl Walter Lauprecht under the direction of h~i.S..Dissertatioti C om mittee, and approved by all its members, has been presented to and accepted by the Graduate School, in partial fulfillment of requirements for the degree of D O C T O R OF P H I L O S O P H Y Dean September 1966 TO MY PARENTS To Hans Karl Lauprecht in fond memory; to Cecilia Juliana Lauprecht in grati tude which goes beyond verbal expression. ACKNOWLEDGMENTS j | j The writer wishes to express his gratefulness to the jmembers of his dissertation committee for the guidance they I [generously provided during the course of this study and the preparation of this dissertation. To Dr. Everett J. Wyers, the committee chairman, go special thanks. His aid and counsel, not only while this study was being conducted, but throughout the writer's years of graduate study, were always given unhesitatingly |and freely and have proved invaluable. | A deep feeling of gratitude is also felt toward [Dr. William W. Grings. Beyond functioning as a scientist on the committee, it was he who many times indicated the jproper path when matters of administrative nature demanded jmore than routine attention. | Finally, three physiologists are due a sincere "Thank You!" Dr. John P. Meehan, as committee member, gave much needed advice during the final phases of the study; Dr. Nathaniel A. Buchwald of the Brain Research Institute, ;Los Angeles, contributed to the formulation of the problem ! land made materials and laboratory facilities available dur- i jing the early part of the study; and Dr. James P. Henry of |the University of Southern California, Medical School, as an j jinformal teacher and friend, patiently read the many early drafts and helped approach the language of the final text. ii TABLE OF CONTENTS ACKNOWLEDGMENTS . . . LIST OF TABLES . . . LIST OF ILLUSTRATIONS Chapter I. INTRODUCTION ................ "Zeitgeist" 1940, 1950, 1960 Scope of the Present Study Organization of Chapters II. BACKGROUND............................... 6 Caudate Nucleus Early history, identification, and gross anatomy Motoric inhibition as first impression of functional role Generalized behavioral effects following caudate stimulation Generalized behavioral effects following caudate lesions Electrophysiology of the caudate nucleus, general method Electrophysiology of the caudate nucleus, low stimulation rates: below 1/2 pps Electrophysiology of the caudate nucleus, intermediate stimulation rates: from 3 to 10 pps Electrophysiology of the caudate nucleus, high stimulation rates: from 20 to 50 pps iii Page ii vii viii 1 Chapter Page Electrophysiology of the caudate nucleus, very high stimulation rates: above 100 pps Specific and discrete behavioral effects following experimental manipulation of the caudate nucleus, general Caudate stimulation and its effect on operant conditioning response rate Caudate manipulation and its effect on visual discrimination Caudate stimulation and its effect on delayed response performance Caudate stimulation and its effect on extinction Summary of caudate function and the possible use of caudate manipulation as a research tool in the behavioral sciences Partial Reinforcement Effect General The "Humphreys' Effect" and previous observations Theories of partial reinforcement effect Expectancy theory After-effects theory Response unit hypothesis Discrimination theory Secondary reinforcement hypothesis Competing response theory Mediating responses theory Assessment of considerations and variables relevant to partial reinforcement effect Percentage of reinforcement Patterning of presentation of reinforcement Situational peripheral cues Inter-trial interval Delay of reward Non-equality of acquisition sessions Conclusion iv Chapter III. i i IV. HYPOTHESIS DEVELOPMENT AND EXPERIMENTAL TESTS ......... .......... General introduction Working definition of partial reinforce ment effect Analysis of explanatory attempts of PRE and their categorization Discrimination theories Habituation theories Requirement to be met for development of PRE as demanded by discrimination theories and habituation theories Analysis of the essentials of NR trials as related to PRE theories Analysis of the essentials of the discrimination processes during extinction which contribute to PRE The role of caudate stimulation in modifying PRE Experimental hypotheses Hypothesis A: Reduction in partial reinforcement effect due to caudate stimulation on non-reinforced acquisi tion trials Hypothesis B: Enhancement of partial reinforcement effect due to caudate stimulation on extinction trials Experimental Tests Test for Hypothesis "A": Experiment I Test for Hypothesis "B": Experiment II EXPERIMENTAL METHOD AND RESULTS............ 95 Experiment I Subjects Source and pre-experimental treatment Preparation Instrumentation Procedure Habituation and training at 100% reinforcement Control extinctions Page 70 v Chapter Page Partial reinforcement retraining Post PR extinctions Results of Experiment I Discussion of Experiment I Experiment II General Instrumentation Subjects Procedure Acquisition training with 100% reinforcement Control extinctions Retraining with 50% partial reinforce ment Extinctions post PR training in discrete trials Results of Experiment II Discussion of Experiment II Discussion General Relation of this study to contemporary work The neurophysiological mechanisms of caudate induced inhibitory events General The interactive process at the thalamus Summary V. DISCUSSION AND SUMMARY 157 LIST OF REFERENCES 183 vi I I ! LIST OF TABLES Table Page I. Target Sites and Coordinates Used for Stereo taxic Electrode Implants in Experimental Animals.................................... 9 8 II. Comparison of Percentages of Trials Responded to during Extinction ........................ 146 vii LIST OF ILLUSTRATIONS Figure Page 1. Schematic Diagram of the Striate Body, Showing Location of the Caudate Nucleus in Man ... 8 2. Caudate Spindles as Typically Displayed on an Oscilloscope ............................. 21 3. Example of Caudate Spindles as Typically Displayed Electrographically .............. 22 4. Control EEG. Typical Electrographic Recording of an Alert Animal not Undergoing Any Brain Stimulation ..... .................... 23 5. Example of Typical "Recruiting Patterns," as Displayed Electrographically .............. 25 6. Electrographic Record as Typically Obtained from Animals Being Caudate Stimulated at 300 Pulses Per Second.................... 28 7. Electrographic Record as Typically Obtained from Animals Being Caudate Stimulated Simultaneously at % and 300 Pulses Per Second................................... 30 8. Electrographic Record as Typically Obtained from Animals Being Caudate Stimulated Simultaneously at 5 and 300 Pulses Per Second................................... 31 9. Modification of Operant Response Rates Due to Caudate Stimulation at h Pulse Per Second and 300 Pulses Per Second................ 34 Figure 10. Target Regions for Subcortical Electrodes Shown by Blackened Dots on Diagramatic Cross Sections Taken from Jasper and Ajmone Marsan (1952).................... 11. Experimental Animal Prior to Electrode Implantation Procedure ................ , 12. Animal Positioned in Kopf Stereotaxic Instrument Ready for Pre-Surgical Prepara tion ............................. . . . , 13. Animal Prepared for Operative Procedure . . , 14. Exposure of Skull......................... 15. Skull Openings Have Been Drilled over Brain Target Sites. Lowest Openings in Photograph Extend through the Animal's Frontal Sinus Cavities. Next Higher Set of Openings Over lies the Caudate Nuclei Targets .......... 16. Electrode during Its Stereotaxic Placement. All Electrodes but One Have Been Positioned and Cemented to Skull by Means of Kadon (R) Dental Cement ........................ 17. Electrode Leads as Connected to a Cannon (R) 25-point Receptacle ...................... 18. Canon (R) Receptacle Attached to the Animal's Skull with Kadon (R) Dental Cement .... 19. Animal's Scalp Sutured to Surround and Partially Enclose the Base which Holds the Canon (R) Receptacle .................... 20. Animal One Day Post Operatively ..... 21. X-Ray of an Experimental Animal after Implantation, Illustrating Positioning of Cortical Electrodes and Extent of Brain Penetration by Concentric Deep Electrodes Page j 97 99 100 101 102 103 104 105 106 107 108 109 ix Figure 22. "Task and Reward Box" as Presented to Experi mental Animals in Experiment I ............ 23. Time to Extinction Curves for Non-Stimulated Animals in Experiment I .................. 24. Lever Press Responses to Extinction Curves for Non-Stimulated Animals in Experiment I . . . 25. Time to Extinction Curves for Stimulated Animals in Experiment I .................. 26. Lever Press Responses to Extinction Curves for Caudate Stimulated Animals in Experiment I . 27. "Task and Reward Box" as Presented to Experi mental Animals in Experiment II. 28. Comparison of Responsiveness to Four Types of Extinction trials by Animals in Experiment I I ........................................ I I Page j j 112 118 119 121 122 149 x CHAPTER I i INTRODUCTION j "Zeitgeist" 1940, 1950, 1960 The following paragraphs are offered to place this dissertation in perspective within present trends of related research: About a decade ago, Boring (1957) described (1) how the goals of scientific effort are to a great extent governed by the forces of Zeitgeist and (2) how the one | jparticular ideology, which enjoys maximal acceptance at any igiven time, precipitates entire sequences of specific and jrelated events across the various disciplines. The !"ideology and spirit of the times" are felt more or less i acutely by all members of the scientific community; but jthey are particularly responded to by those who act as the I synthesizers and articulators of scientific thought. In the contemporary symposia of research workers in the medical and behavioral sciences, these synthesizers and leaders have expressed recent Zeitgeist in their calls for 1 2 Interdisciplinary research combining the faculties of physiology and psychology. Their hope is that interdisci plinary effort will provide some answers to explain the I I intricately interwoven processes of both disciplines which underlie complex molar behavior. The current orientation ! toward psychophysiological research was articulated by Penfield (1957) when he said: The time has come when clinical hypotheses should be of use to physiologists, psychologists, and anatomists. Advances in the understanding of the relationship of brain action to the mind must depend upon cooperative study. Clinicians who seem to have fumbled so long with patients in ward and operating room have something to give the laboratory scientist and much to gain in return. | Subsequently, Jasper (19 61) described the advances made in this direction: The neurological sciences have undergone a remark able expansion during the past decade. In addition to i rapid progress in technical developments, there has I been a growing collaboration in various specialties | focused on the problem of brain function: neurophysi- I ology, neurology, neurosurgery, anatomy, psychology, psychiatry, neurochemistry, and electronics and commu nications engineering. i i Jasper continued: j The need for interdisciplinary collaboration in brain research on an international basis has been recognized (and acted upon) by UNESCO in the formation of the International Brain Research Organization (IBRO), which held its first formal executive meeting in Paris in October, 1960. 3 For the future, Jasper predicted: The implications of . . . neuropsychological efforts are, of course, enormous. After a quarter i | century of relative quiescence, neurobehavioral studies | are again the [italics] avenue that can lead both psychologists and neurophysiologists to sensible choices among disparate alternatives produced by the prodigious store of data. The ideational drought that extreme positivism and classical behaviorism imposed on systematic studies of behavior is over. Scope of the Present Study It is within the framework of this orientation toward interdisciplinary effort that the research for this I idissertation was conducted. Neurophysiology contributed i the recognition of the functional role of the caudate I nucleus in control of behavior. Experimental psychology provided the intriguing paradox called "Partial Reinforce- ment Effect (PRE)" as well as the many theoretical attempts which are oftentimes at odds with each other when trying to explain this puzzling phenomenon. By applying the insights gained from neurophysiology to the investigation of the psychological problem, and Partial Reinforcement Effect (PRE) is observed in j extinctions which follow intermittently rewarded training. PRE causes subjects to perform longer and/or better than parallel controls trained with consistently administered rewards. through the use of direct brain stimulation, allowing manipulation of the experimental animal at the cerebral level, data were gained which appear of relevance to the i , behaviorist and neurophysiologist. Hopefully, the findings: 1. Contribute to psychological theory by support- i ! ing some hypotheses and by arguing against j others. 2. Help understand caudate nucleus function in animals afforded an essentially free environ ment . j 3. Reconfirm the use of brain stimulation as a valuable method of psychological research. i i Organization of Chapters The two sets of literature relevant to the present i [problem are reviewed in Chapter II, Background. In order i jto develop the theoretical aspects of this dissertation and to be able to point to the interplay of the neurophysio- logical and psychological considerations, Chapter II first traces the early concepts of caudate involvement in motor control through to the current appreciation of caudate nucleus function. It then continues with an outline of i jthe partial reinforcement effect paradox. In contrast to 5 the material dealing with the caudate nuclei, the prepara tion of the resume of partial reinforcement effect was made easier by the literature surveys of Stanley and Jenkins (1950) and later of Lewis (1960). Some of the questions jposed by the partial reinforcement effect paradox were developed with the aim of showing their considerable and controversial inconsistencies. Chapter III, Hypothesis Development and Experi mental Tests, indicates the logic for combining recent insights of caudate function with the partial reinforcement problem. The following Chapter IV presents the Experi mental Method and Results. Final Chapter V, Discussion and Summary, attempts an outline of the gains made from this dissertation research. i CHAPTER II BACKGROUND I I Caudate Nucleus Early History, Identification, and Gross Anatomy | According to Lewy (1942), the earliest observations dealing with the caudate nuclei were probably made by Sir i Thomas Willis in his Cerebri Anatoma of 16 64. It contains an account of two prominent masses in the foremost portion of the brain stem, which Sir Thomas described as being located one within each hemisphere and whose appearance and shape he compared to gluteal parts. From this description the term "nucleus caudatus" or "tailed nucleus" arose, as Lewy relates further, in a sequence of erroneous transla tions of Sir Thomas1 work from its original Latin into i jmedieval Greek and then again into the Arabic of Avicenna. I The Latin "glutia" became the corresponding Greek "oura," which in the final translation appears as "tail," thereby leading to the current name for the caudate nuclei. 7 Modern literature accounts for the name of these structures in the more direct way of literally describing i I lone of their prominent features. As Figure 1 illustrates i for man, each nucleus caudatus forms a prominent dorso- ventrally aligned loop within each cerebral hemisphere and j consists of the head of the nucleus, or the capital portion, and its tail, or the caudal portion. Anatomically, the entire nucleus seems to envelop the lentiform nucleus, composed of the putamen and the globus pallidus. It is, however, separated from these structures by interposed fibers of the internal capsule. Lateral to the lentiform nucleus are found fibres of the external capsule and, ultimately, the claustrum. Continuous with the tail of the caudate nucleus is the aggregate of several distinct amygdaloid nuclei. Customarily they are all considered in Icombination as "the amygdala." Although it is in intimate apposition with the tail of the caudate nucleus, and although fibers connect it with the head of the caudate' nucleus, the amygdala is of different cellular origin and i |is functionally a separate and independent structure. As jrecently described by Lesse (1959), the amygdala plays an important role in the processes underlying emotion. When a transverse cut is made through a specimen 8 LENTIFORM NUCLEUS: \ GLOBUS ^ PALLIDUSj” PUTAMEN f - HEAD OF CAUDATE NUCLEUS FIBERS OF INTERNAL CAPSULE AMYGDALA Figure — Schematic diagram of the striate body, showing location of the caudate nucleus in manB (Drawing made after Voneida, 19 60)„ 9 brain at the level of the caudate nucleus, the exposed area in which it and adjacent structures lie appears striped to |the naked eye. This led early anatomists to call the t entire region the "corpus striatum." In contemporary anatomy, all members of the striate body are also con sidered parts of the basal ganglia. The following breakdown shows the various struc tures of the corpus striatum and its surroundings: Basal Ganglia: I 1. Corpus Striatum | a. Caudate Nucleus b. Putamen ) ) Lentiform Nucleus c. Globus Pallidus) d. Fibers of the Internal Capsule (these fibers do not form a basic ganglion) j 2. Claustrum | 3. Amygda (complex of nuclei) Motoric Inhibition as First Impression of Functional Role In spite of the early anatomical identification of the striatum and the caudate nuclei, their functional role remained obscure until quite recently. Except for a suggestion by Friedrich Rezek (1897) that the corpus 10 striatum was somehow involved in the control of reflexive movements, there was little progress in the experimental I [delineation of the function of the caudate nuclei until i I jTower (1935) gathered data which indicated that they had Jinhibitory functions related to extrapyramidal, but not i pyramidal processes. Following Tower's publication, a renaissance of interest in the striatum, led by Drs. F. A. and C. C. Mettler, occurred. Their findings generally buttressed the notion of caudate nucleus control of motoric joutflow. They cited as evidence that simultaneous elec- i trical stimulation of the caudate nucleus and of the motor cortex led to a supression of that motor event which was usually elicited by the motor-cortical stimulus alone (Mettler and Mettler, 1941). In addition, the Mettlers reported that ambient, spontaneous activity of an experi mental animal was also lowered as a consequence of caudate i I (stimulation. They stated: . . . (1) if an animal were struggling, stimulation of the caudate nucleus of putamen stilled this activity j and (2) if a specific movement was induced (phasic j flexion of the forepaw) by stimulation of the cortex, j this movement was reduced in amplitude, frequency or duration of movement and, sometimes, stopped altogether by stimulation of the caudate nucleus or putamen . . . j Mettler, Ades, Lippman, and Culler (1939), further described the typical way in which this caudate j . ." ...... n induced modification of motor activity seemed to take place {as a ". . . melting away of the cortical effect." Their j Jstatement implied a possible shift of control of behavior jfrom upper cortical layers to lower brain centers as a jconsequence of caudate nucleus outflow. Mettler and his co-workers stressed that the observed inhibitory effect upon cortical function was by no means abrupt. The effect was furthermore not marked by discrete temporal limits. It could not be associated easily with the duration of stimulation, nor could it be defined by its onset or cessation. Instead, the inhibiting phenomenon was described as an abatement of the cortically initiated movement over a brief interval of time. In addition to this inhibition of an electrically induced movement, the authors also reported as a result of caudate stimulation a "significant reduction" in amplitude and frequency of vigorous, spontaneous motor activity. Although the authors did not quantitatively state the degree of loss of activity, this latter observation of the generalized slow-down of behavior in an experimental animal was explored in further detail. 12 Generalized Behavioral Effects Following Caudate Stimulation In 1941, the same year in which F. A. and C. C. iMettler published one of their major papers, Gerebtzoff (1941) independently reported similar findings. Using jrabbits as subjects, he found that cortically induced masticatory movements were inhibited upon caudate stimula tion. This observation is fully congruent with those of the Mettlers. Gerebtzoff further reported that the caudate- induced inhibition of masticatory movements did not cease when the caudate stimulation was discontinued, but that it persisted for at least 30 to 40 seconds more, so that a j prolonged interference with mastication resulted. This extended cessation of an almost reflexive activity perhaps gave initial impetus to the notion that the caudate nucleus has more than just a .temporary controlling influence on descending motor impulses. The protracted inhibition of eating movements, as reported by Gerebtzoff, can well be evidence for suppression of a distinct behavior pattern i jrather than of a mere motor event. Freeman and Krasno (1940) furnished further support for the notion of a generalized inhibitory caudate effect. These authors reported that bipolar faradic stimulation of 13 the anterior caudate nucleus has a demonstrable effect on bladder tone, respiration, and the sweat glands. Since ithese autonomic events are among the indices of an organ ism's level of arousal, as outlined by Freeman (1948), Duffy (1951), Lindsley (1951), and Malmo (1959), among others, it can be concluded that the caudata influence general arousal levels in some inhibitory way yet to be understood. Only Forman and Ward (1957) presented a point of view in conflict with the conclusions just drawn. These authors found that "... inhibitory phenomena resulting from stimulation of the caudate nucleus were not demon strable under any conditions." Their findings, however, have remained unsupported. In particular, because they used electrical stimulation of an intensity sufficiently I high to cause contraversive movements. Their stimulus frequency, furthermore, was 60 cycles per second! As is discussed later, this rate falls beyond the range of i | stimulus frequencies or pulse repetition rates which cause inhibition. Forman and Ward's findings are really quite in line with observations made in studies yet to be reported. i Totally offsetting Forman and Ward's doubts, the induction of sleep in experimental subjects due to 14 i I stimulation of the caudate nucleus has remained the most j striking evidence for utmost inhibition of ongoing behavior.; Initially, Akert and Anderson (1951) reported sleep in cats j I : I following 5 pulses per second (pps) stimulation of the head ! of the nucleus. Then Heath and Hodes (19 52) were success ful in similarly causing sleep in monkeys and in man. j Lately Buchwald, Wyers, Okuma, and Heuser (1961) have demonstrated the same phenomenon in cats. Generalized Behavioral Effects Following Caudate Lesions The literature reviewed on the previous pages has emphasized studies investigating the effect of electrical j stimulation of the caudate nucleus on behavior. It has shown how a restraining influence upon ongoing overt activity, particularly that with heavy motoric content, is i [exercised by electrical stimulation of the caudata. For the sake of completeness, studies are reviewed next, showing the effects of lesions or ablations of caudate nucleus tissue on behavior. Richter and Hines ] (1938) gave one of the earliest reports describing the effect of striatal lesions on motor behavior. Upon removal of unilateral prefrontal cortex in monkeys, these authors observed only a slight increase in motoric activity levels; 15 but when the tip of the striatum"*- was removed, the slight j I rise in activity changed and became an immediate and marked activity increase. A subsequent detailed study was conducted by Mettler and Mettler (1942), investigating the interactions between cortical and striatal lesions. As part of these experiments, the Mettlers observed the behavioral effects caused by bilateral damage to the caudate nuclei. The damage was produced either by extirpation or by destruction of tissue following injections of formalin and alcohol. For the caudate lesioned preparations, two types of animals were used. In one group, cortical lesions had been made earlier. The other group was cortically intact. Both groups were observed during the survival period which lasted only a few days following the bilateral caudate damage. The Mettlers reported that even during the immedi ate post-operative period, and while the animals were still comatose, pronounced hypertonia was evident; it changed to incessant movement, once anesthesia wore off. } Margaret Kennard (1944) conducted additional early j studies investigating the effects of caudate lesions on i -^-Consisting primarily of the head of the caudate nucleus! 16 ! behavior, using monkeys and chimpanzees. She varied the j I I damage to the overlying structures by modifying approach i paths to the caudate and reported findings compatible with j | i those of the Mettlers. Her numerous experiments indicate j j that caudate nucleus lesions can contribute to motoric I hyperactivity and hypertonia. Subsequent work has confirmed the findings of the early workers. Forman and Ward (195 7) attributed hyper kinesia and extensor hypertonia to lesions placed in the caudate nucleus. More recently, Davis (1958) reported J the occurrence of spontaneous activity in rhesus monkeys following caudate lesions. Dean and Davis (1959) have i reconfirmed this observation. Also in monkeys, Denny-Brown (1962) has made the observation that hyperkinesia is exhibited in the presence of other monkeys or of the observer following the removal of the caudate nuclei by suction. Romanovskaya (19 57) reported that caudate damage caused by the injection of molten paraffin not only pro duced disturbances in motor coordination, but also in ! cutaneous and deep sensation in rabbits and dogs. i ^ i | Rakic, Buchwald, and Wyers (1962) provided the final evidence to be noted for the general inhibitory potential of caudate outflow. Rakic and his colleagues 17 attempted to induce electrographic seizure activity in cats by stimulating the caudate nuclei using low rate stim- j ulus frequencies. In all cases their attempts had negative i I ; j results, a finding quite in line with expectation. However, when Eserine (R) brand of physostigmine was introduced into t i the caudate nuclei via a cannula, "grand-mal" seizure patterns in the animals1 EEGs occurred in response to the stimulation. By injecting Eserine (R) into the nuclei, Rakic et al. presumably prevented the caudata from func tioning in their usual inhibitory role, permitting seizures to develop. Injection of physostigmine into the caudate nuclei in this instance essentially represented a revers ible chemical lesion and temporarily eliminated "restrain ing" caudate efferents, so that the induction of the caudatogenic seizure became possible. This is in distinct contrast to the customarily encountered difficulty with which seizures by means of non-tetanizing caudate stimula tion are produced. On the basis of the lesioning experiments reviewed, i it may be concluded that caudate function restrains not only motoric outflow, but also lowers the general level of ! activation. ' ' ' ' " ' " " " 18 Electrophysiology of the Caudate Nucleus, General Method Present day neurophysiology was made possible by Hans Berger's observation of the electroencephalogram (Elektrenenkephalogram, as he called it) in 1929, and the use of localized deep brain stimulation as outlined by Hess in 1932. The advances made by Berger (1929) and Hess (1932) typically characterize the contemporary approach to the electrophysiology of the brain in general and of the caudate nucleus in particular. The method consists in principle in exciting neuronal systems by means of elec trical stimulation at a specific point target. The consequences of this stimulation are then observed by using highly sensitive pick-up and display devices. These proce dures may be paired with the production of electrical coagulation and/or chemical destruction of selected tissues. Advances in instrumentation, especially since World War II, have made great refinements in the control of stim ulus parameters possible (Becker et al., 1961). The import ance of differentiation between effects due to variations i i in stimulus frequency, voltage intensities, current inten sities, and even wave-shapes, has been repeatedly stressed by Lilly et al. (1952), Lilly et al. (1955), Lilly (1961), and also by Buchwald and Ervin (1957), among others. By means of the approaches outlined above, as applied to the caudate nucleus, a series of studies has yielded electrophysiological information which can be categorically grouped. It is presented below. The find ings cited next have all either been observed or replicated by this writer in experiments preparatory to the present study. Electrophysiology of the Caudate Nucleus, Low Stimulation Rates: Below 1/2 PPS Shimamoto and Verzeano (1954), Purpura, Hausepian, and Grundfest (1958), and most recently Buchwald et al. (1961a) have reported that two temporally distinct elec trical events can be observed in the cortex, the thalamus, and other brain sites upon appropriate electrical stimula tion of the Cqiudate nuclei. Bilateral stimulus electrodes are placed stereotaxically within the head of each caudate nucleus, and a single shock is then administered period ically (1/2 pps or less) at moderate intensity, e.g., 0.5 i i jmillisecond (msec) and 8 volts (V). | | Following the caudate shock, there appears in the i electrographic display first a short latency evoked poten tial. Then, once this potential has subsided, and after . . .......__....-......................... - 20 I about 150 to 250 milliseconds, a train of higher voltage rhythmic activity at 12 to 14 cycles sets in. This burst of EEG synchrony can last from 1/4 second to several sec onds. Its cessation is marked by a gradual diminution of amplitude, as shown in oscillographic recordings in Figure 2 and also electrographically on EEG tracings in Figure 3. Figure 4 is included as a standard against which to compare Figure 3 and other, subsequent illustrations showing the effects of caudate stimulation on EEG record ings. In the recent literature, dealing with caudate nucleus findings, this train of rhythmic oscillations has been referred to as a "caudate spindle." Caudate spindles can be elicited easily and con sistently, although the required stimulus levels may vary in amplitude, duration, and repetition rate (or all three), depending on the general level of arousal of the experi mental animal. In the laboratory, one finds frequent and spontaneous changes in the animals' arousal levels which require stimulus adjustments. Occasionally, some workers i counteract drastic fluctuations and lower the spindle ! jthreshold by means of an anesthetic agent, such as Nembutal (R). The latter manipulation, however, is practical only in acute experiments. 21 "CAUDATE SPINDLES" (recorded oscillogjcaphicallv) 1, Single spindle marks the onset of stimulus 1 Z Time base;I H * sec. Figure 2a— Caudate spindles as typically displayed on an oscilloscopes 22 R-L SMC*. 'H L SM Ch. ~ —^ | J ( ^ ^ W b ^ ^ ^ J v ^ ^ A f r v \ f ' j I n o i f f . 1 *'_____ I I I I I n o i f f . if I J \f RSM - Qcc.Cx. L H ip . (W f o l j M » f * W i-STIMULUS — — -hscof© Figure 3.— Example of caudate spindles as typically displayed electrographically. L SM C k . - I noiff. In o if f . R S M “ Q c C .C k. L H)P* CStPOLi) >^‘ X < e V u f V w V V r tW v ^ / ’ , c V‘ JV , ': : ^ ^ * ^ ^ 9 —4 s @ c c m > Figure 4„— Control EEG0 Typical electro graphic recording of an alert animal not undergoing any brain stimulation„ 24 Continuous or prolonged periodic stimulation of the caudate nucleus with spindle shocks causes the EEG of an ! alert animal to become one typical of the animal at rest or at sleep. It is accompanied by an increased spindle- responsiveness to stimulation of the caudate. As stimula tion is maintained, it is quite often necessary to reduce stimulus strength if electrographic spindles are to be achieved which correspond to those earlier observed. Electrophysiology of the Caudate Nucleus, Intermediate Stimulation Rates: From 3 to 10 PPS When caudate stimulation is increased from a low repetition rate to a frequency of about 4 pps or more, then the between stimulus time is insufficient to allow caudate spindles to develop. Instead, recruiting patterns appear as shown in Figure 5. Recruiting is characterized by a slow waxing and waning of the observed high amplitude potentials, and is believed to be the result of synchronous rhythmic discharge of thalamic pacemaker systems. It has been described in detail by Morison and Dempsey (1942). Cortical recruiting responses as a consequence of caudate stimulation have been reported by Terzuolo and Stoupel (1953), Umbach (1959), and Buchwald, Wyers, Okuma, and Heuser (1961) . _____________________________________________ _ t o - _ R»L SM Cx. I noiff. ___________________________________ r s m % , ; * m J m m ^ M / / m / M m / / / RSM - Q c c . G c . L H ip. c b ip o l j | .STIMULUS ETC.----O 9 S sscoj© ■ A i ’VVV*>* v^Vi \\X W m n iw u w — — — m m ^ 1 * * i » 1 1 : i i i * * . ' i • • . i i * 1 ! I I I I , . : • I I I ' I » 1 f : : ! '• ' » » s I i i , : ; . i « Figure 5.— Example of typical "Recruiting Patterns," as displayed electrographically. Note that a line has been drawn in, connecting the peaks of waves in the RSM - Occ. Cx. trace to indicate waxing and waning of potentials. 26 Electrophysiology of the Caudate Nucleus,High Stimulation Rates; From 20 to 50 PPS Increasing the stimulation rate to from 20 to 50 pps, and using the same spindle intensity, pulse duration, and amplitude, leads to electrographic records virtually devoid of EEG intelligence. Stimulus artifacts, as a rule, are the only visible trace. Yet remarkable changes in behavior are to be observed. The experimental animals appear as if frozen at that phase of executing any overt act at which the stimulation commenced. This "freezing in its tracks by the animal," as it were, has been earlier called "the arrest reaction" by Hunter and Jasper (19 49) and reconfirmed by Buchwald, Wyers, Lauprechtr and Heuser (1961). Electrophysiology of the Caudate Nucleus, Very High Stimulation Rates: Above 100 PPS Raising the pulse repetition rate of caudate stimu lating shocks, still at spindle intensity, to values in the low hundreds changes the EEG pattern earlier observed at 20 to 50 pps only with regard to the frequency with which the stimulus artifacts occur. Genuine electrical activity of the brain is essentially undetectable in the electro-mechanical write-out. Behaviorally, however, drastic changes can be observed in the experimental animals. The arrest reaction, described in the previous paragraph, gives way to circus movements and the animal turns contralaterally to the caudate nucleus being stimulated. When bilateral stimula tion is used, the movements are contralateral to the side receiving higher effective stimulation. The animal soon assumes a position which can be described as catatonic, unless stimulation ceases promptly. It recovers only gradually, even after the stimulation is stopped. The electrographic and behavioral events just out lined are produced by stimuli of high repetition rates at spindle intensity. If, however, one reduces the amplitude and pulse duration to 1/10 of the values used to cause spindling, then the EEG and behavioral phenomena are mark edly different. The electrographic record again shows legible information. Almost invariably it is low ampli- i tude, fast activity, characteristic of alert animals (Figure 6). This latter finding has often been reported with the stimulation frequency adjusted to 300 pps (Buchwald and Ervin, 1957; Buchwald et al., 1961d; and Lauprecht, 19 61). 28 R-L SM Ch. L SM Ck. - I W D I F F . '__________ ' R SM Ok f t A f r W 5 ^ In d if f . __________ _____________________ L H ip (Be f q l j <biCCM> Figure 6 . — Electrographic record as typically obtained from animals being caudate stimulated at 300 pulses per seconds 29 In the experiments of Buchwald and co-workers (1961d) and Lauprecht (1961) very high frequency stimula- ! jtion at 300 pps was paired with simultaneously administered i caudate stimulation at low (spindle) and intermediate (recruiting) rates. This can be accomplished either by i using one electrode for the simultaneous stimulation, or by means of separate electrodes, both oriented to the caudate nucleus. It effectively blocks the primary EEG and behavioral phenomena associated either with the spindle stimulation or the recruiting shocks (Figures 7 and 8). | Because 300 pps caudate stimulation counteracts I i jthe inhibitory effects caused by the lower rates of stimu lation, it is most useful in control situations. Experi ments, for instance, investigating the effects of slow rate stimulation on behavior often include a condition in j Iwhich the slow stimulation is maintained, but in which its i i I effect is negated by simultaneously applying the 300 pulses per second (Buchwald et al., 1961c; Lauprecht, 1961). ! I Specific and Discrete Behavioral j Effects Following Experimental Manipulation of the Caudate Nucleus, General i - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | | The preceding section has outlined methods for the 30 R"L SM Gt. ^ / v " ° I n d i f f . R SM Cx. - ^ Im d if f . RSM L KiP.tBFQL.) £° STIMULUS 9 taicos^) 300 PPS. STI M. Figure 7.— Electrographic record as typically obtained from animals being caudate stimulated simultaneously at \ and 300 pulses per second. 31 R"L SM Cx. L SM C s . - I n d i f f . • ' 1 / . 1 ■ ■ « ■,.■■■■■ ■ , • : ; f ' ' J. jj , .' : 1 : R SM C x . - - y In d i f f . v ■ ' ' • ' R SM - O c c . C x , / ^ ^ ^STIMULUS 9 ISECOND | ■ ■ ■ 300 PPS. STtM...... -f> Figure 8.— Electrographic record as typically obtained from animals being caudate stimulated simultaneously at 5 and 300 pulses per second. experimental manipulation of the caudate nuclei using stereotaxically implanted electrodes. Emphasis was placed on parametric stimulus control in terms of frequency (or pulse repetition rate), amplitude (or voltage intensity), and pulse width (or duration). Various electrographic findings were listed and some of the concomitant behavioral changes indicated. Next, the relationship between specific stimuli and changes in behavior will be presented. Caudate Stimulation and Its Effect on Operant Conditioning Response Rate Animals with chronically implanted subcortical and cortical electrodes were trained in a modified Skinner box to perform lever pushes for a drop of milk, and training was continued until a high level of proficiency was reached (Buchwald et al., 1961c; Lauprecht, 1961). Once the task was performed smoothly and efficiently, experimentation was begun by introducing conditions in which the animals were caudate stimulated. It was found that with spindle producing stimula tion presented at low rates, i.e., 1/2 pps, the animals performed their task at a lower rate than in the control situation and occasionally stopped lever pressing com pletely. With blocked spindle stimulation (1/2 pps and "... 33 ~ simultaneous 300 cycle stimulation) the animals performed as if without stimulation. Finally, with 30 0 cycle stimu lation at spindle blocking parameters alone, the animals also pushed the lever as if without stimulation. Appro priate controls ruled out motor interference as an explana tion of the effects, and the conclusion was that caudate stimulation at parameters sufficient for spindling is capable of inhibiting performance of a learned task, probably by inhibiting the cerebral processes necessary to perform this task. Behavioral inhibition was not demonstrable in the absence of caudate spindles, whether they were blocked by the superimposition of 300 cycle stimulation, or when the low frequency caudate shock was of insufficient voltage and/or duration (Lauprecht, 1961). Figure 9 illustrates some of these findings. Results consistent with those just presented were reported by Knott, Ingram and Correll (1960). Caudate Manipulation and Its Effect on Visual Discrimination j In another experiment the effect of caudate stimu lation on performance of a simultaneous visual discrimina tion task was tested (Buchwald, Wyers, Carlin and Farley, 19 61) . Cats had to respond to a light stimulus at either 34 RESPONSE RATE DURING Cd. STIMULATION CONTROL L + R Cd. 1/2 ~ L + R Cd. 1/2 ~ CONTROL OFF Figure 9 Modification of operant response rates due to caudate stimulation at h pulse per second and 300 pulses per second* ^24949037195 35 the right or left of a central partition in the feeding alcove of a training box in order to obtain milk. Follow ing initiation of each trial, access to the milk cup was i j briefly blocked by a restraining gate. During this pre response interval, and presumably while discrimination of the visual stimuli was going on, the experimenters adminis tered continuous caudate stimulation at 5 pps. During the early phases of this task, some decre ment in the accuracy of visual discrimination was apparent. Once the task had been learned well, however, interposing caudate stimulation did not appear to affect response accuracy. This suggests the possible alteration of perceptual processes during early discrimination learning due to caudate stimulation. In particular, these findings lend support to the authors1 notion that caudate stimula tion interferes predominantly with short term memory, since accuracy was reduced primarily during the early trials, when memory for the visual cues was not yet firmly fixed. ! | The caudate stimulation did not affect accuracy of firmly j |established responses resulting from prolonged training and subserved by long term memory. That the early incorrect responding was not attributable to motor interference can be concluded from the finding that whenever stimulation 36 was used, the response, once initiated (although slowly), was carried out smoothly and without evidence of motor i 1 impairment. j j The authors further reported that if the caudate I stimulation ceased prior to terminating the presentation of the visual cue, then performance of the discrimination was not impaired. Similar findings have been reported for monkeys by Rosvold and Delgado (195 6), who found no decrement in well established visual discrimination per formance due to caudate stimulation or lesions. Dean and Davis (1959) also reported no loss in color discrimination by monkeys due to caudate lesions. Caudate Stimulation and Its Effect on Delayed Response Performance Buchwald, Wyers, Carlin, and Farley (1961) also investigated the effect of caudate stimulation on delayed responses in their study of visual discrimination. Although somewhat inconclusive due to deterioration of the animals' performance, the results suggested a definite impairment of delayed response behavior. This was inter preted as indicating some direct interference with immediate memory as a result of caudate stimulation. Again, these findings correspond with those of Rosvold and 37 Delgado (1956); whose monkeys showed a loss of accuracy in a delayed response test following caudate stimulation or | electrocoagulation. j | i I : Also in monkeys, Dean and Davis (1959) investigated! i J the effect of caudate lesions on delay of response. Their i | results showed a decrement in response accuracy congruent with the previous findings. Beyond this, a graded effect, depending on whether the lesion was uni- or bilateral, was found. Damage in only one caudate nucleus resulted in a partial loss of delayed response ability. Correct responses still occurred after 5 seconds but not after 15 seconds. For cats, Thompson (1958) has reported an impair ment of delayed reversal learning due to caudate stimula tion. In these experiments, the reversal was made only if the cat responded correctly in 4 out of 5 trials. Thompson found interference with learning in four of five experimental animals, stimulated following response, presumably due to interference with memory. Caudate Stimulation and Its Effect on Extinction The effect of caudate stimulation on extinction was investigated by Wyers, Buchwald, Rakic and Lauprecht (19 62), using cats. The animals were taught to push a lever for a drop of milk, and training was continued until they performed with minimal rate variation. Then they were I subjected to successive extinctions under three experi mental conditions: (1) spindle stimulation of the caudate nucleus; (2) 300 cycle stimulation of the caudate nucleus; and (3) no stimulation. It was found that extinction performance can be modified by caudate manipulation. Spindle stimulation increased both ,time and number of responses to an extinc tion criterion, while 300 cycle stimulation caused the animal to extinguish more rapidly than in a control situation. These effects persisted through prolonged experimental runs, although the number of responses and time to criterion became less as the experiment progressed. Resistance to extinction, therefore, can be changed as a function of caudate stimulation. The conclusion suggested by these findings is that caudate stimulation hinders any new learning involved in extinction. 39 Summary of Caudate Function and the Possible Use of Caudate Manipulation as a Research Tool in the Behavioral Sciences i I From the literature one may draw some specific conclusions (1) about the functional role of the caudate nuclei in control of behavior and (2) about the effects of electrical stimulation of the nuclei on experimental s\ab jects. The early anatomists believed the caudate nucleus to be in some way involved in motor control of the organ ism. At about the turn of the century, appreciation of the functional role of the caudate nuclei was expanded by the notion that they were instrumental in controlling muscle tonicity and thereby affected reflexive movements (Rezek, 1897). In the 1940's, the restraining control believed to i be exerted by the caudate nuclei on motor efferents was i extended to general levels of arousal (Freeman and Krasno, 1940; Mettler and Mettler, 1941; Gerbtzoff, 1941). During the last decade, interpretations of caudate function have been further modified by findings indicating that lesions, as well as stimulation of the caudate nucleus, can cause gross behavioral changes (Akert and Anderson, 1951; Heath and Hodes, 1952; Buchwald and Ervin, 1957; Forman and Ward, 40 1957; Davis, 1958; and Dean and Davis, 1959). The most frequently reported effects of caudate lesions are marked increases in general activity and i spontaneous movements, hyperkinesia, and extensor hyper tonia (Rosvold and Delgado, 1956; Forman and Ward, 1957; Davis, 1958; Dean and Davis, 1959). Electrical stimulation of the caudate nucleus results in inhibition of general activity, slowness in reacting to environmental conditions, and sleep (Akert and Anderson, 1951; Heath and Hodes, 1952; Buchwald and Ervin, 1957). Recent studies suggest that the caudate nucleus is involved in mental process sequences supporting complex behavior (Buchwald et al., 1961c; Wyers et al. , 1962, 1963). If caudate stimulation is such that the parameters and the rate of stimulation lead to the development of spindles in the electrocorticogram, then a decrement in the following can be expected: (1) speed of response initia tion (reaction time), (2) rate of learning, (3) rate of operant responding, (4) accuracy in visual discrimination, (5) alternation performance, (6) delay of response, and (7) rate of extinction. If stimulation is given at 300 cycles and at low intensities, which are nevertheless high enough to block spindling, then mild arousal effects such as desynchronization and behavioral alerting can be observed. If spindle producing stimuli are combined with 300 cycle spindle blocking stimulation, some EEG desyn chronization may be observed, but the animal tends to perform as if not stimulated at all. From these findings it became evident that the normal CNS processes of an experimental subject can be modified through the use of the appropriate caudate stimu lation. The experimental work reported earlier has shown that in this regard caudate spindle shocks are particularly effective. When used repetitively at rates of up to 5 pps, they lead to the marked inhibition of general behavior earlier described in detail. When the caudate shocks are used at rates of less than 1 pps, or even singly at crucial times in a behavior sequence, the inhibitory effect is much more subtle. With this type of caudate stimulation (either intermittent or carefully programmed within a time sequence of events), there is no observable slowness in onset of the animal's behavior or of motor impairment. Instead, the effect of the caudate shock now seems limited purely to higher mental processes. Especially affected are the CNS processes subserving short term memory, while long term memory suffers no interference from caudate shocks (Rosvold 42 and Delgado, 1956; Thompson, 1958; Dean and Davis, 1959; Buchwald, Wyers, Carlin, and Farley, 1961). This difference in degree to which long and short term memory are affected by the caudate spindle shocks suggests the use of this stimulation as a research method. Its application in behavioral experiments makes it possible to assess the effects of specific peripheral events on behavior. One can, e.g., allow subjects to develop full memory for some events and to learn their significance by means of repeated pre sentations; on the other hand, memory for the event signif icance can be impaired in other subjects (or the same subjects in a parallel situation) by administering a cau- I date spindle shock when neural register for the event is believed to be taking place and the memory trace is being formed. Manipulating memory processes in this way provides a tool to determine to what extent some specific environ mental situations are functionally related to behavior. Partial Reinforcement Effect | General The second major body of background information deals with the "Partial Reinforcement Effect" (PRE) which is observed during extinction as a consequence of partially reinforced (PR) acquisition training. The effect is fre quently called "paradoxical" because it does not conform to common sense reasoning. Animals in training are custom arily rewarded immediately after having performed the behavior which they are being trained to do. If one believes that a behavior, which is to be learned, is 2 strengthened by an appropriate reward, then it follows that the behavior, if rewarded every time it occurs (100% or continuous reinforcement), would be expected to be stronger than if rewarded only part of the time it occurs (non-continuous or partial reinforcement). Paradoxically, however, behavioral responses acquired in continuously i reinforced training do not persist longer in subsequent extinctions than those learned in partially reinforced acquisition training; the latter consistently show the jhigher resistance to extinction. The "Humphreys1 Effect" and Previous Observations Humphreys (1939) is customarily credited with the discovery of the partial reinforcement effect, and in the years following his original paper the term "Humphreys' 2 This belief is implicit in the use of a reward! 44 Effect" was used extensively to describe PRE. Prior to Humphreys publication, however, analogous i effects had already been identified and described. An early observation was reported by Pavlov (1927) in his "Conditioned Reflexes." He stated that a dog's salivary i responses remained essentially as strong if rewarded only at every third occurrence as if reinforced upon every occurrence. Skinner (1933) also gave an early report on behavior observed under conditions of partial reinforcement. He described a method of periodic reconditioning which involved the administration of non-continuous, but periodic rewards. Subsequently, Skinner (1936) described having trained animals with reinforcement at a fixed ratio, thus clearly using a partial reinforcement technique. Of less empirical orientation, but nevertheless directly relevant to partial reinforcement, is a theoreti cal paper by Tolman and Brunswick (1935), titled "The Organism and the Causal Texture of the Environment." Here the authors deal with the probabilities of cause and effect relationships existing in the environment and living space of a behaving organism. On theoretical grounds they criticize the then prevalent practice of animal experi menters of dichotomizing the administration of 45 reinforcement. They argue against treatments of animals in such a way that correct responses always lead to reinforce ment and incorrect responses never do. Tolman and | jBrunswick point out that such a sharp dichotomy almost never j lexists in the organisms' environment. Instead, the organ ism operates in situations presenting varying probabilities of obtaining a reward. Since in a natural environment any one of these probabilities rarely ever reaches unity, a j istrong argument is made for the use of non-continuous i I jreinforcement in experimental situations. I ; The observations of Pavlov and of Skinner, as well i | las the theoretical considerations of Tolman and Brunswick, i vindicate that toward the end of the 1930's psychologists were prepared to make use of partial reinforcement in jexperimental work. Humphreys' (19 39) article, therefore, i j I received widespread notice. In his experiment, Humphreys ! investigated the eye-blink reflex in humans. His subjects i junderwent a classical conditioning series using a light as jthe conditioned stimulus (CS) and a puff of air to the jeyelid as an unconditioned stimulus (UCS). The conditioned i (response (CR) was the eye-blink. i Three experimental groups were used. In group 1, CS and UCS were paired ninety-six times. For group 2, the 46 CS light was presented ninety-six times, but was followed by the puff of air to the eyelid only forty-eight times, [on a random selection of half the presentations. The | remaining group 3 was conditioned with the 100% reinforce ment schedule used for group 1, but the total number of CS-UCS pairings corresponded to the partially reinforced group 2. Thus, forty-eight light presentations, each paired with reinforcing air puffs, were given to group 3. Administration of the trials for group 3, however, was spaced so that the time to administer the forty-eight i reinforcements was equal to the time used to administer ninety-six reinforcements to group 1. The three types of conditioning sessions brought all three groups to identical performance levels. Each group was then submitted to an extinction series of twenty- four trials lacking reinforcement. After completion of this series, the performance of the conditioned response by group 2 was better than that of the two other groups. Partial reinforcement during acquisition had increased resistance to extinction. Theories of Partial Reinforcement Effect Expectancy theory.— To account for his results, 47 Humphreys (1939) reasoned that as a consequence of experi ence gained in a learning situation the behaving organism forms expectancies which come to govern his future behavior. During 100% reinforced training, for example, an expectancy of regular reinforcement is formed. When an extinction' session follows, then the subject's expectancy of regular reinforcement must change to one of regular non-reinforce ment, if he is to adapt to the realities of the new environmental situation. If, on the other hand, during the acquisition period only partial reinforcement is given and an extinction session then follows, the required shift in expectancies is one from irregular reinforcement to regular non-reinforcement. Humphreys stipulated that there is a difference in the ease with which these shifts can be made. In particu lar, he believed that the shift from regular reinforcement to regular non-reinforcement is easier than from irregular reinforcement to regular non-reinforcement. The occurrence of PRE was, therefore, a consequence of the level of difficulty presented by the shift and of the prolonged time period during which the shift in expectancies is accom plished. 48 After-effects theory.-“ -Humphreys 1 partial reinforce ment effect was incompatible with stimulus-response (S-R) learning theory. This theory predicted that a response, consistently followed by reward, would be strengthened, and a response, lacking reward, would be weakened. To account for the "Humphreys Effect," C. L. Hull in a memo randum to his seminar in the spring of 1941 (referred to by Jenkins and Stanley, 1950) assumed that the stimulus after effects consequent to any non-reinforced acquisition trial, as experienced by the subject, became conditional stimuli for the next reinforced trial. In any subsequent extinc tion session, the stimulus compound after the first extinction trial would be nearly identical to the one earlier experienced after a non-reinforced acquisition trial. The response, earlier conditioned to this stimulus compound, would thus be elicited by the after-effects of the non-reinforced extinction trials. On the other hand, if non-reinforced trials were not experienced during acquisition, the stimulus compound characteristic of [extinction trials would not have acquired the capacity to elicit the response. Hence, the strength of response following partial reinforcement would be greater than after regular reinforcement, and, consequently, greater 49 resistance to extinction would be expected for a partially reinforced response. i Sheffield (1949) tested the theory. She adminis- j i tered 50% and 100% reinforcement to two major groups of subjects. Each of these were broken down into two sub groups. One received massed trials, the other distributed acquisition trials. For the groups receiving the massed trials, Sheffield believed that the after-effects should form connecting links between non-reinforced and reinforced I trials and lead to PRE. On the other hand, she thought the after-effects would dissipate during the 15 minute period interposed between the acquisition trials for the dis tributed practice group. The results supported her hypothesis. For subjects trained under massed acquisition trial conditions, the 50% group exhibited greater resist ance to extinction when compared with the group having received continuous reinforcement. For the groups trained under distributed acquisition trials resistance to extinc tion did not differ significantly. Later experiments by Wilson, Weiss, and Amsel (1955), however, did not confirm the work of Sheffield. They, in turn, gave rise to a new theoretical conceptuali zation of PRE which is outlined later. Other evidence 50 also often quoted as repudiating the after-effects theory comes from the series of experiments conducted by Weinstock (1954, 1958), who on two occasions demonstrated partial reinforcement effect even when the intertrial interval during spaced acquisition trials was twenty-four hours. Weinstock has also proposed a theory to account for partial reinforcement effect, as will be outlined later. Response unit hypothesis.--An explanation of partial reinforcement effect which is conceptually close to the after-effects theory was advocated by Mowrer and Jones (1945). It has its origin in Skinner's "The Behavior of Organisms" (1938) and stems directly from his operant conditioning procedure using partial reinforcement. Skinner (1938) pointed out that in training situations with ! reinforcement at a fixed ratio the fixed number of responses between reinforcements form one single task unit. If during operant conditioning four lever presses must be made to obtain a reward, then all should be considered as one response unit. He justified this by reasoning that the stimuli which result from a non-reinforced response form part of a stimulus chain, terminally leading to the reinforcer. 51 Skinner's book (19 3 8) preceded Humphreys' paper on partially reinforced conditioned eyelid responses by one year. Thus, he had no need to deal with the "Humphreys' Effect." His idea of chained responses forming a unit must, nevertheless, be considered one of the bases of the response unit hypothesis accounting for PRE, as it was later advanced by Mowrer and Jones (19 45) after conducting the experiment which is outlined next. Four groups of rats were trained in an operant lever-pushing task, but the ratio of reinforcement was different for each group. One group was rewarded after every lever press, another was rewarded after every second press, still another after every third press, and the last after every fourth response. When these groups were subjected to subsequent extinction sessions, the results supported Mowrer and Jones' predictions. Although groups with partial reinforcement performed better than the group trained under continuous reinforcement, it was found that this superiority disappeared if all non-rewarded prerein forcement responses and the rewarded response were combined into one response unit. In this case, the partially reinforced animals performed as many response units as the continuously rewarded group gave individual lever presses. 52 Mowrer and Jones believed that the crux of the PRE problem rested in the definition of "response1 1 prior to a reward. Going beyond Skinner, they saw all acts preceding the reinforcement as part of the response unit, whereas Skinner (1938, p. 300) had earlier only stipulated similar acts or lever presses to be part of a response unit. They contended that subjects would learn that only performance of the entire chain of varied responses would bring forth the reward. From this response unit hypothesis it follows that if the chain were somehow interrupted, partial reinforce ment effect should be lost. Experiments testing this were conducted by Denny (1946) and by Sheffield (1949). Both introduced a time interval between non-reinforced and reinforced trials, stipulating that it would prevent the perception of all between reinforcer acts as a single unit. In both experiments subjects given non-continuous rewards and trained with inter-trial delays failed to show the higher resistance to extinction which would be expected following partial reinforcement acquisition. Straight forward, as the response unit hypothesis seems to be, it has not been widely adopted, and Mowrer and Jones themselves prefer the discrimination hypothesis, 53 outlined next. Discrimination theory.— Mowrer and Jones (1945) i ! also proposed the discrimination theory as a possible explanation of PRE. The theory holds that resistance to extinction is a function of the similarity between the acquisition situation and any subsequent extinction situa tion. The greater the similarity, the higher resistance to extinction. The more dissimilarity, the lower resistance to extinction. For a training situation with partial reinforce ment, this means that a subject is exposed to trials with and also without a reward during acquisition. Thus, the acquisition and the extinction situations have the elements of lack of reward in common, making it difficult for the subject to discriminate between the two. Extinction in this case is a relatively slow process, and resistance to extinction should be high. Conversely, the discrimination becomes very easy for a subject trained under conditions of 100% reinforcement. During later exposure to extinc tion, all trials will lack reinforcement, and the differ ence between the acquisition and extinction becomes i maximally discriminable. In this case, extinction would 54 occur rapidly, and resistance to extinction would be low. If the hypothetical considerations of discrimina- I jtion theory were true, then one would expect human or ani mal subjects who are trained under the lowest ratios of reinforcement to exhibit the highest resistance to extinc tion. Within limits this is true. In a number of studies it was demonstrated that if 10 0% reinforcement was used during acquisition, then resistance to extinction was relatively low. The latter increased, however, as percent age of reinforcement was reduced. Resistance to extinction reached its peak when percentage of reinforcement was between 40 and 80. At percentage levels below 40% rein forcement, resistance to extinction decreased again. Among the studies reporting these observations is one by Grant and Shipper (1952). Human subjects were employed in a conditioning experiment using the eye-blink as the response, with a light as the conditioned stimulus and a puff of air to the eyelid as the unconditioned one. Subjects showing the highest resistance to extinction had been reinforced at the 50% and 75% levels. Another study was reported by Grant, Hake, and Hornseth (1951). In a verbal conditioning situation, human subjects were again reinforced at varying percentage levels. Highest 55 resistance to extinction was in this case shown by the 25% reinforcement group, exceeding that exhibited by the 50, |75, or 100% groups. Further, there is the study of Lewis | (1952), using children of kindergarten age as subjects in an experiment designed like a gambling situation, in which i plastic toys, such as cowboys or football players, were used as reinforcers, administered at 0, 50, 60, and 100% reinforcement levels. Resistance to extinction of the 50 and 60% groups significantly exceeded the other two groups. The studies outlined above relied primarily on differences in ratio of reinforcement to vary the degree of similarity or dissimilarity between acquisition and extinc tion sessions. Differences between the two situations, how ever, can also be introduced experimentally by patterning the administration of the reinforcer. Studies of this type have been conducted by Longnecker, Krauskopf, and Bitterman (1952) investigating GSR in humans, and by Tyler, Wortz, and Bitterman (1953) using rats as subjects in an elevated runway and jumping stand, leading to the reinforcer. Both showed that if a simple alternating pattern of reinforce ment and non-reinforcement was used during the partially reinforced acquisition, then extinction occurred more quickly than if a random, non-patterned reinforcement had 56 been given. Their explanation for this observation was that the serial patterning during acquisition was learnable by the subject. After having been learned, the knowledge of the existing pattern aided the process of discrimination, once extinction was begun. A third group of experiments by Bitterman, Fedder- son, and Tyler (1953) and Elam, Tyler, and Bitterman (1954) sought support for the discrimination hypothesis by using peripheral and environmental cues as the major elements differentiating the acquisition from the extinction situa tions. In this group of experiments, animals received rewards on reinforced trials in one type of box and non rewards in a box very different from the reward box. For the subsequent extinctions the animals were divided into subgroups. One of these was extinguished with the box earlier used for the administration of rewards. The other group underwent extinction with the box earlier used for non-rewards. The experimenters believed that non-rewarded extinction trials, given in the earlier reward box, repre sented a large change between the acquisition and extinc tion situations and should facilitate discrimination. On the other hand, extinction trials given in the earlier non reward box should not be experienced by the animals as 57 radically different from the earlier non-reward acquisition trials. Therefore, discrimination between acquisition and Jextinction in this second treatment was believed to be ! ! difficult. In line with prediction, animals extinguished in the earlier reward box showed inferior resistance to j extinction when compared with those undergoing extinction in the earlier non-reward box. The experiments just outlined, using differences ! ] in ambient and incidental stimulation by providing separate reward and non-reward boxes, are crucial. They j I strongly support the discrimination hypothesis, but beyond ithat, they also argue against the secondary reinforcement I hypothesis of Denny (1946) which is outlined next. Secondary reinforcement hypothesis.--Denny1s (1946) Ipaper introduced the secondary reinforcement hypothesis as l an explanation for the partial reinforcement effect. It is jbased on an experiment in which animals were trained to ! Irun a T-maze using 50 and 100% reinforcement. When extinc- | jtion sessions were later performed, subjects of the two groups were treated differentially. For some animals, all 58 cues which were believed to have secondary reinforcing properties (Hull, 1943) were allowed to remain. For others, |secondary reinforcement cues were minimized. Denny had t stipulated that during extinction the secondary reinforce ment cues, which had been associated with the primary reinforcement, would tend to maintain the animal's perform ances. In the absence of the secondary reinforcement cues — other things being equal— Denny predicted lowered resist ance to extinction. His experimental results were as expected. One observation of Denny's, however, was somewhat discrepant. Contrary to expectation, Denny found slight, but still superior resistance to extinction for those animals who had been continuously rewarded during training and then extinguished with minimized secondary reinforce- i i ment cues. Perhaps this can be ascribed to the fact that he used an intertrial interval of twenty to thirty minutes. His trials, therefore, were spaced, rather than massed. Thus his observations and the subsequent ones of Sheffield ! (1949) are quite consistent. Sheffield also reported resistance to extinction to be somewhat (though not significantly) better after spaced, continuously reinforced acquisition trials. Certainly, these observations of Denny 59 and of Sheffield serve to highlight the importance of intertrial interval duration. In the years following the proposal of the second ary reinforcement explanation for PRE, impressive data have accumulated which are at odds with it. The experiments cited most frequently are those of Bitterman, Fedderson and Tyler (1953) and Elam, Tyler, and Bitterman (1954), which were outlined in detail in the section dealing with discrimination theory. In both experiments, some animals were extinguished in boxes in which secondary reinforcement cues were minimized, since the boxes had earlier been used only for non-reinforced trials during acquisition. Still partially reinforced animals showed higher resistance to extinction than animals extinguished in the compartments in which only reinforcement had earlier been given and in j which secondary reinforcement cues should be plentiful. It is because of findings of the sort just men tioned that the secondary reinforcement hypothesis is rarely used as an explanation of partial reinforcement phenomena. j Competing response theory.— Among the most recent theories attempting to account for partial reinforcement, 60 we find the competing response theory of Weinstock (19 54). This author trained rats to run an "L" shaped alley to reach food. The reward was administered either 10 0, 80, 50, or 30% of the time. Intertrial intervals were held at twenty-four hours. When tested for resistance to extinc tion, the 30% group was highest. The other groups were ranked 50, 80 and finally 100%. In accounting for his findings, Weinstock reasoned that animals trained under partial reinforcement exhibit competing responses during jthe non-reinforced acquisition trials. These competing [responses, essentially consequences of non-reinforcement, habituate and finally drop out. This means, that animals, reinforced only inter mittently during acquisition, should adapt to the noxious after-effects of non-reinforcement presumably responsible for the elicitation of the competing responses while still learning the rewarded responses during the acquisition series. If these same after-effects should occur during a later extinction, the animal would tend to tolerate them. They would not interfere with the animal's tendency to respond to those cues which signal the onset of a trial. \ Thus, all the responses which would otherwise compete for expression in performance during extinction trials, having 61 habituated out, would not impair the response conditioned earlier. Habituation, which is contingent upon non reinforcement, is therefore used to account for the partial reinforcement effect. As used by Weinstock, this process must undoubtedly take time. Therefore, it should vary as a function of the animal1s stay in the environment in which competing responses first develop and then drop out due to habitua tion. Stanley and Clayton (1955) and Hulse (1958) have performed experiments to test this. They argued that if habituation takes place during the stay in the goal situa tion, then habituation should be maximized if the animal is forced to remain in this situation for a prolonged period of time. To test this, one group of animals was retained in the goal area for thirty seconds following their response, while another group was removed immediately following the responses. When subsequently tested for resistence to extinction, there was no difference between jthe two groups. These results weaken Weinstock's hypothe- I sis, but only if habituation is a function of the post response time spent in the goal area. | A different study by Fehrer (1956) supports iWeinstock. Fehrer reported that thirty seconds delay both before and after administering the reinforcement increased resistance to extinction. Weinstock (1958) himself has 'also reconfirmed his earlier findings. | i Mediating responses theory.— The most recently jformulated major theory to account for partial reinforce ment effect is the mediating response theory developed by Amsel (1958). It is based in the main on experiments conducted earlier by Amsel and Roussel (1952), Amsel and jWard (1954), Wilson, Weiss, and Amsel (1955), and Amsel and i i ^Hancock (1957). In general, the crucial manipulation in i j jthese experiments is as follows: Animals are trained to i I |run through two serial runways for a food reward. Each runway has its own goal box, and initially a reward is given after the animal has run each of the two sections. :This procedure is used to elicit the behavior to be learned, i 1 [i.e., traversing the runway. Training is continued until i jperformance stabilizes. Then the reinforcer is withheld in f |the first goal box for some animals on half the trials, 1 thus imposing a 50% partial reinforcement regimen. Changes l in the performances in the second alleyway, as a conse- i iquence of reward or non-reward following the first run, are jthen evaluated. 63 Amsel and his co-workers believed they observed the ! development of an emotional response in the experimental ! t : animals following non-reinforced trials in the first run way. They thought this response was due to frustration caused by thwarting of the animal's food-seeking behavior. i When the response developed, the second runway was traversed at higher speed than earlier observed. This enhancement of behavior, following partial reinforcement, has been called ! jthe "Frustration Effect." In extinction sessions, those i I ! animals trained under partial reinforcement in the first l j irunway were compared with others which had always been i ; reinforced. The partial reinforcement group exhibited less performance decrement (running speed and latency) than the 100% reinforced one, provided that many (eighty-four) [acquisition trials had been administered. After only few j(twenty-four) acquisition trials, the groups did not differ j j [markedly. | ] Briefly summarized, Amsel's mediating response explanation of PRE starts with frustration as an emotional [response, due to the thwarting of goal directed activity. i | jupon repeated trials it becomes conditioned to the stimuli [ [existing at the start of each trial. Once the frustration (and emotion) becomes part of the over-all stimulus-response 64 chain of the performance, it mediates the occurrence of each behavioral component of the learned behavior. When these responses become evident, they are then interpreted 1 las signs of higher resistance to extinction, following partially reinforced acquisition. By contrast, continu ously reinforced acquisition gives no opportunity for the mediating frustration response to develop. If an extinc tion session were to follow the training period of continu- ious reinforcement, there would be no mediating frustration responses and the performance in extinction would be inferior. Assessment of Considerations and Variables Relevant to Partial Reinforcement Effect The previous pages cite seven theories explaining jpartial reinforcement effect. They range from the earliest I concepts of Humphreys (19 39) to the most recent ones of j Amsel (1958) and co-workers. Each theory explains some facet of partial reinforcement. However, no single theory | |fully explains the partial reinforcement effect paradox. | | As Lewis (1960) says, this is most likely due to ;the difficulty of simultaneously varying or controlling all l i l the relevant factors. It has been feasible to vary only 65 one or two factors independently along isolated dimensions. iThe consequent changes in independent variations, however irelevant, do not meet the requirements for conclusive parametric laws. Lewis expresses current thought when he says that his attempt to explain the partial reinforcement i effect had not been realized, even after a most intensive review of PRE literature. At one point he states: . . . conflicts and contradictions in data contribute their share of confusion, but probably more important has been the absence of the right kind of data. In the data section of this paper an attempt was made to discover parametric laws. Except in a very few instan- | ces this was impossible. i Lewis elaborates that most experiments have con- i l centrated on "theory testing." As a result there are a number of experiments, all using different apparatus, which combine two or three groups in factorial designs, whose jresults point to inadequacy of current theory. With five jor more points, and a small number of constants, the i parameters relating to a particular dimension can be described with some precision. However, so far these I l I [requirements have not been fulfilled and, as Lewis stresses ! further, an understanding of partial reinforcement remains ! distant. Among the dimensions along which experimental manipulation has most often been performed, we find the |following ones. _________ ___________________ ___________ 66 Percentage of reinforcement.--Most frequently, percentages used include 0 and 100% with two other inter posed intermediate levels. As was pointed out earlier, the I percentages of reinforcement which appear most efficacious for PRE lie between the 40 and 80% levels (Lewis, 1960). i Patterning of presentation of reinforcement.— As important in the development of PRE as the percentage of reward is the pattern of distribution of reinforced and non-reinforced trials during acquisition. The studies of i |the Bitterman group support the generalization that random j or highly complex patterns of reinforcement and non- ireinforcement lead to higher resistance to extinction than do regular or simple ones. Situational peripheral cues.— Allied to the patternr ing of presentation of reinforcement is the notion that idissimilar situations will also lead to differences in PRE. Situation-specific events are primarily a function of methodology. Thus, differences in experimental design are among contributing causes for non-corresponding observa- | Itions. This is particularly true when, for instance, data j gathered in a human verbal conditioning experiment such as that of Grant, Hake, and Hornseth (19 51) are compared with 67 those obtained in studies using rats in a runway. Subject !as well as task differences contribute to the difficulties in cross comparison of results. i Inter-trial interval.— Crucial to the after-effects f land mediating response theories is the inter-trial interval period. To date, resolution of conflicts about inter- i trial interval duration and, therefore, concerning massed iversus spaced trials in partial reinforcement situations has not been possible. The studies of Sheffield (1949) and : of Weinstock (1954, 1958), attempting to assess the import- |ance of temporal relationships between trials, have in fact j led to two opposed theories of PRE. Delay of reward.— Delay of reward as a crucial variable was mentioned and elaborated upon the discussion of secondary reinforcement theory. In support of delayed i reward as a manipulation mediating partial reinforcement effect, the study of Crum, Brown, and Bitterman (19 51) i deserves mention. The delays used were 0 versus 30 sec onds. Unfortunately, a generalization can not be unequiv- locally drawn from the study because Logan, Beier, and | jKincaid (1956) have subsequently failed to confirm higher resistance to extinction with a delayed reward when a 68 9 second delay was compared with a 0 second control condi tion. On the other hand, Logan et al. (1956) were success-j i ful in replicating the earlier study by Crum, Brown, and jBitterman (1951). The question of the importance of delay | |of reward, therefore, remains open. i Non-equality of acquisition sessions,— Although not a dimension along which independent variation seems pos sible, there is one further problem which must be faced in studies dealing with PRE, and which affects all studies equally. It stems from the fact that when groups of subjects which have received reinforcement at dissimilar percentage levels are to be compared, then it is not possible to equate number of trials and reinforcements at ;the same time. These groups can be equated either: (1) on jnumbers of trials, with the group at a higher ratio of reinforcement receiving more reinforcers than the other; or (2) on number of reinforcements, with the group at the I lower reinforcement ratio receiving the larger number of ! trials. j Customarily, the problem of inequality of this sort j 'is handled by holding the number of trials constant across i i groups. In this fashion, the partially reinforced subjects 69 receive a smaller number of reinforced trials. Conclusion j The previous paragraphs have served to illustrate some factors and variables which must be taken into con- jsideration when evaluating findings in PRE research. As has been stressed, all factors attendant upon methodology, type of task, independent variables, and dependent vari- jables selected for observation, tend so to affect the I (occurrence of partial reinforcement effect that a single, i jbut more far-reaching and general manipulation of crucial i variables seems most desirable. Phrased differently, one can say that there is need for a single experimental pro cedure which would modify simultaneously many of those aspects of the acquisition period, which, singly or in iinterplay, are involved in either causing or modifying i partial reinforcement effect. In addition, the needed experimental procedure should lend itself to independent jvariation. One such manipulation, amenable to experimental i luse, is provided by stimulation of the caudate nucleus. CHAPTER III j HYPOTHESIS DEVELOPMENT AND EXPERIMENTAL TESTS General Introduction The previous chapter reviewed the anatomy of the caudate nucleus and electro-physiological findings relevant to its functions in the control of behavior. It stressed the use of electrical stimulation of the caudata as a means I !of altering the central state of an experimental animal. Such alteration appeared to be inhibitory in nature and was often accompanied by deterioration of voluntary behavior, primarily due to disruption or impairment of immediate memory trace processes. Interference with the immediate memory processes of experimental subjects makes it possible to investigate the effect of selected peripheral events upon the animal's behavior. In one condition, for example, animals can be exposed to a specific experimental procedure with concur rent stimulation of the caudate nuclei. In another I |condition, the same treatment can be administered, but i 70 71 without the electrical input to the nuclei. In the first situation, the general level of alertness is lowered and memory for the peripherally occurring events is impaired. In the second, the level of arousal remains normal and memory processes are unencumbered. This procedure can help to determine the effect of situational variables on changes in observed behavior. j The previous chapter also dealt with the background of the partial reinforcement effect (PRE) and the diffi culties encountered in explaining it to universal satis faction. In the following pages, the caudate stimulation procedure will be related to factors believed to be operating in the formation of PRE; a hypothesis will be presented, and experimental tests will be outlined. j Working Definition of Partial Reinforcement Effect Partial reinforcement effect (PRE) is the phenome non of increased resistance to extinction due to partially reinforced (PR) acquisition training. By definition, PRE can only occur when non-reinforced (NR) jtrials were part of the preceding acquisition situation. One cannot speak of PRE in extinction if the earlier acquisition training did not contain at least one 72 non-reinforced trial. Therefore, the NR trials are crucial to the development of PRE. ! Analysis of Explanatory Attempts |of PRE and Their Categorization I I j I i Discrimination theories.--The attempts to explain PRE, outlined in the previous chapter, can Be divided into two categories. Theories in one category rely primarily on a process of discrimination between acquisition and extinction to explain the effect. They include the [response units hypothesis, the secondary reinforcement I hypothesis, the discrimination hypothesis, the after effects hypothesis, and the expectancy hypothesis. All of these view PRE essentially as a failure of the subject (S) to discriminate between acquisition and extinction trials. ;They argue that the subject performs on the latter as if |they were the former. The failure of S to distinguish l ! jbetween the two situations is attributed to situational I 1 1 cues common to trials of the acquisition and of the [extinction series. The cues are virtually identical for S NR acquisition trials and the subsequent extinction trials and the extent to which the trials are perceived as being alike determines development of PRE. The effect, therefore, 73 is contingent upon events during acquisition, as well as during extinction. During extinction, the regular sequence ! I of non-reinforced trials promotes discrimination of extinction from acquisition and is responsible for the decrement in response obtained. It is the rate at which this discrimination develops which determines the PRE. Habituation theories.— The competing response hypothesis and the mediating response hypothesis form the category of habituation theories. In this category of theories, the events occurring during the acquisition NR trials, or as their immediate consequence., are regarded the determinants of PRE. Such events can be the process of habituation of competing responses during NR trials (Weinstock, 1954, 1958) or the formation of frustrative drives (Amsel, 1958). Both are conditional to and func tionally related to the acquisition trials. The presence of NR trials promotes greater strength of the response acquired. During extinction, each individual non- ! reinforced trial weakens response strength and is thereby i i I responsible for the progressive decrement in response ] i 1 obtained. It is the rate at which response strength | Iweakens that determines the PRE. 74 Requirement to Be Met for Development of PRE As Demanded by Discrimination Theories and Habituation Theories i . . . . . . . In line with the above, an experiment which brought i jthe requirements of the discrimination theories of PRE lunder experimental control should include non-reinforced I (NR) trials during acquisition and provide for a clear cut discriminative distinction between acquisition and extinc tion. The latter should influence the S's resistance to | i |extinction. In an experiment controlling the requirements i of habituation theories, it would be essential only that i non-reinforced trials be given during acquisition training. The habituation theories do not rely on discriminative i processes when explaining PRE, so an experimentally con trollable discriminative distinction between acquisition and extinction would not be required. Comparing the essential conditions needed to bring the requirements of both groups of theories under experi mental control, only the presence of non-reinforced trials | jappears in common. In view of this, the nature of NR i itrials in PR acquisition training is considered next. I ! i I i 75 Analysis of the Essentials of NR Trials as Related to PRE Theories A clear definition of the character of NR trials in PRE studies must answer two questions: (1) What are the factors associated with NR trials which define them beyond the mere statement that: "It is a trial on which reward is lacking"? (2) What role do these factors play in the explanation of PRE? One event which must always be related or contrib utes to the PRE is the registration in the nervous system of effects of the NR trial and the lack of reinforcement experienced in association with the trial. This neural iregistration is the representation in the nervous system of a series of changes in stimulation encountered by S throughout the various phases of the NR trial. I | In terms of operant conditioning, the series might I run as follows: The animal would learn to respond first to a chain of stimuli starting with those emanating from the experimental compartment. If the animal had been trained before, these essentially non-specific cues would communicate to it that an experimental session was about to begin. Then cues signifying the start of experimentation would occur. These new stimuli could be the result of the 76 presentation of a lever and/or the onset of any auditory or visual signals accompanying it. A third group of stimuli in the Skinner box would be that which the animal experienced during its approach to the lever and during the execution of lever pushes. It consists primarily of proprioceptive and kinesthetic cues associated with performing the response. In addition, changes in the relationship of the earlier cues to the ! animal would lead to new stimuli. Thus, as the animal moved, the perceived size and brightness of a light might change and the loudness of any existing auditory cue might be altered. These changes, coupled with the proprioceptive I cues, represent the chain of stimulus events marking the animal's approach to the lever, and execution of a lever response. j At the time of the lever push, and immediately following it, the animal would perceive and register the cues associated either with the reward or the occurrence of non-reward. In this phase one would find relay clicks and other noises inherent in the operation of any feeding or reward device; there would also be cues associated with the changing effects of the spatial environment as the animal moved toward the cup. Finally, and after the reward had been administered, there would be perception and recording in the brain of effects of all cues associated with con- |summatory behavior as the animal would drink or eat its reward. In non-rewarded trials, the stimulus sequence per- i ceived by the animal would be the same as before, except for the complex of stimuli which had directly resulted from consummatory behavior. Instead, the animal would now perceive stimuli resulting from its search for the expected reward. This would include all those stimuli arising from the animal's reaction to the lack of reinforcement. Expressed differently, and in terms of Weinstock's compet ing response theory, these stimuli would stem from clawing the lever, and/or orienting toward other points of fixation in the operant box, and/or searching for the reward in a ( spot not earlier used for a reward. Applying Amsel's ) frustration hypothesis, such cues during an NR trial period would become conditional to the developing frustrative drive. For the set of "Habituation Theories" the processes jnecessary for PRE would have had their full play at this jstage. i For the set of "Discrimination Theories," the events so far described would lead only to the registration 78 of neural reference standards against which the neural effects of cues in extinction would be compared when they were later experienced. Thus, during acquisition, as viewec by Mowrer and Jones, the animals would learn to respond in response units, comprised of the combined motoric effects of at least one NR trial and a rewarded one which would then have to be compared with the combined motoric effects accumulated over a series of extinction trials. As stipu- I jlated by Sheffield, they would learn to associate after- |effects of non-rewarded trials with rewarded ones, and I i during extinction these after-effects would gradually be associated with the effects of following non-rewarded trials. As hypothesized by Mowrer and Jones, they would form the perceptual standards, representing acquisition, against which the later events of the extinction session i i would be discriminated. According to Denny, situational stimuli would acquire secondary reinforcement values and would govern the animal1s later behavior during extinction if present and discriminated. And, finally, as stipulated by Humphries, the subjects would form expectancies of the reward ratios which would later have to undergo a shift i during extinction. In all of these cases, the development of an 79 integrated compound of stimulus effects, dependent on the |occurrence of a sequence of extinction trials, is postu- | jlated. It is the difference between these integrated i stimulus effects and those occurring during acquisition which controls the rate of response decrement, and hence PRE, during extinction. The importance of stimulus effect integration is detailed in the next paragraph. In con trast hereto, such sequential stimulus effects are not postulated for the habituation theories, and each extinc tion trial by itself is taken as reducing a previously determined response strength by some amount. Hence, for these theories, response decrement and PRE during extinc tion are (1) associated with differences in response strength existing prior to the initiation of extinction and (2) governed by the reduction of response strength attributable to every single extinction trial. Analysis of the Essentials of the Discrimination Processes during Extinction which Contribute to PRE The partial reinforcement effect, as conceptualized i by discrimination theorists, depends on two factors. The first is that a standard of reference develops and is registered neurally during acquisition. The second is 80 that the series of events which take place during extinc tion be evaluated and compared with acquisition events. During extinction, there must be a discrimination between the series of extinction trials and the earlier series of acquisition trials. This means, that all effects of all cues existing in the sequence of trials during extinction are integrated sequentially by the subject and that this one integrative effect compound is then compared with the corresponding integrative effect compound earlier experi enced during acquisition. Only when the difference between i these two is perceived by the subject can one assume a discrimination between acquisition and extinction to take place. The rate at which the integration of sequential extinction trial effects takes place, and the ease or difficulty with which the total integrative effect com pound is formed, governs the PRE. In addition, this integration and the discriminations which it serves are learnable and they undergo improvement during the course of repeated extinction sessions. Therefore, the processes become more facile on subsequent extinctions and lead to a i progressive decrement in resistance to extinction. 81 The Role of Caudate Stimulation in Modifying PRE The method of employing caudate stimulation as an experimental procedure to independently control and thereby evaluate the processes leading to PRE is outlined next. Specifically, it is proposed that caudate spindle stimula tion be given to subjects during those phases of the exper iment which are believed to be crucial to PRE. If the caudate is stimulated at the time when reward is being withheld during NR acquisition trials, the neural registration of the effects of the non-reinforcement i j Iwould be impaired. Thus, following this treatment, both habituation and discrimination theories would predict PRE should not occur in a subsequent extinction. Specifically, two subject groups would be trained to do a task, using identical partially-reinforced acqui- i i jsition sessions. One group, however, would be caudate | i stimulated on non-reinforced trials, while the other would not. The stimulated group should show less resistance to extinction than the group trained with the same partial reinforcement conditions, but without caudate shock. Animals would also be trained under partial rein forcement acquisition but without caudate stimulation. 82 They would then be divided into two groups and subjected to extinction sessions. If during these sessions one of the groups received caudate stimulation, then the animal's i discrimination between the extinction and the earlier acquisition trials would be impaired. This weakening of the discrimination would be due to several effects. First, the influence of non-reward during extinc tion would be lessened by the occurrence of caudate I spindling together with the lack of reward. Second, the animal's memory of earlier extinction trials would be impaired, so that the neural-reference standard representing extinction would not be easily formed. Third, the caudate stimulus would impair those higher CNS events which underlie the discrimination process. The comparison of new stimulus patterns, representing the extinction session, with those stemming from acquisition trials should thus be weakened. These three processes would all tend to maintain I the animal's performance and enhance resistance to extinc tion, according to discrimination theory. This is because the development of the neural reference standard, based on the integrative registration of the sequential effects of non-reinforced extinction trials, would be impaired. As 83 indicated earlier, this integrative standard represents the combined effects of the sequence of extinction trials i i i land, in particular, those due to all the types of stimula tion arising from repeated reactions to the absence of reward. i On the other hand, and given the same conditions, habituation theory would not predict enhancement of resistance to extinction due to caudate stimulation occurring on extinction trials. This is because the learn ing responsible for extinction performance and PRE is presumed to be well established before extinction begins. Since caudate stimulation impairs only immediate (or short-term) memory processes, it would not modify the extent of PRE as governed by long-term memory processes. Instead, in an arrangement of conditions permitting long term (or permanent) memory processes their full sway, resistance to extinction would be reduced only by the effect of each individual non-reinforced extinction trial on response strength. Such an arrangement is described below. Experimental Hypotheses The experimental hypotheses given next follow directly from the foregoing definition of PRE, the 84 breakdown of processes thought to be essential to its development, and the use of caudate stimulation to evaluate them. Hypothesis A: Reduction in Partial Reinforcement Effect Due to Caudate Stimulation on Non-Reinforced Acquisition Trials "If single spindle eliciting shocks are adminis tered to the caudate nucleus following response during the NR trials of partially reinforced acquisition training, the neural registration of the non-reinforcement effects will be impaired. This will result in a decrement in the PRE during a subsequent extinction session." , Any reduction in PRE would support discrimination as well as habituation theories and, thus, not differen tiate between them. It would, however, establish the efficacy of caudate stimulation in altering PRE. Hypothesis B; Enhancement of Partial Reinforcement Effect Due to Caudate Stimulation on Extinction Trials "If single spindle eliciting shocks are delivered to the caudate nucleus following response during an extinction session, preceded by partially reinforced acquisition training, discrimination processes responsible 85 for extinction response decrement will be weakened. Conse quently the PRE will be enhanced." An increase in PRE would support discrimination i theories and weaken habituation theories. For the former the caudate shock would interfere with the short-term memory processes serving the integration of sequential extinction trial effects and thereby prevent the discrimi nation of extinction from acquisition. For the latter, depending only on long-term memory processes, the single caudate shock, affecting only short-term memory, should remain ineffective in changing resistance to extinction or PRE. Experimental Tests Test for Hypothesis "A": Experiment I Hypothesis "A" states in essence that PRE, or the enhancement in resistance to extinction, which animals exhibit following partially reinforced acquisition, is a function of the events connected with the occurrence of NR trials during training. In detail, the problem of experimental test of Hypothesis "A," is three-fold: First, it must be shown how animals perform in extinctions following a continuously reinforced acquisition procedure. This extinction per formance becomes the control measure against which the experimental extinctions following partially reinforced acquisitions are to be compared. Second, the development of PRE must be demon strated. Toward this end, animals must undergo extinctions which have been preceded by par tially reinforced acquisition sessions. PRE is demonstrated if the animals now show higher resistance to extinction than they had exhib ited earlier when tested after the continuously rewarded training sessions. Finally, it must be shown that PRE does not occur, or is diminished, if the NR trials during partial reinforcement acquisition are paired with caudate stimulation. To accomplish this, animals must undergo extinctions which have been preceded by partially reinforced acquisition sessions. These animals should show less PRE than corresponding animals 87 trained identically, but without the caudate stimulation. The next chapter on Method and Results gives speci fications and describes procedures in detail, while the section on discussion (page 173) includes considerations dealing with the intrasubject assessment of PRE. Briefly ^ stated, cats were chosen as subjects, and the lever press ing response for a drop of milk was selected as the behavior in which dependent variation was evaluated. Experiment I may be presented in program form, as a test of Hypothesis "A," as follows: Phase 1: Ss trained to stable lever press cri terion with continuous reinforcement. Phase 2: Ss extinguished. Phase 3a: Ss divided into 2 groups: A and B. Phase 3b: Group A retrained to stable lever press criterion under 1:2 ratio of reinforce ment . Phase 3c: Group B retrained to stable lever press criterion under 1:2 ratio of reinforce ment with caudate shock on NR trials. Phase 4a: Group A extinguished. Any increase in extinction responsive ness over that obtained in Phase 2 represents PRE.______________________ 88 Phase 4b: Group B extinguished. Lack of PRE is demonstrated if extinc tion responsiveness is equal to or less than control responsiveness (Phase 2). Test for Hypothesis "B": Experiment II Hypothesis "B" states in essence that PRE which animals exhibit following partially reinforced acquisition training is enhanced if caudate stimulation is administered during the extinction trials. The animals selected for Experiment II, testing Hypothesis "B," were cats, as in Experiment I. Their task consisted in performance of lever responses which led to a drop of milk when reinforced. The task differed from Experiment I, in that the lever was offered in discrete trials, rather than in the free responding situation. The reasons for this change are given later. It is important for Hypothesis "B" that differences between acquisition and extinction trials be easily discriminable. Therefore, the subjects were presented with more than just minimal incidental cues. By providing the added stimuli, linked to experimental events, discrimina tion was facilitated. The "enriching" stimuli selected 89 were a 1 second tone and a flashing light. They marked the onset of the trials. The following experimental criteria also have to be satisfied to test Hypothesis B. 1. Caudate stimulation must occur during extinc tion trials. Accurate timing of the caudate stimulus in relation to the onset of the enriching cues during extinction is important and makes the use of a discrete trial situation (rather than a free responding situation) necessary. This is because in a free respond ing lever press situation it is difficult to define what constitutes a trial. The task used for Experiment I was therefore modified so that in Experiment II the animals could operate the lever only during discrete periods of time. Each of these time periods was considered to represent one trial. Trials were separated from one another by appropriate inter-trial intervals. During these intervals, the response lever was inaccessible to the animal and the reward cup was covered. 90 It is also crucial to Hypothesis "B" that discriminations be performed by the animal subjects. To cue this discrimination, the enriching stimuli were provided. The 1 second tone (at 2000 cycles) and the flashing light (at 10 flashes per second) thus supplemented the incidental cues occurring when the lever and food cup were exposed to the animals. Hypothesis "B" deals with PRE; but before any measure of PRE can be taken, the animals' extinction performance following a continuously rewarded acquisition series must be determined. This, then, becomes the control against which the subsequent extinctions are compared. The development of PRE must be demonstrated. To achieve PRE, animals must be trained under a partial reinforcement regimen and then be subjected to an extinction session. PRE is demonstrated if the animals show higher resist ance to extinction now than was earlier exhibited following the continuously rewarded acquisition sessions. 91 5. Ultimately, PRE is to be enhanced by caudate shock, following the onset of extinction trials. An increase in PRE would be success fully demonstrated if in this situation responses showed higher resistance to extinc tion than under the conditions outlined in paragraph 4. 6. The foregoing paragraphs 4 and 5 indicate that when testing Hypothesis "B" two types of extinction trials must be administered to the experimental animals— some with and others without paired caudate stimulation. Since the extinctions are conducted in discrete trials, it becomes possible to give the two types of trials in a randomized sequence to all animals. Administering the extinction trials as discrete events also makes it possible to observe the effect of the enriching peripheral stimuli on PRE. To this end, these stimuli are presented on half of the extinction trials and withheld on the other half. When stimulation (0) and non-stimula tion (0) conditions and cue (C) and non-cue (N) 92 conditions are combined, four types of trial result, to which the animals' responsiveness is testable during extinction. The four types of extinction trials are described below: Type 0C: Lever presentation, tone, lights, but no caudate stimulation, no food. Type 0C: Lever presentation, tone, lights, caudate stimulation, no food. Type 0N: Lever presentation only, no tone, no lights, no caudate stimulation, no food. Type 0N: Lever presentation only, no tone, no lights, caudate stimulation, no food. Hypothesis "B" is confirmed if resist ance to extinction is found to be highest with caudate stimulation and with "enriching" stimuli (0C) and resistance to extinction with out caudate stimulation and without "enriching" stimuli (0N) is determined to be lowest. The extinction measures obtained in sessions with out caudate stimulation but with "enriching" stimuli (0C) and also with caudate shocks but lacking the "enriching" stimuli (0N) should 93 fall somewhere in between. The procedures and specifications for Experiment II are given in the next chapter on Method and Results. In program form, the protocol for Experiment II, as a test of Hypothesis "B," reads as follows: Phase 1: Ss trained to criterion in lever press task, using discrete trial periods. Reinforcement is given for each lever press during each trial period (1:1 ratio reinforcement). Tone and lights signal the onset of each trial. Phase 2: Ss extinguished to criterion. All four types of trials are administered in blocks of four, with each block containing each trial once in randomized position. i Phase 3: Ss retrained to criterion in lever press task, using discrete trial periods. Reinforcement is given for each lever ! i | press only during one half of the dis- l | j crete trial periods, as randomly chosen (1:2 ratio reinforcement). Tone and i I lights signal the onset of each trial. 94 Phase 4: Ss extinguished to criterion. Four types of trials administered: 0C, $C, 0N, 0N. CHAPTER IV EXPERIMENTAL METHOD AND RESULTS Experiment I Subjects Source and pre-experimental treatment.— Five male and seven female cats were obtained from the Long Beach Branch of the Los Angeles County animal pound. They were quarantined for three weeks and immunized against infec tious enteritis (cat distemper) and infectious feline pneumonitis. At the end of quarantine they were trans ferred to individual cages in the quarters housing experimental animals. Preparation.— The twelve subjects were prepared for brain stimulation and EEG recording by implanting stereo- taxically oriented electrodes. Concentric bipolar elec trodes were used in subdural structures for stimulation and recording. Ball-tipped silver wire electrodes were used for cortical EEG recording, while in some animals monopolar 95 96 electrodes were also employed for subcortical recording. The coordinates for electrode placement were taken from the atlas of Jasper and Ajmone Marsan (1952). For all animals, electrodes were oriented to the head of each caudate nucleus (bipolar electrodes), the nuclei ventralis anteriori (bipolar as well as monopolar needle electrodes), and the anterior sigmoid and occipital cortices (surface electrodes). In addition to these points, other locations were also used for recording, allowing a detailed electrographic check-out of each animal. All animals did not participate in all experiments due to illness or head plug accidents, and six animals had been lost by the end of all experimentation. Figure 10 and Table 1 show the target sites and coordinates for all electrodes used in experimental animals, while Figures 11 to 21 depict the course of the implant procedure. Three animals were prepared each week for four weeks. Anti biotics were administered for five days after the implanta tion during a two-week recovery period. Instrumentation Experiment I and Experiment II used the same basic instrumentation. For training and testing of the animals, 97 c s GC F r . 2 0 S«c. 310 FV 4,0 ? « » « • « < sa&Srs?; See. 2 SO Fr 7.0 LP Sot. 2 4 0 F r . 7.5 See. 150 F r . 12.0 See. 70 F r . Ib.O Figure 10.— Target regions for subcortical electrodes shown by blackened dots on diagramatic cross sections taken from Jasper and Ajmone Marsan (1952). TABLE I TARGET SITES AND COORDINATES USED FOR STEREOTAXIC ELECTRODE IMPLANTS IN EXPERIMENTAL ANIMALS Target Structure Target Sites Atlas Coordinates A/P L H 3 Experimental 4 7 8 Animal 9 Number 10 11 R Cd 16.0 5.0 +5.0 X X X X X X X L Cd 16.0 5.0 +5.0 X X X X X X X R VA 12.0 3.5 +1.5 X X X X X X X L VA 12.0 3.5 +1.5 X X X X X X X R LG 7.5 10.0 +3.0 X - X X X X X L LG 7.5 10.0 +3.0 X - X X X X X R Araygd. 12.0 10.0 -5.0 - - - - - X ~ R GC 4.0 2.0 0.0 - X - X X X - L GC 4.0 2.0 0.0 - X X X X - X R CM 7.0 2.0 +1.0 X - - - - X - L CM 7.0 2.0 +1.0 X - - - - X - R med. Hip. 2.0 7.0 +6.0 X - - - - - - L med. Hip. 2.0 7.0 +6.0 - X X - - - - R lat. Hip. 2.0 9.0 +5.0 - - - X - X X L lat. Hip. 2.0 9.0 +5.0 - X - X X X X L HvM 12.0 1.0 -6.0 - X - - - - - R SM Cx. 28.0 4.0 - X X X X X X X L SM Cx. 28.0 4.0 _ X X X X X X X R Occ. Cx. - 4.0 3.0 - X X X X X X X L Occ. Cx. - 4.0 3.0 - X X X X X X X L Assoc. Cx. 8.0 14.0 - — X — — _ — 99 Pigmsre lip.'— £Kp©rlm@ntal aaimal ps’ ios’ to slscteod© implantation pff©e©dns:e0 100 Figure 12«— Animal positioned in Kopf stereotaxic instrument ready for pre-surgical preparation. 101 C “ < * ^ v » Figure 13*— Animal prepared for operative procedure* 102 ■ ' ' i Figure 14«— Exposure of skull» 103 :V Figure 15e— Skull openings have been drilled over brain target sites» Lowest openings in photo graph extend through the animal“s frontal sinus cavities. Next higher set of openings overlies the caudate nuclei targets. 104 Figure 16 8— Electrode during its stereotaxic placement* All electrodes but one have been posi tioned and cemented to skull by means of Kadon (R) dental cement 8 105 Figure 1 7 Electrode leads as connected to a Cannon (R) 25-point receptacleo 106 m m & m S * Figure 18,— Cannon (RJ receptacle attached to the animal Bs skull with Kadon {Rj dental cement. 107 Figure 190— Animal's scalp sutured to surround and partially enclose the base which holts the Cannon (R) receptacles 108 I S * RSl®® ^ ' - v . - ^ ' . ^ . T ^ i s ■ ' • " v r t ? r..k ;/r-tt'’ £ r W I t s ® * Fignr© 200— Animal one day post operatively 109 -w-;| M 8 & & '"hjaj Figure 21„--X-ray of an a&perlmeat&l animal after implantation* illustrating positioning of cortical electrodes and extent of brain penetration by eoneentri© deep electrodes0 For clarity* a relief® effect was obtained by double printing the original film with a dispositive* the latter superimposed * but off enact register. 110 i a shielded and sound attenuated (-70 db. at 2000 cps.) box provided an electrically and auditorily isolated environ ment. Visual isolation was established by darkening the laboratory, while illuminating the box interior with four D.C. powered automotive light bulbs; negative line to ground prevented introduction of 60 cycle noise. Grass S-4 physiological stimulators with Stimulus Isolation Units (SIUs) were used to administer caudate shocks. The subjects' EEGs were recorded on a Model H 930-P4 Grass EEG machine. Brain stimulation and recording occurred through a multiconductor cable attached to the animal's headplug. A custom made "Task and Reward" box was used. The response lever could be presented and retracted and a food cup covered and uncovered by means of separate solenoids. The animal's lever presses were sensed by a Sonotone stereo crystal pick-up cartridge, Model 8TDS, mechanically linked to the lever. Responses were recorded with a Model C-2 Gerbrands Harvard cumulative recorder. A Davis liquid feeder pump was used to deliver a drop of cream, milk, and water mix (ratio 1:1:2) on rein forced lever presses. Programming of the response- reinforcement ratio, as well as the administration of Ill caudate shocks, was provided by Gerbrands tape controlled switches. Figure 22 shows the "Task and Reward" box as presented to the subjects. ; Procedure Habituation and training at 100% reinforcement.— Each subject was habituated to the experimental compartment by allowing it to explore it for an hour on five separate occasions. During these periods the head cable was routinely attached, and EEG recordings were taken. Then the training sessions were begun. The desired lever press response was shaped by manually rewarding the animal for progressive fractional response approximation. A drop of cream-milk-water mix in the food cup was the reinforcement. Once the response was obtained, the animal was allowed to press at will. Each response was reinforced {1:1 rein forcement ratio) by delivery of 0.1 cc of the milk mix. The animals1 responses were displayed on the cumulative recorder in increments of 100 responses. Training was continued on alternate days, allowing 500 lever presses per session, until the base line distances (time) for all five of the 10 0 response increments fell within 30% of one 112 SLIDING COVER FOOD CUP RETRACTABLE RESPONSE lbver Figure 22»— "Task and Reward Box" as presented to experimental animals in Experiment I, 113 another. Then five more training sessions were given to allow the animal to accumulate 2500 more responses. The end of each training session was marked by withdrawal of the lever and covering of the food cup. Control extinctions.--Following training, each animal was given a series of control extinction (Eq) ses sions. These differed from the acquisition sessions only in that the reinforcer was withheld after either 200 or 300 reinforced lever presses until an extinction criterion was met. The criterion was an inter-response period equal to three times the time interval taken by the animal to make the last 100 reinforced responses prior to extinction. After criterion was met, as a rule, the animal again tested the lever as time went on, and, upon finding milk-mix, returned to pressing. It was allowed to continue for either 300 or 200 terminal reinforced responses. The combined initial pre-extinction and terminal post extinction lever presses totalled 500 per session. The number of initial lever presses (either 200 or 300) was haphazardly varied for successive extinction runs. Five hundred reinforced lever presses were given between extinc tion sessions. The total of reinforced lever presses 114 between extinctions, therefore, could range from 900 to 1100. At least five control extinctions were given each I i animal. For some animals who showed particularly high resistance to extinction, added sessions were administered. In some cases these included electrical stimulation of the caudate nuclei. These added extinctions were not used for the later analysis of results. Also excluded from further use were data obtained from an animal whose response rate was inordinately slow, for example, 20 minutes for 100 responses and two animals whose EEGs were unacceptable. Partial reinforcement retraining.— Retraining at 50% partial reinforcement was begun following the control extinction sessions. Two groups of animals were formed. Group A received conventional PR lever press training. Group B was treated identically in all respects, with the addition of a self administered caudate shock of sufficient intensity to cause a caudate spindle (Buchwald et al., 196Id) presented 165 msec after every non-reinforced (NR) lever press. For these shocks, the caudate electrodes were paralleled and connected to the Grass stimulator. It was expected that Group B would not show 115 enhancement of extinction performance. To make inter- subject differences in extinction responsiveness work ! against this expectation, the animals who had shown the greater resistance to extinction in the earlier control sessions were assigned to Group B. Conversely, the subjects with lower response tendencies were assigned to Group A in which PRE was expected to develop. In both cases findings in agreement with expectation would have to result from the experimental treatments in spite of behavior characteris tics opposed to the expected results. The PR retraining procedure closely paralleled the earlier 100% reinforced training. Reinforcement was at a 1:2 ratio and automatic relays turned on the S-4 Grass stimulator on NR trials for subjects of Group B. The determination of spindle threshold parameters for these subjects always occurred before presentation of response lever and reward cup. Spindling was monitored throughout the experiment and maintained by stimulus readjustment when needed. Stimuli were always kept below any noticeable motor twitch. Training was continued till the animals had performed 25 00 skilled lever presses with less than 30% variation in time for each 100 response increment. 116 Post PR extinctions.--Following the 50% PR training each animal was given 5 extinction sessions, EPRR. These sessions were patterned after the earlier control extinc tion sessions, Ec. Again either 200 or 300 training presses preceded extinction, but at 50% reinforcement. The extinction criterion remained as before, and either 300 or 200 post-extinction PR training presses were again allowed. Five hundred partially reinforced reacquisition lever presses were interposed between extinction sessions. The animals in Group B always received caudate stimulation following non-reinforced responses. The total number of training presses between extinctions was again between 900 and 1100. Results of Experiment 1 Each animal1s extinction performance following partial reinforcement was compared with its corresponding earlier control extinction performance following 100% reinforcement. Individual extinction curves for the four animals constituting Group A are shown in Figures 23-A through 23-D. These figures present time to the extinction criterion for each of the five extinction sessions follow ing continuous reinforcement in comparison with those 117 following partial reinforcement. Time to criterion for the i ! post partial reinforcement extinctions exceeds those for I |the corresponding control extinctions in each case for i janimals number 3 and number 7. For animals number 9 and t I i |number 11 the control extinction value exceeded the post partial reinforcement extinction value for the first comparison only. In each of the four succeeding extinction comparisons these animals gave results similar to those obtained with number 3 and number 7. j Figures 24-A through 24-D plot the number of lever- I |press responses to the extinction criterion given by each i i ! ! animal for each of its five corresponding pairs of extinc- | tion sessions. Animals number 3 and 7 gave more responses following partial reinforcement in each of the five extinction comparisons. Animal number 9 (Figure 24-C) jperformed more responses prior to attainment of the extinc tion criterion following partial reinforcement in four of the five extinction comparisons. In the case of the third I control extinction, more responses were obtained than during the third post partial reinforcement extinction. A jdifference in favor of increased responsiveness following jpartial reinforcement is less evident in the case of i |animal number 11, although Figure 24-D shows that in this 1X8 Time (decimin.) to Extinction 150, 0. Time (decimin.) to Extinction 150« 100- Abort 1 2 3 4 S' Extinctions i i"*3 4 Extinctions Animal 3 Animal? Extinctions Extinctions Animal 11 Animal 9 Figure 230”"-Tirae to extinction curves for no n~ s fcImu 1 ated animals in Experiment IQ 119 To 270^ Number of Lever Presses to Extinction 50J Number of Lever Presses to Extinction 150= 100 J 50-J 0 = » * 5 "°EPRR a a B Extinctions Animal 3 To 172 B 226 To 317 Abort Animal 7 T T J T T Extinctions 3 i t Extinctions Animal 11 Animal V Figure 240““Lever press responses to extinction curves for non-stinralated animals in Experiment I0 120 animal also the rate of decrease in responses to criterion was less rapid following partial reinforcement than after continuous reinforcement. Thus, Figure 24-D indicates that during the fourth and fifth post partial reinforcement extinctions animal number 11 gave more responses to cri terion than during the corresponding extinction sessions following continuous reinforcement. Figures 2 3-A through 23-D and Figures 24-A through 24-D indicate that the customary increase in resistance to extinction following partially reinforced acquisition training was obtained with each of the four animals in Group A. A similar graphic analysis of the results obtained with the three animals subjected to condition B is presented in Figures 25-A through 25-C and Figures 26-A through 26-C. These animals received a caudate spindle shock following each non-reinforced response during their partially reinforced training. Figures 25-A through 25-C show the results obtained for each animal comparing time to criterion for corresponding extinction sessions following i continuous and partially reinforced training. Figures 26-A through 26-C present the results obtained comparing number of responses to criterion. Time (decimin.) fo Extinction j5c 121 Time (decimin.) to Extinction I51 100. 5 0 . A nim al 4 < 3 0 9 B a 9 0 9 a 0 s 0 9 " H - H - i - Exti Animal 10 notions " H h H - A n im a l 8 5 * E xtinctions '*C RRstim sti»ulat^ 9^ f r - W S p tOi S tinotion ourves for * p 122 Number of Lever Presses to Extinction 5Q_ Extinctions Animal 4 Number of Lever Presses to Extinction To 252^ To ?93 too. 5 0. Extinctions Animal 10 b ■ i M J l j I B B I I l J I l . l H B l t ' H I V I' f- . . 1 2 3 4 5 Extinctions Animal '& EptR stim .Ec Figure 260— Lever press responses to extinction curves for caudate stimulated animals in Experiment 1. 123 In contrast to Group A, the caudate stimulated animals of Group B show no generally consistent pattern of performance during their post partial reinforcement extinc tions. Figure 25-A for animal number 4 shows that time to criterion after continuously rewarded training exceeded the corresponding extinction measures following PR training in four of five cases, but that on the third comparison the post PR extinction time was longer than its matching control. Responses to criterion for animal number 4 are shown in Figure 26-A. Here only the first comparison of paired extinction responses shows a smaller number follow ing partially reinforced training than after continuously reinforced acquisition sessions; on all subsequent extinc tions, the post PR measures exceeded the earlier controls. For animal number 4, time to criterion measures show general lack of PRE, while the responses to criterion suggest a higher lever-press activity following PR training. For animal number 8 (Figure 25-B), time to extinc tion criterion following partially reinforced training (with caudate spindle shock on NR trials) exceeded the control times on the first three extinctions, but it was shorter than the controls on the last two. The rather steep rate of decline in the curve for post partial 124 reinforcement extinctions does not indicate the higher resistance to extinction one would expect with PRE. The numbers of responses {Figure 26-B) for animal 8 obtained in post PR extinctions were less than those obtained in con trol extinctions only in the first and fifth extinction sessions, while in the second, third, and fourth sessions they exceeded the corresponding control measures. Again the rapid decline of the curve representing the post partial reinforcement extinctions does not imply any increase in resistance to extinction associated with PRE. Of all animals in Group B, number 10 shows the expected lack of PRE most clearly. For this animal, time to criterion (Figure 25-C) and responses to criterion (Figure 26-C) following PR training with caudate stimula tion on non-reinforced trials were both lower than the corresponding control measures for the first four extinction comparisons. Only on the fifth comparison do the time and response measure post PR training exceed the control values. The graphs for group B, Figures 25-A through 25-C and Figures 26-A through 26-C, indicate that an increase in resistance to extinction following partially reinforced acquisition training does not develop fully if the 125 non-reinforced trials during this training are paired with ja caudate spindle shock. When time to criterion is used to I express extinction performance, animals number 4 and 10 (Figures 25-A and 25-C) give results which show the absence of a clear PRE. For both of these animals, four of five control extinction measures exceeded the ones for post partial reinforcement extinctions. The results for animal number 8, however, (Figure 25-B) are equivocal in that the first three comparisons imply PRE, while the final two do not. Responses to criterion as the measure of extinction performance are shown in Figures 26-A through 26-C. These graphs are indicative neither of the presence or absence of PRE following partially reinforced training with the caudate stimulation. Animal number 4, Figure 26-A, shows higher ‘ responsiveness in a series of four out of five comparisons jwith earlier controls, while animal 10 shows lower respon siveness in a similar series of four out of five compari sons. For animal number 4 the non-consistent case occurs on the first pair of extinctions, for animal number 10 it happens on the last. Animal number 8 is again ambiguous; jits first and fifth comparisons show lower post partial i i reinforcement responsiveness, while in the second, third, 126 and fourth comparisons this responsiveness exceeds the control measures. j As a summary, the graphically presented findings ifor the animals in Group A demonstrate the development of i i -the customarily observed increased resistance to extinction following partially reinforced acquisition training (PRE), whether expressed as time to criterion or as number of ! |responses to criterion. For Group B they show that the ] ! Sadministration of caudate spindle shocks on non-reinforced | i trials 16 5 msec after the response seriously interferes with the development of PRE. Statistical analyses confirm the graphically pre- | sented findings. The sign test (Guilford, 19 56; Siegel, 1956; Walker and Lev, 1953) was used to determine signifi cance of results. For each animal of Group A, the compari- jsons made in the graphs were repeated by pairing times to icriterion for extinctions following training under continu- I |ous reinforcement with the corresponding times obtained t |following partially reinforced acquisition sessions. The i |calculated levels of significance for each animal are as l I I shown below: 127 Level of Significance for Animal Number Observation Eq < Eppp 3 p = 0.031 7 p = 0.031 9 p = 0.188 11 p = 0.188 When corresponding pairs of extinction measures for all animals are entered into the tabulation for the sign test, the resulting level of significance is p < 0.01. For Group B, the same statistical procedure was used, except that the extinction measures post partial reinforcement were those obtained following the training of the animals with the caudate spindle shock administered on non-reinforced trials. The resulting levels of signifi cance calculated for each animal are as shown below: Animal Number Level o£ Significance for Observation Ec < EPRRstim 4 p = 0.969 | 8 p = 0.500 | 10 p = 0.969 i | When corresponding pairs of extinction measures for i jail animals are used in one test, the resulting level of |significance is p = 0.94. 128 i j Discussion of Experiment I I The findings of Experiment I demonstrate the j |dissimilarity in extinction performance of the animals in Group A and in Group B, following 50% partially reinforced acquisition training. The training for Group A animals was conventional, while animals in Group B received a caudate spindle shock 165 msec following every non-reinforced lever press. ! ! In Group A there was the development of PRE. In i Group B the customarily expected increase in resistance to extinction failed to develop. The findings are evident when Figures 23-A through 23-D for Group A and Figures 25-A i through 25-C for Group B are evaluated. The graphs show time to criterion as the measure of extinction performance, |either following the continuously or partially reinforced jtraining which included the caudate shock on NR trials for i I Group B only. i i { For animals number 3 and number 7 of the non- 5 caudate stimulated Group A perfect PRE has occurred, while ianimals number 9 and number 11 show an inversion i I (Ec > EpRR) on the first comparison of paired extinctions only. For animals number 4 and number 10 of Group B, the comparisons of paired extinctions fail to indicate PRE. For each of these animals, four of the post partial rein forcement extinctions show less resistance to extinction ithan was observed, in their corresponding control extinc tions. The single reversal for animal number 4 occurred on i |the third comparison pair, while for animal number 10 it | |was observed on the fifth comparison. | The graph for animal number 8 is ambiguous since the first three comparisons would favor PRE, while the last two would not. Had resistance to extinction developed in i janimal number 8 due to partially reinforced training, one |would have expected higher extinction measures on the fourth and fifth extinction sessions. Aside from the absolute time values represented by the curves, their rate of decline is important. For the animals of Group A, the slopes of extinction curves follow- |ing partial reinforcement are all less steep than those for j jthe corresponding control extinction curves. This is the 3 jresult expected whenever resistance to extinction increases i jdue to partially reinforced acquisition training. For |Group B, the curves representing both types of extinctions i I for each animal drop at about the same rate, indicating a jsimilar resistance to extinction for the control as well as I the post PR extinctions. 130 Figures 24-A through 24-D for Group A and Figures 26-A through 26-C for Group B compare the numbers of re sponses made by each animal during the five pairings of the two types of extinctions. For Group A, animals number 3 and number 7 again show findings consistent with PRE, animal number 9 shows four comparisons of extinctions in favor of PRE, with a reversal on the third pair, and animal number 11 has the first three comparisons reflecting a lack of PRE, while the fourth and fifth comparisons show an increase in resistance to extinction. For caudate stimulated Group B, animal number 4 shows the last four comparisons of extinction measures in support of PRE, while the first comparison indicates lack of PRE. Almost the exact opposite is seen in animal number 10, where the first four comparisons imply lack of PRE, while the last comparison is in the direction of increased resistance to extinction. Animal number 8 shows the first and last comparisons of pairs away from PRE, while the middle three comparisons are in the PRE direction. Inspection of the extinction curves, showing responses to criterion as their data points, reveals such a high variability in the number of presses made by the animals that any judicious statement comparing the slopes - - - - 1 3 1 of Ec versus Epj^ or EpRRst^m curves is ruled out. The interpretation of results of Experiment I, in general, based on numbers of responses to criterion, suffers also I from this high variability. It is in great part attribu table to the experimenter's choice to keep the response lever as sensitive as possible to the touch of an animal. High sensitivity appeared necessary to insure that none of any animal's responses went without reward in the continu ously reinforced acquisition sessions, or was not considered in the 50% reinforcement scheme, or would not be recorded. Unfortunately, the high lever sensitivity also led to artificial inflation of response counts. This 1 occurred, for instance, when an animal tapped the lever at a rate close to the frequency at which the system oscil lated inherently. In this case, the vibration following several lever presses appeared as added multiple responses. Once this was recognized, the apparatus was modified to prevent these oscillations. The equipment corrections involved mainly adjustments of relay armature springs, of r the mechanical linkage between lever and response sensing crystal, and of the electrical coupling of amplifiers in series. The over-all system responsiveness was altered with every change, however, and every change inevitably 132 contributed to the high variability seen in the number of responses.^ i The equipment variables which reduced accuracy of | jresponse counts did not affect the accuracy of measuring jtimes to criterion, since during these times no responses : were made. Each animals' resistance to extinction, there fore, was validly obtained in the time to criterion measures, and for this reason, the statistical analyses i were also based on these time measures. The choice of the sign test to determine signifi cance of results rested on the considerations which follow. Inspection of Figures 23-A through 23-D and Figures 25-A through 25-C reveals high intersubject variability, even in the time to criterion measures. Sidman (19 60) points out that in Skinnerian operant performance situations large intersubject differences are common, and he attributes them to the behavioral history of the organism (page 153). It is stressed that variation among subjects in identical situations can often derive from differences in the adap- i jtive responses earlier made by the subjects to changes in their unique environments. ^For Experiment II a completely revised "Task and Reward" box was used to avoid these instrumentation difficulties. 133 For the present experiments, mature cats of unde termined age were obtained from a public city pound. They could earlier have been docile house companions or aggres sive alley strays, and their histories of adaptive behaviors to life stresses remained unknown. Their certain differ ences in background, however, undoubtedly contributed to the marked variation among subjects. Their variability, coupled with the small number of subjects, precluded the use of parametric statistics. Sidman (1960) elaborates this point by stating that "... group statistical proce dures generally operate against a baseline of intersubject variability. If, for example, the difference between two treatments is less than the intersubject variability between each of two groups, the difference is not considered significant." For situations of this sort, findings should be analyzed using intrasubject replications and results interpreted in terms of within group performances. In addition, the high intersubject variability and spread of individual scores did not permit the assumptions necessary for parametric tests; a normal distribution of the observed extinction measures could not be presupposed, and the animals could not be considered as coming from a parametrically defined population. Therefore, tests of 134 statistical significance resting on the assumption of normal distribution of observed events, such as Student's t-ratio, would not apply. Neither would those non- j parametric tests be suitable which pool and rank the experimental data, such as the Mann-Whitney U test (Siegel, 1956). In these, for instance, data points on essentially parallel curves, which never overlap, but which have the same acceleration, would be telescoped and assigned alter nate ranks. This would result in near equal sums of ranks and imply erroneously that the observations were purely due to chance. A test for significance of results most suitable for the data shown here should evaluate changes in perform ance for corresponding experimental sessions for each animal. It should compare Ec measures with corresponding EPRR measures for Group A and Ec measures with correspond ing EpRRstim measures for Group B. The test offered by Siegel (19 56) for situations of this sort is the sign test. He says in part: It is particularly useful for research in which quantitative measurement is impossible or infeasible, but in which it is possible to rank with respect to each other the two members of each pair. The sign test is applicable to the case of two related samples when the experimenter wishes to estab lish that two conditions are different . . . The test does not make any assumptions about the form of the distribution of differences, nor does it assume that the subjects are drawn from the same population. The different pairs may be from different populations with respect to age, sex, intelligence, etc.; the only requirement is that within each pair the experimenter has achieved matching with respect to the relevant extraneous variables. As was noted before, one way of accomplishing this is to use each subject as his own control. Walker and Lev (1953) suggest the use of the sign test when the investigator cannot make, or prefers to avoid the assumption that observed differences are normally and independently distributed with common variance and that the differences are measured on a scale in which intervals are equal. Walker and Lev stress that the sign test is well suited to data for which judgment between a pair of lobservations is possible. All of these stipulated requirements for the sign test were met, and its use appeared justified. Since the comparisons of paired observations were always made for measures derived from the same animal, matching of extra neous subject variables was achieved. As a summary of the results of Experiment I it can be stated that in each animal of Group A the occurrence of 136 PRE was unmistakable, and that each animal had indeed per formed as expected. PRE was demonstrated as a consequence i i !of 50% partially reinforced acquisition training. For the animals of Group B, which received a caudate spindle shock on non-reinforced trials during their 50% partially reinforced acquisition training, the development of PRE was questionable. It was clearly not present in animals number 4 and number 10, and ambiguous in animal number 8. ' The performance of the animals in Group B was nevertheless generally as expected, and the effect of caudate stimula tion on NR trials as interfering with PRE was demonstrated. The conclusiveness of Experiment I was weakened somewhat by the fact that the number of responses as a measure of extinction performance suffered from the equipment con founding already described. To overcome this in later i Experiment II, a new "Task and Reward" box was used, in which a stationary touch plate replaced the hinged and swivellable lever. This eliminated many of the problems inherent in the lever to crystal linkage and allowed use of the equipment at essentially one constant sensitivity level. In spite of the intermittent equipment difficul ties, however, Experiment I allowed some important conclu sions to be drawn. One is that the operant behavior 137 selected for performance by the animals, once partially reinforced, was sufficiently sensitive to demonstrate the development of the PRE (Group A). The second conclusion is that CNS processes occurring during acquisition, as defined by the "Habituation Theorists” (detailed in the previous chapter) are importantly related to the formation of PRE, since pairing caudate spindle shocks with the NR trials during acquisition impaired the PRE (Group B). These results were consistent with the position of the "Habitua tion Theorists" and with one of the experimental hypotheses. Beyond this, however, the findings furnished proof that the assumption made about the effects of caudate stimulation as being disruptive to CNS processes was correct. Hence, these findings gave justification for the proposed use of the caudate spindle shock in Experiment II, which is out lined next. Experiment II General Experiment I demonstrated the efficacy of caudate spindle stimulation in disrupting crucial CNS processes responsible for the formation of PRE. Pairing this stimu lation with NR trials during partially reinforced 138 acquisition training lessened the increase in resistance to extinction which customarily follows this training. The failure of PRE to develop was consistent with assumptions of the discrimination as well as habituation theories of PRE. For both, the occurrence of the caudate shock pre vented the neural register of the NR trial. This kept the behavioral changes (particularly PRE) from taking place which result from non-reinforcement during training. Thus, Experiment I furnished proof for the crucial need of non reinforcement and its perception during acquisition, if PRE is to develop. The identical predictions made by both theories precluded a differentiation between their conceptual validities based on the results of Experiment I. Neverthe less, the experiment established the equality of both theories in terms of the acquisition contingencies needed for the occurrence of PRE. This equality permitted the planning of Experiment II, in which the role of extinction events on PRE would be determined. In particular, it allowed the assignation of a critical role to the sequence of trials in extinction for the development of PRE. Using the caudate spindle shock (whose effectiveness had been proved in Experiment I) to disrupt the integration of 139 sequential extinction trial effects would now determine the theoretical correctness of one or the other group of theories of PRE. As was outlined earlier, the disruption of this integration due to the interference with short term memory by caudate stimulation during extinction would affect PRE only if it depended on discrimination processes based on the integrated complex of sequential trial effects. Impairing these discriminations, which would distinguish extinction from acquisition, should favor PRE if the discrimination theories are correct. On the other hand, since long term memory is not affected by caudate stimula tion, its administration during extinction should leave PRE unaltered if the habituation theories are correct. For these, the resistance to extinction would be dependent only on the decrement in response strength resulting from the presentation of each non-rewarded trial in extinction. Instrumentation The basic equipment of Experiment I was also used for Experiment II with the following changes. The "Task and Reward" box was modified to improve its reliability of operation and to allow animals more efficient lever presses. In place of the former projecting lever, a new one, 140 fashioned like a touch plate, was installed to the right and left of the food cup, and a single cover was used to expose or block the touch plate and food cup (Figure 27). | Also modified were the automatic control circuits to allow each animal to lever press in discrete trial periods. Gerbrands film strip and Time-O-Lite Model P 59 timers were used to program the trial duration for 20 and the inter trial interval for 10 seconds. (Film strip punch variation led to effective trial durations of from 17 to 23 seconds). Only during the trial period were the touch plate and food cup exposed. At the initiation of each trial, a visual cue of 10 light flashes, spaced over one second, could be presented on a 5x5 inch rear projection screen immediately behind and above the food cup by means of a Grass Photo stimulator. A simultaneous 2000 cycle tone of 1 second duration could also be given. Single caudate spindle shocks could be automatically administered 165 msec after the onset of any selected trial. The Gerbrands cumulative i recorder was used for response recording and its reset feature operated at the end of each trial. A "time-out" timer (Time-O-Lite P 59), set to less than 0.2 sec was Used to prevent response recording and delivery of rein forcement due to triggering impulses from vibration when 141 RE&R PROJECTION SCREEN orara SLIDING COVER POOD CUP Figure 27,— "Task and Reward Box" as presented to experimental animals in Experiment II, Note that the manipulandum is now a touch-plate positioned at each side of the reward cup. 142 the cover for the touch plate and food cup was retracted. Subjects The animals used for Experiment I were also used for Experiment II. Procedure Acquisition training with 100% reinforcement.— Subjects were habituated to the modified experimental compartment in several pre-experimental sessions. During these, and for all subsequent experimentation, they wore their head-cable. Occasional EEGs were taken while the animals habituated, and equipment tests were performed. Necessary electrical adjustments were made to insure proper instrument performance and clean EEG recording. Following habituation and preliminary testing, training was begun. For each animal the correct touch plate response was shaped, as in Experiment I, by manually rewarding progres sive fractional responses with a drop of milk. Once the animal had learned to respond, the food cup and lever were covered by E for brief time intervals with increasing frequency, until the automatic programming of discrete trials could be used without disrupting the 143 animal's performance. After the start of automatic trial administration, training was continued until the animals responded on at least 90% of the trials. At this perform ance level, each animal accumulated a total of 500 trials at the rate of 50 trials per session. Within each trial every response was milk-mixture reinforced. During train ing, each trial onset was paired with the visual and auditory cues. Caudate stimulation was never used. Control extinctions.— After training, control extinctions were run. Animals were placed into the experi mental compartment as for acquisition, but before the first presentation of touch plate and food cup, their spindle threshold to caudate stimulation was determined and the lowest stimulus parameters sufficient for consistent spindling were set. Then one cued and milk-mix rewarded trial period was given manually to test cat and equipment. After an interval of one half to two minutes, the extinc tion trials were administered. Four types were used. With (C) or without (N) cues and with (0) or without (0) stimu lation, resulting in 0C, 0C, 0N, and 0N trials. Trials were given in blocks of four, each comprised of the four types of trials. Within each block of four, each trial 144 occupied a different position, and a different first trial was used for each cat and for each of its extinction sessions. Extinction trials were continued during each session, until the animal failed to respond throughout two complete blocks of trials. Thus, at the end of each extinction session, and before being removed from the experimental compartment, each animal experienced at least 8, and possibly up to 11 trials, on which it did not respond. Eight non-response trials resulted if the ani mal's last response had been on the last trial of the previous block of four; eleven occurred if the last response had been on the first trial of the previous block of four. During the control extinctions, all animals gave their last response within the first three blocks of trials. To express the animal's extinction performance, the number of trials of each type, to which the animal had responded within these three trial blocks, was expressed as a percentage of the trials presented in these blocks. Thus, if an animal had responded to all trials of a given cate gory, its extinction performance would be called 100%; if i jresponses had been made only to two thirds of the trials i [presented, the extinction performance would be called 145 ’ ] 67%, et cetera. This percentage was calculated for each of the four types of trials and the resulting four measures were identified as E^c, E0C, E^N, and E0N. A total of four extinction treatments was given each animal at inter extinction intervals of twenty-three hours, and the extinc tion measures were derived for each of these treatments. The measures are shown in the upper half of Table II. (The lower portion of the table applies to extinctions following PR training.) Each table entry represents the responsive ness by the animal listed at the top of the table to the type of trial indicated immediately below it during the extinction treatment identified in the left column. The mean responsiveness over all four extinction treatments for each animal and by trial type is also shown. Retraining with 50% partial reinforcement.— Follow ing the control extinction sessions, retraining was begun under 50% partial reinforcement. The milk-mix reward was given or withheld by the Gerbrands tape controlled switch on randomly sequenced trials; i.e., on rewarded trials all lever presses were reinforced, on NR trial sessions no lever press ever was. All acquisition trials, whether reinforced or not, were cued by the tone and ten light TABLE II COMPARISON OF PERCENTAGES OF TRIALS RESPONDED TO DURING EXTINCTION {Top of table applies to extinctions following 100% reinforced training; lower portion applies to extinctions following PR training) Extinctions Post 100% Reinforcement; Animal 11 Animal 3 Animal 8 First : 33 67 Second: I 67 Third : Fourth: £; 1234 133 133 133 100 167 100 133 133 8.25 16.5 33.25 8.25 16.5 25.0 33.25 33 .25 33.25 58.5 33.25 Extinctions Post 50% Reinforcement; Animal 11 Animal 7 Animal 8 Animal 3 100 100 100 100 First : 100 Second: 1100 100 100 Third : Fourth: 133 233 234 300 234 133 234 266 200 166 33.0 16.5 33.25 41.75 16.5 58.5 58.5 66.5 50.0 75.0 58.5 58.5 146 147 flashes. Training was continued till the subject had responded in 500 PR trials at least with 90% performance consistency. Extinctions post PR training in discrete trials.— 1 — " \ -------------- The extinction sessions which followed discrete trial PR training were identical in all respects to those given following the corresponding 100% reinforced training. As was the case in the earlier control extinctions, trials during each extinction session were administered until the animal had not responded throughout two complete trial blocks. During these extinctions, which followed the PR acquisition training, the animals performed on more trials during the first three trial blocks (thus evidencing PRE) than during the earlier control extinctions, and responses now occurred on trials beyond the first three trial blocks, again indicating PRE. This observation was true for all animals in the first post PR extinction treatment, and for three of four animals in the second post PR extinction treatment. By the third treatment, however, all animals again withheld their response within the first three trial blocks, and during the fourth treatment this was also the case. As for the control extinctions, each animal's 148 extinction performance was again expressed as the percent age of trials presented during the first three trial blocks, to which the animals made a response. This per centage was calculated for each of the four trial catego ries in each of the four extinction sessions and identified as E^CPRR' e0CPRR' e0NPRR' or E0NPRR- The calculated percentages are shown in the lower portion of Table II, and the entries correspond to the earlier control extinction percentages. Any increase in responsiveness demonstrates the occurrence of PRE. The calculation of the percentages using only the first three trial blocks allowed the compar ison of the post PR extinctions with the earlier control extinctions based on an equal number of trials. Results of Experiment II The findings of Experiment II are graphically demonstrated in Figure 28. For each animal it shows the comparison made between post PR extinction performance and the corresponding control extinction performance. The values used are the means for the percentages calculated earlier for each type of trial in each extinction treatment (Table II). Each comparison appears in a separate row for the four types of trials administered. Post PR extinction 149 TYPE OF TRIAL 0C RESPONSIVENESS TO FOUR TYPES OF TRIALS. POST PR (isoraraQiffi!) EXTINCTION MEASURES ARE SHOWN ABOVE THOSE OBTAINED POST 100% REINFORCED TRAINING ( esssmsassBSBa) . ANIMAL NUMBER 3 __________________________________________________ 0C --------- -------------------- 0N 0N 0C ANIMAL NUMBER 7 0 5 0 B B u r n B ra ira I B S ) rai am B B ! I S 5S OS m B S 3 ires I M O C S S J H S S k«i 0C aaaM aKE} 0N rsi ssn isi ira rS 0N No Reap. I no Reap. ANIMAL NUMBER 8 0C 0C * > iu ( b i n p s B S jc ™ rraKBsjm (menoEic^Riiessi^Dgri 0N M P a a i s i s a B ? M g P H a ia ia |a ,! W M |3 1 ------------------------------— ---------------------------— 0N ANIMAL NUMBER 11 0C 0C yj%IJ8H||83jgjBflJ^»W^B}>g&BSaf^tairamfmEa BBBlEia I I — 0N silsinRSU^i Eti iej E ia isa ira ES ^ .. . . . 0N i 0 10 20 30 40 50 60 70 80 90 100 Percent Response| Figure 28„•— Comparison of responsiveness to four types of extinction trials by animals in Experiment II. measures are represented by the broken horizontal lines. The solid bars on which they rest in turn show the control extinction measures. PRE is indicated whenever the broken line extends beyond the solid bar. The scale along the bottom edge of Figure 28 allows one to read the percentage of responsiveness of each animal to each extinction situa tion. From Figure 28 it can be seen that in Experiment II animal number 3 developed PRE in all types of trial cate gories, and animals number 7, 8, and 11 in all categories except the one lacking cues and without the caudate shock (0N). Animal number 7 never responded to this type of trial, animal number 8 responded less in post PR extinc tions than in the earlier controls, and animal 11 responded to the same extent in post PR and control extinc tions. Summarizing from the sixteen comparisons made in Figure 28, one can make the following general conclusions: (1) highest resistance to extinction was always associated with trials using the caudate shock and the presentation of cues (0C); (2) next to highest occurred on cued trials lacking stimulation (0C); (3) still less resistance to extinction developed on trials using the caudate shock without the cues (0N); and (4) least resistance to 151 extinction was observed on trials lacking both the caudate shock as well as the cues. To test the significance of these observations, it was again elected, as in Experiment I, not to make any assumptions about the normality of distribution of the extinction measures nor about the parametric definition of the subject population. Therefore, the pairs of corres ponding extinctions for all animals were combined and levels of significance for the findings were again calcu lated by means of sign tests. The following levels of significance resulted for the observation: E0CPRR > E0C P <0*01 2. E0CPRR > E0C P ^ 0.01 3* E0NPRR ^ e0N P < 0.01 4* E0NPRR > E0N P = 0.375 The implications from these findings for the two groups of theories of PRE are elaborated next. Discussion of Experiment II Several clear observations were made in Experiment II. The first is that the addition of a single caudate spindle shock 16 5 msec after the onset of a trial results in an enhancement of response strength. This enhancement 152 occurred without fail in every post partial reinforcement extinction and was independent of whether the trial was cued or non-cued. In the extinctions following continu ously reinforced training, the caudate stimulus effect, enhancing response strength, was also generally true, although for animal number 8 responses made in 0C and 0C trials were equal and for animal number 11 responses in 0N trials were less than in 0N trials. The second observation is that the presence or absence of "enriching" cues also relates to the occurrence of responses during extinction. The animals' responsive ness to trials with cues was always higher than to corres ponding trials lacking the cues in post partial reinforce ment extinctions. In the earlier control extinctions, which had followed continuously reinforced acquisition, this finding was also generally true, although animals number 7 and 8 responded equally to 0C and 0N trials. The clearness with which stimulation and cue effects appeared in post PR extinctions and the occasional inconsistencies which cropped up following 100% reinforced training allow one immediate conclusion. It is that the A higher resistance to extinction following PR training with its concomitant higher number of responses allowed both - - - ■ ■ 153 of these variables to exert their full sway in the control of extinction behavior, whereas their effect was attenuated j j by the relative lability and rapid cessation of response following the continuously reinforced training. In terms of theory testing, the findings of Experi ment II, combined with those of Experiment I, allowed a value assessment of the habituation and discrimination theories. From Experiment I it had been learned that the presence of NR trials was crucial to the development of PRE, since the impairment of their neural register by caudate stimulation prevented PRE from developing fully. This finding was consistent with the assumptions of the discrimination and habituation theories. Both would pre dict that if the consequences of the NR trial were disrupted, then the PRE would not occur. In addition, the fact that the effect was blocked as a consequence of the administration of caudate stimulation confirmed the neuro- physiological assumptions made for the caudate spindle shock as being antagonistic to CNS memory trace events. Finally, Experiment I provided a needed check on later Experiment II, in which a critical role in the control of PRE was being attributed to the sequence of extinction trials and their effects which the animal was believed to 154 integrate into a perceived compound. This compound of integrated sequential stimulus effects during extinction trials is compared with the similar compound of effects formed during acquisition. A rapid formation of the integrated compound during extinction facilitates the discrimination of extinction from acquisition and lessens PRE. Interfering with the short term memory necessary for the compound formation by means of caudate stimulation, however, impairs the discriminability of extinction from acquisition. Hence, an increase in PRE would result if discrimination theories were correct. No change in PRE should occur if the habituation theories were to hold. Had there not been the prevention of PRE in Experiment I due to the caudate stimulation on NR acquisition trials, the further testing of extinction events on PRE as conducted in Experiment II would not have been possible. As it was, PRE did occur in Experiment II and was furthermore enhanced by caudate stimulation. This can only have resulted from the impairment which the caudate shock imposed on the formation of the integrative sequential stimulus effect compound. The subsequent difficulty for the subject to distinguish extinction from acquisition led to PRE. Hence, the results of Experiment II support the 155 discrimination theories of PRE. At the same time they argue against the habituation theories. For these latter, j the extinction performance was to be governed solely by the response strength established in acquisition. It would deteriorate only due to the reduction in response strength caused by each single non-reinforced extinction trial. Therefore, the single caudate shock on stimulated trials, interfering with short term but not long term memory, should have had no effect. Since PRE occurred, the assump tions of habituation theories are inconsistent with results. Finally, the results of Experiment II give rise to some further considerations. The consistent rank ordering of extinction performance in the four types of trials earlier described, indicates that the extinction processes in these four kinds of trials are independent of one another. If this is so, one must assume that not only the enriching cues, but also the caudate stimulus shock was discriminable to the animals. Support for this assumption has recently been provided by Buchwald et al. (1965), This discriminability of the caudate shock would also account for the higher responsiveness observed on £fN trials over 0N trials. The higher responsiveness due to the stimulus implies that it affected the animal in more than one way. As stipulated earlier, it would still interfere with short term memory, but in addition it would directly affect i sensory processes by also being linked to the complex of stimuli associated earlier with reinforced trials. Thus, pairing the caudate shock with the enriching cues, which have a well established cue significance, would lead to a new discriminable stimulus compound. The results of the experiment indicate further that this new compound was discriminable, since 0C and 0C differed. Ultimately, it could be argued that the effect of the caudate stimulus in Experiment II was due to an acquired capability to elicit the animal's response customarily evoked by the new stimulus compound. If this were so, it could be equated to the occurrence of disinhibition. Dis- inhibition is itself, however, the result of a short term memory impairment, so that the justification for the use of caudate stimulation in Experiment II remains. The relevant process which evaluated the appropriateness of either the habituation theory or discrimination theory was the discrimination of extinction from acquisition. The findings that the poorer (or more difficult) the discrimi nation, the more responses were obtained gives unequivocal support to the discrimination theories of PRE. CHAPTER V DISCUSSION AND SUMMARY Discussion General Since the "Humphreys Effect" or "Partial Reinforce ment Effect (PRE)" was first described and introduced into psychological literature in 1939, seven major theories have been proposed to account for its occurrence. The first five of these depend for their explanation on processes of perception and discrimination. They hold that during partially reinforced acquisitions the subject first experi ences a sequence of rewarded and non-rewarded trials as well as all stimuli arising from this sequence. Then the subject perceives the effects of the sequential events and, upon repeated trial presentations, integratively combines these multiple, consecutive, individual effects into a single complex which can be considered an integrated stimu lus effects compound. The formation of this compound is a CNS process, since it depends on the neural registration of 157 158 the trial sequences and of the sequential trial effects. Once formed, the compound subserves the processes hypothe sized by the five theories as being crucial to PRE. Specifically, it applies to each theory as follows: 1. It represents the reward to non-reward contin gencies on which are based the reward expectan cies which govern later behavior in extinction for "Expectancy Theory." 2. Its sequential integration of effects links non-reinforcement to reinforcement on which the "After-Effects Theory" is based. 3. The sequential integration of the effects of repeated non-rewarded trials with the rewarded one forms the compound which corresponds to a "Response Unit" in the "Response Units Theory." 4. The total integrated stimulus effect compound represents the acquisition situation against which the extinction must later be contrasted for the "Discrimination Theory." 5. Finally, the integrative stimulus effect com pound must be presumed to contain the links which give some of the individual stimuli secondary reinforcement potential necessary for 159 the "Secondary Reinforcement Theory." It is implicit in all these theories that the acquisition integrated stimulus effect can be remembered, for in subsequent extinction sessions it forms the stand ard against which the extinction events are evaluated. In order that this evaluation can take place, a similar inte grated stimulus effects compound, based on the sequence of extinction trials, must first be formed. Only then can the comparison be made between extinction and acquisition. All five theories hold that a high degree of similarity between extinction and acquisition makes for a difficult discrimin- ability between both and will tend to maintain the learned response. Conversely, they state that the more discri minable extinction is from acquisition, the quicker the acquired response will drop out. Since there is more similarity between PR acquisition and extinction than there is between 100% reinforced acquisition and extinction, it follows that more responses should occur after PR training than after continuously reinforced training. This is PRE. The five theories outlined all rely on discrimina tion as their central process in the formation of the PRE. For convenience, they were grouped and called "Discrimina tion Theories." The remaining other two theories to ~ 160 explain PRE hypothesize entirely different processes as being basic to its occurrence. As described in detail in the earlier review of literature, Weinstock (1954, 1958) believed that whenever NR trials occur during partially reinforced training, the subject has an opportunity to perform responses which are alternates to the one being acquired. Since these alternate, competing responses in the NR trials are never reinforced, they eventually drop out, or habituate. In later extinctions, when the subject experiences nothing but NR trials, the competing responses have already habituated and can no longer interfere with the performance of the learned, earlier rewarded response. Therefore, it occurs more often following PR training than after 100% reinforced training. In the latter case, the subject would not have been exposed to NR trials during acquisition, and the competing responses would not have habituated. During extinction they would occur in place of the prime response. Hence, the animal would perform the prime response less often, and resistance to extinction would be lower following continuous training than after PR training. Weinstock's competing response theory does not rest on discrimination of extinction from acquisition. Instead, following PR training (in which competing 161 responses have habituated), the course of extinction depends only on the strength of response resulting from initial training and the cumulative diminution of response strength which occurs as a consequence of each NR trial in extinction. Manipulating discriminative processes during extinction in a subject would, therefore, not change the resistance to extinction of its response. Finally, the "Mediating Response Theory" proposed by Amsel (19 58) shares a similarity with the "Competing Response Theory." In Amsel's concepts, the NR trials in PR acquisition are situations in which the animal's goal- directed behavior is thwarted and the animal is frustrated. This frustration is considered to be an emotional response with drive properties and becomes conditioned to the cues existing in the task situation. Once the frustrative drive has developed after a necessary minimum number of trials (usually in excess of 20), it enhances the performance of the learned behavior. In contrast hereto, the mediating frustrative drive cannot develop in 10 0% reinforced acqui sitions which lack the NR trials. Therefore, response performance in a subsequent extinction is poorer than if the extinction had been preceded by PR training. For Amsel's theory, as for Weinstock's, events in acquisition 162 NR trials are the sole determinants of response strength during extinction, and extinction will occur when the non-- reinforcement experienced on each extinction trial has decreased strength of response to an ineffective level. Again, no active process of discrimination need be per formed by the animal to distinguish extinction from acqui sition and to change its behavior to the withholding of reward more appropriate for the extinction situation. In the present experiment, the two final theories have been grouped and called the "Habituation Theories." The fact that the "Discrimination Theories" rely on the formation of the integrative sequential trial effect during extinction, while the "Habituation Theories" do not, make it possible in the present experiments to plan a test for the validity of either group. The plan rested on the potential of caudate spindle shocks to impair immediate (or short-term) memory processes, while leaving long-term memory unimpaired. By administering the caudate shock during extinction, the short-term memory needed to link trial effects in the formation of the integrative sequen tial trial effects compound would be disrupted and the discrimination between extinction and acquisition hampered. If Discrimination Theories were valid, PRE should thereby - 163 be enhanced. Since the shock would not affect long-term memory, PRE should occur unaltered if the Habituation Theories were to be correct. i Before these alternatives could be tested, however, a set of logically related working assumptions had to be formulated and their truth had to be determined. Toward this end, present Experiment I was performed. First, it was necessary to demonstrate the immediate memory blocking potential of the caudate spindle shock. If the assumption were valid that the shock "wiped from immediate memory" very recent events by disrupting their neural register, then pairing of the shock with non-reinforcement during acquisition should impair PRE according to both theory groups. When Experiment I bore this out, the capability was proven for the caudate spindle shock to block the jmemory formation for an event which it followed by 165 msec. In addition, by wiping from memory the effects of this event, i.e., non-reinforcement, and by preventing PRE, Experiment I had proven the need for NR trials during acquisition if PRE is to occur. Finally, Experiment I allowed the assignation of a critical role to the sequence of extinction trials in governing PRE, if this effect were later to be modified in Experiment II due to caudate 164 stimulation in extinction trials. In formulating the foregoing assumptions, some alternatives had to be considered as well: In case PRE had not been prevented in Experiment I by the administration of the caudate spindle shock 165 msec after every non reinforcement during the PR acquisition, then the later Experiment II would never have been possible. If, on the other hand, PRE had not been enhanced in Experiment II by imposing the caudate shock 165 msec after the onset of an extinction trial, then one would have had to assume that the caudate shock, in preventing PRE in Experiment I had affected more than only the immediate memory trace. An additional effect, interfering with the reward and non reward consequences in general, would then also have been indicated. Since, however, PRE in Experiment II was clearly shown, the function of the caudate spindle shock could be limited to its disruption of short-term memory processes. Furthermore, the following assumptions became tenable: 1. Had PRE occurred normally in Experiment II and had it not been enhanced by the spindle shock, then the "Habituation Theories" would have been supported and "Discrimination Theories" would have been weakened. 165 2. Had PRE shown enhancement due to the spindle shock, then the "Discrimination Theories" would be consistent with results and the "Habi tuation Theories" would be inconsistent. For all foregoing considerations relating to the use of the caudate shock, its timing in the sequence of extinction events was important. In Experiments I and II the shock occurred 165 msec after the event whose memory register or sequential integration into an effects com pound was to be disrupted. For Experiment I, this time had been determined empirically in pilot studies. In these, stimulus delay time had been adjusted to make the caudate spindle coincide with the licking of the milk-mix reward. The simultaneous occurrence of spindle and non-reward during PR acquisition for Group B proved to be effective in decreasing PRE. For Experiment II, the caudate shock had to be timed so that an animal1s perception of the onset of each trial was not impaired. Administering the shock 165 msec after the trial onset caused the spindle to develop about 400 msec after the start of the trial. This time lag was believed to be sufficient to allow the perception of the trial cues. Since, furthermore, the spindle did not interfere with long-term memory, the well established 166 significance of the cues at the onset of the trial should then elicit the response. If the postulates of the "Habituation Theories" were true, response decrement in extinction would then take place only due to the individual NR trial effects in reducing response strength. For these theories the occurrence of the caudate shock would remain inconsequential to PRE. On the other hand, because the spindle does interfere with short-term memory, it would prevent the integration of the sequential trial effects necessary for the "Discrimination Theories." This in turn would hinder discriminability of extinction from acquisi tion and lead to an enhancement of PRE. The results of Experiment II are proof that the time relationships between cues and caudate spindle shock, as well as the frequency of its occurrence, were adequate to test the hypothesized differences between the two groups of theories. As was pointed out in the earlier review of caudate literature, an enhancement of extinction performance in a free responding lever press situation can be observed if the animal is caudate stimulated during extinction (Wyers, Buchwald, Rakic and Lauprecht, 1962). This type of increased resistance to extinction was never elicited 167 clearly with pulse repetition rates of less than 1/2 pps. It follows, therefore, that caudate shocks every 30 seconds l (1/30 pps), as might have occurred in Experiment II if two ! trials with caudate stimulation followed one another, would I not in themselves lead to a similar increase in resistance i to extinction. Rather, one must conclude that the effect observed in present Experiment II was truly due to inter ference with the discriminative process postulated by the "Discrimination Theories." Further support of this con clusion lies in the fact that the four types of trials used in Experiment II were responded to differently. If the caudate shocks per se had led to increased extinction performance, this latter increase should have been the same on all equally stimulated trials. Since it was not, the importance of discrimination for PRE is again stressed. Additionally, the differential responsiveness to the four types of trials further suggests that the differ ent trial types were discriminable by the animals. For the enriching cues involved, this discriminability had been expected. For the caudate shock the discriminability now appeared implicit. The recent findings by Buchwald et al. (1965) give experimental support to the notion that the caudate shock can indeed be perceived by the animal. If 168 this perception of the shock takes place, then the shock must contribute to the course of PRE in at least two ways. Primarily it would act through the disruption of immediate memory. In addition it would operate by means of sensory processes. The caudate shock, in repeated presentations, would become part of the enriching cue complex, whose cue significance for reward had earlier been well established. Upon occurring, the shock would thereafter favor the evocation of the response earlier associated with the cues alone. Should this series of assumptions be true, then it could be argued that the caudate shock had acted as a dis- inhibiting stimulus during extinction and thereby enhanced PRE. This possibility cannot be ruled out. Nevertheless, it does not weaken the present experimental results, since the process leading to disinhibition in extinction is itself explainable as being a failure of short-term memory for the existing extinction situation. As long as dis inhibition would impair the relevant discriminability between extinction and acquisition, it too, could serve to distinguish between the two types of theories. Finally, the problem of possible current spread as having facilitated PRE needs consideration. It could be argued that current spread from the caudate nucleus to the internal capsule at the time of stimulation might have led to a motor event whose occurrence served to initiate the response. Related observations made earlier weaken this argument: 1. When current spread into the internal capsule appears to take place at all, the result is usually a diffuse and generalized twitch of the animal. It inter rupts , rather than supports any meaningful ongoing motoric act. In addition, this twitch was not observed to initiate the response in lever press experiments. Beyond this it must further be assumed that even if the current spread did not lead to a noticeable twitch, it would act in a general ized way and not serve to mediate a specific response. 2. Stimulation, when used in the present experi ment, occurred at parameters fully correspondent to those employed in parallel experiments. In these, any deleteri ous effect on performance of a response due to current spread was never observed. 3. The responses made in the present experiment i l very often occurred relatively late in the trial, for example, five to ten seconds after trial onset. By that time, the triggering effect of current spread for the response, if at all true, should have been dissipated 170 and appears very unlikely. Relation of This Study to Contemporary Work The prime conclusion made possible by the current experiments is that the explanations of PRE which are based mainly on discrimination during the extinction session appear to describe the most important and crucial process in the formation of this effect. An added point to be drawn from the present findings is that the events associ ated with the NR trial during a partially reinforced acquisition cannot by themselves fully account for PRE, nor determine its extent. The writer, therefore, is very much inclined to favor the "Discrimination Theories" of PRE over the "Habi tuation Theories." This preference is strengthened by recent findings of other workers, even though they them selves questioned to some extent the full appropriateness of discrimination theory. Particular reference is made to the recent work of Jenkins (1962). He devised an experiment, in which groups of pigeons were trained (1) under continuous reinforcement and (2) under some conditions of partial reinforcement. Then the animals which had experienced partial 171 reinforcement training were given a session of continuously reinforced trials. Ultimately, both groups underwent an extinction series. The purpose of adding the 100% training following the partial reinforcement training in one of the groups was to make the transition from rewarded acquisition trials to the subsequent extinction trials identical for both groups. The changes in situational cues to be dis criminated at the time of onset of extinction were, there fore, the same for both groups of animals. Jenkins found PRE for the animals which had been partially reinforced when he compared their extinction performance with the control group baseline, although trials at a 100% level of reinforcement immediately pre ceded their extinction. He further reported that if there was any change in PRE, as a result of interposing continu ously reinforced trials between the partially rewarded acquisition and extinction, then this change was in the direction of enhancing PRE, rather than in lessening it. Jenkins concludes from his findings that "... the abruptness of the transition from training to extinction is obviously not the critical functional difference between regular and partial reinforcement." Jenkins' conclusion is not at odds with the findings of the present experiment, 172 for which the "abruptness of the transition from training to extinction" was not considered crucial. Instead, the present experiments demanded the formation of the integra tive sequential stimulus effect over a sequence of extinc tion trials. The reason why this integrative effect in Jenkins' study need not have been markedly different from the one formed during acquisition because of the inter position of 100% reinforced trials prior to extinction is described below. Findings similar to those of Jenkins have also been reported by Theios (1962). He was led by his results to state that "... discrimination theory cannot ade quately account for PRE." Both authors apparently do not give full consider ation in their explanations to the fact that the partial and then continuous reinforcement schedule in itself constitutes a training regimen following the partial i reinforcement paradigm. That is to say, that in their training regimen all the trials in which only occasional reinforcement was the consequence of the animals' responses formed a specific block of trials. This block then alter nated with another series of trials (100% reinforced) in which reinforcement was always the consequence of the 173 animals' responses. Viewing both trial blocks in combina tion as the acquisition situation, one could define it as consisting of discrete training periods, marked by low (partial) or high (continuous) levels of reinforcement which alternated with one another. By looking at the training given to subjects by Jenkins and by Theios in this way, their studies and the present dissertation experiments gain much in similarity. The PR training sessions adminis tered to the experimental animals in the present study during Experiment II were indeed training periods (discrete trials) which differed only in the level of reinforcement. Reinforcement levels alternated randomly between 0% and 100%. As was the case in Jenkins' study, PRE was observed. Based on the logic outlined in the previous para graph, the findings of Jenkins and of Theios, as well as those reported in this dissertation, would all appear to be congruent. While relating this present study to contemporary work in PRE research, two further points need to be made. In the present experiments, the occurrence or absence of PRE was determined by intrasubject comparisons of the ex tinction performances of animals following training at 100% reinforcement and then again at 50% reinforcement. This 174 approach differs from the intergroup comparisons customarily encountered in the literature. The choice to rely on intrasubject comparisons rested on the following factors: Earlier studies of rates of responding or of extinction under conditions of caudate stimulation, which had used brain implanted cats as subjects, all employed the subjects as their own controls (Buchwald et al., 1961a; Lauprecht, 1961; Wyers et al., 1962). In the present experiments this methodological approach was maintained. The underlying logic for this approach was that it minimized confounding of results due to idiosyncratic behaviors of the animals which might be attributable directly to the implant proce dure per se, or to possible differences in stimulated neural cell aggregates because of variation in electrode position (however minute) at the target, or in polarization, or in electrode tip exposure. Added justification for the use of intrasubject comparisons rests in the realization that high intersubject variability would make, across animal compari sons ambiguous, as has been discussed by Sidman (1960) and as outlined starting on earlier page 132. The second fact to be mentioned is that the devel opment of the experimental design for this study was governed in part by the swiftness with which trained cats 175 were known to stop responding during extinction, after apparently having gained insight into the essentials of extinction situations. This meant that for Experiments I and II of this study a procedure was desirable which did not require prolonged responsiveness by the animals over a long series of extinctions. Allowing for this, control measures of the animals' extinction performances following 100% rewarded acquisition training were always obtained first. Thus, these control measures were not weakened by any other, previous extinction treatments and represented pure baseline extinction performances. Then the animals were retrained with partial reinforcement and again extin guished. It was felt that the experience of 10 0% reinforce ment would neither seriously interfere with the later development of PRE nor weaken the experiment if PRE, in fact, developed. It is anticipated that future work will give further clarification about the appropriateness of intrasubject assessment of PRE. The Neurophysiological Mechanisms of Caudate Induced Inhibitory Events General.— The positive results obtained in present Experiments I and II, based on the caudate spindle shock employed, convincingly reaffirm the use of electrical 176 stimulation of specific neural aggregates in the brain as a profitable method for the independent manipulation of experimental animals. Some consideration of the mechanisms i which underlie the effectiveness of caudate stimulation have been formulated by Heuser et al. (19 61), outlined by Lauprecht (1961) , modified by Buchwald et al. (1962) , and elaborated by Wyers (1963). In the main, these interpretations rest largely on the neurophysiological data reported in the earlier section on Background, Chapter II. In addition, they are based on the notion presented by Buchwald et al. (19 62) that there are at least two major brain systems involved in the control of an organism's momentary state of activation. These systems, in order to give the organism functional stability, provide a source of activating influences and also of inhibitory influences, both being in constant interplay. A suggestion that this might be so, on purely logical grounds, has also been advanced by Ashby (1954). The first system, contributing the arousal influ ences, is believed to be the ascending reticular activating system of Moruzzi and Magoun (1949). Since electrical stimulation of the reticular formation, using 300 pulses per second (pps), leads to an EEG virtually identical with 177 that of highly aroused subjects, the authors concluded that the reticular formation primes those processes which are believed to lead to activation of an organism. Much research has been triggered by the theoretical implications resulting from the findings of Moruzzi and Magoun. In particular, Lindsley and his associates have engaged in experiments in which the ascending reticular arousal system (ARAS), as defined by Lindsley (1951) was activated by electrical stimulation of the midbrain reticu lar formation. Fuster (19 58) reported that high frequency stimula tion of the reticular formation improved discrimination performance and reaction time in monkeys. Lindsley and Griffiths (1957) have also demonstrated that stimulation of the reticular formation with high frequency pulses increases cortical resolution of two light flashes pre sented in very close succession. Thus, excitation of the reticular formation has been identified experimentally as a means of "activating" behavior and of "arousing" the living organism. But the reticular formation is not the only locus at which electrical stimulation leads to behavioral arousal and a concomitant desynchronization of the EEG. Several 178 other subcortical sites have been found, which, upon stimulation, yield the electrographic and behavioral signs of activation. Murphy and Gellhorn (1945) , for example, reported particular involvement of the posterior hypothalamus in Activation, and Jasper (1949) described arousal as a result of electrical stimulation of the diffuse thalamocortical projection system. Hess (1954), too, reported "ergotrophic" effects from areas in the posterior and mesial part of the hypo thalamus, which he named hypothalamic "dynamogenic" fields. Finally, Monnier et al. (19 60) have published findings dealing with the activating effects of stimulation of thalamic nuclei. The cumulative evidence of these results could suggest that the reticular formation is perhaps not the unique locus for the production of electrographic desyn- chreny and behavioral arousal. Lindsley (1960), however, resolved these findings, which at first appear to be at odds, by postulating a relationship between the thalamic and reticular events in which the former lead to "bounced-back" potentials at the ARAS. In his 19 60 discussion, he suggested that the 179 observable resultants from subcortical stimulation are the consequences of influences of these sites of stimulation upon "... the reticular formation in the lower brain stem and, through it, the ascending reticular activating system (ARAS)." Lindsley further reported that arousal effects triggered by even mild stimulation of the ARAS override those cortical responses which are usually elicited by competing sub-cortical stimulation. In a hierarchical ordering of the arousal producing structures of the brain, the midbrain reticular formation would be in first place. The system thought to contribute the inhibiting influences needed to keep the ARAS in check is believed to be the caudate loop as described by Heuser et al. (1961) and by Buchwald et al. (1962). Upon electrical stimula tion, this system exhibits effects which offset those usually ascribed to the influences of a stimulated ARAS. It has been suggested by Buchwald and his co-workers that this second, inhibitory system includes the caudata and the non-specific thalamic nuclei. For purposes of this discussion, it will be called the diffuse inhibitory caudato-thalamic system (DICTS). 180 The interactive process at the thalamus.— Since the ARAS and DICTS overlap at the thalamic level, there is great likelihood that one of the interaction processes between the two systems takes place at this common site. Buchwald et al. (19 62) suggest that there is a modification of afferent impulses at the thalamus, and in particular the non-specific nuclei, while the afferents are ascending toward the cortex, and that this is the process making caudate effect possible. The afferents, coming from the reticular formation, upon being modified by caudate efflux, ultimately carry to the cortex the interaction product of ARAS and DICTS. It is believed that during normal states of activa tion cortico-caudate loops govern the amount of caudate outflow. Ideally, it is maintained at a level which yields an ARAS-DICTS interaction product allowing the organism to operate at the optimal level of activation. At this level, its higher mental processes should occur normally. In particular, the short-term memory trace formation would also take its normal course. With the driving of the caudate outflow by electrical stimulation, the caudate associated inhibitory processes become more predominant. If the stimulation is prolonged and moderately fast (about 181 5 pps), the over-all behavior of the animal can be slowed down. With the single shocks used in the present experi ments, however, the general level of activation was not impaired. Still, the stimulus proved sufficient to disrupt the neural events on which short-term memory processes are based. Disrupting these short-term memory processes was instrumental in modifying PRE. Summary The two experiments of this study sought insight into the basic processes underlying the partial reinforce ment effect (PRE) often evidenced in the behavior of animals and man. This frequently observable phenomenon exists whenever some behavior is learned in a situation in which the reward for the behavior being acquired is neither always immediate, nor consistent. Paradoxically, behavior acquired under these non-continuous reinforcement contin gencies persists more stably once the reinforcement never again occurs, as in extinction, than if the behavior had been learned with continuous (100%) reinforcement. The PRE paradox has challenged theorists in psychology for close to twenty-five years. Seven major attempts have been made to explain it: 182 1 . The Expectancy Theory 2. The After-Effects Theory 3. The Response Units Theory 4. The Discrimination Theory 5. The Secondary Reinforcement Theory 6 . The Competing Response Theory 7. The Mediating Response Theory In the present experiments, the theories were divided into two categories. The first five were combined in one, the last two in the other group. The process considered crucial by the proponents of the first five theories is essentially one of discrimina tion of extinction from acquisition. In the main, these five theories hold that PRE is evidenced as a consequence of the similarity of non-continuously reinforced acquisi tion situations to any subsequently experienced extinction sessions. To the extent to which the subject is unable to differentiate between these two situations, his behavior in an extinction environment will be as it was in the earlier acquisition environment, and PRE will thus be evidenced. The proponents of the last two theories believe that the direct consequences of the non-reinforced trials which the subject must experience during partially 183 reinforced acquisition are the solely critical ones for PRE. They are either (1) habituation of interfering and con flicting responses, caused by the non-occurrence of the reinforcer; or (2) the formation of a "frustrative drive," which develops for the same reason. In this dissertation, the beliefs of the two opposing groups have been identified as the "Discrimination Theories" or "Habituation Theories" to explain PRE, and the present experiments tested the extent to which either of the two types of postulated processes, in fact, contributed to PRE. The experimental method employed for this study included the manipulation of brain events in experimental animals, while the crucial psychological interactions were believed to be taking place. In particular, electrical stimulus shocks to the caudate nuclei (caudate spindle shocks) were used, which had been demonstrated to be dis ruptive to higher mental processes and to impair short-term memory, while not affecting long-term memory. In Experiment I, the caudate shock was paired with non-reinforced trials during a partially reinforced acquisition session. A decrement in PRE was the result. 184 In Experiment II, the caudate spindle shock was employed during extinction sessions which had been preceded by non-continuously reinforced acquisition sessions. An assessment of PRE showed that it had been generally enhanced. The results of the current experiments favor an explanation of PRE which is based primarily on the discrim ination of compound sequential effects of extinction trials from the compound sequential effects earlier experienced in partially reinforced acquisition trials. In the discussion some possible brain mechanisms which may underlie the effective use of the caudate spindle stimulation in psychological research are outlined. LIST OF REFERENCES LIST OF REFERENCES Akert, K. and Anderson, B. Experimenteller Beitrag zur Physiologie des Nucleus Caudatus. Acta physiol. Skandinav., 1951, 2£, 281-298. Amsel, A. The role of frustrative nonreward in non- continuous reward situations. Psychol. Bull., 1958, 55_, 102-119. Amsel, A. and Hancock, W. Motivational properties of frustration: III. Relation of frustration effect antedating goal factors. J. exp. Psychol., 1957, 53, 126-131. Amsel, A. and Roussel, J. Motivational properties of frustration: I. Effect on a running response of the addition of frustration to the motivational complex. J. exp. Psychol., 1952, £3, 363-368. Amsel, A. and Ward, J. S. Motivational properties of frustration: II. Frustration drive stimulus and frus tration reduction in selective learning. J. exp. Psychol., 1954, £8, 37-47. Ashby, W. R. Design for a brain. New York: John Wiley and Sons, 1954. Becker, H. C., Peacock, S. M., Jr., Heath, R. G., and Mickle, W. A. Methods of stimulation control and con current electrographic recording. In Sheer, D. E. Electrical stimulation of the brain. Austin: Univer sity of Texas Press, 1961. Berger, H. Uber das Elektrenenkephalogram des Menschen, I. Arch, f. Psychiat. u. Nervenkr., 1929, £7, 527-570. Bitterman, M. E., Fedderson, W. E., and Tyler, D. W. Secondary reinforcement and the discrimination hypoth esis. Amer, J. Psychol., 1953, 66, 456-464. 186 187 Boring, E. G. A history of experimental psychology. New York: Appleton-Century-Crofts, Inc., 1957. Buchwald, N. A. and Ervin, F. R. Evoked potentials and behavior. A study of responses to subcortical stimula tion in the awake, unrestrained animal. Electroenceph. clin. Neurophysiol. , 1957, 9_, 477-496. Buchwald, N. A., Heuser, G., Wyers, E. J., and Lauprecht, C. W. The "Caudate-Spindle" III. Inhibition by high frequency stimulation of subcortical structures. Electroenceph. clin. Neurophysiol., 1961a, T3_, 525-530. Buchwald, N. A., Wyers, E. J., Carlin, J., and Farley, R. E. Effects of caudate stimulation on visual discrimination. Exp e r im. Neuro 1. , 1961b, _4, 23-36. Buchwald, N. A., Wyers, E. J., Lauprecht, C. W., and Heuser, G. The "Caudate-Spindle" IV. A behavioral index of caudate-induced inhibition. Electroenceph. clin. Neurophysiol., 1961C, 13, 531-537. Buchwald, N. A.f Wyers, E. J., Okuma, T., and Heuser, G. The "Caudate-Spindle" I. Electrophysiological properties. Electroenceph. clin. Neurophysiol., 1961d, 13, 509-518. Buchwald, N. A., Rakic, L., Wyers, E. J., Hull, C., and Heuser, G. Integration of visual impulses and the "Caudate Loop." Experim. Neurol., 1962, 5_, 1-20. Buchwald, N. A., Horvath, F. E., Soltysik, S., and Romero- Sierra, C. Caudate influences on behavior and the EEG. Paper presented to the Am. Electroenceph. Soc., Oct., 1964, Electroenceph. clin. Neurophysiol., 1965, 18_, 529 Crum, J., Brown, W. L., and Bitterman, M. E. The effect of partial and delayed reinforcement on resistance to extinction. Amer. J. Psychol., 1951, 64, 228-237. Davis, G. D. Caudate lesions and spontaneous locomotion in the monkey. Neurology, 1958, 8, 135-139. 188 Dean, W. H., and Davis, G. D. Behavior changes following caudate lesions in rhesus monkey. J. Neurophysiol., 1959, 22, 524-537. Denny, M. R. The role of secondary reinforcement in a partial reinforcement learning situation. J. exp. Psychol., 1946, 36, 373-389. Denny-Brown, D. The basal ganglia. Oxford Neurological monographs. London: Oxford Univ. Press, 1962. 1 Vol., cit. 105-118. Duffy, E. The concept of energy mobilization. Psychol. Rev., 1951, 58, 30-40. Elam, C. B., Tyler, D. W., and Bitterman, M. E. A further study of secondary reinforcement and the discrimination hypothesis. J. comp, physiol. Psychol., 1954, 47, 381-384. Fehrer, Elisabeth. Effects of reinforcement and of pre- and post-reinforcement delays on learning and extinction. J. exp. Psychol. , 1956, 5^2, 167-176. Forman, D. and Ward, J. W. Responses to electrical stimu lation of caudate nucleus in cats in chronic experi ments. J. Neurophysiol. , 1957, 2Q _, 230-244. Freeman, G. L. and Krasno, R. Inhibitory functions of the corpus striatum. Arch. Neurol, and Psychiat., 1940, 44, 323-327. Freeman, G. L. The energetics of human behavior. Ithaca: Cornell University Press, 1948. Fuster, J. M. Effects of stimulation of the brain stem on tachistoscopic perception. Science, 1958, 127, 150. Gerebtzoff, M. A. Contribution a la physiologie du corps strie. Arch. Internat. Physiol., 1941, 53., 333-352. Grant, D. A., Hake, H. W., and Hornseth, J. P. Acquisition and extinction of a verbal conditioned response with differing percentages of reinforcement. J. exp. Psychol., 1951, 42, 1-5. 189 Grant, D. A. and Schipper, L. M. The acquisition and extinction of conditioned eyelid responses as a func tion of the percentage of fixed-ratio random reinforce ment. J. exp. Psychol. , 1952, _43, 313-320. Guilford, J. P. Fundamental statistics in psychology and education. New York: McGraw-Hill, 1950. Heath, R. G. and Hodes, R. Induction of sleep by■stimula tion of the caudate nucleus in Macacus rhesus and man. Trans. Amer. Neurol. Assoc. , 1952, 11_, 204-210. Hess, W. R. Beitrage zur Physiologie des Hitnstammes. I: Die Methodik der lokalisierten Reizung und Auschaltung subcorticaler Hirnabschnitte. Leipzig: Thieme, 19 32. Hess, W. R. Brain mechanisms and consciousness, ed. Adrian, E. D., Bremer, F., Jasper, H. H., and Delafrasnaye, J. F. Oxford: Blackwell, 1954. Heuser, G., Buchwald, N. A., Wyers, E. J., and Lauprecht, C. W. The "Caudate-Spindle" i;E. The "caudate spindle" and the caudato-cortical evoked response. An investi- , gation into their interaction and pathways. Electro- enceph. clin. Neurophysiol. , 1961, 1_3, 519-524. Hull, C. L. Principles of behavior. New York: Appleton- Century, 1943. Hulse, S. H., Jr. Amount and percentage of reinforcement and duration of goal confinement in conditioning and extinction. J. exp. Psychol. , 1958, 5j5 , 48-57. Humphreys, L. G. The effect of random alternation of reinforcement on the acquisition and extinction of conditioned eyelid reactions. J. .exp, Psychol., 1939, 25, 141-158. Hunter, J. and Jasper, H. H. Effects of thalamic stimula tion in unanaesthetized cats: Arrest reactions and petit mal-like seizures, activation patterns and generalized convulsions. Electroenceph. clin. Neuro physiol ., 1949, 1, 455-473. 190 Jasper, H. H. Diffuse projection systems: The integra tive action of the thalamic reticular system. Electroenceph. clin. Neurophysiol., 1949, 1, 405-420. Jasper, H. H. Implications for the neurological sciences, in D. E. Sheer: Electrical stimulation of the brain. Austin: University of Texas Press, 1961. Jasper, H. H. and Ajmone-Marsan, C. A stereotaxic atlas of the diencephalon of the cat. Ottawa: The National Research Council of Canada, [date supplied: 1952]. Jenkins, W. 0. and Stanley, J. C., Jr. Partial reinforce ment: a review and critique. Psychol. Bull., 1950, 47, 193-234. Jenkins, H. M. Resistance to extinction when partial reinforcement is followed by regular reinforcement. J. exp. Psychol., 1962, 6_4, 441-450. 1 Kennard, M. Experimental analysis of the function of the basal ganglia in monkeys and chimpanzees. J. Neuro physiol . , 1944, l_f 127-148. Knott, J. R., Ingram, W. R., and Correl, R. E. Effects of certain subcortical lesions on learning and performance in the cat. A. M. A. Arch. Neurol., 1960, 2, 476-484. Lauprecht, C. W. Electrical stimulation of the caudate nucleus as a method for the inhibition of behavior. Unpublished Master's thesis, University of Southern California, 1961. Lesse, H. Amygdaloid electrical activity during a condi tioned response. Proc. First Intern. Congr. Neurol. Sc. , 1959, _3, 177-180. Lewis, D. J. Partial reinforcement in a gambling situation. J. exp. Psychol., 1952, 4^3, 447-450. Lewis, D. J. Partial reinforcement: a selective review of the literature since 1950. Psychol. Bull., 1960 , 57, 1-28. 191 Lewy, F. H. The basal ganglia and their diseases. Associ ation for Research in Nervous and Mental Diseases, Baltimore, Williams and Wilkins Co., 1942, 21., 1-20. Lilly, J. C. Injury and excitation by electric currents; A. the balanced pulse-pair waveform, in D. E. Sheer: Electrical stimulation of the brain. Austin: Univer sity of Texas Press, 1961. Lilly, J. C., Austin, G. M., and Chambers, W. W. Threshold movements produced by excitation of cerebral cortex and efferent fibers with some parametric regions of rectangular current pulses- J. Neurophysiol., 1952, 15, 319. Lilly, J. C., Hughes, J. R., Alvord, E. C., Jr., and Galkin, T. W. Brief, noninjurious electric waveform for stimulation of the brain. Science, 1955, 121, 468-469. Lindsley, D. B. Emotion. Handbook of Experimental Psychology, ed. S. S. Stevens. New York: Wiley, 1951. Lindsley, D. B. Attention, consciousness, sleep and wake fulness. Handbook of physiology, sec. I, Neurophysi ology, Vol. Ill, ed. J. Field, H. W. Magoun, and V. E. Hall. Washington: Am. Physiol. Soc., 1960. Lindsley, D. B. and Griffiths, R. Study quoted by D. B. Lindsley. Psychophysiology and motivation. Nebraska Symposium on Motivation, ed. M. R. Jones. Lincoln: University of Nebraska Press, 19 57. Logan, F. A., Beier, E. M. and Kincaid, W. D. Extinction following partial and varied reinforcement. J. exp. Psychol. , 1956, 52_, 65-70. Longnecker, E. D., Krauskopf, J., and Bitterman, M. E. Extinction following alternating and random partial reinforcement. Amer. J. Psychol., 1952, 65, 580-587. Malmo, R. B. Activation: A neurophysiological dimension. Psychol. Rev., 1959, 66, 367-386. 192 Mettler, F. A., Ades, H. W., Lipman, E., and Culler, E. The extrapyramidal system: An experimental demonstra tion of function. Arch. Neurol, and Psychiat., 1939, 4±r 984-994. j Mettler, F. A. and Mettler, C. C. Role of the neostriatum, j Am. J. of Physiol., 1941, 133, 594-601. Mettler, F. A. and Mettler, C. C. The effects of striatal injury. Brain, 1942, 6_5, 242-255. Monnier, M., Kalberer, M., and Krupp, P. Functional antagonism between diffuse reticular and intralaminary recruiting projections in the medial thalamus. Exper. Neurol., 1960, 2, 271-289. Morison, R. S. and Dempsey, E. W. A study of thalamo cortical relations. Am. J. Physiol., 1942, 135, 281- 292. Moruzzi, G. and Magoun, H. W. Brain stem reticular forma tion and activation of the EEG. Electroenceph. clin. Neurophysiol., 1949, 1, 455-473. Mowrer, O. H. and Jones, H. M. Habit strength as a func tion of the pattern of reinforcement. J. exp. Psychol., 1945, 43^, 293-311. Murphy, J. P. and Gellhorn, E. The influence of hypo thalamic stimulation on cortically induced movements and on action potentials of the cortex. J. Neuro- physiol., 1945, £, 341-364. Pavlov, I. P. Conditioned reflexes. Trans, by G. V. Anrep. London: Oxford University Press, 1927. Penfield, W. The reticular formation and clinical neurology. Designated discussion in Reticular formation of the brain. Henry Ford Hospital International Symposium. Toronto: Little, Brown and Co., 1957. Purpura, D. P., Hausepian, E. M., and Grundfest, H. Analysis of caudate-cortical connections in neuraxially intact and "telencephale isole" cats. Arch. ital. Biol., 1958, 96, 145-167. 193 Rakic, L., Buchwald, N. A., and Wyers, E. J. The caudate nucleus and inhibition. A : Biochemical aspects of caudate inhibition. Proc. intern, union, physiol, sci. Leiden, 1962, Vol. 2, Abstr. 1100. Rezek, P. Ein primaeres polymorphes Sarkom des Gehirns. Jb. Psychiat. Neurol, 1897, 1£, 40-75. Richter, C. P. and Hines, M. Increased spontaneous activity produced in monkeys by brain lesions. Brain, 1938, 61, 1-16. Romanovskaya, E. A. Certain consequences of destroying the caudate nuclei in rabbits and dogs. Bull, exp. Biol. Med., 1957, 4£, 1066-1069. Rosvold, H. E. and Delgado, J. M. R. Effect on delayed- alternation test performance of stimulating or destroying electrically structures within the frontal lobes of the monkey's brain. J. comp, physiol. Psychol.,1956, 51, 437-444. Sheffield, V. F. Extinction as a function of partial reinforcement and distribution of practice. J. exp. Psychol., 1949, 39, 511-525. Sidman, M. Tactics of scientific research. New York: Basic Books, Inc., 1960. Siegel, S. Nonparametric statistics. New York: McGraw- Hill, 1956. Shimamoto, T. and Verzeano, M. Relations between caudate and diffusely projecting thalamic nuclei. J. Neuro physiol. , 1954, 17, 278-288. Skinner, B. F. Resistance to extinction in the process of conditioning. J. gen. Psychol., 1933, _9, 420-429. Skinner, B. F. The extinction ratio and its modification by a temporal discrimination. Psychol. Bull., 1936, 33, 784. 194 Skinner, B. F. The behavior of organisms. New York: Appleton-Century, 193 8. Stanley, W. C. and Clayton, F. L. An experimental test of Weinstock's hypothesis concerning the effect of partial reinforcement and extinction. Psychol. Rep., 1955, 1, 421-424. Theios, J. The partial reinforcement effect sustained through blocks of continuous reinforcement. J. exp. Psychol. , 1962, 64^, 1-6. Thompson, R. The effects of intracranial stimulation on memory in cats. J. comp, physiol. Psychol. , 1958, 51, 421-426. Tolman, E. C., and Brunswick, E. The organism and the causal texture of the environment. Psychol. Rev., 1935, _42, 43-77. Tower, S. S. Dissociation of cortical excitation from cortical inhibition by pyramidal section. Brain, 1935, 58, 238-255. Tyler, D. W., Wortz, E. C., and Bitterman, M. E. The effect of random and alternating partial reinforcement on resistance to extinction in the rat. Amer. J. Psychol. , 1953, 66, 57-65. Umbach, W. Zur Electrophysiologie des Caudatum der Katze: Elektrische Reizung und Krampfauslosung in verschie- denen Grosshirnstrukturen und ihre Beziehungen zum Nucleus caudatus. Arch. Psychiat. Nervenkr., 19 59, 199, 553-575. Voneida, T. J. An experimental study of the course and destination of fibers arising in the head of the caudate nucleus in the cat and monkey. J. comp. Neurol., 1960, 115, 75-82. Walker, H. M. and Lev, J. Statistical inference. New York: Henry Holt and Co., 1953. 195 Weinstock, S. Resistance to extinction of a running response following partial reinforcement under widely spaced trials. J. comp, physiol. Psychol., 1954, 47, 318-322. Weinstock, S. Acquisition and extinction of partially reinforced running response at a 24-hour inter-trial interval. J. exp. Psychol., 1958, J56, 151-158. Willis, Sir Thomas. Cerebri anatoma cui accessit nervorum descripto et usus. London: Roycroft, 1664. Wilson, W., Weiss, E. J., and Amsel, A. Two tests of the Sheffield hypothesis concerning resistance to extinc tion, partial reinforcement and distribution of practice. J. exp. Psychol., 1955, 5j0 , 51-60. Wyers, E. J., Buchwald, N. A., Rakic, L., and Lauprecht, C. W. The caudate nucleus and inhibition: Caudate inhibition and extinction. Proceedings of the XXII. Int. Congress of Physiol. Sc., Sept. 9-17, 1962. Leiden, the Netherlands, Vol. 3, Free comm. 1101. Wyers, E. J., Lauprecht, C. W., and Buchwald, N. A., Abstract: Learned behavior and stimulation of the caudate nucleus. 17th Int. Congr. Psychol., Abstr., Washington D. C., August 20-27, 1963.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

The Effects Of Prior Part-Experiences On Visual Form Perception In The Albino Rat

PDF

Habituation Of The Multiple Unit Discharge Response To White Noise Stimulation In The Unanesthetized Rabbit

PDF

The Effects Of Ether And Electroconvulsive Shock On One Trial Appetitive And Adversive Learning

PDF

Human Performance As A Function Of The Joint Effects Of Drive And Incentive Motivation

PDF

Classical Discrimination Conditioning As A Function Of Probability Of Reinforcement

PDF

Transfer Of The Partial Reinforcement Extinction Effect Across Tasks In Normal And Retarded Boys

PDF

The Effect Of Discriminability On The Partial Reinforcement Effect In Human Gsr Conditioning

PDF

The Effect Of Irrelevant Environmental Stimulation On Vigilance Performance

PDF

The Effects Of Transverse Accelerations And Exponential Time-Lag Constants On Compensatory Tracking Performance

PDF

Verbal Reports Of Emotional States And Onsets And Offsets Of Conditioned Stimuli

PDF

Stress Response Modification And Autohypnotic Techniques

PDF

Clinical Implications Of Verbal Learning Without Awareness: An Experimental Study

PDF

Recognition Accuracy In The Method Of Single Stimuli: A Test Of An Operational Definition Of The Distinctiveness Of Stimuli

PDF

Sex-Role Preferences Of Early Adolescents In Relation To Adjustment

PDF

The Effects Of Reinforcement Pattern, Intertrial Interval Regularity, And adaptation On The Orienting Response And Classical Gsr Discrimination Conditioning

PDF

The role of cues in the arousal of anxiety

PDF

The Effect Of Motor Ability Loss On Cognition And Emotion

PDF

Human Gsr Classical Conditioning And Awareness Of The Cs-Ucs Relation

PDF

Effects Of Success And Failure On Impulsivity And Distractibility Of Three Types Of Educationally Handicapped Children

PDF

A Factor-Analytic Study Of Military Leadership

Asset Metadata

Creator Lauprecht, Carl Walter (author)

Core Title The Modification Of Partial Reinforcement Effect As A Consequence Of Electrical Stimulation Of The Caudate Nucleus In Cats

Degree Doctor of Philosophy

Degree Program Psychology

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag OAI-PMH Harvest,psychology, experimental

Language English

Contributor Digitized by ProQuest (provenance)

Advisor Wyers, Everett J. (committee chair), Grings, William W. (committee member), Meehan, John P. (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c18-227814

Unique identifier UC11359933

Identifier 6700410.pdf (filename),usctheses-c18-227814 (legacy record id)

Legacy Identifier 6700410.pdf

Dmrecord 227814

Document Type Dissertation

Rights Lauprecht, Carl Walter

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA