Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Data-driven modeling of the hippocampus & the design of neurostimulation patterns to abate seizures
(USC Thesis Other)
Data-driven modeling of the hippocampus & the design of neurostimulation patterns to abate seizures
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
DATA-DRIVEN MODELING OF THE HIPPOCAMPUS & THE DESIGN OF NEUROSTIMULATION PATTERNS TO ABATE SEIZURES A doctoral dissertation by ROMAN A. SANDLER University of Southern California Viterbi School of Engineering Department of Biomedical Engineering Committee Members: Vasilis Z. Marmarelis, Ph.D (Chair) Dong Song, Ph.D Maryam M. Shanechi, Ph.D Micheal C.K. Khoo, Ph.D For Degree Conferral of December 2015 Acknowledgements I wish there were more occasions where culture has institutionalized a formal custom to express gratitude to all those who helped me along my path. William Arthur Ward once said, "Feeling gratitude and not expressing it is like wrapping a present and not giving it." I truly hope that the following people have felt my deep sense of appreciation for them all along. However, if I have failed in that, I would like to explicitly express my gratitude for them now. First and foremost I would like to thank my parents, Alla & Alex who are responsible for so much of who I am and who have instilled much of my love for science. From a very early age, my mom, originally an aerospace engineer, pushed me to develop my skill in mathematics which established a foundation that has lasted me well until today. To my dad, a true engineer and for whom no problem was too intimidating. The two summer I spent doing appliance repair with him were perhaps the most "experimental" research I have done. To my sister Lena, a true role model all my life, both academically and personally. To my grandmother, Yelizaveta, whom I love so much. To all my cousins and entire extended family who have given me so much love and support throughout my life. To my advisor, Vasilis Marmarelis who introduced me to the scientic world and instilled in me a passion for truth. I am incredibly grateful for the amount of unconditional support Vasilis has given me these four years to pursue my ideas. 1 Even when I was working on projects he was less than passionate about, he still allowed me the academic freedom to pursue them. This is a truly rare quality in academia, and has made my time in grad school very interesting, smooth, and stress free. I would also like to thank Dr. Dong Song, who has always given me generously of his time and has always provided me fresh insight and perspective. To all my friends and collaborators in grad school. To Rob Hampson's lab which provided the data that drives so much of this work. Particularly to Dustin Fetterho with whom I have developed a very fruitful friendship and collaboration. To Yue, Kunling, and Brandon from the BMSR lab. To all the guys from the math journal club, which I am very proud to have helped start. I would list all your names, but am too afraid to leave anyone out... However, I would like to single out Brian Robinson and Kunling Geng whose presentations on group regularization and simulated annealing have very positively impacted my own research. Finally, to Tami, who has sat through countless hours listening to me ramble about my research. Thank you for providing the patience, support, and love to get me through these years. You truly have enriched my time here. 2 Contents Contents 3 List of Figures 8 1 Introduction 11 2 Review of Relevant Biological Topics 14 2.1 Hippocampal Reentrance . . . . . . . . . . . . . . . . . . . . . . . . 14 2.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.1.2 Anatomy & Physiology . . . . . . . . . . . . . . . . . . . . . 17 2.1.3 Relation to Epilepsy . . . . . . . . . . . . . . . . . . . . . . . 20 2.1.4 Relation to Learning & Memory . . . . . . . . . . . . . . . . 25 2.2 Hippocampal Theta Rhythm . . . . . . . . . . . . . . . . . . . . . . 28 2.2.1 Theta: Mechanisms . . . . . . . . . . . . . . . . . . . . . . . 28 2.2.2 Theta: Learning & Memory . . . . . . . . . . . . . . . . . . . 32 2.3 Deep Brain Stimulation for Epilepsy . . . . . . . . . . . . . . . . . . 34 3 System Identication of Point-Process Neural Systems Using Prob- ability Based Volterra Kernels 39 3.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3 3.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.3.1 Model Conguration . . . . . . . . . . . . . . . . . . . . . . . 44 3.3.2 Model Estimation . . . . . . . . . . . . . . . . . . . . . . . . 45 3.3.3 Relationship to Poisson-Wiener Kernels . . . . . . . . . . . . 48 3.3.4 PBV Kernels for Correlated Point-Process Inputs . . . . . . . 49 3.3.5 Model Validation . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.3.6 Data Procurement . . . . . . . . . . . . . . . . . . . . . . . . 52 3.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.4.1 Synthetic Poisson Inputs . . . . . . . . . . . . . . . . . . . . . 53 3.4.2 Convergence & Overtting . . . . . . . . . . . . . . . . . . . 54 3.4.3 Model Robustness . . . . . . . . . . . . . . . . . . . . . . . . 57 3.4.4 Correlated Inputs . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.4.5 Real Hippocampal Data . . . . . . . . . . . . . . . . . . . . . 61 3.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 3.6 Relation of PBV Kernels with Poisson-Wiener Kernels . . . . . . . . 69 3.7 Derivation of Wiener Kernels for Correlated Gaussian Inputs . . . . 71 4 Understanding Spike Triggered Covariance using Wiener Theory for Receptive Field Identication 75 4.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 4.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 4.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 4.3.1 Wiener-Volterra Kernels . . . . . . . . . . . . . . . . . . . . 80 4.3.2 STC & Wiener Kernels . . . . . . . . . . . . . . . . . . . . . 83 4.3.3 Principal Dynamic Modes . . . . . . . . . . . . . . . . . . . 86 4.3.4 Correcting for Correlated Inputs . . . . . . . . . . . . . . . . 90 4 4.3.5 Associated Nonlinear Functions . . . . . . . . . . . . . . . . 92 4.3.6 Simulations & Model Assessment . . . . . . . . . . . . . . . 94 4.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 4.4.1 Illustrative Cases . . . . . . . . . . . . . . . . . . . . . . . . 96 4.4.2 General Trends . . . . . . . . . . . . . . . . . . . . . . . . . 100 4.4.3 Correlated Inputs . . . . . . . . . . . . . . . . . . . . . . . . 103 4.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 5 In-Vivo Predictive Relationship from CA1 to CA3 in the Rodent Hippocampus 116 5.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 5.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 5.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 5.3.1 Experimental Protocols and Data Preprocessing . . . . . . . 119 5.3.2 Model Conguration and Estimation . . . . . . . . . . . . . . 121 5.3.3 Model Selection . . . . . . . . . . . . . . . . . . . . . . . . . . 122 5.3.4 Model Validation . . . . . . . . . . . . . . . . . . . . . . . . . 123 5.3.5 Kernel Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 124 5.3.6 Principal Dynamic Mode Analysis . . . . . . . . . . . . . . . 125 5.3.7 Statistical Analysis . . . . . . . . . . . . . . . . . . . . . . . . 125 5.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 5.4.1 Estimated Models . . . . . . . . . . . . . . . . . . . . . . . . 127 5.4.2 General Trends . . . . . . . . . . . . . . . . . . . . . . . . . . 127 5.4.3 Bidirectional Predictive Power & Dynamics . . . . . . . . . . 129 5.4.4 Bidirectional PDM Analysis . . . . . . . . . . . . . . . . . . . 133 5.4.5 Theta and Gamma Power . . . . . . . . . . . . . . . . . . . . 135 5.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 5 5.5.1 CA1!CA3 In uence . . . . . . . . . . . . . . . . . . . . . . . 136 5.5.2 Modeling Methodology . . . . . . . . . . . . . . . . . . . . . . 140 5.5.3 Rhythms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 5.5.4 Potential Applications . . . . . . . . . . . . . . . . . . . . . . 143 6 Single-Stream Closed-Loop Modeling of the Hippocampus 145 6.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 6.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 6.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 6.3.1 Experimental Protocols & Data Preprocessing . . . . . . . . 149 6.3.2 Closed-Loop Point-Process (CLPP) Model . . . . . . . . . . . 149 6.3.3 Feedforward Input-Output Models . . . . . . . . . . . . . . . 150 6.3.4 Random Narrowband Spike-Train Generation . . . . . . . . 154 6.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 6.4.1 Estimated Volterra-PDM subsystems . . . . . . . . . . . . . . 155 6.4.2 Stimulation with Model Prediction Residuals . . . . . . . . . 156 6.4.3 Stimulation with Random Poisson Inputs . . . . . . . . . . . 157 6.4.4 Aect of Trigger Threshold on Resonant Modes . . . . . . . 162 6.4.5 Stimulation Testing . . . . . . . . . . . . . . . . . . . . . . . 163 6.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 6.5.1 Closed-Loop Modeling . . . . . . . . . . . . . . . . . . . . . . 169 6.5.2 Hippocampal Reentrance . . . . . . . . . . . . . . . . . . . . 171 6.5.3 Theta Emergence . . . . . . . . . . . . . . . . . . . . . . . . 172 6.5.4 Epilepsy and Seizure Abatement . . . . . . . . . . . . . . . . 174 7 Designing Patient-Specic Optimal Neurostimulation Patterns for Seizures from Human Single Unit Hippocampal Data 178 6 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 7.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 7.2.1 Experimental Setup and Preprocessing . . . . . . . . . . . . . 181 7.2.2 Modeling and Analysis . . . . . . . . . . . . . . . . . . . . . . 183 7.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 7.3.1 Reconstructed Neuronal Network . . . . . . . . . . . . . . . . 191 7.3.2 Seizure Initiation and Classication . . . . . . . . . . . . . . 195 7.3.3 Identifying Optimal Neurostimulation for Seizure Abatement 198 7.3.4 Responsive Neurostimulation . . . . . . . . . . . . . . . . . . 202 7.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 7.4.1 RNN for Neurostimulation Design . . . . . . . . . . . . . . . 204 7.4.2 Modeling Methodology and Limitations . . . . . . . . . . . . 206 7.4.3 Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 7.5 Supplementary Figures . . . . . . . . . . . . . . . . . . . . . . . . . 209 References 215 7 List of Figures 2.1 Hippocampal Anatomy Schematic . . . . . . . . . . . . . . . . . . . 16 2.2 Hippocampal Reentrance after EC Lesion . . . . . . . . . . . . . . . 18 2.3 Hippocampal Reentrance in Epileptic Slices . . . . . . . . . . . . . . 22 2.4 CA3 stimulation prevents reentrant ictal activity . . . . . . . . . . . 23 2.5 Reentrance observed in-vivo during sleep states . . . . . . . . . . . . 26 2.6 CA1 Theta resonance . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.7 Theta phase in multiply recorded hippocampal regions . . . . . . . . 31 2.8 Theta Reset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.1 PBV and Volterra Schema . . . . . . . . . . . . . . . . . . . . . . . . 55 3.2 Comparison of 3 Models . . . . . . . . . . . . . . . . . . . . . . . . . 56 3.3 Convergence and overtting analysis . . . . . . . . . . . . . . . . . . 58 3.4 Robustness Analysis 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 59 3.5 Robustness Analysis 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.6 PBV Correlated Inputs . . . . . . . . . . . . . . . . . . . . . . . . . 61 3.7 PBV for Hippocampal Data 1 . . . . . . . . . . . . . . . . . . . . . . 63 3.8 PBV for Hippocampal Data 2 . . . . . . . . . . . . . . . . . . . . . . 64 4.1 STC Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 4.2 Comparison of Methods . . . . . . . . . . . . . . . . . . . . . . . . . 110 8 4.3 Adelson-Bergen Model . . . . . . . . . . . . . . . . . . . . . . . . . . 111 4.4 RF Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 4.5 STC vs PDM example . . . . . . . . . . . . . . . . . . . . . . . . . . 113 4.6 Convergence & Overtting analysis . . . . . . . . . . . . . . . . . . . 113 4.7 Correlated Inputs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 4.8 Correlated Inputs 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 5.1 Hippocampal anatomy schematic . . . . . . . . . . . . . . . . . . . . 120 5.2 Model conguration schema . . . . . . . . . . . . . . . . . . . . . . . 123 5.3 Representative CA1!CA3 model . . . . . . . . . . . . . . . . . . . . 128 5.4 Monte Carlo simulation example . . . . . . . . . . . . . . . . . . . . 129 5.5 All signicant kernels . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 5.6 Kernel power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 5.7 Bidirectional predicitve power comparison . . . . . . . . . . . . . . . 132 5.8 Connectivity grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 5.9 Timecourse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 5.10 PDMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 5.11 Band power dierences . . . . . . . . . . . . . . . . . . . . . . . . . . 137 5.12 Granger causality schematic . . . . . . . . . . . . . . . . . . . . . . . 138 6.1 Closed-loop model schema . . . . . . . . . . . . . . . . . . . . . . . . 151 6.2 Open-loop model conguration. . . . . . . . . . . . . . . . . . . . . . 152 6.3 Volterra-PDM Subsystem Model . . . . . . . . . . . . . . . . . . . . 154 6.4 Obtained Volterra-PDM Subsystems . . . . . . . . . . . . . . . . . . 156 6.5 Bidirectional Predictive Power . . . . . . . . . . . . . . . . . . . . . 156 6.6 CLPP System driven by residual input . . . . . . . . . . . . . . . . . 158 6.7 CLPP Model driven by broadband inputs . . . . . . . . . . . . . . . 159 9 6.8 Comparison of Open-Loop and Closed-Loop systems driven by iden- tical broadband inputs e3 and e1. Note robust theta rhythms only arise in the closed-loop case. . . . . . . . . . . . . . . . . . . . . . . . 160 6.9 Synthetic cosine CLPP system . . . . . . . . . . . . . . . . . . . . . 161 6.10 Aect of modifying trigger threshold on CLPP output . . . . . . . . 163 6.11 Eect threshold has on synthetic cosine system . . . . . . . . . . . . 164 6.12 CLPP system driven by random narrowband inputs . . . . . . . . . 165 6.13 Single-Input Closed Loop systems . . . . . . . . . . . . . . . . . . . . 167 6.14 RNB vs RIT response comparison . . . . . . . . . . . . . . . . . . . 168 7.1 Graph & Example System . . . . . . . . . . . . . . . . . . . . . . . . 195 7.2 Seizure Induction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 7.3 Simulated Annealing Results . . . . . . . . . . . . . . . . . . . . . . 200 7.4 Simulated Annealing Monte Carlo Analysis . . . . . . . . . . . . . . 202 7.5 Responsive Stimulation . . . . . . . . . . . . . . . . . . . . . . . . . 203 7.6 Human Electrophysiology . . . . . . . . . . . . . . . . . . . . . . . . 210 7.7 Regularization Path . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 7.8 All NET Kernels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 7.9 Simulated Annealing Results . . . . . . . . . . . . . . . . . . . . . . 213 7.10 Periodic Stimulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 10 Chapter 1 Introduction The rst chapter of this manuscript is a review of various biological topics which will be covered, including hippocampal reentrance, hippocampal theta rhythms, and deep brain stimulation (DBS) for epilepsy. The remainder of the manuscript consists of ve self-contained chapters, each in the format of a journal article. This ve chapters can be divided into two parts. The rst deals with novel theoretical insights into nonparametric modeling of point-process systems such as those found in the brain. If one takes the reductionist view of the brain, at a certain level of abstraction, the brain can be seen as a point-process information processor. Specically, the current neuroscience dogma states that all information in the brain is encoded is the precisely timed sequences of action potentials in populations of neurons. This leads to several prominent questions amongst computational neuroscientists: How is sensory experience transformed into binary spiketrains? Can we decode those binary spiketrains to learn about the outside world? What is the dynamic interrelationship between dierent neurons? The second chapter deals with the question: how is information transformed between dierent neurons. Or, on a more abstract level, how can one identify the 11 mathematical transformation between two (input and output) binary sequences by observing only their spiketimes. Our group at USC has done much research on this topic and has pioneered the use of many nonlinear Volterra methods for this purpose. However, one limitation of much Volterra modeling is its inaccessibility to nonspe- cialists. This chapter, which is published in the Journal of Neuroscience Methods (Sandler et al., 2015), presents the probability-based Volterra (PBV) kernels, which are meant to be a robust and interpretable versions of the aforementioned Volterra kernels. On a deeper level, these kernels are shown to be equivalent to the previously explored Wiener kernels when both input and output are binary timeseries. The PBVs are shown to compare quite favorably to multiple existing methods on both synthetic and real hippocampal data. The third chapter deals with the question of how sensory information is encoded into neural sequences. Specically, many groups have used a computational method called Spike-triggered covariance (STC) to identify the receptive elds of both visual and auditory neurons. There has been debate in the literature regarding the theoret- ical foundation of the method and when the method may be properly applied. This chapter derives an explicit relationship between STC and Wiener kernels - showing that the former is simply a modied version of the latter. This deep connection thus allows STC to benet from all the prior theoretical convergence work done with Wiener kernels and makes a bridge between several previously unconnected concepts in the literature such as STC and principal dynamic modes (PDMs). The results of this chapter have recently been accepted for publication in the Journal of Vision. The remainder of the manuscript applies the nonparametric modeling techniques described above towards actual applications in neuroscience. The fourth chapter (Sandler et al., 2014) uses Volterra modeling to infer connectivity in the CA3!CA1 12 pathway. Although several studies have explored the causal connection from CA1 to CA3 on a network level, using local eld potentials, and usually in the context of epilepsy, this is the rst study to explore this connection on a single neuron level in the nonpathologic, nonperturbed brain. Understanding this long neglected pathway has the potential to critically add to our understanding of navigation, memory, and epilepsy. The last two chapters present, what is in my opinion, the most important work I have done while at USC: using data-based modeling to design a realistic hippocampal network to serve as a testbed for designing ecient neurostimulation patterns for the abatement of seizures. The fth chapter presents the initial work towards this goal. It builds of the work of Chapter 4, and studies more in depth the closed-loop nature of the hippocampus via a fully closed-loop point process (CLPP) model between CA3 and CA1. In this model, it was found that spontaneous theta rhythms emerge akin to epileptic seizures. Then stimulation patterns are identied which suppress this seizure-like activity. The nal chapter builds of the previous chapter by incorporating a large network of 24 rather than 2 neurons, thus allowing the generation of much more complex emergent seizure like activity, and much more realistic design of unsynchronized independant neurostimulation patterns. This is perhaps my most clinically relevant work, and I believe has the potential to truly help many suering from intractable epilepsy. 13 Chapter 2 Review of Relevant Biological Topics 2.1 Hippocampal Reentrance 2.1.1 Introduction No structure in the brain is as implicated in memory as the hippocampus. The primary input and output of the hippocampus is the entorhinal Cortex (EC), which acts as a relay between the hippocampus and the neocortex. According to several theories of hippocampal function, sensory information is fed to the hippocampus via the EC, which leads to short-term/working memory. Then this information is fed back to the neocortex via the EC in order to be consolidated into long-term memory (Buzs aki, 1996). However, in addition to sending hippocampal information back to the neocortex, the EC also relays this information back to the hippocam- pus in the so called hippocampal-entorhinal loop. Given the centrality of the both the hippocampus and EC in memory formation and consolidation, this reentrance pathway is presumed to have an important functional role. However, given the 14 vast amount of scientic literature on the hippocampus (over 100,000 articles on PubMed), relatively few studies have dealt with hippocampal reentrance. Most studies on hippocampal information ow focus on the traditional trisynaptic path- way by which information ows from the EC through the hippocampus and ignore the feedback connections which turn the trisynaptic pathway into a closed-loop. The traditional view of hippocampal information processing is the trisynaptic pathway by which information ows from the EC to the Dendate Gyrus (DG) via the Per- forant path, from the DG to CA3 through the mossy bers, and from CA3 to CA1 through the Schaer collateral path. Most studies on the hippocampus focus on this feedforward pathway and largely ignore the feedback connections which turn the hippocampus into a closed-loop system. Why have these vital connection been ignored? Depending, on the path taken, there are three to ve synapses connectnig CA1 to CA3 through the EC. To put this number in perspective, there are less synapses going from a sensory neuron in the ngers to the primary somatosensory area in the brain. Thus, the complicated multisynaptic nature of the hippocampal reentrance pathway has made it dicult for neuroscientists to understand the role of this pathway both qualitatively and quantitatively. The obvious approach of study- ing this relationship under controlled conditions in in-vitro slice preparations has proven challenging due to the diculty in preparing slices where the hippocampal- entorhinal connections remain intact (Barbarosie and Avoli, 1997). In-vivo studies are also challenging due to the diculty of recording activity from all the various structures involved, including multiple layers of the EC and virtually all parts of the hippocampus. Every study which has dealt with this phenomena has studied it on a population level using local eld potentials rather than single neuronal ac- tivity. While this approach can help elucidate the role of this pathway in highly synchronized hippocampal states like slow wave sleep and epilepsy, it leaves open 15 the question of the functional signicance of this pathway in the nonpathological nonperturbed brain. In this section we will review the relevant literature to date on the hippocampal reentrant pathway. In particular, we will focus on three questions: what is the anatomy and physiology of hippocampal reentrance? Specically, what paths does information leaving the hippocampus take when reentering the hippocampus. What role does the hippocampal closed-loop play in epilepsy? Finally, what role does hippocampal reentrance play in learning and memory? Figure 2.1: (A) Horizontal rodent hippocampal slice showing anatomical locations of the areas dealt with in our model. (B) Detailed schematic representation of anatomical con- nectivity of hippocampus. Black and red lines show excitatory and inhibitory connections, respectively. Notice that information from CA1 may enter CA3 via a direct inhibitory con- nection (bold red line), or via multisynaptic paths through the entorhinal Cortex (bold black line) 16 2.1.2 Anatomy & Physiology The rst hints on hippocampal reentrance came from anatomical studies which used various staining methods to characterize the interwoven neuronal connections be- tween the hippocampus, subiculum, and EC (Fig. 5.1b). The rst such study in rodents was done by Hjorth-Simonsen (1971) which discovered CA3 pyramidal cells projecting to the EC. Later studies found hippocampal CA3, CA1, and subicular pyramidal cells projecting to the deep layers of the EC (Swanson and Cowan, 1977; Swanson, Wyss, and Cowan, 1978; Srensen and Shipley, 1979). An important later anatomical study was done by Tamamaki and Nojyo (1995) which showed that the topography of the hippocampal reentrant loop is preserved - ie, that information coming from a local subpopulation of EC neurons will reenter that same local sub- population and not any other. This feature is essential if reentrance is to be studied in a closed-loop model on the single neuron level as is proposed in this study. Also, it should be noted that Sik et al. (1994) found backprojecting interneurons from CA1 to CA3. Although these interneurons do not contribute to reentrance per say, their eects cannot be distinguished from reentrant phenomena in the context of causality when only recording from CA3 and CA1 as is done here. Physiological studies of reentrance have mostly recorded local eld potentials (LFPs) in-vivo from hippocampal and EC subregions using 1-2 depth electrodes. The rst such study was done by Deadwyler et al. (1975) who showed that induced population spikes in CA3 will spread through the EC and reenter the Dentate Gyrus (DG), thus conrming the reentrance hypothesis which at the time was based only on anatomy. Deadwyler conrmed that the reentrant pathway was through the EC by showing that when the EC was chemically lesioned, reentrance no longer occurred (Fig. 2.2). He also quantied the loop-time, or the time it takes for activity from any given area in the hippocampal-entorhinal loop to reenter that area. The loop- 17 time was found to be 18-25ms. This was later conrmed by Buzs aki (1989) and Wu, Canning, and Leung (1998). Figure 2.2: [Adopted from Deadwyler et al. (1975)]. Aect of EC lesion on reentrance. Left column shows eld potential responces to commisural stimulation at various depths of the DG granule cell layer (CL). Arrows point to reentrant phenomena. Right and middle columns show same recordings after EC lesion. As can be seen, the late reentrant activity no longer occurs because the path from CA1 back to the DG through the EC is no longer viable. Due to an incomplete understanding of hippocampal-entorhinal anatomy at the time, Deadwyler attributed reentrance to the direct CA3!EC projections (Hjorth- Simonsen, 1971). The next major physiological reentrance study came from Finch et al. (1986) who stimulated multiple areas in-vivo, including the DG, CA3, and Subiculum, and measured the amount of time electrical volleys from those areas took to reach the EC. This study was the rst to propose that reentrance may si- multaneously take many pathways, and thus the hippocampus should not be seen as a simple closed-loop structure, but rather a heavily nested loop structure. The con- current operation of multiple reentrant paths was conrmed in several other studies 18 (Kloosterman, Haeften, and Silva, 2004; Wu, Canning, and Leung, 1998; Bartesaghi and Gessi, 2003). Brie y, the path information takes in the hippocampal-entorhinal loop make be broken down into two strands. First, the information must leave the hippocampus and enter the EC. This may happen by several routes including CA3!EC (Deadwyler et al., 1975), CA3!CA1!EC (Tamamaki and Nojyo, 1995), or CA3!CA1!Subiculum!EC. Second, information must leave the EC and reen- ter the hippocampus. This may happen through the traditional trisynaptic path- way, via the DG (Stringer and Lothman, 1992; Wu, Canning, and Leung, 1998), or through the temporoammonic pathway which connects the EC directly with CA1 (Bartesaghi and Gessi, 2003; Jones, 1993; Yeckel and Berger, 1990). Signicant work by Kloosterman, Haeften, and Silva (2004) characterized the dynamics between the deep layers (IV & V) of the EC where hippocampal inputs arrive and the supercial layers (II III) from whence EC stellate cells project to the hippocampus. Finally, several studies by Bartesaghi were signicant for making a rst step in modeling the hippocampal-entorhinal loop by means of tting the loop synapses with sigmoid functions (Bartesaghi, Gessi, and Migliore, 1995; Bartesaghi, Migliore, and Gessi, 2006). The aforementioned physiological studies on hippocampal-entorhinal interac- tions, spanning four decades, have made much progress. In particular they have conrmed that not only can information leaving the hippocampus return to the hip- pocampus via the EC, it can also reverberate in the hippocampal-entorhinal loop. They have also characterized the various pathways information can take within the loop and quantied the minimal amount of time information takes to get around the loop. However, these studies are also all fundamentally limited due to the elec- trophysiological techniques they use to observe neural activity. Namely, all these studies have used depth electrodes to monitor LFP activity after electrical stimula- 19 tion. LFPs are a measure of population synchrony and appear when large groups of neurons depolarize or re together such as during slow-wave sleep, seizures, and after electrical stimulation (Buzs aki, Anastassiou, and Koch, 2012). During normal awake states, however, the hippocampus is dominated by independant local compu- tations and large scale neural synchrony is absent (with the exception of neuronal rhythms such as theta and gamma). Thus, LFP recordings cannot be used to study hippocampal-reentrance in the nonpathologic, nonperturbed brain. Here, other techniques must be used. One promising method is optical imaging which can be used to record the activity of every point in a 2D slice preparation. Such methods have been used to observe activity in hippocampal-entorhinal slice preparations and have conrmed that after stimulation neural activity will reverber- ate in the EC and in the hippocampal-entorhinal loop (Iijima et al., 1996; Ezrokhi et al., 2001). As this technology improves it may provide enough spatial resolution to monitor individual neurons. However, it will still be limited to in-vitro slice prepara- tions. Another promising method is the use of multi-electrode arrays (MEAs) such as those used in this study. MEAs are promising since they can simultaneously record several neurons in multiple regions. Even though, the amount of neurons MEAs are able to record are but a small fraction of those found in the hippocampal subregions, the fact that topography is preserved around the hippocampal-entorhinal loop may mean that one could potentially record from all the loop neurons in-vivo. 2.1.3 Relation to Epilepsy Epilepsy is the condition of chronic seizures, which in turn are characterized by pathological hypersynchronous activity in neural populations. The traditional view of partial epilepsy maintains that seizures occur in the seizure focus and then spread to adjacent regions by overcoming those regions' natural inhibitory mechanisms such 20 as GABAerbic interneuron networks (Westbrook, 2000). One of the most common areas of seizure initiation is the hippocampus (Bertram, 2013). This has motivated the hypothesis that the hippocampal-entorhinal loop may become unstable and lead to the aberrant hypersynchronous oscillations which characterize seizures. The rst to suggest this hypothesis was Buzs aki et al. (1991) who lesioned the subcortical connections of the hippocampus in-vivo and observed the emergence of spontaneous oscillations along the hippocampal-entorhinal loop. It was hypothe- sized that subcortical inputs, such as those from the thalamus, modulate the loop, and in their absence, the loop may easily degenerate into an unstable state. The rst to specically study the hippocampal-entorhinal loop in the context of epilepsy was Pare, Llinas, et al. (1992). There, an in-vitro epilepsy model was constructed by inducing ictal (seizure-like) activity in hippocampal-entorhinal slices via tetanic stimulation of the perforant path. It was found that ictal activity initiated in CA3 and then spread to the EC (see Fig. 2.3). However, a fullscale hypersynchronous seizure only developed when the ictal activity spread to the DG and back to the hip- pocampus. Thus, the DG was hypothesized to be a sort of 'hippocampal gatekeeper' whose high threshold prevented pathologic activity from entering the hippocampus from the EC (see also (Yeckel and Berger, 1990; Jones, 1993)). A similar study was repeated by Stringer and Lothman (1992) in-vivo. In that study it was shown that Mossy Fiber lesions, which 'open' the hippocampal-entorhinal loop, prevented seizures from developing in the hippocampus. Later studies by Avoli and colleagues drew into question the unstable nature of the hippocampal-entorhinal loop. Using chemical models of epilepsy (4-AP and Pilo- carpine) which suppress natural hippocampal inhibitory mechanisms, they were able to successfully induce ictal and interictal activity in in-vitro hippocampal-entorhinal slice preparations (Barbarosie and Avoli, 1997; Nagao, Alonso, and Avoli, 1996). 21 Figure 2.3: [Adopted from Pare, Llinas, et al. (1992)]. (A,B) Simultaneous recordings ob- tained from several regions along hippocampal-entorhinal loop. Activity was recorded during ictal activity induced by tetanic stimulation. As can be seen in (B) ictal activity sequentially oscillates along the loop from CA3!CA1!EC!DG. Interictal events are small bursts of hypersynchronous activity between bouts of seizures; their functional role and clinical signicance is still under debate (Fisch, 2003). In these studies it was found, contrary to Pare, Llinas, et al. (1992), that interictal activity originates in CA3, while ictal activity originates in the EC. Fur- thermore, the interictal activity prevented ictal activity from the EC from spreading into the hippocampus. Thus, the hippocampal-entorhinal loop was found to be a defense against seizures. This can be seen in Fig. 2.4 where Schaer-Collateral 22 (SC) lesions (which disconnected CA3 from CA1) actually induced ictal activity by opening the hippocampal-entorhinal loop. Finally, it was found that after the SC lesion, 1Hz stimulation to CA1, as a surrogate for CA3 interictal activity could suc- cessfully ward of seizures. This successful treatment motivates the concept that the hippocampal-entorhinal loop can be exogenously disturbed/manipulated by electri- cal stimulation in order to produce desirable eects such as seizure abatement. In recent years there has been a huge eort to apply this concept clinically in the form of DBS (Sun, Morrell, and Wharen Jr, 2008). Figure 2.4: [Adopted from Barbarosie and Avoli (1997)]. Aects of Schaer collateral lesion on Seizure frequency. (A) Recording of CA3, EC, and DG in in-vitro 4-AP epilepsy model. Left shows interictal activity in prelesioned slice. Middle shows that after SC lesion ictal activity develops in DG. Right shows that after EC lesion, no more ictal activity is observed. (B) Expanded traces of recordings shown in A. Although Barbarosie and Avoli and Pare, Llinas, et al. obtained good results in in-vitro preparations, the role of the hippocampal-entorhinal loop in actual in-vivo epilepsy is much less clear. An important study by Bragin et al. (1997) showed that 23 in-vivo ictal activity did not just oscillate around the hippocampal-entorhinal loop in a xed, consistent manner as supposed by the previous studies. In the study, epileptic activity was induced in-vivo by means of tetanic stimulation of the per- forant path, similar to Stringer and Lothman (1992). Here, however it was found that hypersynchronous reverberations were randomly initiated in several areas. Fur- thermore, seizures would terminate in dierent areas at dierent times. Finally, EC lesions did not abolish seizures as presupposed by the previous works, but simply altered their routes through the loop. The above study suggests that to fully understand hippocampal-entorhinal seizure activity, the closed-loop paradigm is not enough; rather, one must take into account the highly complex nested loop structure of the hippocampus (Fig. 5.1). One should consider four hierarchal levels. First, on a cellular level, individual interneu- rons and pyramidal cells are capable of busting behavior which can be induced by the extracellular medium (Jeerys et al., 2012). Second, intraregional interactions between interneurons and pyramidal cells can give rise to unstable dynamics and lead to aberrant oscillations (Lopantsev and Avoli, 1998) Third, as emphasized by the previously discussed studies, interregional interactions can spread and sustain ictal activity. The hippocampal-entorhinal loop is a prime example of this. Finally, interactions between the hippocampus and other structures, particularly subcortical structures such as the Thalamus must be considered (Bertram, 2013; Buzs aki et al., 1991). Although, a full understanding of hippocampal seizures must incorporate dynamics on all the hierarchal levels, such a model is currently beyond the reach of current scientic knowledge. It is hoped that a partial understanding can lead to promising new therapies to abate seizures. In particular, in this study, only interre- gional interactions are explicitly taken into account to develop an in-silico epilepsy model which can hopefully elucidate more eective DBS strategies. 24 2.1.4 Relation to Learning & Memory The hippocampus has long been known to have a central role in memory formation and consolidation. The role of the hippocampus in declarative memory was rst established by the pioneering studies of Scoville and Milner (1957) on patient H.M. It was found that after hippocampal resectioning due to intractable epilepsy, H.M. developed complete anterograde amnesia and lost his ability to form new memories. Later, the work of O'Keefe and Dostrovsky (1971) showed the hippocampus also had a central role in spatial memory. The highly in uential study found that a specialized group of cells dubbed place cells selectively red only when the rodent was in a spatial 'receptive eld'. Later, other specialized cells were found including functional cell types (FCTs) which responded only to specic aspects of a predened behavioral task (Hampson, Simeral, and Deadwyler, 1999). Although less attention has been paid to the role of the entorhinal Cortex in memory, several studies have also identied specialized cells in that region including grid cells and head-direction cells (Hafting et al., 2005; Sargolini et al., 2006). Given the central role of both the hippocampus and EC in memory, virtually no studies have explored the role of reentrance in memory. This is presumably because learning is a nonpathological, subtle phenomena characterized by independent lo- cal computations which doesn't involve the hypersynchronous neural activity which characterizes seizures and sleep. The lack of hypersynchronous activity means that learning isn't amenable to electrophysiological methods which focus on recording LFPs, which thus far are the only techniques used to study reentrance and the hippocampal-entorhinal loop. However, some studies have provided hints of the role of reentrance in memory. Notably, Buzs aki (1989) was the rst to show that hippocampal reentrance depends on the animal's behavioral state. He showed that reentrance was much more likely to occur during resting states than exploratory 25 (theta-dominated) states. This suggests that behavioral states and environmen- tal stimuli can aect the path of information through the complex hippocampal- entorhinal nested loop. This was conrmed by later studies which showed that the reentrant phenomena are much more common during slow-wave (non-REM) sleep than during waking states (Fig. 2.5) (Zosimovskii, Korshunov, and Markevich, 2008; Zosimovskii and Korshunov, 2012). It should be noted however that these studies used LFPs as a measure of reentrant activity, and thus may not necessarily re ect reentrant activity during local computations on a single-neuron level. An interesting study on the single neuron level was done by Craig and Commins (2005) who showed the the CA1!EC pathway was highly prone to long-term potentiation, which has been implicated in learning and memory. Figure 2.5: [Adopted from Zosimovskii, Korshunov, and Markevich (2008)]. Recordings of rodent ECoG activity during awake (A) and sleep states (B). Notice that electric volleys delivered by commisural stimulation only show reentrance during sleep states. Even though few studies have dealt specically with the relationship of reen- trance and memory/learning, there exists much literature on the general role re- verberatory activity and reentrant circuits plays in memory. Hebb was the rst to hypothesize in his highly in uential 1957 book that persistent reverberatory activ- 26 ity between semi-unied 'assemblies' of neurons could be the biological substrate of short-term/working memory. Since Hebb, several anatomical and computational studies have supported this theory (Amit, 1995; Johnson, LeDoux, and Doyere, 2009; Kaplan, Sonntag, and Chown, 1991; Wolters and Raone, 2008; Lansner and Frans en, 1992; Wang, 2001) Although the hippocampal-entorhinal loop has a more complex anatomical structure than typical cell assemblies, it is optimally wired to support neural reverberations. Furthermore, there is intriguing evidence supporting the role of hippocampal-entorhinal interactions in working memory. In particular, Fell et al. showed recorded ECoG activity from human hippocampus and EC during a word recall task involving spatial memory (Fell et al., 2001; Fell et al., 2003). He found signicantly higher coherence between the hippocampus and EC during successful tasks, suggesting information was reverberating around the hippocampal- entorhinal loop. However, much more work needs to be done to understand the role of reentrance for learning and memory. In particular, the solution to this puzzle will necessarily have involve (1) new large-scale electrophysiological methods which can track in- formation ow through the loop on an individual neuron level, simultaneously in many neurons, and (2) sophisticated computational modeling which can elucidate and reverse-engineer the functional role of the loop. Recent advances in MEAs, such as those used in this study, will presumably help with the former; however, MEAs are unlikely to produce the spatial resolution needed to fully understand the loop. A promising solution is optogenetics and voltage-sensitive dying techniques which may not only record large numbers of specic neuron types, but also selectively perturb the loop (Buzs aki, Anastassiou, and Koch, 2012). It is our hope that the current study may provide a rst step in the sophisticated computational modeling of the hippocampal-entorhinal loop. 27 2.2 Hippocampal Theta Rhythm One of the major themes of modern neuroscience is expounding the role of neural rhythms in cognition and neural information processing. Neural rhythms are charac- terized by oscillatory activity seen in the electric eld surrounding neural tissues. At rst, this rhythmic activity was largely ignored as an insignicant epiphenomenon of complex neural interactions. Pioneering work by Green and Arduini (1954) was the rst to draw major attention to neural EEG rhythms by linking them with atten- tion. Later, as the biophysics of EEG and extracellular potentials were elucidated, it became apparent that neural rhythms are a macrophenomena which emerge from the synchronized activity of large groups of neurons. This led to theories which suggested that synchronized neural activity, the source of neural rhythms, may be the solution to the so-called 'binding problem' of consciousness (Crick and Koch, 1990). These theories were buttressed by several behavioral studies which correlated neural rhythms with attention, episodic memory, spatial memory, and other cogni- tive states. In our proposed study, we suggest that the entorhinal-hippocampal loop may be a contributor to hippocampal theta. Although a thorough review of neural rhythms is obviously beyond the scope of this work, we will brie y review some relevant aspects of neural rhythms. In particular, we will attempt to answer the questions of how the hippocampal theta rhythm arises and what are its behavioral correlates. 2.2.1 Theta: Mechanisms How do potentially hundreds of thousands of neurons bathed in the extracellular medium develop synchronized synaptic activity? Although the answer involves the complex interactions of several biological mechanisms, we will divide these mecha- nisms into three categories: intracellular mechanisms, intraregional mechanisms, and 28 interregional mechanisms. The primary intracellular mechanism for rhythmogenesis is neuronal resonance. This resonance arises from the biophysical properties of the cell membrane and ion channels. Namely, the cell membrane acts as a low-pass (ca- pacitive) component while certain rectifying channels like Potassium channels act as high-pass (inductive) components (Hutcheon and Yarom, 2000). The combination of these two components leads to cells which naturally resonate at specic frequencies. It has been shown that hippocampal CA1 cells have a natural resonant frequency in the theta range (Fig. 2.6) (Leung and Yu, 1998; Pike et al., 2000). Although this resonance doesn't explain spontaneous synchronized theta oscillations, it can explain how 'white' uncorrelated inputs can be transformed into theta-correlated outputs through these resonant cells. It has also been shown that hippocampal in- terneurons have higher resonant frequencies in the gamma range (Pike et al., 2000). Furthermore, it has been shown that the resonant frequencies of the cell soma and dendrites can be dierent and have their own complex interactions (Kamondi et al., 1998). Initially, it was thought that hippocampal theta was a result of projections from the Medial Septum, which is known to be a theta pacemaker (Colom, 2006). Later, it was found that various hippocampal and entorhinal regions can generate theta rhythms even in isolation. First, independent theta rhythms were found in the EC (Alonso and Garcia-Austt, 1987) which presumably arise from the natural theta res- onances of EC stellate cells (Gloveli et al., 1997; Haas and White, 2002; Schreiber et al., 2004). Then, it was found that CA3 could individually generate theta even when it was lesioned from the EC (Buzs aki, 2002; Kocsis, Bragin, and Buzs aki, 1999). Most recently, theta oscillations were observed in a completely isolated in- tact CA1 preparation (Goutagny, Jackson, and Williams, 2009). Signicantly, in that study it was found that when CA1 was lesioned into two halves, two indepen- 29 Figure 2.6: [Adopted from Pike et al. (2000)]. (A) ZAP current is characterized by a continuously changing frequency. (B) CA1 membrane responde to ZAP current. (C) CA1 response in frequency domain. Notice the resonant peak at the theta range (4 Hz). dent theta oscillations arose in each half, suggesting multiple theta pacemakers in each region which entrain each other. How do hippocampal/entorhinal subregions develop synchronized rhythms? The process is still far from understood, however, multiple processes are known to be involved. They include inherent neuronal reso- nances, network interactions between principal cells and interneurons (Rotstein et al., 2005), and other network interactions such as gap junction coupling (Whitting- ton and Traub, 2003). Each of the above subregions, which are endogenous theta pacemakers, are in- terconnected with each other and have complex interactions between their theta rhythms and phases. For example, CA1, the hippocampal region with the most prominent theta oscillations, has three inputs which oscillate at theta frequencies: entorhinal inputs on the CA1 str. lac. mol. layer, Schaer collateral inputs from 30 CA3 on the str. rad. layer, and septal inputs in the str. oriens layer (Buzs aki, 2002). These inputs interact with the endogenous intraregional CA1 theta mechanisms to develop the unique CA1 theta frequency and phase. Interestingly, even though the hippocampus is so interconnected, the various subregions do not entrain each other and maintain their own distinct theta phase dierences which cannot be accounted for by simple synaptic latencies (Mizuseki et al., 2009). Furthermore, interneurons and pyramidal cells within the same region have their own theta phase (Fig. 2.7). These ndings taken together suggest incredibly complex interactions which give rise and characterize neural synchrony. However, this complexity buttresses theo- ries which suggest that neural synchrony may play a very critical role in memory, navigation, and cognition, as will be discussed in the next section. Figure 2.7: [Adopted from Mizuseki et al. (2009)]. Average ring probability of hippocampal and entorhinal principal cells (left) and interneurons (right) recorded from rodent hippocam- pus. Note that the regional phase preferences do not simply re ect loop delays. Furthermore, principal cells and interneurons within the same region maintain their own unique phase preferences. Reference theta phase is that of EC layer 3. 31 2.2.2 Theta: Learning & Memory The rst study showing that hippocampal theta rhythms were linked with learning and memory was done by Winson (1978). The study lesioned the septohippocam- pal connections in rodents trained to a spatial maze task. It found that only those lesions which abolished hippocampal theta decreased the rodents' performance on the behavioral task. That same year, Berry and Thompson (1978) showed that theta oscillation band power before the performance of a behavioral task in rab- bits correlated well with performance on the task ( = :72). Later many other animal studies established the importance of theta oscillations for episodic (experi- ential) and spatial memory (Olvera-Cort es, Cervantes, and Gonzalez-Burgos, 2002; Chrobak, Stackman, and Walsh, 1989; Wiebe and St aubli, 2001). Of particular note was a study by Staubli and Xu (1995) which showed that chemically augmenting hippocampal theta oscillations led to improvements in behavioral task performance. Although at rst the signicance of theta in human learning and memory was con- troversial, many EEG and ECoG studies in epilepsy patients have since conrmed the that the two are related (Kahana, Seelig, and Madsen, 2001; Nyhus and Curran, 2010; Fell et al., 2001; Fell et al., 2003). The mechanisms by which theta rhythms contribute to learning and memory are still largely not understood and are beyond the scope of this work. The reader is referred to several useful reviews (Kahana, Seelig, and Madsen, 2001; Axmacher et al., 2006; Nyhus and Curran, 2010; Buzs aki and Moser, 2013). Here, we will brie y try to introduce a leading theory of theta function which involves two central concepts: theta reset and theta in uenced long term potentiation (LTP) (Axmacher et al., 2006). Most theories of hippocampal function assumes that the hippocampus encodes working and spatial memory by binding information from various sensory modalities in the form of potentiated synapses, and that this information is then 32 gradually fed to the neocortex for long-term memory consolidation (Buzs aki, 1996; Nyhus and Curran, 2010). In this paradigm, cells which are concurrently active during novel stimuli presentation 're together' and encode the memory through long-term potentiation in the classical Hebbian fashion (Hebb, 1949; Amit, 1995). Later, when the same stimuli presents itself, these cell assemblies are already po- tentiated and respond to the stimuli, thus facilitating memory retrieval (Axmacher et al., 2006). One of the open questions with this theory is how the hippocampus indices LTP in concurrently ring cells only during novel stimuli and not during all other awake states. One of the most promising answers is through the concept of theta reset. It has been shown both in animals (Givens, 1996) and humans (Tesche and Karhu, 2000) that the theta phase of the hippocampus 'resets' during the presentation of novel stimuli (Fig. 2.8). Such phenomena have also been explored in our group on a single cell level through the concept of "functional cell types" which are time- locked to various phases of the DNMS behavioral task (Hampson, Simeral, and Deadwyler, 1999). Although the mechanisms of this phase reset are unknown it has been hypothesized that it is induced via inputs from the EC and neocortex (Axmacher et al., 2006). Once a global theta rhythm is induced in the hippocampus during novel stimuli it can then facilitate LTP in concurrently ring cells (Axmacher et al., 2006). Theta has been shown to be the optimal oscillation frequency to induce LTP in several studies and in several locations including the perforant path (Deadwyler et al., 1975; Jones, 1993), the EC (Gloveli et al., 1997), and CA1 (Nyhus and Curran, 2010). Theta facilitates LTP by opening NMDA channels during its peak and thus causing a Calcium in ux into the cell which leads to the molecular cascade ending in potentiated synapses. (Huerta and Lisman, 1995). Thus, once an animal is 33 Figure 2.8: [Adopted from Givens (1996)]. Theta reset demonstrated in rodent hippocampus. Top three recordings show individual LFP responses in CA1. Bottom recording shows session average over N=173 trials. Time 0 represents stimulus presentation. Notice how it can be seen that the theta phase resets on stimulus presentation. in an anticipatory state to receive novel stimuli, the hippocampal theta phase is reset and those the concurrently ring cells will successfully encode the memory in the form of potentiated synapses (McCartney et al., 2004). It should once again, however, be emphasized that although the theory presented here has experimental and theoretical justication, the contribution of theta to learning and memory is still an open question with much ongoing research. 2.3 Deep Brain Stimulation for Epilepsy Epilepsy is a neurological disorder characterized by chronic seizures which aects 1-2% of the US population (Begley et al., 2000). The standard method of treatment is using antiepileptic drugs. However, upto 30% of patients do not respond to these drugs; of those who do, many suer serious side-eects such as nausea, dizziness, 34 drowsiness, and weight-gain (Brodie and Dichter, 1996). For these patients, the main option is resective surgery which aims to remove the epileptic focus. Not only are there large inherent acute and long-term risks of such a surgery, there is also a large remission rate within 1-2 years (Engel et al., 2003). Furthermore, many patients are not eligible for such a surgery since their epileptic focus is located in so-called 'eloquent areas' such as the motor cortex or language areas whose removal would result in irreparable cognitive decits. In recent years, neurostimulation has emerged as a promising approach to reduc- ing seizures. For many patients it is currently the only therapy which may alleviate their symptoms. Peneld and Jasper (1954) were the rst to report that electrical stimulation may abate seizures in humans. Since then several human and animal studies have explored this nding in in-vivo and in-vitro experiments. In humans, several stimulation sites have been explored including the cerebellum, anterior tha- lamus, centromedial thalamus, basal ganglia, temporal cortex, and vegal nerves (Saillet et al., 2009; Fisher and Velasco, 2014). Vagal nerve stimulation (VNS) was the rst such therapy to be approved by the FDA in 1997. Since then the FDA has approved the Neuropace R cortical neurostimulation device and is in phase 3 clinical trials (SANTE) for the MedTronic anterior thalamus neurostimulation device. Traditional DBS therapies, including the above mentioned Medtronic device, have used scheduled stimulation patterns which emit xed pulse trains irrespective of wether a seizure is occurring. More recently, responsive DBS therapies have emerged which use real-time signal processing to detect seizure onset and then emit electrical stimulation to abort the oncoming seizure. The Neuropace device, for example, emits a pulse train whenever ECoG activity crosses any of 3 physician dened thresholds. The device has been shown to reduce median seizure frequency by 54%, with 9% of patients being seizure-free for at least 6 months (Heck et al., 35 2014; Ben-Menachem and Krauss, 2014). It should be noted that this clinical trial was done on patients for whom all other options have failed. In order to qualify, each patient had to unsuccessfully have tried at least 2 epilepsy drugs and possibly had prior unsuccessful VNS or resective surgery. Although electrical stimulation for epilepsy has proven itself to be clinically ecacious, the mechanisms by which it works remain a mystery. Several distinct forms of electrical stimulation have been used in animal and human studies and several theories have been proposed for each method. Essentially the goal of all these theories is to show how injection of electrical current can have an inhibitory eect on network dynamics. Many animal studies have attempted to use phase-resetting and chaos control theory to abort seizures using well-places single pulses to breakup the synchronized neural oscillations which characterize ictal events; however, the clinical viability of this method is dubious (Durand and Bikson, 2001). Others have argued that low-frequency stimulation (LFS) below 2Hz may induce long-term depression (LTD) and thus decrease synaptic transmission and population synchrony (Barbarosie and Avoli, 1997). However, LFS has also fared poorly in human trials. High-frequency stimulation (HFS), which has fared the best in human trials, has perhaps the least understood mechanisms of action. It has been suggested that HFS raises the level of extracellular potassium, which in turn raises the neuronal baseline potential and induces depolarization block, a condition whereby the neuron is so depolarized (usually around -40mV) that it can no longer initiate action potentials (Durand and Bikson, 2001). Although several hypothesis for how electrical stimulation suppresses ictal ac- tivity have been proposed, a satisfying answer is still elusive. Furthermore, very few quantitative models have been proposed to elucidate the eects of neural stimula- tion on large neural populations. Two exceptions are a large scale neural network 36 model on the eects of single pulse stimulation on bursting activity (Anderson et al., 2009) and a neural mass model on the eects of xed frequency stimulation on the thalamocortical loop (Mina et al., 2013). A computational model may not only serve to elucidate the fundamental biological mechanisms of DBS, but also as- sist in parameter selection, which is currently done by physicians using brute-force approaches. Any computational model should aim to answer three unresolved DBS ques- tions. First, why does the optimal frequency for seizure suppression vary so greatly between dierent brain regions? Second, why do some frequencies initiate seizures while other frequencies suppress seizures. This phenomena has been noted in the rodent thalamus where LFS (8Hz) induces seizures, while HFS (100Hz) suppressed seizures (Mirski et al., 1997). Conversely, in the human caudate nucleus, LFS (4- 6Hz) suppressed seizures, while HFS (50-100Hz) induced seizures (Chkhenkeli and Chkhenkeli, 1997). It should be noted that using electrical stimulation to initi- ate seizures is known as kindling and is one of the most studied epilepsy models (Stringer and Lothman, 1992). Finally, how can seizures be suppressed by vastly dierent frequency parameters? For example, Kinoshita et al. (2005) showed that both scheduled LFS (1Hz) and HFS (50Hz) of the cortical focus in humans can reduce seizure frequency. These unresolved issues of seizure generation and suppression suggest highly complex networks of interacting elements at dierent hierarchies which are region specic. Such a network would be very dicult to model using a parametric ap- proach such as that used in Mina et al. (2013) due to the huge number of parameters and our limited understanding of the mechanisms at work. We propose that a much more ecient approach would be to employ a data-driven nonparametric model such as that described in chapter 6. A nonparametric approach has the benets of 37 not being based on any priori assumptions which may later on be disproven (Song, Marmarelis, and Berger, 2009). 38 Chapter 3 System Identication of Point-Process Neural Systems Using Probability Based Volterra Kernels 1 3.1 Abstract Background: Neural information processing involves a series of nonlinear dynami- cal input/output transformations between the spike trains of neurons/neuronal en- sembles. Understanding and quantifying these transformations is critical both for understanding neural physiology such as short-term potentiation and for developing cognitive neural prosthetics. New Method: A novel method for estimating Volterra kernels for systems with point-process inputs and outputs is developed based on elementary probability the- 1 This chapter has been published in Journal of Neuroscience Methods: Sandler et al. (2015) 39 ory. These Probability Based Volterra (PBV) kernels essentially describe the prob- ability of an output spike given q input spikes at various lags t 1 ,t 2 ,...t q . Results: The PBV kernels are used to estimate both synthetic systems where ground truth is available and data from the CA3 and CA1 regions rodent hippocam- pus. The PBV kernels give excellent predictive results in both cases. Furthermore, they are shown to be quite robust to noise and to have good convergence and over- tting properties. Through a slight modication, the PBV kernels are shown to also deal well with correlated point-process inputs. Comparison with Existing Methods: The PBV kernels were compared with ker- nels estimated through least squares estimation (LSE) and through the Laguerre expansion technique (LET). The LSE kernels were shown to fair very poorly with real data due to the large amount of input noise. Although the LET kernels gave the best predictive results in all cases, they require prior parameter estimation. It was shown how the PBV and LET methods can be combined synergistically to maximize performance. Conclusions The PBV kernels provide a novel and intuitive method of charac- terizing point-process input-output nonlinear systems. 3.2 Introduction Information in the nervous system is encoded in precisely timed sequences of neu- ronal action potentials. A central goal of computational neuroscience is to under- stand how these series of spike trains are processed in the brain. In particular, three fundamental problems can be identied: how the brain maps external stimuli onto neural spike trains (encoding), how neural spike trains map onto bodily response (decoding), and nally how the spike trains of one neuronal population are mapped onto another. These problems are not just important for increasing our understand- 40 ing of the brain, but they also have immediate applications to neuroprosthetics and brain-machine interfaces which aim to replace damaged brain regions in order to restore lost cognitive function. In particular, the encoding problem is central to neuroprosthesis which aim to replace damaged sensory systems such as the visual (Weiland, Cho, and Humayun, 2011), auditory (Loeb, 1990), and vestibular systems (Di et al., 2010). The decoding problem is central to motor prosthesis which aim to compensate for lack of movement either by reactivating muscle movement (Loeb et al., 2001) or by controlling external articial actuators such as a computer cursors (Hochberg et al., 2006; Wolpaw and McFarland, 2004). Finally, the third problem is directly relevant to prosthesis which aim to replace lost cognitive function within the central nervous system and which have made much headway in recent years (Hampson et al., 2012b; Berger et al., 2012; Marmarelis et al., 2012). This prob- lem is distinct from the previous two in that both the input and the output signals are binary spike trains. Thus, this problem necessitates a highly robust and ver- satile modeling approach which may adequately describe the nonlinear dynamical transformation between these spike trains. Traditional physiological modeling has often revolved around parametric ap- proaches which incorporate a priori knowledge/assumptions about the system into model structure. Such models can be extremely useful and have successfully shed light on synaptic transmission and somato-dendritic integration (Dittman, Kreitzer, and Regehr, 2000; Dayan and Abbott, 2001; London and H ausser, 2005); however, their use is limited to environments where one may condently assume the underly- ing structure of the model. In the context of neuroprosthesis, when recording from arbitrary neurons in the brain, one may not make such assumptions since even if the neuronal presynaptic and postsynaptic dynamics are known, the anatomical con- nections between the recorded neurons are unknown. These connections are critical 41 to the input-output dynamics of the neurons and include the number and location of the synapses between them and whether they are connected directly, through intermediate neurons, or both. Thus, this problem necessitates a nonparametric approach which estimates the model directly from the input-output data records without making any a priori assumptions of the connections. An additional ben- et of the nonparametric approach is that the model will not change with future discoveries (Song, Marmarelis, and Berger, 2009; Song et al., 2009b). The nonpara- metric approach has a long history in sensory neuroscience and has been successfully applied to diverse areas such as the retina (Marmarelis and Naka, 1973), visual cor- tex (Rapela, Mendel, and Grzywacz, 2006; Touryan, Felsen, and Dan, 2005), and auditory cortex (Eggermont, 1993; Slee et al., 2005). The problem of modeling transformations between spike trains necessitates a nonlinear approach not only because this transformation is intrinsically nonlinear due to the presence of a threshold (Marmarelis, Citron, and Vivo, 1986), but also because nonlinear interactions are known to occur in the nervous system between pairs and triplets of input spikes (Dittman, Kreitzer, and Regehr, 2000; Song, Mar- marelis, and Berger, 2009). A powerful technique for characterizing nonlinear sys- tems is the Volterra/Wiener functional approach which has had a long history in biological physiomodeling (Marmarelis and Marmarelis, 1978; Marmarelis, 2004). Several adaptations of this technique have been developed to solve the encoding problem, where a continuous signal is mapped onto a spike train. In particular, much attention in recent years has been given to this approach to map the receptive elds of retinal cells in the form of spike-triggered covariance (STC) (Schwartz et al., 2006). Also, several adaptations have been developed for the decoding problem which maps a spike train input to a continuous output (Steveninck and Bialek, 1988; Krausz, 1975; Marmarelis and Berger, 2005). In the present paper, we propose a 42 Volterra functional style technique which is exclusively designed for systems with spike-train inputs and outputs. Although previous methods have attempted to solve this problem (Zanos et al., 2008; Song et al., 2007; Marmarelis et al., 2013b), they have been, with few exceptions (Marmarelis, Zanos, and Berger, 2009), methods designed for continuous data and later applied to spike train data. In this paper, we propose a nonparametric nonlinear Volterra-style model to de- scribe spike train transformations. Our approach uses elementary probability theory to estimate Volterra style functionals, termed the Probability Based Volterra (PBV) kernels, which describe the nonlinear dynamical transformation between input and output spike trains. In brief, the q th order PBV kernel describes the probability of an output spike givenq input spikes at various lagst 1 ,t 2 , ... t q . These PBV kernels are shown to be equivalent to the Poisson-Wiener Kernels, and thus are capable of describing arbitrary nite memory nonlinear spike train transformations. The PBV method is then compared with two other methods found in the literature of estimating Volterra kernels. These methods are evaluated in terms of predictive power, convergence, overtting, robustness to noise, and response to correlated in- puts. Finally, the methods are applied to real spike train data obtained from the rodent hippocampus. 3.3 Methods Nonparametric modeling approaches for point process systems aim to characterize a black box or unknown functional which transforms the input spike train into the output spike train. Characterizing a black box can be broken down in to the prob- lems of model conguration and parameter estimation. In physiological systems, the problem of model conguration is further constrained by the need for such models to be physiologically plausible and prone to interpretation. Several methods exist 43 for characterizing systems with point process inputs and outputs (Zanos et al., 2008; Song et al., 2007; Marmarelis, Zanos, and Berger, 2009; Marmarelis et al., 2013b). Here we chose the well-known Volterra framework to characterize our model and we developed a novel technique to estimate our Volterra kernel parameters which is customized to systems with point process inputs and outputs and is based on ele- mentary probability theory. The Volterra kernels estimated by this technique will be referred to as the Probability-based Volterra (PBV) kernels. 3.3.1 Model Conguration In the Volterra functional expansion of an arbitrary nonlinear system, the system output, y[t], is described as the sum of the outputs of a series of functionals with increasing orders of nonlinearity: y[t] = Q X q=0 F q (x[t];k q [ 1 ::: q ]) (3.1) Where F q (x[t];k q [ 1 ::: q ]) is the q th order Volterra functional, which is essentially the q th order convolution of the input x[t] with the q th order Volterra kernel. It should be noted that here the kernels and input have been discretized in order to handle real data, much like in traditional spike-triggered analysis. Each Volterra functional describes interactions between up to q input spikes. For example, the rst order kernel can be seen as weighing the contribution to the output of a single pulse t lags prior to the output. Thus, the rst order Volterra functional describes a linear system and the rst order Volterra kernel is equivalent to the well-known impulse response function. The second order Volterra functional, and the rst non- linear functional, describes second order nonlinear interactions between two input pulses, such as paired pulse facilitation and depression. Although theoretically we 44 could construct an innite order (Q=inf) system, in practice this is computationally impossible and physiologically unrealistic. Here, we curtail our system to a second order nonlinear system (Q=2) as higher order nonlinearities have been shown to pro- vide only slight improvements in modeling the nervous system (Song et al., 2007; Lu, Song, and Berger, 2011). Thus: ^ y c [t] =k 0 + M X =1 k 1 []x[t] + M X 1 =1 M X 2 =1 k 2 [ 1 ; 2 ]x[t 1 ]x[t 2 ] (3.2) Where ^ y c [t] is the predicted continuous output, andM is the system memory which describes how many lags of the input past aect the output. k 1 [] andk 2 [ 1 ; 2 ] are the 1st and 2nd order Volterra kernels which are equivalent to the PBV kernels if estimated by the method described herein. Because the above equation will gener- ate a continuous output and we are dealing with a point process system, we must threshold the continuous output, ^ y c [t] such that: ^ y[t] =Th(^ y c [t]) = 8 > > < > > : 1; if ^ y c [t]>T 0; if ^ y c [t]T (3.3) In this study, the threshold is set so that the number of spikes in the predicted spike train, y c [t], equals the number of spikes in the true spike train, y[t]. 3.3.2 Model Estimation Several methods of estimating Volterra kernels exist in the literature. Norbert Wiener was the rst to propose a method of estimating the kernels by orthogonal- izing the Volterra functionals with respect to Gaussian white noise (GWN) inputs, giving rise to the well-known Wiener Series (Wiener, 1966). The Wiener series was rst popularized when the computationally feasible cross-correlation technique was 45 introduced as a method of estimating the Wiener kernels (Lee and Schetzen, 1965; Marmarelis and Marmarelis, 1978). Since then several techniques have been devel- oped which allow ecient estimation of Volterra kernels using shorter data records and arbitrary inputs. Here we develop a novel and intuitive technique for estimat- ing the Volterra kernels for systems with point-process inputs and outputs which we call the Probability Based Volterra kernels. We later show that these kernels are essentially equivalent to the Wiener kernels specialized for point-process inputs. The PBV kernels are estimated in a two-step process. First the q th order condi- tional probability kernels (CPKs) are calculated. Then the PBV kernels are obtained by removing lower order eects from the CPKs. The q th order CPK is dened as the probability of an output spike given q input spikes at t 1 ,t 2 ,...t q lags prior to the output. The zeroth order CPK is dened as the probability of an output spike without any information of the input. Thus it is just the mean ring rate (MFR) of the output: CPK 0 =P (y[t]) = y (3.4) The rst order CPK is: CPK 1 [] =P (y[t]jx[t]) = P (y[t]\x[t] P (x[t]) (3.5) In this section we will assume that the point process input is an uncorrelated Poisson process with rate x. Later, in section 3.3.4 we will show how one may partially correct the kernels for correlated/colored inputs. With this in mind, eq. 5 becomes: CPK 1 [] = P (y[t]\x[t] P (x[t]) = E(y[t]x[t]) x (3.6) It should be noted that the 1st order kernel is a scaled version of traditional spike triggered average (STA) (De Boer and Kuyper, 1968). Similarly, the second 46 order CPK is: CPK 2 [ 1 ; 2 ] =P (y[t]jx[t 1 ]\x[t 2 ]) = P (y[t]\x[t 1 ]\x[t 2 ] P (x[t 1 ]\x[t 2 ]) (3.7) Notice thatCPK 2 is only dened whent 1 6=t 2 , since fort 1 =t 2 =t, the numerator, y[t]\x[t]\x[t] reduces to y[t]\x[t]. Thus, CPK 2 is set to 0 when t 1 =t 2 . The same constraint applies to all higher order CPKs when any pair of lags is identical. This can also be intuitively understood by noting that the diagonal of the second order Volterra kernel describes the contribution of the square of the input pulse. Since we are dealing with binary point process signals, the square of the input equals the input, and thus the diagonal of the 2nd order kernel actually describes 1st order (linear) eects. Once again, by assuming a Poisson process input, eq. 7 becomes: CPK 2 [ 1 ; 2 ] = 8 > > < > > : E(y[t]x[t 1 ]x[t 2 ]) x 2 ; for 1 6= 2 0; for 1 = 2 (3.8) Now, in order to obtain the PBV kernels lower order eects must be removed from the CPK kernels. This step can intuitively be understood since the conditional probability of a an output spike given a set of n 1 , n 2 ,...n q input spikes includes the probability of an output spike given every subset of lower order. For example, the probability of an output spike given an input spike includes the probability of an output spike ring spontaneously on its own. So, to isolate the rst order dynamics, we must remove the zeroth order dynamics. Thus: PBV 0 =CPK 0 (3.9) PBV 1 [] =CPK 1 []PBV 0 = E(y[t]x[t]) x y (3.10) 47 Similarly, to isolate second order eects, we must remove 0th and 1st order dynamics: PBV 2 [ 1 ; 2 ] =CPK 2 [ 1 ; 2 ]PBV 1 [ 1 ]PBV 1 [ 2 ]PBV 0 (3.11) 3.3.3 Relationship to Poisson-Wiener Kernels Marmarelis et al derived the Poisson-Wiener (PW) functional expansion for arbi- trary nonlinear systems with point process inputs and continuous outputs (Mar- marelis and Berger, 2005). The PW kernels are obtained by orthogonalizing the system nonlinearity with respect to Poisson inputs, akin to how the Wiener kernels are obtained by orthogonalizing the system nonlinearity with respect to Gaussian white noise. However, the mathematically sophisticated derivation of the P-W ker- nels has limited their utilization in the broader neuroscience community. In appendix 1, we show that the PBV kernels are essentially equivalent to the PW kernels, albeit with a dierence of scale: PW 1 [] = 1 (1 x) PBV 1 [] (3.12) PW 2 [ 1 ; 2 ] = 1 2(1 x) 2 PBV 2 [ 1 ; 2 ] (3.13) This equivalence serves two purposes. First it establishes a solid theoretical footing for the PBV kernels on the basis of Wieners orthogonalization approach. Second, it provides an optimal scaling of the PBV kernels to achieve maximal predictive power. However, it should be emphasized that the PBV kernels are distinct from the PW kernels since the PW kernels are exclusively designed for systems with uncorrelated Poisson inputs and continuous outputs, while the PBV kernels are designed for systems with point-process inputs and outputs. 48 3.3.4 PBV Kernels for Correlated Point-Process Inputs Although ideally, one would probe a system with uncorrelated inputs, much work has been done in recent years to develop system estimation methods for correlated inputs. This is useful because (1) sometimes there is no opportunity to stimulate the system, and all one can do is record the naturally occurring endogenous system activity (Berger et al., 2012), and (2) much work in sensory neuroscience has shown that one can estimate a sensory neurons receptive eld using much shorter data records by simulating the statistics of its natural stimuli, which usually have higher order statistical moments (Touryan, Felsen, and Dan, 2005). The PBV kernels are derived for uncorrelated point-processes (i.e. Poisson processes), and would have biases if they were estimated with correlated point- process inputs. Any attempt to analytically derive PBV kernels for correlated point-processes would be extremely dicult, if not impossible, due to the ill-dened analytical nature of such processes as they are used here. Korenberg (1990) has previously derived corrections for Wiener kernels estimated using correlated Gaus- sian inputs (see appendix 2). The method relies on the decomposition property of higher order moments of Gaussian processes, which does not apply to correlated point-process inputs. The main result of correction method is shown below: ! k 1 = 1 x ! k 0 1 (3.14a) K 2 = 1 x K 0 2 1 x (3.14b) Where ! k 0 1 and K 0 2 are the biased/naive Wiener 1st order kernel vector and 2nd order kernel matrix estimated using correlated inputs, ! k 1 andK 2 are the corrected Wiener kernels, and 1 x is the inverse of the Toeplitz autocorrelation matrix of x[t]. It should be noted that the correction in eq. 14a is identical to that derived 49 by (Theunissen et al., 2001) for STA. Although eq. 14 was derived for correlated Gaussian processes, we have found empirically that it can be used, with slight modication, to partially correct PBV kernels estimated from correlated point-process inputs. Namely, the modied form of eq. 14 is: ! PBV 1 = 1 x ! PBV 0 1 (3.15a) PBV 2 = 1 x PBV 0 2 (3.15b) Where once again the apostrophe indicates the biased PBV kernels. This correction implicitly imposes a smooth prior on the second order kernel by multiplying the true correction (eq. 14) with the input autocorrelation matrix. Imposing this smooth prior has been shown empirically to improve results (see Fig. 3.6,3.8). 3.3.5 Model Validation Two metric were used to evaluate the goodness-of-t of the estimated PBV models. ROC curves, which evaluate the performance of a binary classier, plot the true positive rate against the false positive rate over the putative range of threshold values for the continuous output, y (Zanos et al., 2008). The area under the curve (AUC) of ROC plots are used as a performance metric of the model, and have been shown to be equivalent to the Mann-Whitney two sample statistic (Hanley and McNeil, 1982). The AUC ranges from 0 to 1, with 0.5 indicating a random predictor and higher values indicating better model performance. The second metric used was the Pearson correlation coecient, , between the estimated prethreshold output, ^ y c [t], 50 and the true output, y[t]: y;^ yc = E[(y y)(^ y c m ^ yc )] y ; ^ yc (3.16) Where m ^ yc is the mean of ^ y c [t], and y and ^ yc are the variances of y[t] and ^ y c [t] respectively. The and AUC metrics were chosen as they measure the similarity between a continuous prethreshold signal and a spike train. The continuous prethreshold signal was chosen over adding a threshold trigger and comparing true output spike train with an output postthreshold spike train for two reasons. First, this allows us to avoid specifying the threshold trigger value, which relies on the somewhat arbitrary tradeo between true-positive and false-negative spikes (Marmarelis et al., 2012). Second, similarity metrics between two spike trains often require the specication of a binning parameter to determine the temporal resolution of the metric. (Victor and Purpura, 1997; Rossum, 2001). In order to compare the performance of the PBV model with other methods in the literature, we estimated the 2nd order Volterra model using two additional techniques. First, we estimated the kernels using the least squares estimation (LSE) technique (Korenberg, Bruder, and Mcllroy, 1988; Marmarelis, 2004; Lu, Song, and Berger, 2011). This method nds the kernels which minimizeE[(y ^ y c ) 2 ] by solving a matrix inversion problem. The standard LSE technique can be seen as projecting the input onto a delta function basis. An alternative, known as the Laguerre expan- sion technique (LET), projects the input onto a set of L Laguerre basis functions and once again use least squares estimation to obtain the basis coecients (Mar- marelis and Orme, 1993). The Laguerre basis set, which was rst suggested for use in physiological systems by Norbert Wiener Wiener, 1966, is dened on [0;1], and is composed of increasing oscillations followed by an exponential decay, as is typical 51 of many physiological systems. Indeed, this method, which has been successfully applied to several nonlinear physiological systems, has been shown to drastically re- duce the amount of free parameters needed in the 2nd order Volterra model (Valenza et al., 2013; Kang et al., 2010; Marmarelis, 2004). In order to use the LET, one must specify the alpha parameter of the Laguerre basis functions. Here, the ground truth alpha was used for synthetic systems and no eort was made to search for alpha. In real rodent systems, alpha and L were determined by tting the estimated 1st and 2nd order PBV kernels. This is a novel technique which has not been previously reported in the literature. An important advantage of both the LSE and LET tech- niques is that by solving the inverse problem they are implicitly able to deal with correlated inputs without requiring any additional processing steps; however, this feature comes at the expense of additional computational power. 3.3.6 Data Procurement In this study the PBV method has been applied to both synthetic systems where ground truth is available and to real data recorded from the rodent hippocampus. In order to generate condence bounds for the predictive power of the models when applied to synthetic data, a Monte Carlo approach was used with several trials (N=30) of randomly generated systems. Each system was obtained by randomly generating coecients for L=3 Laguerre basis functions for the rst and second order kernel. The output was generated by feeding a random point process input through the system and then thresholding the output so that the MFR of the output equaled the MFR of the input (i.e. y = x). To generate correlated point process inputs we fed GWN through a lowpass lter and then thresholded the output. Rodent hippocampal data was collected in the labs of Dr. Deadwyler and Dr. Hampson at Wake-Forest University and has been described in detail in our previous 52 publications (Hampson et al., 2012b). Brie y, N=10 Male Long-Evans rats were trained to criterion on a two lever, spatial Delayed-NonMatch-to-Sample (DNMS) task. Spike trains were recorded in-vivo with multi-electrode arrays implanted in the CA3 and CA1 regions of the hippocampus during performance of the task. Only neural activity from trials where the rat successfully completed the DNMS task was used. Spikes were sorted, time-stamped, and discretized using a 2 ms bin. Spike train data from 1s before to 3s after the sample presentation phase of the DNMS task was extracted and concatenated into one time series. 3.4 Results 3.4.1 Synthetic Poisson Inputs As an illustrative example consider the simulated 2nd order point process system shown in Fig. 3.1. Here, the input, x[t], shown in Fig. 3.1a is a 100,000 sample discretized Poisson process with probability x = 0:2 of ring. This input is fed through the 2nd order Volterra system in Fig. 3.1b and then thresholded to generate the outputy[t], shown in Fig. 3.1c. The input and output signals were then used to estimate the PBV kernels, shown in Fig. 3.1d. As can be seen, the estimated PBV kernels are able to reproduce the dominant features of the ground truth 1st and 2nd order kernels. For example, both 1st order kernels have two critical points, the rst negative and the second positive, followed by an exponential decay. However, the kernels have slight dierences. For example, the rst critical point of the ground truth kernel occurs at bin 3, while that of the estimated kernel occurs at bin 2. These dierences are not just the results of nite data records, but also due to the fact that the thresholding operator makes the system have an innite order nonlinearity, while the PBV kernels are truncated to the 2nd order. This truncation results in 53 slightly biased 1st and 2nd order kernels due to their absorbing the eects of higher order nonlinearities. As can be seen however, these biases are relatively minor. The input was fed through the estimated kernels to generate the predicted continuous output ^ y c [t]. As can be seen from Fig. 3.1e, the estimated prethreshold output compares quite favorably with the true spike train, y[t]. This can also be seen from the ROC plot in Fig. 3.2c, where the estimated PBV system had a near perfect AUC of .993. The output was then thresholded to produce the predicted spike train output, ^ y[t], shown in the bottom of Fig. 3.1e. The 2nd order Volterra model was also estimated using the LSE and LET meth- ods, as shown in Fig. 3.2. As can be seen, these methods were also able to suc- cessfully estimate the true kernels; however, like the PBV technique, they were also somewhat biased due to the absorption of higher order nonlinearities. For example, both the LSE and LET 1st order kernels had a peak in the 2nd bin which was absent from the true 1st order kernel. The LET kernels came closest to approximating the true kernels with as little noise as possible. However this was partly because the ground truth dynamics were designed from Laguerre basis functions and the alpha parameter was known beforehand, which is rarely the case in reality. The stan- dard LSE kernels were much more similar to the PBV kernels, albeit the LSE 1st order kernel was much noisier. Predictively, the LSE and LET techniques slightly outperformed the PBV technique (Fig. 3.2c), however at the expense of greater computational power (i.e. matrix inversion). 3.4.2 Convergence & Overtting In experimental physiological system identication, the experimenter requires crite- ria to know how much data is needed to estimate a valid model. Usually, this choice is constrained by the inherent nonstationarities of physiological systems. Model 54 0 5 10 15 −1 −0.5 0 0.5 1 1.5 0 10 20 0 10 20 −1 0 1 2 0 5 10 15 −0.2 −0.1 0 0.1 0.2 0.3 0 10 20 0 10 20 −0.05 0 0.05 Input x[t] True Output y[t] Estimated Spike Train Output y p [t] Ground Truth System Estimated System Estimated Continous Output S[t] (A) (B) (C) (D) (E) Figure 3.1: Representative ground truth and estimated system. Input spike train (A) is fed through the 2nd order nonlinear Volterra system, dened by the 1st and 2nd order kernels shown in (B), and then thresholded to generate the output in (C). The input and output point-process time series are then used to obtain the estimated PBV kernels shown in (D). The input is then fed through the estimated system to generate the estimated output in (E). The top plot in (E) shows the estimated continuous output (^ y c [t]) overlaid on the true output (in grey). The red vertical line shows the estimated threshold trigger value. The bottom plot in (E) shows the estimated spike-train output, ^ y[t]. convergence or data hungriness describes the amount of data needed to successfully model the system. Overtting describes how well the model estimated on the given dataset will generalize to other datasets acquired from the same system. Monte Carlo style simulations were performed to evaluate the convergence and overtting properties of the three models. N=30 trials were conducted with randomly gen- erated synthetic systems as described in methods. For each synthetic system, the three estimation methods were applied to randomly generated training/in-sample data of various lengths from 200 to 15,000 bins. Then the predictive performance of the three estimated kernel sets was evaluated on testing/out-of-sample data of 55 Figure 3.2: Comparison of results from 3 models for system in Fig. 3.1. (A) Ground truth and 3 estimated 1st order kernels. All 1st order kernels were normalized by their power to make comparison easier. Plot (B) shows the same for 2nd order kernels. Note that the diagonal of the 2nd order LET and LSE kernel was added to the 1st order kernel as these diagonals re ect 1st order dynamics (see text). (C) ROC plots for all three models. Note that the FPR (x-axis) extends only to .01 rather than 1 as in traditional ROC plots since the models were so accurate. equal lengths. The predictive power of the training and testing sets (solid and dashed lines respectively) is shown in Fig. 3.3 for the metric. The convergence of the data is described by the shape of the testing set curve in Fig. 3a, and in particular how quickly this curve approaches a steady-state value. The LET model converges re- markably quickly, and achieves 98% of its maximal value with only 200 data points (Fig. 3.3b). As noted before however, this assumes that the ground truth dynamics 56 can be expressed as a sum ofL Laguerre basis functions and that the alpha param- eter can be known with sucient accuracy. The LSE and PBV kernels converge at approximately the same rate and they achieve over 90% of their maximal value with 1000 data points. Overtting is re ected by the dierence in predictive power between training and testing sets. Once again the LET model has an extremely small amount of overtting, even with extremely small data records (<4% overt- ting with 200 bin data). The PBV and LSE models show more overtting. They require 1000 data points in order to converge to >90% of their training value. The LSE model in particular is unreliable for shorter data records. In fact, for short records under 500 bins, the LSE method often failed due to ill conditioned matrices, which are common with binary data. At steady state the predictive power of the PBV, LSE, and LET models as measured by were .797, .821, and .821, respectively (Fig. 3.3c). This conrms statistically what was seen in Fig. 3.2: that the PBV kernels are competitive with the LSE and LET kernels. 3.4.3 Model Robustness In experimental settings noise is an unavoidable reality which may arise in a vari- ety of ways, including imperfect instrumentation and/or from unobserved inputs. Thus, any input-output model must be robust, i.e. be able to accurately estimate the underlying system even in the presence of noise. In many neuroprosthetic appli- cations there is a large convergence of input neurons onto any given output neuron and only a small portion of these input neurons may be recorded. The unrecorded input neurons can be modeled as input noise. Thus, In order to examine the robust- ness of the PBV method, the PBV kernels where estimated with various amounts of spurious spikes added to the true/observed input spike train. Fig. 3.4 shows the estimated PBV kernels for the system in Fig. 3.1 with 0, 50%, and 150% spu- 57 Figure 3.3: Convergence and overtting analysis. Plot (A) shows the predictive power of both training (solid lines) and testing (dashed lines) sets for data of various lengths ranging from 200 to 15,000 data points. (B) Indicates the performance of the training set, normalized by its maximal (steady-state) performance. All points in A and B were averaged over N=30 trials. (C) Boxplot of median and quartiles of steady state performance. rious spikes added. As can be seen, the underlying form of the kernels remains remarkably consistent. The added noise manifests itself as high frequency jitter in the estimated kernels rather than a change in form. This corresponds with theo- retical ndings that Wiener kernels are unbiased with uncorrelated Gaussian noise (Marmarelis and Marmarelis, 1978). In low noise environments, the most robust model was the LET, since the kernels were conned to the Laguerre subspace and thus were immune to the high frequency jitters which permeated the PBV and LSE kernels. In high noise environments (<150% spurious input spikes), the PBV kernels proved to be the most robust since the least squares regression technique on which both the LET and LSE method rely is not well suited for input noise (S oderstr om, 1981). Although considering such high noise levels may seem super uous, it is actually quite necessary in neuroprosthetic 58 Figure 3.4: Degradation of PBV kernels with noise. PBV kernels where estimated for system in Fig. 3.1 with various amounts of spurious spikes added to the input. Degradation of 1st order kernels are shown in A, while degradation of 2nd order kernels are shown in (B). applications where the experimenter may have access to only a small handful of the hundreds of thousands of input neurons which may synapse onto the output neuron. It should be noted that although we only analyzed the robustness of the models to additive uncorrelated noise in the output, other types of noise exist including correlated noise, multiplicative noise, and output noise. 59 Figure 3.5: Robustness analysis. (A) Reduction of predictive power of models as increasing amounts of spurious spikes are added to the training set input. (B) Performance of training set normalized by their performance in noise-free environments. (C) Estimated 1st order kernels with 200% spurious spikes added to input. Notice the PBV kernel is the least aected. All points in (A) and (B) were averaged over N=30 trials. 3.4.4 Correlated Inputs Correlated point-process inputs were generated by ltering Gaussian white noise through a lowpass lter and thresholding the lter output. The autocorrelation of x[t] is shown in Fig. 3.6a. The estimated naive and corrected PBV kernels are shown in Fig. 3.6b,c. As can be seen, the corrected PBV kernels comes much closer to reproducing the ground truth kernels than the naive PBV kernels, establishing that the correction technique described in methods is indeed eective. However, the corrected PBV kernels are still far from the ground truth kernels. As mentioned previously, this is because the correction technique used is adapted from Gaussian inputs and used only heuristically with correlated point processes. The obtained LSE and LET kernels obtained for the colored inputs are also shown in Fig. 3.6. No correction needs to be applied to these kernels since they solve the inverse problem and can thus handle all types of inputs. 60 Figure 3.6: Results for correlated point-process inputs. (A) Representative interval of corre- lated input x[t] (above) and autocorrelation of x[t] (below). (B) Ground truth and estimated 1st order kernels. Note that the naive PBV kernel fails to detect peak in the rst bin. (C) Estimated ground truth and 2nd order kernels. (D) ROC plots of estimated models. Note that correction PBV kernels were able to achieve signicant improvement over naive PBV kernels. 3.4.5 Real Hippocampal Data The PBV method was applied to 10 datasets of CA3 and CA1 recordings from the rodent hippocampus (see methods). The input spike train had more power in the lower frequency ranges (<8 Hz) than higher frequency ranges and thus was not broadband/Poisson. The data was split in a training set and a testing set (33% of data). A representative example of the kernels estimated through the dierent estimation methods for one of the datasets is shown in Fig. 3.7, while the predictive power of these kernels is shown in Fig. 3.8a-c. The average predictive power over all datasets for training and testing sets, as measured by AUC is shown in Fig. 3.8d. 61 As can be seen, the time-course of the decay of the 2nd order naive (non- whitened) PBV kernel is longer than that of the 2nd order whitened PBV kernel. This longer time course is the result of the correlated input biasing the estimated PBV kernel. By using the whitening technique described in section II.D, this bias is minimized. The success of the whitening method can be seen in Fig. 3.8, where the corrected PBV kernels clearly outperform the naive kernels. Interestingly, the PBV kernels actually performed slightly better in the testing set. This is an artifact of the inherent nonstationarities of physiological data, e.g. the selected testing interval had less noise than the training interval. The obtained LSE kernels were found to be extremely noisy. Thus, LASSO regularization (Tibshirani, 1996) was used to limit the number of free parameters in the 2nd order kernel. Nonetheless, the obtained regularized LSE kernels, shown in Fig. 3.7a,d, were found to be quite noisy and dicult to interpret. This can also be seen in Fig. 3.8d, where the LSE kernels generalized to the testing set poorly. Notably, the LSE method had the greatest dierence in predictive power between training and testing sets. This is presumably because, as shown in Fig. 3.5, the LSE kernels are very unreliable in environments with high levels of input noise. Rodent CA3 pyramidal cells, which are estimated to have on average 380,000 input synapses (West, Slomianka, and Gundersen, 1991), can be presumed to have large amounts of unobserved inputs, which in the modeling context is equivalent to noise. Although the LET also relies on least squares estimation, by conning the kernels to the Laguerre basis space, the LET avoids many of the issues of the LSE technique. In order to obtain good results with the LET technique, however, two conditions must be met: the underlying system dynamics must be expressible with Laguerre basis functions and the alpha and L parameters must be known with sucient accuracy. Both of these conditions were satised by rst tting the obtained PBV 62 kernels with Laguerre basis functions and selecting an ideal alpha and L. As can be seen in Fig. 3.7c, the PBV kernels can be excellently t with alpha=0.8 and L=2. The LET was then used with these parameters to calculate the kernels shown in Fig. 3.7a,b. As can be seen in Fig. 3.8d, the obtained LET kernels gave the best results for both the training and testing set data. The obtained corrected PBV and LET kernels were found to be very similar. Both 1st order kernels show an excitatory eect of the input onto the output with a time constant of roughly 30ms. Furthermore, both showed strong 2nd order nonlin- ear facilitatory eects. This 2nd order facilitation corresponds to the well-studied short-term potentiation (STP) or paired-pulse facilitation commonly seen in neural systems. This phenomena has been studied previously in the context of nonlinear Volterra modeling (Song, Marmarelis, and Berger, 2009; Song et al., 2009b). Figure 3.7: Estimated 1st order (A) and 2nd order (B) kernels for real rodent hippocampal data. Note ground truth is not available. (C) Shows the results of tting the estimated corrected 1st order PBV kernel with Laguerre basis functions. As can be seen, the parameters L=2 and alpha=.8 gave excellent results. 63 Figure 3.8: Predictive performance of models on real data. (A) Training set ROC curves (B) testing set ROC curves (C) Training and testing performance as measured by . Note that LSE model has the most overtting, while the remaining models are able to successfully avoid tting noise. (D) Mean + SEM predictive power for N=10 hippocampal datasets. Note that regularized LSE was used for all these cases. 3.5 Discussion In this paper, we have proposed and demonstrated the ecacy of the PBV method- ology for modeling point-process single-input single-output systems such as those found in the nervous system. The PBV kernels are derived in a novel and intu- itive way using elementary probability theory, which will hopefully make them more accessible to the broader neuroscience community, which is largely unfamiliar with nonlinear systems theory. The PBV kernels are obtained using a nonparametric/data- driven approach, which is ideal for situations where a priori information about the system cannot be assumed. Such situations commonly occur in neuroprosthetics 64 which record/stimulate arbitrary neurons without knowing their anatomical con- nectivity. The only assumptions of the PBV method is that the system under question be stationary and have nite memory. The PBV method has shown itself to be remarkably robust to noise. Even with 150% spurious spikes in the input spike train (i.e. only 40% of true input spike train is observed), the estimated PBV kernels have shown only minor decreases in performance (see Fig. 3.4,3.5). The PBV method has also been shown to have good convergence properties. Although the exact rate of convergence of any point- process model depends on the MFR rather than the recording length, we have shown that with synthetic systems of Poisson rate parameter of 0.2, the PBV kernels give excellent results with only 1000 data points, and with real hippocampal systems the PBV kernels achieved adequate performance with only 3 minutes of recording. The PBV method was compared with two other previously used methods to model point process systems: the least squares estimation method and the Laguerre expansion technique. The PBV method is most naturally compared with the LSE method since both models have the same number of parameters and do not constrain the kernels to any xed basis space as the LET method does. The LSE kernels had slightly better predictive power in noise-free synthetic systems (Fig. 3.3); however, they were shown to perform poorly in simulations featuring high input noise (Fig. 3.5). This was failing was conrmed in real rodent hippocampal data where the 2nd order regularized LSE kernel had a high amount of jitter and was dicult to interpret (Fig. 3.7,3.8). Additionally, the PBV kernels can be estimated for Poisson inputs using much less computational power since they do not solve the inverse problem. The LET kernels, like the LSE kernels minimize mean squared error (MSE); however, while the LSE kernels can be viewed as expanding the input on a basis of 65 delayed delta functions, the LET method expands the input on a set of L Laguerre polynomials, whose rate of exponential decay depends on the alpha parameter. It should be noted that when alpha=0 (i.e. immediate decay), the LET kernels become equivalent to the LSE kernels (Valenza et al., 2013). The LET kernels outperformed the other two models with regards to predictive power, convergence, and overtting. However, the use of the LET technique assumes (1) that the underlying dynamics can be expressed by the sum of L Laguerre polynomials and (2) the alpha and L parameters can be known with sucient accuracy. Examples of where the 1st assumption would fail include systems with high order nonlinearities such as a sig- moid (see discussion below) or systems with both fast and slow dynamics (Mitsis and Marmarelis, 2002; Mitsis et al., 2002). In these cases, the PBV method would be preferred over the basic LET method. In cases where the rst assumption is met, such as rodent hippocampal CA3!CA1 dynamics (Song et al., 2007; Zanos et al., 2008), the PBV method can be used to estimate the alpha and L parameters which are required for proper use of the LET method. In previous work, these parameters were estimated using a global search (Lu, Song, and Berger, 2011). Here, we have shown that these parameters can be found more eciently by rst estimating the PBV kernels, which require no parameters, and then tting the obtained PBV ker- nels with Laguerre polynomials to optimize for alpha andL (Fig. 3.7c). As another example, the LET kernels in Fig. 3.6 were reestimated using the alpha and L pa- rameters obtained through this method. The predictive power of the kernels with estimated parameters diered from that of the kernels with ground truth parameters by less than 4%. Thus, the PBV and LET methods can be combined synergistically to maximize performance. It should be noted that characterizing the nonlinear transformation between sets of neurons is conceptually very similar to the goal of identifying the receptive elds 66 of sensory neurons such as V1 cells and auditory cells, which falls under the 'en- coding problem' mentioned earlier. The main dierence is that sensory neurons are assumed to transduce continuous stimuli from the outside world such as light and sound, whereas the neurons modeled here are assumed to only transduce the spik- ing history of their input neuron. Mathematically, the receptive eld identication problem takes the form of identifying a function which maps a continuous stimulus onto a binary spike train, while the systems identication problem maps a binary spike onto another binary spike train. This dierence is signicant in our context as the probabilistic framework for estimating the Volterra kernels presented here is inapplicable with continuous inputs. Although they are not explicitly compared, the PBV method has many similarities to reverse correlation methods used in receptive eld identication such as spike triggered averaging (STA) and spike triggered co- variance (STC) (De Boer and Kuyper, 1968; Steveninck and Bialek, 1988; Schwartz et al., 2006). In fact, the rst order PBV kernel is identical to the STA (albeit with a dierence in scale). The second order kernel however is distinct from the STC co- variance matrix, as the former is formed from the average value of the output with two input spikes at various lags, while the latter is the covariance of the STA matrix. Furthermore, in STC analysis, the covariance matrix is only an intermediate step in acquiring the set of linear lters of a system. A 2nd order PBV model characterizes the response of a system to all pairs of input spikes at all temporal separations. This includes many well-known phenomena in nervous systems such as facilitation and depression, which are both under the um- brella category of short-term potentiation (STP). This motivated applying the PBV method to data recorded from the CA3 and CA1 regions of the rodent hippocam- pus, where STP is well known to occur and has been ascribed to accumulation of residual calcium and vesicle depletion in the presynaptic terminal (Liley and North, 67 1953; Katz and Miledi, 1968; Betz, 1970). As hypothesized, nonlinear 2nd order eects were found in the form of facilitation. The observed facilitation justies the utility of using nonlinear dynamical models when studying the nervous system, as opposed to linear dynamical models which are unable to describe such phenomena. It should be noted that Song et al (Song et al., 2007) showed 3rd order nonlinear eects in the hippocampus which could in theory be modeled by the 3rd order PBV kernel. However, including the 3rd order kernel showed only minor improvements in predictive power and estimating the 3rd order kernel would require much longer data records. Finally, we should note that while in this paper the PBV method was applied to neuron action potential data, the method can be used to model arbitrary point process systems including highway trac data (Alfa and Neuts, 1995; Krausz, 1975), heartbeat dynamics (Barbieri et al., 2005), and communications data (Frost and Melamed, 1994). One fundamental limitation of the PBV approach is that it is designed for sys- tems whose underlying dynamics can be characterized by a 2nd order Volterra series. This excludes a large number of systems such as neural networks which have a sig- moid nonlinearity (Bishop et al., 2006). Furthermore, the 2nd order kernel, which is 2-dimensional can often be challenging to interpret, which is essential in biologi- cal physiomodeling. One solution is the Principal Dynamic Mode (PDM) approach which decomposes the 2nd order Volterra series into a Wiener cascade composed of a linear lterbank (the PDMs) followed by a static nonlinearity in a manner similar to what is done in STC analysis (Marmarelis, 2004). The PDMs are chosen as the most ecient linear basis set of the system. In SISO systems, the PDMs can be thought of as the eigenvectors of the combined 1st and 2nd order kernel matrix. For point-process systems, histogram based approaches have been successfully used to estimate the static nonlinearity. This allows the static nonlinearity to take on 68 any arbitrary shape, including sigmoidal, and thus allow the estimated PDM model to cover a much broader set of nonlinear systems. Furthermore, the 1-dimensional nature of the PDMs makes them much more prone to interpretation than the kernel approach. In conclusion, with little additional computational power, the estimated PBV model can be converted into the more powerful and interpretable PDM model, as will be done in future work. Additionally, in future work the current model will be expanded to cover systems with multiple inputs, in a manner similar to how Wiener kernels were expanded for multiple inputs (Marmarelis and Naka, 1974). 3.6 Relation of PBV Kernels with Poisson-Wiener Ker- nels Here we will show the equivalence between PBV kernels and the Poisson-Wiener kernels derived in Marmarelis, 2005 (Marmarelis and Berger, 2005). The 0th PW kernel is exactly equivalent to the 0th PBV kernel, ie: PW 0 =PBV 0 = y (3.17) The 1st PW kernel is dened as: PW 1 [] = 1 2 E(y[t]z[t]) (3.18) Wherez[t] is the demeaned Poisson input, i.e. z[t] =x[t] x and 2 is is the variance of z[t], ie: 2 =E(z 2 [t]) = x(1 x) (3.19) 69 Thus, PW 1 is: PW 1 [] = 1 2 E(y[t]z[t]) = 1 x(1 x) E(y[t]x[t x]) = 1 1 x [ E(y[t]x[t]) x y] = 1 1 x [CPK 1 []CPK 0 ] = 1 1 x PBV 1 [] (3.20) This is exactly eq. 11 in the text. Like the 2nd order PBV kernel, the 2nd order PW kernel is undened for t 1 =t 2 . For t 1 6=t 2 , the second order PW kernel is: PW 2 [ 1 ; 2 ] = 1 2 2 2 E(y[t]z[t 1 ]z[t 2 ]) (3.21) Once again, the 2nd order PW kernel can be shown to be a scaled version of the 2nd order PBV kernel: PW 2 [ 1 ; 2 ] = 1 2 2 2 E(y[t]z[t 1 ]z[t 2 ]) = 1 2 2 2 E(y[t]fx[t 1 ] xgfx[t 2 ] xg) = 1 2 x 2 (1 x) 2 E(~ y~ x 1 ~ x 2 ~ y~ x 1 x ~ y~ x 2 x + ~ y x 2 ) = 1 2(1 x) 2 [ ~ y~ x 1 ~ x 2 x 2 ~ y~ x 1 x ~ y~ x 2 x + y] = 1 2(1 x) 2 [CPK 2 [ 1 ; 2 ]CPK 1 [ 1 ]CPK 1 [ 2 ] +CPK 0 ] = 1 2(1 x) 2 PBV 2 [ 1 ; 2 ] (3.22) Where in the above derivations, ~ y,~ x 1 , and ~ x 2 , where used to respectively represent y[t], x[t 1 ], and x[t 2 ] in order to promote clarity. 70 3.7 Derivation of Wiener Kernels for Correlated Gaus- sian Inputs In this section, we show how to correct the Wiener kernel estimated using the cross- correlation technique (Lee and Schetzen, 1965) from correlated Gaussian inputs. Namely, the biased Wiener kernel must be deconvolved with the autocorrelation matrix of the input. The proof will follow along the lines of that shown in (Mar- marelis and Marmarelis, 1978; Korenberg and Hunter, 1990). Furthermore, it will be shown where exactly the proof breaks down for correlated point-process inputs. To begin, we will assume that the arbitrary nonlinear functional which trans- forms the input time-series x[t] into the output time-series y[t] can be expressed in the form of a Volterra series: y[t] = 1 X q=0 G q fk q ;x[t]g (3.23) wherefk q g are the set of Volterra kernels,fG q g are the set of Volterra functionals, andx[t] is an arbitrary zero-mean correlated random process. Later it will be shown what the implications are if x[t] is a correlated Gaussian process or a correlated point-process. For the purpose of this proof, we will assume a quadratic nonlinearity (Q = 2). This assumption has been shown to be quite reasonable in most cases, including point-process systems (Marmarelis, Citron, and Vivo, 1986; Marmarelis, 2004). As a preliminary to the main proof, we will rst derive the expected mean of 71 the output y[t]: y =E(y[t]) =E(k 0 + X n k 1 [n]x[tn] + X X n 1 ;n 2 k 2 [n 1 ;n 2 ]x[tn 1 ]x[tn 2 ]) = k 0 + X n k 1 [n]E(x[tn]) + X X n 1 ;n 2 k 2 [n 1 ;n 2 ]E(x[tn 1 ]x[tn 2 ]) (3.24) Now, the middle term is 0 since x[t] is zero-mean, giving: k 0 + X X n 1 ;n 2 k 2 [n 1 ;n 2 ] x [n 1 n 2 ] (3.25) We begin the proof by dening the 2nd order cross-correlation, which is essen- tially equivalent to the 2nd order Wiener kernel (Lee and Schetzen, 1965) xxy =E(y 0 [t]x[t 1 ]x[t 2 ]) = (3.26) where y 0 [t] is the demeaned output (ie y 0 [t] =y[t] y). Eq. 3.26 is then expanded as: E(y 0 [t]x[t 1 ]x[t 2 ]) = E(fy[t] ygx[t 1 ]x[t 2 ]) = E(y[t]x[t 1 ]x[t 2 ])E( yx[t 1 ]x[t 2 ]) (3.27) 72 The denition of y[t] from Eq. 3.23 is now substituted into Eq. 3.27: E( 2 X q=0 G q fk q ;x[t]gx(t 1 )x[t 2 ]]E[ yx[t 1 ]x[t 2 ]] = E(k 0 x[t 1 ]x[t 2 ]) +E( X n k 1 [n]x[tn]x[t 1 ]x[t 2 ]) +::: :::E( X X n 1 ;n 2 k 2 [n 1 ;n 2 ]x[tn 1 ]x[tn 2 ]x[t 1 ]x[t 2 ])E( yx[t 1 ]x[t 2 ]) (3.28) Eq. 3.28 can then be simplied by substituting Eq. 3.24 and noticing that the 1st terms cancel out, thus resulting in: E( X n k 1 [n]x[tn]x[t 1 ]x[t 2 ]) +::: :::E( X X n 1 ;n 2 k 2 [n 1 ;n 2 ]x[tn 1 ]x[tn 2 ]x[t 1 ]x[t 2 ])::: ::: X X n 1 ;n 2 k 2 [n 1 ;n 2 ]E(x[tn 1 ]x[tn 2 ]) (3.29) At this point we can see the dierence between correlated gaussian processes and correlated point-processes. The latter are mathematically ill-dened and thus their higher-order moments cannot be simplied any further analytically. Gaussian pro- cesses, however, are entirely dened by their 2nd order moments. Thus, very nice simplication properties exist for higher order moments of Gaussians (Marmarelis and Marmarelis, 1978). All odd order moments of Gaussians are zero so the rst term of Eq. 3.29 vanishes. Furthermore, fourth order moments of Gaussians can be simplied via the Gaussian decomposition property (Marmarelis and Marmarelis, 73 1978). Thus, Eq. 3.29 becomes: E( X X n 1 ;n 2 k 2 [n 1 ;n 2 ]x[tn 1 ]x[tn 2 ]x[t 1 ]x[t 2 ]) X X n 1 ;n 2 k 2 [n 1 ;n 2 ]E(x[tn 1 ]x[tn 2 ])] = 2 X X n 1 ;n 2 k 2 [n 1 ;n 2 ] x (n 1 1 ) x (n 2 2 ) + x ( 1 2 ) X X n 1 ;n 2 k 2 [n 1 ;n 2 ] x (n 1 n 2 )::: ::: X X n 1 ;n 2 k 2 [n 1 ;n 2 ]E(x[tn 1 ]x[tn 2 ]) = 2 X X n 1 ;n 2 k 2 [n 1 ;n 2 ] x (n 1 1 ) x (n 2 2 ) (3.30) This is our nal results and shows that the 2nd order cross correlation of a correlated Gaussian input and the output is the 2d convolution between the input autocorre- lation matrix and the 2nd order Wiener kernel. Thus, the unbiased Wiener kernel can be attained by multiplying the biased Wiener kernel (which is dened by the second order cross-correlation of Eq. 3.26), K 2 , with the inverse of the Toeplitz autocorrelation matrix, x (Korenberg and Hunter, 1990): ~ K 2 = 1 x K 2 1 x (3.31) Where ~ K 2 is the corrected 2nd order Wiener kernel. Empirically, for point-process systems it was found that a modied form of Eq. 3.31 gives better results (see section 3.3.4): ~ K 2 = 1 x K 2 (3.32) 74 Chapter 4 Understanding Spike Triggered Covariance using Wiener Theory for Receptive Field Identication 1 4.1 Abstract Receptive eld identication is a vital problem in sensory neurophysiology and vi- sion. Much research has been done in identifying the receptive elds of nonlinear neurons whose ring rate is determined by the nonlinear interactions of a small number of linear lters. Despite more advanced methods which have been proposed, spike-triggered covariance (STC) continues to be the most widely used method in such situations due to its simplicity and intuitiveness. Although the connection be- tween STC and Wiener/Volterra kernels has often been mentioned in the literature, 1 The results of this chapter are accepted for publication for the Journal of Vision 75 this relationship has never been explicitly derived. Here we derive this relationship and show that the STC matrix is actually a modied version of the 2nd order Wiener kernel which incorporates the input autocorrelation and mixes rst and second or- der dynamics. It is then shown how, with little modication of the STC method, the Wiener kernels may be obtained, and from them the Principal Dynamic Modes (PDMs), a set of compact and ecient linear lters which essentially combine the STA and STC matrix and generalize to systems with both continuous and point- process outputs. Finally, using Wiener theory, we show how these obtained lters may be corrected when they were estimated using correlated inputs. Our correction technique is shown to be superior to those commonly used in the literature for both correlated Gaussian images and natural images. 4.2 Introduction A central goal of neuroscience is to answer the question 'what does the underlying neuron or neural population compute?'. Since in most neural systems, the output of the system is exclusively considered to be its spike rate and timing, the above question can be rephrased as 'what makes this neuron or neural population re?'. This question, oftentimes referred to as system identication or receptive eld iden- tication, has been a major area of research, particularly in sensory neuroscience where one begins with a strong a priori notion of what stimuli or stimulus modality the cell is sensitive to. Early attempts to characterize neural systems attempted to parameterize their responses to one or two stimuli parameters and then describe the cell response through various forms of tuning curves. Although such tuning curve approaches are useful to gain a rst glance approximation of the system, they fail to take into account the full complexity of neural responses, and prove woefully inadequate when the input stimuli occupy a high-dimensional space, such as the 76 spatiotemporal receptive elds of complex cells in the visual cortex. The advancement of random inputs such as Gaussian white noise (GWN) to characterize systems in the 60's and 70's represented a major advance in the system identication problem (Marmarelis and Marmarelis, 1978). Such inputs facilitated the development of non-parametric models whereby the system response to arbitrary stimuli could be predicted with only very minor prior assumptions about the nature of the underlying system. In particular, the Volterra/Wiener framework achieved success in modeling many nonlinear physiological systems, including mapping the nonlinear receptive elds of auditory (Aertsen and Johannesma, 1981; Eggermont, 1993) and vision (Emerson et al., 1987; Citron and Marmarelis, 1987) neurons. How- ever, perhaps due to the diculty of interpreting the higher order Volterra kernels, and the need to use many kernels to characterize certain nonlinear systems, the Volterra/Wiener approach never gained the popularity of linear approaches which approximate the underlying nonlinear system with a single linear lter. In sensory neuroscience this usually took the form of the spike-triggered average, or STA (De Boer and Kuyper, 1968). Later the STA approach was expanded by adding a static nonlinearity to map the STA output to the neural response in the so-called LNP model which was used successfully to characterize the dynamics of retinal ganglion cells (Chichilnisky, 2001) However, the limitations of STA based approaches soon became apparent when it was realized that the response of certain neurons where a nonlinear function of multiple linear lters. For example, in the popular Adelson-Bergen energy model, the output is determined by the sum of the square of the outputs of two gabor lters (Adelson and Bergen (1985), see Fig. 4.3b); thus the STA of such a system is zero since it responds equally to positive and negative contrasts. In other less severe cases, the system may be approximated by the STA but multiple lters would still 77 be required to completely characterize it. The most popular method to obtain these multiple lters is the spike-triggered covariance (STC) method, which builds of the STA framework by taking the covariance rather than the mean of all spike triggering stimuli. Although other more complex methods have been used to obtain these lters (Sharpee, Rust, and Bialek, 2004; Rapela, Mendel, and Grzywacz, 2006; Paninski, 2003; Saleem, Krapp, and Schultz, 2008; Pillow and Park, 2011), non have been as popular as the relatively straightforward and simple STC method which has been used in diverse areas such as modeling Hodgkin-Huxley dynamics (Arcas, Fairhall, and Bialek, 2001), vision (Rust et al., 2005; Touryan, Felsen, and Dan, 2005; Fairhall et al., 2006), audition (Slee et al., 2005), insect movement (Steveninck and Bialek, 1988; Fox, Fairhall, and Daniel, 2010), and olfaction (Kim, Lazar, and Slutskiy, 2011). A excellent review of both STA and STC for characterizing receptive elds appears in Schwartz et al. (2006). Parallel to the development and popularization of STC in sensory neuroscience, the concept of Principal Dynamic Modes (PDMs) was developed in the area of dynamical systems identication as a system specic linear lter set to eciently describe the linear dynamics of nonlinear systems (Marmarelis and Orme, 1993; Marmarelis, 1997; Marmarelis, 2004). The PDMs are estimated from a 2nd or- der Volterra model and provide a compact representation of the model which is much more amenable to physiological interpretation. The PDMs have been used to successfully characterize renal autoregulation (Marmarelis et al., 1999), cerebral hemodynamics (Marmarelis, Shin, and Zhang, 2012), heart-rate variability (Zhong et al., 2004), spider mechanoreceptor dynamics (Marmarelis, Juusola, and French, 1999), the schaer collateral pathway in the hippocampus (Marmarelis et al., 2013b; Sandler et al., 2013; Sandler et al., 2014), and most recently, V1 receptive elds (Fournier et al., 2014). As recently pointed out (Fournier et al., 2014), the PDMs 78 and STC are functionally equivalent as they both aim to identify an ecient linear lterbank for the system under study. The main functional dierence being that PDMs, derived from the Volterra framework, can be used for systems with both continuous and point-process outputs, while STC lters, being an extension of the STA approach, are restricted to systems with point process outputs. Although the connection between STC and the 2nd order Wiener kernel is of- ten mentioned in the STC literature (Schwartz et al., 2006; Samengo and Gollisch, 2013), this connection has never been explicitly derived. In this paper, we derive this connection and show that the STC matrix is actually a modied version of the 2nd order Wiener kernel which incorporates the input autocorrelation and mixes 1st and 2nd order dynamics. Then, we will show how the STC method can be corrected to obtain the 2nd order Wiener kernel and PDMs while still retaining its elegance and simplicity. The obtained Wiener kernel, unlike the STC matrix which is simply an intermediate step, is highly interpretable and provides important information about the system nonlinearity which may not be apparent from the STC lterbank and associated nonlinearity. To provide guidance for experimentalists, several prop- erties of the PDMs are examined including their predictive power, robustness to noise, overtting, and 'data-hungriness', or how much data they need in order to converge. Finally, we show how the obtained PDMs may be corrected when they are estimated using correlated inputs. This is a pertinent issue in sensory neuro- science where complex correlated inputs such as natural images are commonly used to characterize systems. It is shown that the correction procedure advocated here is superior to many of those commonly used in the literature for both correlated Gaussian inputs (such as blurred images) and truly natural images obtained from an existing database. 79 4.3 Methods 4.3.1 Wiener-Volterra Kernels The Volterra functional expansion is a method of identifying arbitrary nonlinear nite-memory systems akin to the way a Taylor series can be used to approximate arbitrary functions. Thus, in any input-output system, the output y[t] can be ex- pressed in terms of the input, x[t], and a series of increasing-order nonlinear lters, known as the Volterra kernels, k q [ 1 ::: q ], as: y[t] =k 0 + M X =1 k 1 []x[t] + M X 1 =1 M X 2 =1 k 2 [ 1 ; 2 ]x[t 1 ]x[t 2 ]+ ::: Q X q=3 M X 1 =1 ::: M X q =1 k q [ 1 ::: q ]x[t 1 ]:::x[t q ] (4.1) Although in this form, the Volterra series describes a dynamic system where the input is temporal in nature (i.e. x[t]), the Volterra framework can just as easily be applied to systems where the input is spatial or spatiotemporal. Thus, the spatial version of Eq. 4.1 for the 1st two terms is: y[t] =k 0 + M X n=1 k 1 [n]x[n;t] + M X n 1 =1 M X n 2 =1 k 2 [n 1 ;n 2 ]x[n 1 ;t]x[n 2 ;t] +::: (4.2) where in the context of receptive eld identication, x[n;t] is the n th pixel of the input image at time t. For the remainder of this section, we will assume a spa- tial system, even though any intuitions may just as easily be applied to temporal systems. The 1st order Volterra kernel, k 1 [n] can be seen as weighing the contribution of a single pixel of the input to the output. Thus, the rst order Volterra functional 80 describes a linear system and the rst order Volterra kernel is equivalent to the well- known impulse response function in engineering and the spike-triggered-average in neuroscience (see Eq. 4.6). The second order Volterra kernel, and the rst nonlinear kernel, describes second order nonlinear interactions between two input pixels. Each value of the kernel (depicted as a surface), for instance k 2 [n 1 ;n 2 ] at lags n 1 and n 2 , represents the weight (i.e., relative importance) of the product of pixel values x[n 1 ;t]x[n 2 ;t] for spatial systems, or time lagsx[tn 1 ]x[tn 2 ] for temporal systems in reconstructing the system output, y[t]. Thus, a large positive value of k 2 [n 1 ;n 2 ] implies strong mutual facilitation of the input-lagged valuesx[tn 1 ] andx[tn 2 ] in the way they aect y[t] (Marmarelis, 2004). In vision research, 2nd order dynamics are commonly used to describe center-surround inhibition and phase insensitivity, while in temporal neural systems,they are commonly used to describe paired-pulse facilitation and depression (Krausz, 1975; Song, Marmarelis, and Berger, 2009). Likewise the q th order kernel describes the nonlinear interactions between q input pixels. Norbert Wiener was the rst to propose a method of estimating the kernels by orthogonalizing the Volterra functionals with respect to Gaussian white noise (GWN) inputs, giving rise to the well-known Wiener Series (Wiener, 1966). How- ever, the Wiener series was not popularized until the computationally feasible cross- correlation technique was introduced as a method of estimating the Wiener kernels (Lee and Schetzen, 1965; Marmarelis and Marmarelis, 1978). Thus, for GWN in- puts, theq th order Wiener kernel was shown to be simply a scaled version of theq th order cross-correlation, x q y [n 1 :::n q ]: h q [n 1 :::n q ] = 1 q! 2q x q y [n 1 :::n q ] = 1 q! 2q E(y 0 [t]x[n 1 ;t]:::x[n q ;t]) (4.3) where y 0 [t], is the demeaned output, h q is the q th order estimated Wiener kernels 81 and 2 is the power (variance) of the GWN input. In particular, the 1st two Wiener kernels are: h 1 [n] = 1 2 xy [n] = 1 2 E(y 0 [t]x[n;t]) (4.4a) h 2 [n 1 ;n 2 ] = 1 2 4 xxy [n 1 ;n 2 ] = 1 2 4 E(y 0 [t;t]x[n 1 ;t]x[n 2 ;t]) (4.4b) Although these equations apply only when using GWN inputs, with slight mod- ications, they can be used to obtain Wiener kernel estimated from a wide class of inputs such as correlated Gaussian inputs (see section 4.3.4), Poisson processes, and uniform inputs (see discussion) (Marmarelis and Marmarelis, 1978; Marmarelis and Berger, 2005). Asymptotically, or in the limit of innite data, the estimated Wiener kernels are the optimal kernels to minimize mean squared error (MSE). It should be noted that outside of certain specic cases such as when the underlying system only has 2nd order dynamics, the Wiener kernels are not generally equivalent to the Volterra kernels, and equations exist to interchange between the two (Marmarelis, 2004). Although many physiological systems do have higher order dynamics, and thus theoretically the Volterra and Wiener kernels are distinct, in practice, due to computational constraints, usually only up to the 2nd order Wiener model is esti- mated. In this situation, the obtained Wiener kernels are the best approximation to the Volterra kernels, and thus this distinction will be ignored for the remainder of this study. Volterra modeling has a long history in neuroscience research in diverse areas such as the retina (Marmarelis and Naka, 1973; Citron and Marmarelis, 1987), the auditory cortex (Eggermont, 1993), the hippocampus (Song et al., 2007), and motor control (Jing et al., 2012). 82 4.3.2 STC & Wiener Kernels STC has been a popular method in sensory neuroscience to characterize nonlinear systems. It aims to identify an optimal subspace of linear lters which are then nonlinearly combined to determine the output ring rate. The fundamental idea behind STC, and its precursor, STA, is that spike triggering stimuli,f~ x(t n )g have dierent statistical moments than mean stimuli, and that these moments can be used to characterize the system. Thus, STA is dened as the mean (1st order moment) of the spike triggering stimuli: ~ A =E[~ x(t n )] = 1 N N X n=1 ~ x(t n ) (4.5) where N is the total amount of spikes. Equivalently, the STA can also be shown to be a scaled version of the 1st order cross-correlation, xy [n], and, under a wide class of inputs, the 1st order Wiener kernel, h 1 (Eq. 4.4, (Rieke et al., 1999)). The critical step is to note that the sum of the spike triggered stimuli ( P N n=1 ~ x(t n )) is equivalent to the sum of all stimuli times the output ( P T t=1 y(t)~ x(t)) This is true because when an input stimuli does not elicit a spike, y[t] will be zero and thus that stimulus will not in uence the total sum. The derivation is as follows: ~ A = 1 N N X n=1 ~ x(t n ) = 1 N T X t=1 y(t)~ x(t) = T N E[y(t)~ x(t)] = 2 m y h 1 (4.6) where m y = N T is the mean of y[t]. Likewise, The STC matrix, ^ C, is dened as the covariance (2nd order moment) of the spike triggering stimuli (Schwartz et al., 2006), i.e.: ^ C =Cov[~ x(t n )] = 1 N 1 N X n=1 [~ x(t n ) ~ A][~ x(t n ) ~ A] T (4.7) 83 It should be noted that while this is the most commonly used variant of the STC matrix, other variants exist such as removing the projection of STA from~ s(t n ) (Rust et al., 2005; Petersen et al., 2008). The STC linear lters are then dened as the signicant eigenvectors of ^ C. Although it is commonly asserted in the literature that the STC matrix, ^ C, is related to the 2nd order Wiener kernel, h 2 , the relationship to our knowledge has never been explicitly dened (although see Bialek and Steveninck (2005) and Park et al. (2013)). Here, we derive this relationship, and show that for GWN inputs, ^ C is actually a modied version of h 2 which intertwines 1st and 2nd order dynamics and incorporates the input autocorrelation. We begin the derivation by noting that the covariance of vector ~ x with mean can be written as E(~ x~ x T ) T . Thus, ^ C =Cov[~ x(t n )] =E[~ s(t n )~ x(t n ) T ] ^ A ^ A T As in the STA derivation, the sum over spike-triggered stimuli is interchanged with that over all stimuli: ^ C = T N E[y(t)~ x(t)~ x(t) T ] ^ A ^ A T Now, y[t] is separated into y 0 [t] +m y , giving: T N E[y 0 (t)~ x(t)~ x(t) T ] + T N m y E[~ x(t)~ x(t) T ] ^ A ^ A T Finally, by noting m y = N T and using Eq. 4.6 and Eq. 4.4b to substitute in the Wiener kernels, we get: ^ C = 2 4 m y h 2 | {z } Scaled h 2 + x |{z} Mod. M 4 m y h 1 h T 1 | {z } Mod. M 1 (4.8) 84 where x is the input autocorrelation matrix. Here, it can be seen that the STC matrix ^ C is composed of a scaled version of the 2nd order Wiener kernel and 2 modifying terms. The rst modication, M , incorporates the input autocorrela- tion, x . In the case of uncorrelated inputs, such as GWN, this modication adds a delta function down the diagonal of ^ C, and thus has little eect on the acquired STC lters (eigenvectors). However, when using correlated stimuli such as natu- ral images, this modication can signicantly alter the results, as will be shown in section 4.4.3 (although here, the cross-correlation estimate for h 2 in Eq. 4.4 will also be biased - see section 4.3.4). This modication as a source of bias has been acknowledged previously in the context of information theory and addressed by removing the 'prior' covariance matrix, or the input autocorrelation matrix, x (Arcas, Fairhall, and Bialek, 2003). The second modication, M 1 , incorporates 1st order dynamics through the term h 1 h T 1 . Thus, the STC matrix mixes 1st and 2nd order dynamics into a single kernel. Despite these modications, the linear lters for GWN obtained from both the Wiener kernel and the STC matrix, when the STA is included, span the same subspace, and thus are consistent and should have roughly equal predictive power. However, these modications remove the interpretability of the 2nd order Wiener kernel from the STC matrix, since the each element of the STC matrix can no longer be viewed as weighing pairwise input correlations. Also, it can easily be shown that if x[t] is not demeaned, then there will be an additional modication to the Wiener kernels which causes a signicant bias. It is dened as: m x 2 (1 + 1 m y )(h 1 (i) +h 2 (j)) +m 2 x (m y + 1) Note that this bias actually changes the shape of the STC matrix and the obtained linear lters, highlighting the absolute necessity of demeaning all stimuli when using STC or Wiener approaches. This is in distinction to STA, where not demeaning the 85 input will only change the baseline of the obtained STA lter, but not its actual shape. The good news is that the Wiener kernels may be obtained by using the 2nd moment of ^ A rather than the covariance of ^ A (i.e. E[~ x(t n )~ x(t n ) T ] rather than E[(~ x(t n ) ~ A)(~ x(t n ) ~ A) T ]). This will eliminate the M 1 modication, leaving only the M modication, which can easily be removed by subtracting the input auto- correlation. Thus, the unmodied Wiener kernel, h 2 can be obtained by: h 2 =M 2 ( ~ A) x = 1 N 1 N X n=1 [~ x(t n )][~ x(t n )] T x (4.9) whereM 2 ( ~ A) refers to the second moment expectation of ~ A. The question between whether to use the covariance or second moment of ^ A is already present in the literature (Schwartz et al., 2006; Samengo and Gollisch, 2013). In fact, another group (Cantrell et al., 2010) previously showed that using the second moment of ^ A (which they referred to as 'non-centered STC') could provide good results when used in the context of high-throughput cell classication, but is inferior to standard STC for spike prediction. We hope that this work conclusively shows that although when used in conjunction with the STA, both the STC and Wiener kernel will yield the same linear subspace, the Wiener kernel adds much interpretability for no theoretical or computational cost. 4.3.3 Principal Dynamic Modes In the above section, it was shown that the STC matrix is a biased version of the 2nd order Wiener kernel,h 2 . This nding, however, does not mean that the linear lters obtained from the STC matrix are biased. To begin the search for an ecient linear lterbank, we note that any arbitrary set of Wiener/Volterra functionals 86 can be represented as a Wiener-Bose cascade model composed of a set of linear lterbanks followed by a multivariate static nonlinearity (see Fig.4.1; Westwick and Kearney, 2003; Marmarelis, 2004). However, unlike the true Wiener kernels, which have a canonical form, from which any deviation will lead to suboptimal predictive power, any Wiener system can be expressed by an innite number of Wiener-Bose cascades so long as the linear lterbanks span the same subspace. Thus, a desirable linear lterbank will be (1) highly predictive, or essentially span the same subspace as the equivalent Wiener-Bose cascade (2) compact, or contain as few lters as necessary and (3) be interpretable, or facilitate scientic discovery. Our group has attempted to nd such a desirable lterbank over the last two decades in the context of dynamical systems identication through the use of Principal Dynamic Modes (PDMs) (Marmarelis and Orme, 1993; Marmarelis, 2004; Marmarelis et al., 2014). The PDMs are a system-specic and input-independent set of linear lters which aim to compactly describe the dynamics of nonlinear systems, and thus are functionally equivalent to the STC lters. To understand the theoretical underpinnings of the PDM approach, one must rst realize that Eq. 4.1 can be expressed in the following quadratic matrix equation: y[t] = ~ x 0 [t] T ^ H ~ x 0 [t] (4.10) where ~ x 0 [t] is the augmented vector of the stimuli at time t: ~ x 0 [t] = [1x(1;t)x(2;t)x(M;t)] T 87 and ^ H is the quadratic matrix dened by: 2 6 6 6 6 6 6 6 4 h 0 1 2 h 1 . . . . . . 1 2 h 1 h 2 . . . . . . 3 7 7 7 7 7 7 7 5 where h 0 =m y is the 0th order Wiener kernel. Now, as in standard STC analysis, the ^ H matrix can be factored via eigendecomposition to obtain: y[t] = ~ x 0 [t] T E 0 E ~ x 0 [t] (4.11) where E is the orthonormal matrix of the eigenvectors of ^ H, and are the corre- sponding eigenvalues. Finally, the L signicant eigenvectors are selected according to some criteria and those eigenvectors, from the second element onward, can be considered the PDMs,f ~ P i ::: ~ P L g. The obtained PDM model can then be expressed as: y[t] L X i=1 i (c i + ~ x[t] T ~ P i ) 2 (4.12) where here~ x[t] is the non-augmented input stimuli vector andc i is the 1st element of the i t h eigenvector. Note that the eect of the c i constant is to oset the parabolic static nonlinearity (see section 4.3.5). The signicance of Eq. 4.12 is that it establishes a strong theoretical foundation for the PDM method by showing that the PDMs are actually a compact representa- tion of the estimated 2nd order Wiener system which actually become equivalent to that system as L is increased. Having established these theoretical underpinnings, other approaches which claim to estimate unbiased linear lters may be considered. Most notably, Samengo and Gollisch (2013) have demonstrated that if one denes 88 the STC lters as the STA and the signicant eigenvectors of the STC matrix, these STC lters will span the same linear subspace whether or not the STA sub- tracted prior to calculating the STC matrix. Their proof demonstrates that even though modication M 1 makes the STC matrix a biased Wiener kernel, the set of linear lters obtained from the STC matrix, provided the STA is included, is itself unbiased. Nonetheless, although both the PDMs derived from ^ H and from the traditional STC approach are unbiased, they cannot be described as compact since there is no guarantee they will be linearly independent - i.e. one of the lters may be dened as a linear combination of the others and thus be super uous. For the PDMs derived from ^ H this occurs because while the eigenvectors of ^ H are by denition linearly independent, the PDMs which are the eigenvectors from the 2nd element onward may be linearly dependant. In the traditional STC approach this occurs because the STA lter may span the same subspace as the signicant eigenvectors of the STC matrix. This issue has been brought up previously in the STC literature and several methods have been proposed on how to eciently combine the STA and STC eigenvectors (Pillow and Simoncelli, 2006; Kaardal et al., 2013). Here, we suggest a method whereby a compact, linearly independent set of lters may be obtained directly from the rst two Wiener kernels (or equivalently, the STA and STC matrix) without having to go through the intermediate step of rst obtaining the signicant eigenvectors of h 2 (or STC matrix). This is done by taking the signicant singular vectors of the rectangular matrix dened as ^ H 0 = [h 1 ;h 2 ] through singular value decomposition (SVD) (Marmarelis et al., 2014). Several methods have been proposed in the STC literature about how to access the signicance of the STC lters. Here, we will follow the nested bootstrap approach advocated in Schwartz et al. (2006) whereby condence bounds are generated for 89 the STC matrix eigenvalues by shuing the input several times with respect to the output and estimating the distribution of the largest and smallest eigenvalue. If the null hypothesis was rejected and the largest (or smallest) eigenvalue was found to be signicant, it is projected out, and the test is repeated for the next largest/smallest eigenvalue. It was found that all nested tests after the rst did not change any signicance assessments, so all gures show condence bounds for the largest and smallest eigenvalue, and any eigenvalues within these bounds are signicant. The same approach was used here with theh 2 eigenvalues and the [h 1 ;h 2 ] singular values with T = 40 bootstrap repetitions for each hypothesis test. Finally, it is important to note that here the PDMs may be obtained from the estimated Wiener kernels since for 2nd order systems, the Wiener kernels are es- sentially equivalent to the Volterra kernels. However, in systems with higher order nonlinearities, the estimated truncated 2nd order Wiener system will be dependent on the specic input used to probe the system - for example, GWN inputs of dif- ferent power level will generate dierent Wiener kernels. In these situations, the PDMs, which are dened to be input-independent, cannot be estimated from these estimated Wiener kernels, since they contain input-dependencies, but only from the Volterra kernels (see discussion) (Marmarelis and Marmarelis, 1978). 4.3.4 Correcting for Correlated Inputs Many system identication techniques, including Wiener kernels and STC, were originally developed for uncorrelated ('white') inputs. However, since then many methods have been proposed to 'correct' systems estimated from these techniques when correlated inputs were used. This is important for two reasons. First, there has been a large trend in sensory neuroscience in the last several years to use naturally occurring stimuli because the system under question was evolutionarily designed 90 for such stimuli and these stimuli tend to elicit more output spikes, thus facilitating system estimation (Schwartz and Simoncelli, 2001; Touryan, Felsen, and Dan, 2005). Also, in many situations the experimenter is unable to stimulate the system and thus must rely on spontaneous activity which generally has correlations (Song et al., 2007; Marmarelis et al., 2013b). Theunissen et al. (2001) was the rst to show that the biased STA, ~ A 0 can be corrected by multiplying it by the inverse of the input autocorrelation matrix, x : ~ A = 1 x ~ A 0 (4.13) Touryan, Felsen, and Dan (2005) proposed that the STC method can be corrected for correlated inputs by rst 'prewhitening' these inputs and then using the standard STC method. Later, Samengo and Gollisch (2013) showed that this was equivalent to multiplying the biased STC matrix with the input autocorrelation matrix as: ^ C = 1 x ^ C 0 (4.14) Now that we have explicitly established the relationship between the STC matrix and the 2nd order Wiener kernel, we may take advantage of existing results in the literature on how to correct Wiener kernels for correlated inputs. Namely, Koh and Powers (1985), showed that Wiener kernels estimated from correlated Gaussian processes may be corrected as shown below: ~ h 1 = 1 x ~ h 1 0 (4.15a) ^ h 2 = 1 2 1 x ^ h 2 0 1 x (4.15b) Note that if we substitute Eq. 4.9 into Eq. 4.15, we obtain the proper correction 91 (ignoring, dierences in scale) for the STC matrix: ^ C = 1 x ( ^ C 0 x ) 1 x (4.16) where here, ^ C refers to the second moment matrix of ~ A (see Eq. 4.9). Although the Koh-Powers correction was derived specically for correlated Gaussian inputs, we show in section 4.4.3, that it is superior to Eq.4.14 for both correlated Gaussian inputs and much more complicated inputs such as natural images. However, it should be noted that taking the inverse of x amplies noise in the input. Thus, the corrected kernel, h 2 , which is obtained by twice multiplying the biased kernel with 1 x , may be extremely noisy. In order to compensate for this, one should use regularization techniques when taking the inverse (Touryan, Felsen, and Dan, 2005). In some cases, it may be more eective to only multiply once by 1 x , in a matter similar to that of Eq. 4.14 in order to avoid high levels of noise in the corrected kernel (Sandler et al., 2015). 4.3.5 Associated Nonlinear Functions As mentioned previously, any nonlinear Volterra-type system can be represented as a Wiener-Bose cascade consisting of a set of linear lters (here the PDMs or STC lters) followed by a multivariate static nonlinearity, F [v 1 (t):::v L (t)], where fv 1 (t):::v L (t)g are the lter outputs (Marmarelis, 2004). In the STC literature, the nonlinearity following these lters has traditionally been estimated using a histogram approach based on Bayes rule: P [spikejv 1 (t):::v L (t)]/ P [v 1 (t):::v L (t)jspike] P [v 1 (t):::v L (t)] (4.17) 92 where the above quotient is estimated by dividing each lter output into B bins and calculating the proportion of times a spike was elicited when the lter output fell into a specic set of bins (Marmarelis et al., 2013b; Schwartz et al., 2006). An advantage of this approach is that since the obtained nonlinearity is not constrained to be a polynomial, it can describe many nonlinearities which would otherwise be dicult to characterize with a Volterra approach, such as a sigmoidal nonlinearity (Bartesaghi, Gessi, and Migliore, 1995). However, a major issue with this approach is that the dimension of the nonlinearity is equal to the number of lters. For exam- ple, even with 3 lters, estimating this nonlinearity becomes unfeasible due to the large number of parameters (B 3 ) - and even if there was enough data to estimate this nonlinearity, it would be very dicult to visualize and interpret. Some researchers have attempted solve this issue by using a priori assumptions of the underlying physiological system to simplify the high dimensional multivariate nonlinearity into simpler forms (Bialek and Steveninck, 2005; Rust et al., 2005). Alternatively, many researchers have attempted to get around this 'curse of dimensionality' by assuming that the lter outputs are independent of each other, or, equivalently, that there are no nonlinear interactions between the L lters. In such a situation, the out- put of each lter would be transformed independently by a 1 dimensional static nonlinearity, which our group has previously dubbed the associated nonlinear func- tion (ANF) (Marmarelis, 2004). Thus, with this 'independence assumption', one would only need to estimate L 1-dimensional nonlinearities (ANFs), each with B parameters, rather than 1 L-dimensional nonlinearity with B L parameters. The Wiener-Volterra framework of nonlinear systems analysis can provide such guidelines. As shown in Eq. 4.12, in a 2nd order system, the output is the weighted sum of the square of the PDM outputs. Thus, the PDMs all contribute additively to the output, y[t], and the 'independence assumption' is satised. If, however, 93 the underlying system has higher order nonlinearities, and the PDMs are acquired from only the rst and second order kernels (as currently done), then these PDMs may or may not have nonlinear interactions or 'cross-terms' which also contribute to the output thus violating the 'independence assumption'. Our group has devel- oped methods for statistically testing whether such cross-terms exist, and, if they do, how one may characterize the system without resorting to a full multivariate nonlinearity (Marmarelis, 2004; Marmarelis et al., 2013b). In this work, however, all of the prethreshold dynamics of the simulated systems are second-order, and thus these methods will not be used. It should be noted that even though the threshold trigger in spiking systems technically makes the nonlinearity 'innite-order', it has been shown that the number of Wiener kernels needed to completely characterize such a system is at most the order of nonlinearity of the prethreshold dynamics (Marmarelis, Citron, and Vivo, 1986). 4.3.6 Simulations & Model Assessment The ecacy of the STC and PDM methods was compared using synthetic spatial and temporal systems where ground truth was available. In order to study the dierences between the two methods more generally, a Monte Carlo approach was used with several trials. In each trial, a physiologically realistic 2nd order system, such as that shown in Fig.1a was obtained by making the 1st and 2nd order kernels equal to the randomly weighted sum of L = 3 Laguerre basis functions (and for the 2nd order case, the outer products of these basis functions) (Ogura, 1985). Unless otherwise noted, the 1st and 2nd order kernel were normalized so that their respective output power would be equal. The output was generated by feeding a random Gaussian input through this system and then thresholding the output. The rst two-thirds of the data was used for model estimation (training set), while the 94 remaining third was used for model assessment (testing set). The PDM/STC lters were also estimated using correlated inputs in order to test the correction methods described in section 4.3.4. Two forms of correlated inputs were used: correlated gaussian processes (with only 2nd order statistical cor- relations) and natural images. Correlated Gaussian inputs were obtained by feeding GWN though a lowpass lter. Natural images were obtained from the dataset used in Touryan, Felsen, and Dan (2005) (see Dan, Felsen, and Touryan (2009)). Two metrics were used to evaluate the goodness-of-t of the estimated models by comparing the estimated continuous output, ^ y[t] with the true binary output y[t]. ROC curves, which evaluate the performance of a binary classier (Caru- ana and Niculescu-Mizil, 2004; Zanos et al., 2008), plot the true positive rate (TPR=P (^ y b [t] = 1jy[t] = 1)) against the false positive rate (FPR=P (^ y b [t] = 1jy[t] = 0)) over the putative range of threshold values, T , such that ^ y b [t] = 8 > > < > > : 1 ^ y[t]>T 0 ^ y[t]T The area under the curve (AUC) of ROC plots is used as a performance metric of the model, and has been shown to be equivalent to the Mann-Whitney two sample statistic (Hanley and McNeil, 1982). The AUC, which is obtained by numerically integrating the ROC curve, ranges from 0 to 1, with 0.5 indicating a random pre- dictor and higher values indicating better model performance. The second metric used was the Pearson correlation coecient, , between the estimated prethresh- old output, ^ y[t], and the true output, y[t]. Unless otherwise mentioned, all model assessment metrics were evaluated on the testing set. 95 4.4 Results 4.4.1 Illustrative Cases In this section, we present three illustrative systems, one temporal and two spatial, where the STC and PDM methods were compared in terms of their ability to re- cover the underlying ground truth linear lterbank and their ability to predict the output spike train. In the rst case, a GWN input was fed through the 2nd order Volterra system in Fig.1a and then thresholded to generate the output spike train. As explained in section 4.3.5, every 2nd order nonlinear system can be decomposed into a Wiener-Bose cascade consisting of a linear lterbank followed by independent quadratic nonlinearities. An equivalent cascade model of the previously mentioned system appears in Fig.4.1b. Note that while both representations of Fig.1 describe the same system, they both showcase dierent aspects of the system. For exam- ple, the large positive central hump negative corners of the 2nd order kernel in Fig.1a show pairwise facilitation for nearby stimuli and pairwise inhibition for stim- uli which are more distally separated. Alternatively, the three linear lters in Fig.1b show three distinct modes, each with their own frequency proles, which contribute quadratically to the output. Ideally, a researcher would use both representations synergistically to obtain a deeper understanding of the nonlinear dynamics than would be possible from either representation in isolation. In order to provide intuition on dierences between the second order Wiener kernel, h 2 , and the STC matrix, ^ C, 4 methods were used to identify the underlying system from the input and output data records. First, the traditional STC approach was used, as dened in Eq. 4.7. The obtained STC matrix is shown in Fig. 2a(i). As noted previously in Eq. 4.8, this matrix has 2 modications from the canonical Wiener kernel, h 2 (shown in Fig.1): the addition of the input autocorrelation, M , 96 Figure 4.1: (A) Input/output data records were generated by feeding GWN (x[t]) through a 2nd order Volterra system dened by a 1st and 2nd order kernel (K 1 andK 2 ) and threshold- ing the continuous output to generate the binary output, y[t]. (B) Equivalent Wiener-Bose cascade model of the system shown in (A) composed of 3 linearly independent lters followed by static quadratic nonlinearities. which can be seen as the 'delta function' down the matrix diagonal, and the inter- mixing of rst and second order dynamics (M 1 ). These modications prohibit any straightforward interpretation or prediction from the STC matrix and relegate it to being an intermediate step for obtaining the STC lters, which are predictive and interpretable. The eigenvalue distribution of the STC matrix, along with the associated boot- strap condence bounds for the largest and smallest eigenvalue (shown in grey) can be seen in Fig.2a(ii). As can be seen, there are clearly three signicant eigenvalues (the condence bounds were also recomputed for the 2nd highest eigenvalue). The set of signicant lters, which is dened as the signicant STC eigenvectors along with the STA, and their respective associated nonlinear functions are shown in Fig. 2a(iii,iv). In Fig.2b, the STC matrix is shown with the input autocorrelation subtracted, as was done in Arcas, Fairhall, and Bialek (2003); thus, only the M 1 modication was present. As can be seen, when the input is uncorrelated ('white'), this modi- 97 cation asymptotically has no eect on the obtained STC lters; when dealing with nite data records, however, it may induce negative artifacts. In Fig.2c, the M 1 modication was removed, resulting in a dierent set of STC matrix eigenvectors. However, as shown by Samengo and Gollisch (2013), when the STA is included, these eigenvectors should span the same linear subspace as the linear lters ob- tained from the traditional STC matrix. Finally, in Fig. 2d, both modications were removed to obtain the estimated canonical Wiener kernel, ^ h 2 (to be compared with the true Wiener kernel, h 2 in Fig.1a). This is the only kernel whose elements can be interpreted as the weight assigned by pairwise interactions of the inputs. The PDMs, obtained by the SVD method, are shown in Fig.2d(ii). The method recovered a compact and ecient set of 3 linearly independent lters. Notice that while the PDMs and STC lters should theoretically span the same subspace, the obtained PDMs much more closely match the linear lters used to generate the out- put (Fig.1b). The projection of spike-triggering stimuli (blue dots) and non spike triggering stimuli (red dots) onto the rst two PDMs can be further seen in Fig. 2g. As can be seen, the rst two PDMs are able to achieve a large degree of separability between the stimuli, although inclusion of the 3rd PDM is needed to achieve near perfect separability. The predictive power of both the kernel models, evaluated using Eq. 4.1, and the Wiener-Bose models (Eq. 4.12) are shown in the ROC plots of Fig. 2e,f for all 4 permutations of the STC method. As expected, the estimated Wiener kernels outperformed all other kernel models (Fig. 2e). This is because, asymptotically, the Wiener kernels can be shown to be optimal. More interestingly, the PDMs outperformed the STC lters, even though they theoretically should span the same subspace (Fig.2f). This occurs since the STA and STC matrix eigenvectors are linearly dependant and thus the ANF cannot be calculated independently for each 98 lter. Note that this occurs even though all eigenvectors were deemed signicant via the bootstrapping method. In order to avoid this issue, one could use cross- validation rather than the bootstrapping approach to determine which lters to include. Using, this method, the 3rd STC matrix lter was shown to oer no additional predictive power and deemed super uous. This slightly raised the STC lter predictive power; however, it was still slightly lower than that of the PDMs (AUC=.985 vs AUC=.991). The 3 PDMs actually have better predictive power than the corresponding Wiener kernel model from which they were derived (AUC=.991 vs AUC=.984). This shows that the rst 3 PDMs contain all the information of the 2nd order Volterra model The next illustrative example examines the well-known Adelson-Bergen energy model (Adelson and Bergen, 1985) where the input is fed through two linear lters, whose output is then squared and summed: y[t] = [( ~ P 1 ~ x[t]) 2 + ( ~ P 2 ~ x[t]) 2 ] (4.18) The schematic of this model is shown in Fig.4.3a. This model is meant to repre- sent the receptive eld of an ideal V1 complex cell, whose response incorporates phase invariance. In this model, there are only 2nd order (quadratic) dynamics. Thus, the STA or 1st order Winer kernel, shown in Fig.4.3b, is zero. This means that the M 1 modication present in the STC matrix is irrelevant, leaving only the input-autocorrelation modication, M . Note that here, unlike in the previ- ous 1-dimensional (temporal) example, visually examining the 2nd order kernel is meaningless since the 2D (spatial) input was arbitrarily rearranged into a 1D vec- tor. However, interpretation of the 2nd order Wiener kernel may still be done by considering only specic parts of it, such as the diagonal (Citron and Marmarelis, 1987). Fig.4.3c,d shows the eigenvalue distribution, linear lters, and nonlinearities 99 of the STC and PDM methods. As can be seen, both methods recover the ground truth lters very accurately. Furthermore, as can be presumed from the similarity of the obtained lters, both methods had roughly equal predictive power. This shows that in the case of uncorrelated inputs, the M modication has an asymptotically negligent eect on the recovered lters. As we shall see later, however, this is not the case for correlated inputs when the input autocorrelation is more complex than a delta function down the kernel diagonal. In the nal illustrative example, a sample V1 cell is constructed with 3 lters: two lters with the same orientation, but out-of-phase, and a 3rd lter whose orientation is 180 0 opposed to the rst two (Fig.4.4a). The recovered STA (Fig.4.4b) was a jumbled version of both orientations and thus was dicult to interpret. Once again both the STC (Fig.4.4c) and PDM (Fig.4.4d) methods were used to characterize the system from the generated input and output data series. The obtained PDMs had slightly better predictive power than the STC lterset (AUC PDM =.985 vs. AUC STC =.98). Furthermore, the PDMs were much closer to the original rotation of the ground truth lters, thus facilitating interpretability (see Pillow and Simoncelli (2006) and Kaardal et al. (2013) which explore this vital issue more deeply). 4.4.2 General Trends The previous three illustrative examples were meant to provide intuition concerning how the linear lterbanks estimated by the PDM and STC methods compared with the underlying ground truth system, and how well these methods were able to sep- arate between spike-triggering and non-spike-triggering stimuli (i.e. their predictive power). In this section, Monte Carlo simulations are used to explore more gener- ally how these two methods compare in terms of their predictive power, overtting, robustness to noise, and convergence or 'data-hungriness'. In all these cases, the 100 recovered models were assessed based on their predictive power rather than on how closely they recovered the ground truth system. This is because neither method promises that their recovered lters will be equivalent to the ground truth lters, but only that they will span the same linear lterspace. This property is implicitly measured by the recovered systems' predictive power since it is reasonable to assume that as the recovered lterspace moves away from the true lterspace, its predictive power will decline. The predictive power of the two models was evaluated on T = 30 randomly generated systems with GWN inputs (see Methods, section 4.3.6). As noted pre- viously, all synthetic systems could be presented as a 2nd order Volterra system, or, alternatively, as a Wiener-Bose cascade consisting of three independent linear lters. In Fig.4.5a, the predictive power was measured as a function of the number included lters. For the STC method, the rst included lter was the STA. As can be seen, the PDMs modestly outperformed the STC lterset for any amount of included lters. Notably, the 1st PDM outperformed the STA lter showing that even in a Wiener-Bose cascade model with a single linear lter, the STA is sub- optimal. Furthermore, as suggested by the example in Fig.2, the predictive power of the STC lterset decreased with the inclusion of the 3rd signicant STC lter (4th lter total) due to the linear dependance between the STA and STC lters. This occurs even though the 4th lter was deemed signicant by the bootstrap method, demonstrating the need for cross-validation as essential for lter selection. Fig.4.5b compares the predictive power of the PDM and STC ltersets (with 3 l- ters). As can be seen, the PDMs outperformed the STC lters in almost every trial ( PDM =:843vs STC =:815;P <:001). In experimental physiological system identication, the experimenter requires criteria to know how much data is needed to estimate a valid model. Usually, 101 this choice is constrained by the inherent nonstationarities of physiological systems. Model convergence or data hungriness describes the amount of data needed to suc- cessfully model the system. Overtting describes how well the model estimated on the given dataset will generalize to other datasets acquired from the same system. To assess these properties, N = 30 trials were conducted with randomly generated synthetic systems as described previously. For each synthetic system, the STC and PDM estimation methods were applied to randomly generated training/in-sample data of various lengths from 200 to 15,000 bins. Then the predictive performance of the estimated lters was evaluated on testing/out-of-sample data. The predictive power of the training and testing sets (solid and dashed lines respectively) is shown in Fig.4.6a for the metric. The convergence of the data is described by the shape of the testing set curve in Fig.4.6a,top, and in particular how quickly this curve approaches a steady-state value. In order to facilitate comparisons between the two methods, their predictive power was plotted as a percentage of their maximal value in Fig.4.6a,bottom. As can be seen, in noise-free conditions, it takes both methods only about 150 spikes to converge to over 90% of their maximal value. The singular value distribution of the PDM method for a sample system is shown in Fig.4.6b. As shown by the red line, with only 200 output spikes the PDM eigenvalues can be distinguished from the remaining eigenvalues. In experimental settings noise is an unavoidable reality which may arise in a variety of ways, including imperfect instrumentation and/or from unobserved inputs. Thus, any input-output model must be robust, i.e. be able to accurately estimate the underlying system even in the presence of noise. Here, the robustness of the PDM and STC methods was assessed by seeing how their predictive power declines as increasing amounts of uncorrelated gaussian noise was added to the input time series. The results are shown in Fig.4.6c. Notice that training set prediction results 102 (solid line) rapidly decline with increasing noise since noise was added only to the training input; however, the models estimated from these noisy input-output time series are still good as can be seen by their prediction results on the testing set (dashed line), where no input noise was added. Both methods proved to be very robust and their predictive power declined by less than 10% even when 60% of the input power was caused by noise. Fig.4.6d shows the PDMs obtained from the system in Fig.1 when the input was composed of 60% noise. As can be seen, the obtained PDMs still faithfully represent the ground truth system. 4.4.3 Correlated Inputs In all of the previous simulations uncorrelated Gaussian inputs were used to stimu- late the model. As discussed previously in Methods, section 4.3.4, in many situations either the experimenter does not have the luxury of designing the inputs, or chooses to use correlated inputs such as natural images. Such correlated inputs will bias the obtained PDM/STC lters necessitating procedures to correct the estimated lters. In this section we will present two illustrative examples, one temporal and one spatial, where the system was stimulated with correlated inputs and how the correction procedures discussed in section 4.3.4 were used to correct the estimated system. In Fig.4.7a, a 2nd order Volterra system was stimulated with a correlated Gaus- sian process and thresholded to obtain a binary output. Fig.4.7b shows the kernels obtained through 3 methods: (I) the biased Wiener kernels obtained through the standard procedure without correction (Eq. 4.4b), (II) the correction approach originally used in Touryan, Felsen, and Dan (2005) and later developed in Samengo and Gollisch (2013) (Eq.4.14), and nally (III) the approach originally derived by Koh and Powers (1985) and advocated here (Eq. 4.16, bottom row). It should 103 be noted that in lterset II, the input autocorrelation modication, M , is not re- moved which is a serious source of bias when using correlated inputs. These dierent lter sets will henceforth be referred to as lterset I, lterset II, and lterset III, respectively. Fig.4.7b-d shows the kernels, eigenvalue/singular value distribution, and obtained lters from all three methods. Note that the corrected Wiener ker- nel from Fig.4.7d(i) obtained near perfect predictive power (as seen in Fig.4.7e), thus validating the correction formula from Eq. 4.16. Note that all methods only showed two signicant eigenvalues, even though there are three ground truth lters. The reason for this is presumably because the 3rd lter is inhibitory and thus the application of a relatively high binary threshold may have obstructed its recovery (Marmarelis, Citron, and Vivo, 1986). As mentioned previously, quality of the ob- tained ltersets must be assessed by their predictive power, which quanties how closely the obtained lterset spans the true lterspace. As can be seen in the ROC plot in Fig.4.7f lterset III has a signicantly higher AUC value than the other two methods. The superiority of lterset III can also be seen by comparing it with the ground truth lters in Fig.4.7a. Its two lters match almost exactly the rst two lters of the system. The two lters of lterset I and II, on the other hand, are strongly biased versions of the rst two ground truth lters It should also be noted that recent work by Aljade et al. (2013) has proposed alternative methods on how to recover optimal linear subspaces when using highly correlated Gaussian inputs, which may further improve results here and possibly recover the 3rd linear lter. In the second illustrative example, the Adelson-Bergen energy model of Fig.4.3a, was stimulated with correlated (blurred) Gaussian images obtained using a 2D Gaus- sian lter of 10 pixels, and 1 pixel variance (Fig.4.8a) and natural images (Fig.4.8b). The recovered lters are shown in Fig.4.8c-d. Once again, no method was able to perfectly recover the original lters. However, once again, by using predictive power 104 as a measure for how close the lterspaces are too each other, it can be seen that in the case of blurred images, lterset III almost perfectly spans the ground truth lterspace, as it achieved near perfect predictive power (AUC=.999). Thus, even though lterset III does not appear visually similar to the ground truth lters, they span the same space. In the case of natural images, lterset III has a much lower predictive power than it did with blurred images; however, it still performs signicantly better than the other two methods. This is to be expected since the correction in Eq. 4.9 was derived to be used exclusively with correlated Gaussian inputs such as those in Fig. 4.8a. In fact, the use of STC with non Gaussian inputs, the obtained biases, and possible solutions have been previously discussed in the literature (Pillow and Park, 2011; Fitzgerald et al., 2011; Rajan and Bialek, 2013). What we have demonstrated is that, at least in the example presented here, the method still works well even with highly non Gaussian inputs such as natural images. Even better results were obtained with uniform and tertiary inputs put through a lowpass lter (results not shown). We should note, however, that strong regularization was needed to obtain good results with natural images. In certain very noisy systems, the methods used to obtain lterset I and II may be more appropriate. Once again, the practitioner is advised to use the predictive power of the obtained lters to validate whatever method they choose to use. 4.5 Discussion Spike triggered covariance (STC) is currently the standard technique in sensory neu- roscience for characterizing nonlinear systems whose output is a nonlinear function of the outputs of multiple nonlinear linear lters/receptive elds. In this work, the STC method is examined through the Voltera/Wiener framework of characterizing 105 nonlinear systems by explicitly deriving the relationship between the STC matrix and the rst two Wiener kernels (Eq. 4.8). Our main nding is that the STC ma- trix is actually a modied version of the 2nd order Wiener kernel which incorporates the input autocorrelation (M ) and mixes 1st and 2nd order dynamics (M 1 ). Fur- thermore, it is shown how one may obtain the Wiener kernels while preserving the simplicity and elegance of the STC method and with no additional computational expense, i.e. for free (see Eq. 4.9). Parallel to the popularization of the STC method in sensory neuroscience, the Principal Dynamic Modes (PDMs) were developed in nonlinear dynamical systems identication as an ecient lterset to describe the linear dynamics of nonlinear systems (Marmarelis, 2004). Functionally, the PDMs are equivalent to the STC lters. However, perhaps because the PDMs have largely (although not exclusively) been used to characterize continuous dynamical systems, such as cerebral hemody- namics, where the context of a 'receptive eld' is not applicable they have largely escaped the attention of sensory neurophysiologists. The PDMs are obtained from a second order Volterra system and attempt to answer an open question in the STC literature: how to combine the STA and STC to obtain the optimal set of linear lters for the system. Although the STC matrix can be considered a 'biased' Wiener kernel in the sense that it is suboptimal to use in Eq. 4.2 (see Fig. 4.2, the STC lterset, which is derived from the STC matrix and is composed of the STA and signicant STC eigenvectors, is nonetheless unbiased and asymptotically spans the same subspace as lters derived from the Wiener kernels - i.e. the PDMs. Given the correspondence between these two ltersets, the question may be asked what benet one gains by using the Wiener kernel/PDMs over the STC matrix/lterset, or at least of establishing the relationship between the two. We suggest the following: rst, unlike 106 the STC matrix which is only an intermediate step to acquire the STC lterset, the Wiener kernel is highly interpretable and may reveal information about the system not apparent from simply examining the linear subspace. Second, the Wiener/PDM method, unlike STC, allows a single unied framework to be applied to systems with both continuous and binary outputs or for the neuronal case, to both subthreshold and suprathreshold dynamics (see Fournier et al. (2014)). Third, one may apply the vast body of theory developed for Wiener/Volterra kernels, particularly their convergence properties, to STC (see below). Finally, the PDM extraction method presented here allows one to recover a linearly independent lterset in a single step rather than rst obtaining the signicant STC lters and then combining them with the STA. Furthermore, the PDMs are shown to have slightly better predictive power than the STC lters, and in many cases may be more interpretable (see Fig. 4 and Cantrell et al. (2010)). By explicitly establishing the relationship between the STC matrix and the 2nd order Wiener kernel, one gains 'for free' all the prior theoretical work done on Wiener theory. For example, it was believed for a long time that the STC method is only valid for Gaussian inputs (Paninski, 2003). Very recently, it was shown that the STC is actually consistent for arbitrary spherical inputs (Samengo and Gollisch, 2013). This result is corroborated by noting that the rst two Wiener kernels are very similar for a wide range of inputs including uniform and discrete-level inputs (such as binary and tertiary); the only dierences would be along the 2nd order kernel diagonal which involves fourth order moments of the input used (Marmarelis and Marmarelis, 1978). However, it can be shown that if systems with higher order nonlinearities are estimated with a 'truncated' second order model, the projection of the higher order nonlinearities onto the rst two kernels will dier based on the statistical properties of the input used. Thus, even though the kernels estimated 107 from Eq. 4.4 are the best possible kernels for a wide range of inputs, they will dier depending on the specic input used, based on how this truncation bias expresses itself (Marmarelis and Marmarelis, 1978). The same comments would apply to any linear lterbank estimated from the Wiener kernels since it would simply be a compact representation of the latter (see Eq. 4.12). Furthermore, Wiener kernels have been derived for Poisson inputs which are not symmetrical (Marmarelis and Berger, 2005; Sandler et al., 2015). These Poisson Wiener kernels are particularly important when one wishes to characterize neurons whose only identiable inputs are the spiking history of their input neurons rather than external stimuli such as an images or sound (Song et al., 2007; Zanos et al., 2008; Berger et al., 2012). Perhaps the most important gain from Wiener theory is how to correct Wiener kernels estimated from correlated Gaussian inputs (Eq. 4.15). This correction scheme was used here and found to be superior to commonly used previous methods both for correlated Gaussian inputs and natural images (Figs. 4.7,4.8), although it should be noted that for natural images with higher order correlations, recently proposed methods may further improve STC results Aljade et al. (2013). In this work, in order to emphasize the relationship between the PDMs and the STC lters, the PDMs were derived from Wiener kernels estimated using the cross- correlation technique (CCT) (Lee and Schetzen, 1965). However, several improve- ments have been made to the PDM method which make the PDMs used in practice far superior to those presented here. First, the CCT is largely obsolete as a method of estimating the Wiener/Volterra kernels (Korenberg, Bruder, and Mcllroy, 1988). Rather, current practice in our group is to estimate the Volterra kernels by rst projecting the input onto a set of ecient basis functions such as the Laguerre basis set (Marmarelis, 1993) or b-splines (Song et al., 2013) and then estimating the ker- nel coecients through least squares estimation (LSE) or generalized linear models 108 (GLMs) (Marmarelis, 2004; Song et al., 2007; Truccolo et al., 2005). The obtained kernels are found to be much more robust than the corresponding CCT kernels and can be estimated using much less data. Most importantly for sensory neurophysiol- ogists, these kernels can be estimated from highly correlated inputs such as natural images without having to apply any further correction schemes (Marmarelis, 1994). Furthermore, the static nonlinearity can also be estimated much more eciently by expanding it on a polynomial basis (Marmarelis, 2004). We have found that a cubic (3rd order) polynomial is sucient to model the nonlinearity of most physiological systems. Another advantage of these estimation techniques is that they enable one to explicitly include interneuronal and spike-history eects, such as bursting and the afterhyperpolarization into the model (Eikenberry and Marmarelis, 2013; Song et al., 2007). Most neurons are in uenced by these eects, and thus any STC l- ters obtained without considering them will be biased, as has been shown in Pillow et al. (2008) In the future, we hope to apply these techniques to the receptive eld identication problem. 109 20 10 K 2 0 0 10 -1 0 1 2 20 STC Matrix 0 5 10 15 -1 -0.5 0 0.5 1 Recovered Filters Filt#1 Filt#2 Filt#3 STA -10 0 10 -0.5 0 0.5 1 Recovered ANFs -10 0 10 20 30 0.5 1 1.5 2 K 2 Eigenvalues 20 10 0 0 10 -0.2 0 0.2 20 STC Matrix - M Φ 20 10 0 0 10 -1 0 1 2 20 STC Matrix - M 1 20 10 0 0 10 -0.2 0 0.2 0.4 20 Wiener Kernel PDM 1 -10 0 10 PDM 2 -10 -5 0 5 10 WIEN Projection False Positive Rate (FPR) 0 0.05 0.1 True Positive Rate (TPR) 0 0.2 0.4 0.6 0.8 1 PDM ROC STC. A=0.978 STC-M Φ . A=0.978 STC-M 1 . A=0.986 WIEN. A=0.991 False Positive Rate (FPR) 0 0.05 0.1 True Positive Rate (TPR) 0 0.2 0.4 0.6 0.8 1 Kernel ROC STC. A=0.732 STC-M Φ . A=0.935 STC-M 1 . A=0.782 WIEN. A=0.984 (A) (i) (E) (F) (G) (iv) (iii) (ii) (D) (B) (C) Figure 4.2: (A-D) Results of 4 estimation methods to characterize system in Fig. 1. Note (ii) shows grey condence bounds for largest and smallest eigenvalues (see section 4.3.3. (E) ROC plots of prediction results using only 2nd order kernels (Eq. 4.2) before they were decomposed into linear lters and static nonlinearities(Eq. 4.12). (F) ROC plots showing prediction results using estimated linear lters & static nonlinearities. Notice that ROC results for STC and STC-M lters are practically identical since the M modication has minimal eect on the obtained lters when using uncorrelated inputs. In both (E) and (F) the light blue line (TPR=FPR) indicates a model with no predictive power. (G) Projection of spike triggering stimuli (blue) and non spike triggering stimuli (red) onto the rst 2 PDMs (shown in D(iii)) note that separation is imperfect even without noise. The 3rd PDM is needed to achieve optimal results. WIEN=Wiener Kernel. PDM=Principal Dynamic Mode. All column axis are equivalent. 110 Figure 4.3: (A) schematic representation of Adelson-Bergen energy model. (B) Recovered STA from energy model contains no information. (C) STC eigenvalues and obtained lters. Note only 2 lters contain information. (C) Wiener kernel eigenvalues and recovered lters. 111 Figure 4.4: (A) Schematic of sample nonlinear Gabor model containing three underlying linear lters. (B-D) are same as in Fig. 4.3. Note STC was only able to detect 2 of the three ground truth linear lters. ANF=Associated nonlinear function. 112 # of Filters 1 2 3 4 Predicitive Power (ρ) 0.65 0.7 0.75 0.8 0.85 STC PDM ρ STC (3 Filters) 0.7 0.75 0.8 0.85 0.9 ρ PDM 0.7 0.75 0.8 0.85 0.9 (A) (B) Figure 4.5: Predictive power of traditional STC vs PDM method. (A) Shows predictive power as a function of included lters. For STC, the 1st included lter was the STA. Notice that including the 4th signicant STC lter actually slightly reduced predictive power. Error bars show S.E.M. (B) Predictive power for T=30 trials. All predictive power results are evaluated on testing set data. ρ 0.5 0.6 0.7 0.8 0.9 Model Convergence & Overfitting Log[# Output Spikes] 2 2.5 3 % of Maximal Performance 60 70 80 90 100 STC PDM Log[# Output Spikes] 1 2 3 4 Eigenvalue Distribution -2 -1 0 1 2 3 Convergence of PDM Eigenvalues ρ 0.4 0.6 0.8 1 Model Robustness % Input Noise 0 20 40 60 % of Maximal Performance 75 80 85 90 95 100 (A) (B) (D) (C) Figure 4.6: (A) Convergence & Overtting analysis. Top shows the predictive power of both training (solid lines) and testing (dashed lines) sets for data of various lengths ranging from 20-3000 spikes (200 to 15,000 bins). Bottom Indicates the performance of the testing set, normalized by its maximal (steady-state) performance. All points in A and B were averaged over N=30 trials. (B) Eigenvalue distribution for a sample system as amount of output spikes are increased. (C) Model robustness measured by predictive power as input noise is increased. % Input noise=(input noise power/total input power)*100. Same layout as (A). (D) Recovered PDMs of sample system in Fig. 1 when the input was composed of 60% noise. Error bars show in (A) and (C) show S.D. 113 Figure 4.7: (A) A correlated gaussian input was put through sample system and thresholded as in Fig. 1. (B-D) results of 3 three dierent correction techniques for I/O data of (A) (see text). (E-F) Predictive power of obtained kernels (E) and linear lters (F). (G) Projection f data onto 1st two lters of lterset III is able to almost perfectly separate the data. 114 2 4 6 8 10 2 4 6 8 10 Sample Correlated Gaussian Input Filterset I Filterset II Filterset III 0 0.02 0.04 0.06 0.08 0.1 0 0.2 0.4 0.6 0.8 1 Kernel ROC FPR TPR I. A=0.885 II . A=0.88 III. A=0.999 0 0.02 0.04 0.06 0.08 0.1 0 0.2 0.4 0.6 0.8 1 PDM ROC FPR TPR I. A=0.886 II. A=0.886 III. A=0.999 2 4 6 8 10 12 2 4 6 8 10 12 Sample Natural Image 0 0.02 0.04 0.06 0.08 0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Kernel ROC FPR TPR I. A=0.703 II. A=0.854 III. A=0.879 0 0.02 0.04 0.06 0.08 0.1 0 0.2 0.4 0.6 0.8 1 PDM ROC FPR TPR I. A=0.802 II. A=0.827 III. A=0.927 (iii) (iv) (i) (A) (B) (ii) (v) (vi) Figure 4.8: Results of three dierent recovery methods onto correlated Gaussian images (A) and natural images (B). (i) shows sample input (ii-iv) show obtained lters of 3 dierent methods (see text) (v-vi) show predictive power of kernels and linear lter models, respec- tively. ROC=Receiver Operative Curve. PDM=Principal Dynamic Mode. 115 Chapter 5 In-Vivo Predictive Relationship from CA1 to CA3 in the Rodent Hippocampus 1 5.1 Abstract Although an anatomical connection from CA1 to CA3 via the Entorhinal Cortex (EC) and through backprojecting interneurons has long been known it exist, it has never been examined quantitatively on the single neuron level, in the in-vivo nonpatholgical, nonperturbed brain. Here, single spike activity was recorded us- ing a multi-electrode array from the CA3 and CA1 areas of the rodent hippocam- pus (N=7) during a behavioral task. The predictive power from CA3!CA1 and CA1!CA3 was examined by constructing Multivariate Autoregressive (MVAR) models from recorded neurons in both directions. All nonsignicant inputs and mod- els were identied and removed by means of Monte Carlo simulation methods. It was 1 This chapter has been published in Journal of Computational Neuroscience: Sandler et al. (2014) 116 found that 121/166 (73%) CA3!CA1 models and 96/145 (66%) CA1!CA3 mod- els had signicant predictive power, thus conrming a predictive 'Granger' causal relationship from CA1 to CA3. This relationship is thought to be caused by a combination of truly causal connections such as the CA1!EC!CA3 pathway and common inputs such as those from the Septum. All MVAR models were then exam- ined in the frequency domain and it was found that CA3 kernels had signicantly more power in the theta and beta range than those of CA1, conrming CA3's role as an endogenous hippocampal pacemaker. 5.2 Introduction Information ow through the hippocampus has long been viewed in the context of the trisynaptic pathway, a series of feedforward synapses from the entorhinal Cortex (EC) through the Dentate Gyrus and area CA3 to area CA1. However, there has long been evidence to support the notion that CA1 may also causally in uence CA3 (see Fig. 5.1). Deadwyler et al. (1975) showed that population spikes from the hippocampus can reenter the hippocampus via the EC, thus giving the rst evidence of a Hippocampal-entorhinal loop through which the CA1!CA3 causal in uence may occur. It was later found that hippocampal reentrance was facilitated by the connections between CA1, the Subiculum, and the EC (Finch et al., 1986). Also, it was shown that the topology between the EC and the hippocampus is preserved, i.e. information which leaves the hippocampus returns to the same hippocampal neurons via the EC (Buzs aki, 1989; Tamamaki and Nojyo, 1995). Additionally, interneurons have been found in CA1 which directly backproject to CA3 (Sik et al., 1994). Much work has been done to associate the Hippocampal-entorhinal loop with sustaining and spreading seizures. Pare, Llinas, et al. (1992) showed in Guinea Pig 117 slice preparations that seizures will only occur once interictal activity is able to reenter the hippocampus through the Dentate Gyrus. The reverberation of ictal activity through the entorhinal-Hippocampal loop was later corroborated in rats using various in-vitro epilepsy models (Stringer and Lothman, 1992; Nagao, Alonso, and Avoli, 1996; Barbarosie and Avoli, 1997). Bragin et al. (1997) showed that ictal activity in-vivo did not simply reverberate around the hippocampal-entorhinal loop in xed cycles, but was probably sustained by multiple nested oscillators, many of which may independently initiate ictal population spikes. Although much work has been done to study the causal connection from CA1 to CA3 in the context of epilepsy, its function in the working nonpathologic brain re- mains largely unknown. Buzs aki (1989) showed that the ability of population spikes to reenter the hippocampus depends on the behavioral state of the animal. Further- more, it has been shown that the CA1!EC pathway is able to undergo long-term potentiation, suggesting a role for this pathway in learning and memory (Craig and Commins, 2005). Furthermore, much experimental and computational research has implicated reverberatory neural activity between bidirectionally connected regions in sustaining working (short-term) memory (Fuster, 2000; Wang, 2001). In partic- ular, this reverberatory activity can lead to rhythmogenesis which has been widely and successfully linked to performance in dierent memory tasks across several ani- mals (Winson, 1978; Berry and Thompson, 1978; Wiebe and St aubli, 2001; Staubli and Xu, 1995). Thus, given the central role of the hippocampus in consolidating working memories to the cortex, it seems reasonable to hypothesize that bidirec- tional causal connections between hippocampal regions play an important role in this process. Although several studies have explored the causal connection from CA1 to CA3 on a network level, using local eld potentials, and usually in the context of epilepsy, 118 no studies have yet explored this connection on a single neuron level in the nonpatho- logic, nonperturbed brain. In the past, our group has developed several multiple- input nonparametric predictive models characterizing the dynamics from CA3 to CA1 for point-process data recorded in rodents during a behavioral task (Song et al., 2007; Zanos et al., 2008; Marmarelis et al., 2012; Sandler et al., 2015). These models have been validated both analytically and in-vivo, in the context of hip- pocampal prosthesis (Berger et al., 2012; Hampson et al., 2012b). In the present study, we use similar models to show that a 'Granger-like' causal relationship exists from CA1!CA3. Furthermore, we show that the predictive power of this relation- ship is as strong as that from CA3!CA1. Lastly, we show how the bidirectional dynamics between CA3 and CA1 can be used to shed light on the emergence and propagation of the hippocampal theta and gamma rhythms. 5.3 Methods 5.3.1 Experimental Protocols and Data Preprocessing Male Long-Evans rats were trained to criterion on a two lever, spatial Delayed- NonMatch-to-Sample (DNMS) task. Spike trains were recorded in-vivo during per- formance of the task with multi-electrode arrays implanted in the CA3 and CA1 regions of the hippocampus. These experiments were conducted in the labs of Dr. Deadwyler and Dr. Hampson at Wake-Forest University and have been described in detail in our previous publications (Hampson et al., 2012b). Only neural activity from trials where the rat successfully completed the DNMS task was used. Spikes were sorted, time-stamped, and discretized using a 2ms bin. Spike train data from 1s before to 5s after the sample presentation phase of the DNMS task was extracted and concatenated into one time series. In total, 74 CA3 cells and 84 CA1 cells were 119 Figure 5.1: (A) Horizontal rodent hippocampal slice showing anatomical locations of the areas dealt with in our model. (B) Detailed schematic representation of anatomical con- nectivity of hippocampus. Black and red lines show excitatory and inhibitory connections, respectively. Notice that information from CA1 may enter CA3 via a direct inhibitory con- nection (bold red line), or via multisynaptic paths through the entorhinal Cortex (bold black line) recorded during 9 sessions in 7 dierent animals (table 5.1). As previous studies have shown that the cell dynamics vary depending whether the trial involved the left or right lever (Hampson et al., 2012b), separate MIMO models were constructed for both types of trials for a total of 166 CA3!CA1 and 145 CA1!CA3 models. Time series lengths varied due to the number of trials in the session and ranged from 30 seconds to 5 minutes. As the motivation behind our modeling is to quan- tify predictive power between the CA3 and CA1 regions, no attempt was made to separate principal cells and interneurons. 120 Table 5.1: Summary of obtained data. Subsession labels LA, LB, RA, and RB refer respec- tively to left hippocampus A-nonmatch, left hippocampus B-nonmatch, right hippocampus A-nonmatch, and right hippocampus B-nonmatch. A-nonmatch and B-nonmatch refer to the right and left lever during the DNMS task. See Hampson et al. (2012b). Last row indicates totals. 5.3.2 Model Conguration and Estimation Nonparametric multiple-input linear autoregressive models were used to model the dynamical transformation between input and output spike trains (see Fig. 5.2). Thus, each model consisted of a feedforward component, re ecting the eect of the N input cells on the output cell and a feedback/autoregressive component re ecting the subthreshold and suprathreshold eects the output cell has on itself. Thus, the output y is calculated as: y(t) = N X n=1 M X =0 k n ()x n (t) + M+1 X =1 k AR ()y(t) (5.1) wherek n re ects the feedforward kernel of input n, andk AR re ects the autore- gressive/feedback kernel. In order to reduce the amount of parameters in the model, we applied the Laguerre expansion technique to expand the feedforward and feed- back kernels over L Laguerre basis functions (Marmarelis and Orme, 1993). This technique has also been shown to dramatically reduce the amount of data needed 121 to estimate accurate dynamic input-output time-series models (Marmarelis, 2004). Thus, the input and output data records were rst convolved with the Laguerre functions: v (l) x i = M X =0 b l ()x i (t) (5.2) v (l) y = M X =0 b l ()y(t) (5.3) whereb l is thel th Laguerre basis function. By rst convolving with the Laguerre basis functions, the dynamical eects of the past input epochs are removed and we are left with a simple regression of contemporaneous data. Substituting equations (2) and (3) into equation (1) we have: y(t) =k 0 + N X n=1 L X l=1 c l;x i (l)v l;x i (t) + L X l=1 c l;y (l)v l;y (t) (5.4) where c l;x i and c l;y are the feedforward and feedback Laguerre expansion coef- cients. All model parameters were estimated using minimum mean square error (MMSE) estimation. The memory of our system was xed at 300ms, in accordance with previous studies (Song et al., 2007; Lu, Song, and Berger, 2011). The Laguerre parameter was xed at 0.84 to re ect this system memory (Marmarelis, 2004). 5.3.3 Model Selection In theory, the most predictive model would include all recorded inputs. However, such a model would be susceptible to overtting, and would not reveal which neurons are causally connected to each other. To overcome this issue a forward step-wise selection procedure was used to minimize overtting and prune out all inputs which are not causally related to the output (Song et al., 2009a). Given an output cell 122 Figure 5.2: Model Conguration. Each model has N point-process inputs which each go through a linear kernel, K i . These inputs are then summed with the output of the feedback kernel, K AR to generate the nal output, y(t), which is a continuous signal and M potential input cells recorded during the same session, the following steps were used to select the N input cells which are causally connected to the output cell. First, the data was divided into training (in-sample) and testing (out-of-sample) sets. Then, M single-input single-output (SISO) models were constructed with each of the potential inputs. The model whose predicted output had the highest correlation, as measured by the Pearson correlation-coecient, , with the actual output was selected. Afterwards, N-1 two input models were constructed using the previously selected input and one of the remaining potential inputs. If any of the inputs were able to raise , the input which raised the most was selected; otherwise, the procedure was ended, and only 1 input was selected. This procedure was repeated until either none of the inputs were able to raise , or all M potential neurons were selected. The N selected neurons were then used as the model input. 5.3.4 Model Validation To avoid overtting, Monte Carlo style simulations were used to select those models which represent signicant causal connections between input and output neurons and do not just t noise (Zanos et al., 2008). The following procedure was used: in each run the real input was replaced with a surrogate Poisson input having the same 123 mean ring rate (MFR) as the real input. A model was then generated between the surrogate input and the real output, and the Pearson correlation coecient, i , was obtained as a metric of performance. T=40 such simulations were conducted for each output and a set of performance metrics,f i g T i , was obtained. Then, using Fischer's transformation, we tested the hypothesis, H 0 , that was within the population of f i g. If this hypothesis could be rejected at the 95% signicance level, the model was deemed signicant. 5.3.5 Kernel Analysis In order to gain understanding of the underlying CA3!CA1 and CA1!CA3 dy- namics, several features of the estimated kernels were analyzed. The total area, both positive and negative, of a kernel re ects the in uence the associated input has on the output, and thus can be used as a measure of the predictive power the given input has on the output. The excitatory index, dened as the ratio of positive area to total area, was used to quantify whether a given input cell has an excitatory or inhibitory in uence on the output cell. Thus, an excitatory index of 1 corresponds to an entirely excitatory input, while an excitatory index of 0 corresponds to an entirely inhibitory input. To quantify how the kernels contribute to neuronal oscillations, the power within the kernels in the given frequency band was calculated. In order to compare the power in a given frequency band between two kernels, the power in the given band was normalized by the total kernel power. These metrics are summarized in table 5.2. 124 Table 5.2: Summary of kernel analysis metrics used 5.3.6 Principal Dynamic Mode Analysis A major issue of data based system identication is how to extract features of the underlying system from subject to subject variability. This issue is particulary pertinent when dealing with large datasets such as those in this study. One approach developed by our group to deal with this issue is to extract the global Principal Dynamic Modes (PDMs) of the system (Marmarelis, 2004; Marmarelis et al., 2013b; Marmarelis et al., 2014). Essentially, the PDMs are an ecient system-specic set of basis functions which parsimoniously describe the linear dynamics of each input- output transformation. The global PDMs were obtained in a two step process: rst, all kernels of each input from every animal were concatenated in a rectangular matrix. Then singular value decomposition (SVD) was performed on the rectangular matrix to obtain all the signicant singular vectors, which are the global PDMs. Here, two sets of PDMs were obtained for both the CA3!CA1 and CA1!CA3 feedforward transformation. It was found that 3 global PDMs were sucient to describe the linear dynamics of each transformation. 5.3.7 Statistical Analysis Unless otherwise noted, the unpaired Mann-Whitney U test was used to access whether signicant dierences exist between two samples. This test was used since 125 it does not assume a normal distribution, and much of our data was found to be skewed/nonnormal. Shift estimates (Hodges-Lehman) and condence intervals were estimated as prescribed by Higgins (2003). In order to estimate the scale estimate, or the ratio between two samples, the data was rst log-transformed and then scale estimate was taken to be the antilog of the shift estimate. In addition to the Pearson correlation coecient, , Receiver Operating Char- acteristic (ROC) curves were used to visualize model performance. ROC curves plot the true positive rate against the false positive rate over the putative range of threshold values for the continuous output, y (Zanos et al., 2008). The area un- der the curve (AUC) of ROC plots are used as a performance metric of the model, and have been shown to be equivalent to the Mann-Whitney two sample statistic (Hanley and McNeil, 1982). The AUC ranges from 0 to 1, with 0.5 indicating a random predictor and higher values indicating better model performance. The and AUC metrics were chosen as they measure the similarity between a continu- ous 'prethreshold' signal and a spike train. The continuous 'prethreshold' signal was chosen over adding a threshold trigger and comparing true output spike train with an output 'postthreshold' spike train for two reasons. First, this allows us to avoid specifying the threshold trigger value, which relies on the somewhat arbitrary tradeo between true-positive and false-negative spikes (Marmarelis et al., 2013b). Also, similarity metrics between two spike trains often require the specication of a 'binning parameter' to determine the temporal resolution of the metric (Rossum, 2001; Victor and Purpura, 1997). 126 5.4 Results 5.4.1 Estimated Models 166 CA3!CA1 and 145 CA1!CA3 multiple-input single-output (MISO) models, spanning 7 animals (table 5.1), were examined to determine whether a predictive re- lationship exists going from CA1!CA3 and how this relationship compares with the established predictive relationship from CA3!CA1. A representative CA1!CA3 model is shown in Fig. 5.3. Monte Carlo style simulations were performed for every estimated model in order to establish signicance (see methods). Fig. 5.4A shows the results of these simulations for the model shown in Fig. 5.3, which was deemed signicant. Fig. 5.4B shows an example of a model deemed insignicant. To see whether the Pearson correlation coecient was a valid metric to use to compare continuous signals and spike trains, the absolute values were plotted against their estimated signicance level (p-value) in Fig. 5.4C. The green line shows the chance that a model of a given value or higher will be deemed signicant. It is clear that higher values mean that a model is more likely to be deemed signicant. Fur- thermore, if the model's value is greater than 0.2, there is a >95% chance it will be deemed signicant; thus, although low values are inconclusive, a value >0.2, can be used to deem the model signicant without undergoing full Monte Carlo simulations. These facts justify the use of to assess the quality of our models. The feedforward and feedback kernels of all the signicant models are shown in Fig. 5.5. 5.4.2 General Trends The predictive power of feedforward and feedback kernels was compared (Fig. 5.6A). Feedback kernels were found to have signicantly more predictive power than feed- forward kernels. This is not surprising given that hippocampal principal cells have an 127 200 205 210 215 220 225 230 235 240 Output y . MFR: 11Hz Input#4. MFR: 2.3Hz Input#3. MFR: 3.7Hz Input#2 . MFR: 7Hz Input#1. MFR: 2.9Hz Time (s) 0 0.02 0.04 0.06 0.08 0.1 0 0.05 0.1 0.15 0.2 0.25 0.3 True Positive Rate False Positive Rate 0 50 100 150 200 250 300 −5 0 5 10 x 10 −3 Time (ms) Input#1 Input#2 Input#3 Input#4 FB (B) (C) (A) Figure 5.3: Representative CA1!CA3 model with 4 inputs and a single output. (a) Raster- plots of input and output activity over 4 minutes. MFR is abbreviation for mean ring rate. (b) Estimated input (feedforward) and feedback kernels. Notice that many kernels have both excitatory (positive area) and inhibitory (negative area) eects. Furthermore, input kernels #1 and #4 display strong theta oscillations. (c) ROC plot showing model predictive power. The light blue line (TPR=FPR) indicates a model with no predictive power. AUC=668.001. For this model, =0.11 estimated 380,000 synapses (West, Slomianka, and Gundersen, 1991), thus severely limiting the ability of any single input cell to determine the output cell's behav- ior. The ability of a cell's past activity to in uence its current behavior, however, is well known. Pyramidal cells not only have absolute refractory periods lasting a few milliseconds but afterhyperpolarization lasting upto several seconds (Spruston and McBain, 2007). Furthermore, the feedback kernels incorporate the well-known intraregional recurrent connections between pyramidal cells and interneurons within the hippocampal CA3 and CA1 regions (Li et al., 1994; Goutagny, Jackson, and Williams, 2009). In Fig. 5.6B, the total area of each kernel was plotted against the MFR of its associated inputs to see whether there was any relationship between input MFR and 128 0.275 0.2755 0.276 0.2765 0.277 0.2775 0 1 2 3 4 5 6 7 8 9 P<.0001 0.277 Fischer Z−Score 0.133 0.134 0.135 0.136 0.137 0 1 2 3 4 5 6 7 8 9 10 P=0.138 0.135 Fischer Z−Score 0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 P=.05 Significance Level Absolute ρ Z−Score of ρ (A) (B) (C) Figure 5.4: (A,B) Examples of Monte Carlo simulations: For each model, 40 surrogate models with random Poisson inputs of the same mean ring rate were generated. The Fischer z-scores of these models, which are derived from , were plotted as a histogram, while the true value is the plotted dashed red line. The P value for the hypothesis that the true value is greater than the simulated values is printed above the graphs. Models were deemed signicant if P<.05. (A) shows the results for the model in Fig. 5.3, which was deemed signicant. (B) shows an insignicant model (C) Scatterplot of absolute values of and their associated signicance values. The green line shows the probability that a model with a given value or greater will be deemed signicant predictive power. It was found that the most predictive cells were those with a MFR below 6 Hz. It is reasonable to assume that these cells correspond to principal cells. This is not surprising given that the primary output of both regions is provided by principal cells. In fact, the set of highly predictive principal cells may correspond to the so-called functional cell types which selectively re in response to a specic type of stimulus in the DNMS task (ie left-nonmatch) (Hampson, Simeral, and Deadwyler, 1999; Hampson et al., 2012b; Hampson et al., 2012a). Given that these cells are found both in CA3 and CA1 it is clear that these cells would have predictive power over each other. 5.4.3 Bidirectional Predictive Power & Dynamics The predictive power, as measured by values, between CA3!CA1 and CA1!CA3 models was compared (Fig. 5.7). As shown in Fig. 5.7A, 121/166 (73%) CA3!CA1 129 0 50 100 150 200 250 300 −0.05 0 0.05 CA3 → CA1 Feedforward Kernels N=312 Time (ms) 0 50 100 150 200 250 300 −0.05 0 0.05 CA1 → CA3 Feedforward Kernels N=242 Time (ms) 0 50 100 150 200 250 300 −0.05 0 0.05 CA1 → CA3 Feedback Kernels N=65 Time (ms) 0 50 100 150 200 250 300 −0.05 0 0.05 CA3 → CA1 Feedback Kernels N=104 Time (ms) (A) (B) (C) (D) Figure 5.5: All signicant feedforward (top) and feedback (bottom) kernels from CA3!CA1 (left) and CA1!CA3 (right) and 96/145 (66%) CA1!CA3 models were found to be signicant. Although a higher proportion of CA3!CA1 models were signicant, this dierence was not itself deemed signicant (2-sample z test, P=.2). The values of all the signicant models in both directions were then compared (Fig. 5.7B). Once again, no signicant dierences were found in the predictive power of the models (P=.27). To better visualize these results in a single animal, bidirectional connectivity grids were constructed between CA3 and CA1 cells recorded in the same session (Fig. 5.8A) (Kim et al., 2011). Each square in the connectivity grid shows the 130 FF FB 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 Total Power 0 10 20 30 40 50 60 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 MFR (Hz) Total Power Relationship Between MFR and Relative Influence CA3→CA1 CA1→CA3 (B) (A) Figure 5.6: (A) Boxplots of total power of all feedforward kernels (FF, N=659) and all feedback kernels (FB, N=203). The average feedback kernels had 2.82 times the area of the average feedforward kernel (MW test, P<.001. CI=[2.5,3.2]). (B) Scatterplot of input MFR vs kernel total area. Notice there was little to no dierence between CA3!CA1 and CA1!CA3 models bidirectional connectivity between two cells. As can be seen, many pairs of cells have bidirectional connectivity (ie between CA3 cell #2 and CA1 cell #3), while other pairs of cells only have unidirectional connectivity (ie CA3 cell #2 in uences CA1 cell #4, but not vice-versa). Other pairs of cells had no in uence on each other (ie CA3 cell #2 and CA1 cell #9). It was found that cells which had a causal connection in one direction were 16% more likely to have a connection in the other direction (Fig. 5.8B. 2 proportion z-test, p<.001, CI=5%). This suggests that not only are the CA3 and CA1 regions bidirectionally connected, but specic cells within those regions are bidirectionally connected. Such topographical preservation of connectivity between individual CA3 and CA1 cells has been previously shown in physiological and labeling studies (Buzs aki, 1989; Tamamaki and Nojyo, 1995). 131 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0 20 40 60 80 100 120 N=96/145~66%<.05 0 20 40 60 80 100 120 P−Values of Model Significance N=121/166~73%<.05 CA3→CA1. N=166 CA1→CA3. N=145 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0 5 10 15 20 Mean=0.179 Med=0.0973 0 5 10 15 20 ρ Values of All Significant Models Mean=0.128 Med=0.0833 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 31 13 CA3→CA1. N=121 CA1→CA3. N=96 (B) (A) Figure 5.7: Comparison of CA3!CA1 and CA1!CA3 predictive power. (A) Histogram of CA3!CA1 (top) and CA1!CA3 (bottom) P-values, acquired from Monte Carlo simulations (see Fig. 5.4). Dashed red line is the 5% signicance level. (B) Boxplot (right) and histogram (left) of the values of all signicant models. The black line shows the median, while the red line shows the mean Finally, the relationship between excitatory levels from CA3!CA1 and CA1!CA3 in bidirectionally connected cells was examined. A positive trend was found between the excitatory levels in both directions (Fig. 5.8C. Spearman's =.44. p<.001). From a control theory standpoint, this suggests that CA3 and CA1 cells are prone to be wired in a positive feedback loop. Such a structure is particularly prone to the unstable oscillations which characterize epilepsy. Indeed, in-vitro experiments have shown such uncontrolled oscillations in hippocampal-entorhinal epilepsy slices (Pare, Llinas, et al., 1992; Barbarosie and Avoli, 1997). Although the predictive power of both pathways was found to be roughly equal, the dynamics of the two pathways were, as to be expected, asymmetrical. In particu- lar, a dierence in the timecourse of the dynamics was seen in the estimated kernels. The mean normalized RMS power of each set of feedforward kernels can be seen in Fig. 5.9. It was found that the CA3!CA1 kernels had signicantly more power in the rst 10ms (=+13.5%, P=.026, see Fig. 5.9), while the CA1!CA3 kernels 132 0.35 0.60 0.05 0.59 0.50 0.37 0.40 0.75 0.44 0.37 0.54 0.54 1.00 0.67 0.61 0.64 0.53 0.21 0.26 0.28 0.61 0.86 0.57 0.87 0.16 0.48 0.54 0.92 0.70 0.45 0.53 0.43 0.26 0.83 0.51 0.48 0.28 0.76 0.36 0.64 0.72 0.38 0.47 0.57 0.89 0.96 0.63 0.26 0.35 0.82 0.68 0.94 0.42 0.89 1.00 0.54 0.74 0.67 0.49 0.69 0.72 0.97 0.99 0.47 0.98 0.13 0.76 0.15 0.39 0.82 0.55 0.72 0.44 0.68 0.62 0.66 0.83 0.42 0.66 0.85 0.62 0.84 0.43 0.64 1.00 1.00 0.57 0.55 0.39 CA1 CA3 1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8 0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 CA3→CA1 Excitatory Index CA1→CA3 Excitatory Index Excitatory Index. ρ s =0.444. P<.001 S N Tot Tot N S 435 172 263 419 231 188 854 403 451 CA1→CA3 CA3→CA1 Inhibitory Connection No Connection Excitatory Connection Top: CA3→CA1 Bottom: CA1→CA3 Top: CA3→CA1 Bottom: CA1→CA3 Top: CA3→CA1 Bottom: CA1→CA3 (A) (C) (B) Figure 5.8: (A) Representative bidirectional connectivity grid. In each square there are two values representing the excitatory indices from CA3!CA1 (top) and from CA1!CA3 (bottom). Blue shading corresponds to net excitatory connections, while salmon shading corresponds to net inhibitory connections (i.e. excitatory index < 0.5) Blank values indicate no signicant causal connection. (B) Table of signicant connections. S=signicant, N=not signicant, Tot=total. (C) Scatter plot of bidirectional excitatory levels and tted linear regression. had more power from 20-40ms (=+14.5%, P=.005). The slower timecourse of the CA1!CA3 pathway is to be expected giver the much longer anatomical path that information from CA1 takes through the EC to reenter CA3 (although as noted, in- formation from CA1 may reenter CA3 directly through backprojecting interneurons, this pathway is presumed to be more sparse than the EC pathway). 5.4.4 Bidirectional PDM Analysis The global PDMs for each pathway were calculated as discussed in methods and are shown in Fig. 5.10. The PDMs can be seen as a system-specic basis of functions which eciently describe the linear dynamics of each pathway. The 1st PDM of 133 Figure 5.9: (A) mean normalized RMS power of CA3!CA1 and CA1!CA3 pathways. (B) The ratios of power of early dynamics (1st 10ms, pink plot) and late dynamics (20-40ms, blue plot) vs total power were compared between both pathways using a two-sample Mann-Whitney test. It was found that the CA3!CA1 pathway has more power in 1st 10ms (=+13.5%, P=.026) whereas CA1!CA3 has more power later on from 20-40ms (=+14.5%, P=.005). Here 31 and 13 refer to CA3!CA1 and CA1!CA3 kernels respectively. both pathways is almost identical and has its peak in the 0th time bin, indicating concurrent ring in both populations. Such concurrent ring presumably arises through common inputs such as those from the EC and has the eect of coding common representations of the environment in both regions. The 2nd and 3rd PDMs are dierent in both pathways and presumably represent direct causal in uences from one region onto the next, The 2nd PDM in both pathways has a delayed excitatory eect, which peaks at 10ms in the CA3!CA1 pathway and at 20ms in the CA1!CA3 pathway. The slower timecouse of the 2nd PDM from CA1!CA3 corroborates what was seen in the RMS power timecourse analysis in Fig. 5.9. It should be noted that several groups have shown that electrical volleys induced by stimulation in CA1 take roughly 20ms to reach CA3 (Deadwyler et al., 1975; Buzs aki, 1989; Wu, Canning, and Leung, 1998). The 3rd PDM in both pathways 134 oscillated in the theta range at 5Hz. However, although both PDMs oscillated at the same frequency, they were 180 0 out of phase, with the CA3!CA1 PDM having an initial excitatory component and the CA1!CA3 PDM having an initial inhibitory component. This PDM phase dierence may provide a mechanism for the nding that CA3 and CA1 principal cells re 180 0 out of phase during the theta cycle (Mizuseki et al., 2009). Namely, the theta phase dierence in cell ring may emerge through a theta phase dierence in the dynamical transformations between the regions, which can be observed in the 3rd PDM. Figure 5.10: Global PDMs for (A) CA3!CA1 pathway and (B) CA1!CA3 pathway. 5.4.5 Theta and Gamma Power Hippocampal theta and gamma rhythms have long been shown to be relevant for normal animal behavior (Winson, 1978; Chrobak, Stackman, and Walsh, 1989; Axmacher et al., 2006), and pathological phenomena such as epilepsy (Pacia and Ebersole, 1997; Doose and Baier, 1988; Medvedev, 2001). However the mechanism by which brain rhythms emerge is still not fully understood. Several mechanisms have been suggested to contribute to rhythmogenesis in the hippocampus, includ- 135 ing intrinsic neuronal resonance properties (Leung and Yu, 1998; Hutcheon and Yarom, 2000), intraregional recurrent networks of interneurons and pyramidal cells (Goutagny, Jackson, and Williams, 2009; Buzs aki and Wang, 2012), and external inputs (Buzs aki, 2002; Colom, 2006). In our model, feedback kernels incorporate both intrinsic neuronal resonant properties and intraregional recurrent connections (Kim et al., 2011). The feedforward kernels, on the other hand, represent the ability of the input region to elicit rhythms in the output region. The mean band power of CA3!CA1 and CA1!CA3 feedforward and feedback kernels was calculated and compared (Fig. 5.11A,C). These mean band powers were found to be surprisingly similar in both directions. This may be because of common inputs to both regions which modulate the rhythmic properties of the region. For every frequency, a two sample Mann-Whitney test was used to check for signicant dierences. Amongst the feedforward kernels, the CA3!CA1 direction was found to have a stronger beta/low gamma rhythm (20-40Hz ,+13.9%, P<.01. CI=[5.7,22]%; Fig. 5.11B). Amongst the feedback kernels, the CA3 cells were found to have stronger feedback theta components (4-6Hz, +23.5%, P<.01. CI=[6.4,39]%; Fig. 5.11D). No other signicant dierences were found. 5.5 Discussion 5.5.1 CA1!CA3 In uence In this report we have shown that the model predictive power in the in-vivo rodent hippocampus from CA1 cells to CA3 cells is just as signicant as the model pre- dictive power from CA3 cells to CA1 cells. Although the predictive power of the connections was not shown to be signicantly dierent, our analysis revealed several dierences in the dynamical nature of the two pathways. On the simplest level, the 136 0.04 0.1 0.16 Feedforward Frequency Band Power Comparison CA3→CA1 Feedforward Kernels CA1→CA3 Feedforward Kernels ← CA1→CA3 | CA3→CA1 → 0 10 20 30 40 50 −20 0 20 Frequency (Hz) % Difference 0.05 0.2 Feedback Frequency Band Power Comparison CA1 Feedback Kernels CA3 Feedback Kernels ← CA3 | CA1→ 0 10 20 30 40 50 −50 0 Frequency (Hz) % Difference 10 20 30 40 50 60 0 5 10 15 20 0 5 10 15 20 Feedforward Kernel Gamma Power CA3→CA1. N=334 CA1→CA3. N=325 31 13 10 20 30 40 50 60 5 10 15 20 25 30 35 40 45 0 2 4 6 8 0 2 4 6 8 Feedback Kernel Theta Power CA1 FB Kernels. N=113 CA3 FB Kernels. N=90 CA1 CA3 5 10 15 20 25 30 35 40 45 (A) (B) (C) (D) Figure 5.11: Frequency power band dierences between CA3!CA1 and CA1!CA3 models. (A) Feedforward kernels. Top: mean band power per frequency for both classes. Bottom: Mann-Whitney scale estimate and condence bounds for every frequency in the 0-50Hz range. Positive values re ect CA3!CA1 model is greater. Dierences are signicant whenever condence bounds do not include 0. Notice CA3!CA1 models have signicantly greater beta/low gamma band power (20-40 Hz, indicated by orange bar). (B) Histograms and boxplot of band power values for the beta/low gamma range which are shown in A. Solid violet line and dashed black line show the mean and median of the data, respectively. The boxplot labels of 31 and 13 refer to CA3!CA1 and CA1!CA3 respectively. (C,D) Same as A,B for Feedback kernels. Notice CA3 neurons have signicantly greater feedback theta components (4-6 Hz, indicated by grey bar) CA1!CA3 pathway had slower dynamics than the CA3!CA1 pathway, which is to be expected given the longer anatomical route information takes to return from CA1 to CA3 (see Fig. 5.9). Further dierences were revealed with PDM and fre- 137 quency analysis (Fig. 5.10 & 5.11). PDM analysis in particular proved useful in seperating concurrent ring presumably caused by common inputs (PDM #1) and slower dynamics presumably caused by direct causal connections (PDMs #2 & 3). The nding that both pathways had no signicant dierence in predictive power is very surprising given the tenuous anatomical connections from CA1 to CA3. This predictive capability presumably arises from the combination of direct anatomical connections from CA1 to CA3 and from common inputs into both regions (Fig. 5.1,5.12). What is clear, however, is that the classical view of hippocampal informa- tion processing, in the form of the trisynaptic pathway, may have to be revised to include return pathways that allow the possibility of CA1 having predictive power on CA3. Figure 5.12: Abstract schematic of CA3 and CA1. Notice CA1 can have predictive power over CA3 via anatomical connections (labeled FB) and through common inputs which project to both regions There exist several direct anatomical pathways by which CA1 may casually in- uence CA3. These connections are summarized in Fig. 5.1B. Interneurons which backproject directly from CA1 to CA3 have been reported (Sik et al., 1994). Fur- thermore, there are several routes through the entorhinal Cortex (EC) which connect CA1 to CA3. Information from CA1 may reach the EC either directly, or through the subiculum (Finch et al., 1986). From the EC, information may go back to CA3 either through the Dentate Gyrus, as in the classical trisynaptic pathway (Naylor, 2002), or directly via the temporoammonic pathway (Jones, 1993; Kloosterman, 138 Haeften, and Silva, 2004; Ahmed and Mehta, 2009). There is evidence that multi- ple paths operate simultaneously (Finch et al., 1986). It should be noted that the hippocampal-entorhinal loop has been shown be fundamental for seizures in in-vitro epilepsy models (Pare, Llinas, et al., 1992; Stringer and Lothman, 1992; Barbarosie and Avoli, 1997; Avoli et al., 2002). In particular, ictal activity has been shown to continually oscillate through this loop, thus maintaining seizures (Pare, Llinas, et al., 1992). The exact role this loop plays in the in-vivo nonepileptic brain, however, remains unclear (Bragin et al., 1997). Furthermore unobserved variables may contribute to the predictive relationship between two regions. In our case, these would take the form of common inputs to both CA3 and CA1 (although there are other possibilities, such as physical per- turbation or temperature). Candidates for common inputs include the EC and the medial septum. In the case of the EC, it should be noted that EC layer II projects to CA3 while EC layer III projects to CA1. There is strong evidence to believe that there is a predictive relationship between these two layers (Kloosterman, Haeften, and Silva, 2004), which is enough to insure a predictive relationship between CA1 and CA3. The medial septum may have a particularly important role in contributing to the CA1, CA3 predictive relationship given its purported role as the 'hippocam- pal pacemaker' (Colom, 2006). It has been shown that hippocampal regions have distinct phase preferences with regards to the theta rhythm (Mizuseki et al., 2009). Although there is evidence that hippocampal subregions can generate theta rhythms in isolation (Kocsis, Bragin, and Buzs aki, 1999; Goutagny, Jackson, and Williams, 2009), an external pacemaker input, such as the septum, may be responsible for maintaining the distinct theta phase preferences. These phase preferences can lead to a bidirectional predictive relationship between the regions. It is likely that both direct anatomical connections and common inputs con- 139 tribute to the CA1!CA3 predictive relationship; however the relative in uence of each remains unknown and cannot, in principle, be inferred by simply observing (recording) the regions. The advantage of the data-driven/nonparametric modeling approach used in this study is that the kernels express the dynamics between two regions even when the underlying mechanisms responsible for the dynamics are un- known. This is because the kernels are derived directly from the data without any a priori assumptions about the dynamics. Furthermore, the kernels will not change with future discoveries (Marmarelis, 2004; Song, Marmarelis, and Berger, 2009; Song et al., 2009b). It should also be noted that the 300ms memory of the kernels al- lows the kernels to capture much more complicated dynamics than simple synaptic transmission, which takes 2-3ms from CA3!CA1 via the Schaer-Collateral path- way and takes anywhere from 18-45ms from CA1!CA3 via the entorhinal Cortex Deadwyler et al., 1975; Buzs aki, 1989. These more complicated dynamics encap- sulate the time-course of NMDA and GABA synaptic transmission (100ms, and 200-500ms, respectively; Wang (2001)), intraregional recurrent connectivity, and the afterhyperpolarization (>1s; Spruston and McBain (2007)). 5.5.2 Modeling Methodology In this paper, a linear (1st order Volterra) Laguerre autoregressive model was used to describe hippocampal dynamics (Marmarelis, 2004). This linear model is dier- entiated from an ARMAX model by the use of the Laguerre basis expansion which reduces model parameters and the amount of data needed to estimate the model. In the past it has been demonstrated that the CA3!CA1 relationship is nonlinear, and that at least a 2nd order Volterra model is needed to describe these nonlin- earities (Song et al., 2007; Zanos et al., 2008). Here, we conned ourselves to a linear model for two reasons. First, our goal was not to make the best predictive 140 model, but only to identify predictive relationships amongst the recorded neurons. Second, linear models lend themselves more easily to interpretation via intuitive metrics such as total power and excitatory index. However, there may be nonlinear causal relationships between neurons which the linear model will not detect. For example, an input neuron may not in uence the output neuron directly, but only through cross-interactions with another input neuron. This causal in uence would be detected in a second order Volterra model in the form of cross-terms, but will not show up in a linear model. Thus, the number of causal connections which were found in this study serve only as a lower bound for the true number of causal connections. 5.5.3 Rhythms Feedforward and feedback kernels were analyzed in the frequency domain to see whether they could shed light on how oscillations emerge within regions (via the feedback kernels) and how these oscillations spread to other regions (via the feed- forward kernels). Before discussing the results, the relationship between the kernels and rhythmogenesis should be claried. The feedback kernels re ect the casual eects cells have on themselves. Minimally, these eects would include the afterhy- perpolarization (AHP) induced in cells after action potentials. The AHP can last upto several seconds and results from the various ionic currents which restore a cell back to equilibrium potential (Spruston and McBain, 2007). It has been shown that the AHP in CA1 pyramidal cells has a theta resonance brought about by I h channels (Leung and Yu, 1998; Pike et al., 2000). Additionally, the feedback kernels would incorporate intraregional recurrent connections between cells such as the ex- tensive recurrent principal cell networks in CA3 (Kim et al., 2011; Li et al., 1994). The feedback kernels may also re ect interregional recurrent connections between cells, such as the connections between CA3 and the Dentate hilar region which have 141 been suggested to give rise to theta oscillations (Buzs aki, 2002; Scharfman, 2007; Gonzalez-Sulser et al., 2011). The feedforward kernels on the other hand re ect all the physiological processes which occur between a spike in the input cell and a spike in the output cell. These minimally include axonal conductance in the input cell and synaptic integration mechanisms in the output cell. They could also include propagation via intermediate cells, such as the multisynaptic route from CA1 to CA3 via the EC. On a functional level, however, the feedforward kernels should be seen as a lter for information ow from one region to the other. Thus, the theta and delta peaks found in the feedforward kernels indicate that information packaged within theta and delta oscillations will be preferentially transferred to the adjacent region (Gloveli et al., 1997). In our analysis it was found that CA3 feedback kernels had more theta power than CA1 feedback kernels. This supports previous experimental studies which have shown that CA3 is able to independently generate strong theta rhythms and that it projects these rhythms to CA1 (Kocsis, Bragin, and Buzs aki, 1999; Buzs aki, 2002). Furthermore it has been shown that individual pyramidal cells and interneurons cyclically re in a consistent phase with the theta eld potential (Mizuseki et al., 2009). This is ideal for our analysis which deals exclusively with spike data from individual cells. The CA3 region has also been shown to be an independent gamma rhythm generator and to project these rhythms to CA1 (Csicsvari et al., 2003; Colgin, 2013). Gamma eld potentials, however, are generated by the coordinated activity of interneurons and pyramidal cell ensembles and not individual pyramidal cells. Thus, while pyramidal cells may have subthreshold gamma oscillations, these are not re ected in their spiking activity since their mean ring rate is well below the gamma range (Mann and Paulsen, 2005; Buzs aki and Wang, 2012). This is presumably why there was no gamma dierence in the feedback kernels, even though 142 it is known that CA3 is a gamma generator. There was however more power in the CA3!CA1 feedforward kernels in the beta/low gamma range. This is in accordance with reports showing that gamma rhythms in CA3 may entrain beta rhythms in CA1 (Bibbig et al., 2007). The agreement of our results with previous physiological studies shows that our method can be used to detect intraregional rhythmogenesis and to probe frequency selective information ow between regions. 5.5.4 Potential Applications In the past, our group has done work to implement a hippocampal cognitive pros- thetic between CA3 and CA1 (Berger et al., 2005; Berger et al., 2012; Hampson et al., 2012b). The principal behind the prosthetic is that the predictive relationship from CA3!CA1 can be identied and CA1 neurons can be articially stimulated to evoke their natural response, even if the direct Schaer-Collateral connection is damaged. This essentially creates an electronic 'neural bypass' for the Schaer- Collateral connection. The prosthetic was successfully developed and used to restore a rat's performance ability during a behavioral task (Marmarelis et al., 2012; Berger et al., 2012). Given the nding that the CA1!CA3 predictive relationship is just as strong as the CA3!CA1 predictive relationship, we suggest the prosthetic idea can be extended to a CA1!CA3 prosthetic, and help restore some of the function of CA3 even when CA3 aerents are damaged. Recurrent connections between brain regions have long been thought to be a critical component for sustaining seizures. Several studies have shown that these recurrent connections can amplify ictal activity and lead to the synchronized oscil- lations which characterize seizures (Wendling et al., 2002; Wendling, 2008; Bertram, 2013). In particular, several studies have directly explored the role recurrent connec- tions between the hippocampus and entorhinal Cortex play in epilepsy (Barbarosie 143 and Avoli, 1997; Boido et al., 2014). So far, however, there has been no attempt to create a data-based closed-loop model of this phenomenon. Such a model can potentially be used to rigorously study how ictal activity arises from recurrent con- nections between these regions. Furthermore, one can use such an 'in-silico' epilepsy model to study what eect external perturbations will have on the seizure and its as- sociated oscillations. This is particularly relevant for deep-brain stimulation (DBS) which aims to abort seizures by perturbing the in-vivo epileptic brain. In this study we have shown that there is a recurrent connection between CA3 and CA1 and sev- eral distinctive features of these dynamics have been identied. Currently, work is underway to use our identied CA3!CA1 and CA1!CA3 dynamics as subsystems in precisely this type of closed-loop model (Sandler et al., 2013). 144 Chapter 6 Single-Stream Closed-Loop Modeling of the Hippocampus 1 6.1 Abstract Objective: Traditional hippocampal modeling has focused on the series of feedfor- ward synapses known as the trisynaptic pathway. However, largely ignored feedback connections from CA1 back to the hippocampus through the Entorhinal Cortex (EC) actually make the hippocampus a closed-loop system. By constructing a functional closed-loop model of the hippocampus, one may learn how both physiological and epileptic oscillations emerge and design ecient neurostimulation patterns to abate such oscillations. Approach: Point process input-output models where estimated from recorded rodent hippocampal data to describe the nonlinear dynamical transformation from CA3!CA1, via the Schaer-Collateral synapse, and CA1!CA3 via the EC. Each Volterra-like subsystem was composed of linear dynamics (Principal Dynamic Modes) 1 This chapter has been submitted for publication to the Journal of Neural Engineering on April 13, 2015 145 followed by static nonlinearities. The two subsystems were then wired together to produce the full closed-loop model of the hippocampus. Main Results: Closed-loop connectivity was found to be necessary for the emergence of theta resonances as seen in recorded data, thus validating the model. The model was then used to identify frequency parameters for the design of neu- rostimulation patterns to abate seizures. Signicance: DBS is a new and promising therapy for intractable seizures. Currently, there is no ecient way to determine optimal frequency parameters for DBS, or even whether periodic or broadband stimuli are optimal. Data-based com- putational models have the potential to be used as a testbed for designing optimal DBS patterns for individual patients. However, in order for these models to be successful they must incorporate the complex closed-loop structure of the seizure focus. This study serves as a proof-of-concept of using such models to design ecient personalized DBS patterns for epilepsy. 6.2 Introduction The hippocampus is amongst the most studied of brain regions and has been im- plicated extensively in physiological functions of learning and memory as well as pathologies such as Alzheimer's disease and Epilepsy. Traditionally, information processing through the hippocampus has been viewed in the context of the trisy- naptic pathway, a series of feedforward synapses from the entorhinal cortex (EC) through the Dentate Gyrus and area CA3 to area CA1. However, there has long been evidence to support the notion that CA1 may also causally in uence CA3 (see Fig. 6.1). Deadwyler et al. (1975) was the rst to show that population spikes from the hippocampus may reenter the hippocampus via the EC, thus giving the rst physiological evidence of a functioning hippocampal-entorhinal loop. Since then, 146 multiple anatomical and physiological studies have elucidated the complex intercon- nected nested-loop nature of the hippocampal-entorhinal formation (Finch et al., 1986; Buzs aki, 1989; Tamamaki and Nojyo, 1995; Iijima et al., 1996; Kloosterman, Haeften, and Silva, 2004; Bartesaghi, Migliore, and Gessi, 2006). Our group has recently shown that there exists a signicant predictive relationship between sin- gle neurons from area CA1 to area CA3 in the rodent hippocampus (Sandler et al., 2014). Furthermore, many studies have proposed the hippocampal-entorhinal loop may serve as the anatomical substrate for the uncontrolled network oscillations which characterize seizures (Pare, Llinas, et al., 1992; Stringer and Lothman, 1992; Nagao, Alonso, and Avoli, 1996; Barbarosie and Avoli, 1997; Boido et al., 2014). Finally, the hippocampal-entorhinal loop has been suggested to support the sort of reverberatory activity which has been theorized to be necessary for working memory, a well-known function of the hippocampus (Hebb, 1949; Fuster, 2000; Kloosterman, Haeften, and Silva, 2004) Past work by our group has led to the development of several nonlinear non- parametric predictive models of the transformation of action potential activity from the CA3 to the CA1 region of the rodent hippocampus (Song et al., 2007; Zanos et al., 2008; Song et al., 2009a; Marmarelis, Zanos, and Berger, 2009; Sandler et al., 2015). Most recently, we have utilized the concept of Principal Dynamic Modes (PDMs) to achieve a more compact representation of our model and to facilitate its physiological interpretation (Marmarelis et al., 2013b; Marmarelis et al., 2014; San- dler and Marmarelis, in press). These 'open-loop' models have been validated both computationally and experimentally in the context of hippocampal neuroprosthetics (Song et al., 2007; Berger et al., 2012; Hampson et al., 2012b). Although much attention has been given to quantitatively understanding the CA3!CA1 dynamical transformation, almost no work has been done to quanti- 147 tatively understand the CA1!CA3 dynamical transformation and the resulting hippocampal closed-loop. Here we propose a nonparametric closed-loop point- process (CLPP) model of the hippocampal-entorhinal formation. The closed-loop model is composed of two bidirectionally connected 'open-loop' PDM-based sub- systems which model the nonlinear dynamical transformation from CA3!CA1 and CA1!CA3. The purpose of the closed-loop model is not to accurately predict the output time-series, as is the case with the aforementioned open-loop models, but rather to study the phenomena which emerge from the closed-loop network cong- uration. For example, the response of the system to external perturbation involves not only the feedforward transformation from CA3!CA1, but also to loop eects arising from the CA1!CA3 feedback connection. Thus, even a highly predictive open-loop model may prove misleading when examining this issue due to its lack of feedback connectivity. Once the closed-loop system is estimated from real hippocampal data, it is used to model 'in-silico' the eects of externally stimulating imposed the hippocampal- entorhinal system with arbitrarily designed stimulation patterns. It was found that under broadband (Poisson) stimulation, theta resonant modes emerged only in the closed-loop system, and not in the open loop system, suggesting that closed-loop connectivity is essential for strong network oscillations to occur. The response of the system was then analyzed for random narrowband spike trains of various frequen- cies. It was found that high-theta/low-alpha stimulation (6-14 Hz) greatly increased output activity, while delta (<5 Hz) and low gamma (30 Hz) signicantly reduced it. We suggest that such a model may prove useful in designing ecient neurostim- ulation patterns which aim to abate the uncontrolled network oscillations which characterize seizures. Some of these results were presented previously in the form of a conference paper (Sandler et al., 2013). 148 6.3 Methods 6.3.1 Experimental Protocols & Data Preprocessing A Male Long-Evans rat was trained to criterion on a two lever, spatial Delayed- NonMatch-to-Sample (DNMS) task. Spike trains were recorded in-vivo with multi- electrode arrays implanted in the CA3 and CA1 regions of the hippocampus during performance of the task. These experiments were conducted in the labs of Dr. Deadwyler and Dr. Hampson at Wake-Forest University and have been described in detail in our previous publications (Hampson et al., 2012b). Only neural activity from trials where the rat successfully completed the DNMS task was used. Spikes were sorted, time-stamped, and discretized using a 12 ms bin. Spike train data from 1s before to 3s after the sample presentation phase of the DNMS task was extracted and concatenated into one time series. 6.3.2 Closed-Loop Point-Process (CLPP) Model The closed-loop model of the reciprocal relation between the CA1 and CA3 regions is congured as shown in Fig. 6.1e. It involves two input-output PDM-based subsys- tems, CA3!CA1 and CA1!CA3, (see section 6.3.3 below) which are reciprocally connected. It also involves two external disturbances, e1 and e3, that are the model- prediction residuals of the two input-output models. These disturbances correspond to the exogenous inputs to the closed-loop model that are not accounted for by the two input-output subsystems. Thus, e1 represents all non-CA3 inputs to CA1, such as the direct pathway from EC to CA1 (Yeckel and Berger, 1990; Jones, 1993). Likewise, e3 represents inputs entering the system between CA1 and CA3 e.g. from the prefrontal cortex and septum (Buzs aki, 1996; Colom, 2006). These external disturbances combine 149 with the respective subsystem outputs, CA1p and CA3p, to form the inputs for the subsequent subsystem, CA1m and CA3m. Note that physiologically, CA3p and CA1p, the subsystem outputs, correspond to the spike activity in CA3 and CA1 which is caused by feedback connections from CA1 and CA3, respectively. Thus, CA3m and CA1m, being the sum of feedback and exogenous spikes, correspond to the spike train signals one would measure experimentally. Essentially, our model decomposes the recorded CA3/CA1 activity into endogenous activity arising from the closed-loop nonlinear dynamics (CA3p and CA1p) and exogenous activity arising from external inputs (e3 and e1). Thus, once the CLPP model is estimated from real data, it can be stimulated with any arbitrary exogenous signals, while preserving the endogenous dynamics. In the beginning of each simulation, all of the signals were initialized based on real data, and then CA3m was fed through the CA3!CA1 subsystem to obtain an updated value for CA1p. Then, in order to skip-ahead, or move up one time step each iteration, the last element of the updated CA1p was concatenated onto the previous CA1p signal This process was then repeated for each iteration of the simulation. To see which results arise purely from the closed-loop structure of our model, as opposed to the nonlinear dynamics of either of the component subsystems, an open-loop model, as shown in Fig. 6.2, was also analyzed. In the open-loop model, the connections between the two subsystems were severed, such that stimulating the open-loop model is equivalent to stimulating both subsystems in isolation. 6.3.3 Feedforward Input-Output Models Two Volterra-like PDM models (see Fig.6.3 were used to describe the spike train transformations from CA3 to CA1 (via the Schaer collateral pathway) and from 150 Figure 6.1: (A) Horizontal rodent hippocampal slice showing anatomical locations of the areas dealt with in our model. (B) Detailed schematic representation of anatomical con- nectivity of hippocampus. Black and red lines show excitatory and inhibitory connections, respectively. (C) Schematic of trisynaptic-loop. (D) High level schematic representation of CLPP model. (E) Detailed systems representation of CLPP model 151 Figure 6.2: Open-loop model conguration. CA1 to CA3 (via the subiculum, entorhinal cortex, and dendate gyrus) (Marmarelis et al., 2013b). In the PDM-based model, the input signal is rst convolved with L linear lters (the PDMs) and the PDM outputs are transformed through static nonlinearities, termed the Associated Nonlinear Functions (ANFs). The sum of the ANF outputs is fed into a Threshold -Trigger operator that generates an output spike if its input exceeds a xed threshold value. The PDMs can be understood as a system-specic basis of functions that eciently describe the system dynamics. The ANFs represent the conditional probability of an output spike given a specic PDM input value and can be viewed as link functions connecting a given PDM with the pre-threshold output. Thus, PDMs with larger amplitude ANFs make more signicant contributions to the output. Input-output spike data were used to estimate all parts of the model with methods previously reported (Marmarelis et al., 2013b). For the same input, the CLPP system will generate a much higher MFR output than its open-loop correlate. This occurs due to the binary nature of the CLPP system; namely, the subsystems may only have excitatory outputs (ie 'positive' spikes) and not inhibitory outputs (ie negative spikes). Thus, in closed-loop mode, the input to each subsystem will not only be the exogenous input, but also additional spikes from the other subsystem, which will lead to more spikes in the output. In order to maintain signals with physiologically plausible MFRs, the thresholds of the 152 subsystems were raised relative to open-loop conditions. The open-loop 'reference' threshold was calculated to maximize true positive spikes while minimizing false negative spikes according to the methods dened previously in Marmarelis et al. (2013b). The predictive power of the models was visualized using Receiver Operating Characteristic (ROC) curves, which plot the true positive rate against the false positive rate over the putative range of threshold values for the continuous output, y (Zanos et al., 2008). The area under the curve (AUC) of the ROC plots was used as a performance metric of the models, and has been shown to be equivalent to the Mann-Whitney two sample statistic (Hanley and McNeil, 1982). The AUC ranges from 0 to 1, with 0.5 indicating a random predictor and higher values indicating better model performance. The AUC metric was chosen as it measures the similarity between a continuous prethreshold signal and a spike train, thus allowing us to avoid the somewhat arbitrary process of choosing a threshold trigger value. Monte Carlo simulations were used to test the null hypothesis that the given CA3 (or CA1) neuron can predict the output CA1 (or CA3) neuron's spiketimes signicantly better than a random neuron of the same MFR (see Sandler et al. (2014) for more info). If the null hypotheses could be rejected with > 99% condence, we can conclude that the two neurons under question are indeed bidirectionally connected, thus warranting closed-loop analysis. In the Volterra-PDM model, the domain of each ANF is limited by the power of the input used to estimate the model. This can be seen by examining the domain of each ANF, which spans a nite interval whose endpoints are determined by the maximum/minimum value of its respective PDM input. This presents a problem in the CLPP model, where at each iteration the Volterra-PDM subsystems are fed a new input which cannot be predicted at the outset of the simulation. Thus, there 153 exists the possibility that the PDM outputs / ANF inputs will exceed the domain of the ANFs, and thus the domain of the Volterra-PDM subsystem. To overcome this issue, the ANFs were extended in both directions by the mean of their last two endpoints. In order to examine the emergent properties of CLPP systems, barebones syn- thetic systems were also analyzed where ground truth was available. These systems reduced the complexity of a full Volterra-PDM system by (1) excluding ANFs, mak- ing the threshold trigger the only nonlinearity, (2) Having only one PDM, and (3) having the two subsystems of the CLPP system to be equivalent. Figure 6.3: Volterra-PDM Subsystem Model 6.3.4 Random Narrowband Spike-Train Generation To generate random narrowband (RNB) spike-train disturbances centered around a given frequency, f 0 , the following algorithm was used: rst, a broadband (Poisson) spike train with a given MFR was generated. Then the spike train was ltered through a butterworth bandpass lter with a passband of 4 Hz centered around f 0 . The lter output (a continuous signal) was then thresholded to generate the nal narrowband spike train. The threshold value was selected such that the MFR of the output narrowband spike trains was equivalent to that of the original broadband spike train. An example of a narrowband spike train is shown in Fig. 6.3.4b (top row). Note that unlike classical linear lters, the harmonics off 0 will also appear to varying degrees due to the nonlinearity of the threshold operator. To our knowledge, 154 this useful and ecient method of attaining narrowband spike trains has not been previously discussed in the literature. 6.4 Results 6.4.1 Estimated Volterra-PDM subsystems The estimated Volterra-PDM subsystems for CA3!CA1 and CA1!CA3 appear in Fig. 6.4. It was found that a set of 3 PDMs (with 3 corresponding ANFs and a threshold value) is sucient for each input-output subsystem. The ROC plots shown in Fig. 6.5A indicate that both models have roughly the same predictive capability. Furthermore, using Monte Carlo simulations, it was found that both neurons are indeed bidirectionally connected (P < :0001 in both cases, see Fig. 6.5B,C). This result is corroborated by our previous study where it was found that the CA1!CA3 predictive power was roughly equivalent to the CA3!CA1 predictive power (Sandler et al., 2014). Although this result may seem surprising given the greater number of synapses and exogenous inputs going from CA1 to CA3, as compared to going from CA3 to CA1, it should be noted that anatomical connectivity is only one aspect of predictive power, which is also in uenced by connection 'impact' and unobserved inputs (see section 6.5.2). We note that the frequency-domain representations of the PDMs exhibit spectral peaks in the theta (4-8 Hz) and delta (<4 Hz) bands. These peaks were found to be critical in dening the resonant characteristics of the closed- loop model. Similar results linking the PDM spectral peaks with the celebrated cerebral rhythms were previously obtained in the primate neocortex (Marmarelis et al., 2013b; Marmarelis et al., 2014). 155 0 50 100 150 −0.5 0 0.5 1 Time (ms) CA3→CA1 PDMs PDM#1 PDM#2 PDM#3 0 50 100 150 −1 −0.5 0 0.5 1 Time (ms) CA1→CA3 PDMs PDM#1 PDM#2 PDM#3 0 10 20 30 40 50 0 0.1 0.2 0.3 0.4 0.5 CA3→CA1 PDMs in Freq. Domain Frequency (Hz) PDM#1 PDM#2 PDM#3 0 10 20 30 40 50 0 0.1 0.2 0.3 0.4 CA1→CA3 PDMs in Freq. Domain Frequency (Hz) PDM#1 PDM#2 PDM#3 0 1 2 3 4 0 0.05 0.1 0.15 0.2 0.25 CA3→CA1 ANFs ANF #1 ANF #2 ANF #3 0 1 2 3 4 0 0.1 0.2 0.3 0.4 CA1→CA3 ANFs ANF #1 ANF #2 ANF #3 Figure 6.4: Obtained Volterra-PDM Subsystems. Top row shows CA3!CA1 subsystem, while bottom row shows that of CA1!CA3. Left column shows PDMs, middle column shows PDMs in frequency domain, and right column shows ANFs. Note that several PDMs have frequency peaks in the theta range. 0 0.02 0.04 0.06 0.08 0.1 0 0.05 0.1 0.15 0.2 0.25 False Positive Rate (FPR) True Positive Rate (TPR) Bidirectional ROC Plots CA3→CA1. A=0.582 CA1→CA3 . A=0.61 0.5 0.52 0.54 0.56 0.58 0.6 0 1 2 3 4 5 6 7 8 CA3→CA1 Significance 0.582 AUC 0.52 0.54 0.56 0.58 0.6 0.62 0 1 2 3 4 5 6 CA1→CA3 Significance 0.61 AUC (A) (B) (C) Figure 6.5: (A) Bidirectional ROC plots. (B,C) Results of Monte Carlo simulations for CA3!CA1 (B) and CA1!CA3 (C). Histogram plots the predictive power (AUC) of N=40 random inputs with the same MFR as the true input. The AUC of the true input is shown by the dashed red line. In both cases the true AUC was greater than the random AUC's with P <:0001 6.4.2 Stimulation with Model Prediction Residuals In order to better illustrate the role of the disturbances and intraloop signals in the CLPP model, model prediction residuals were rst used to stimulate the estimated 156 CLPP system. The residuals were dened as: e 3 =CA3CA3 p (6.1) e 1 =CA1CA1 p (6.2) where CA3/CA1 are the actual recorded spike trains and CA3p/CA1p are the sub- system predictions. The results are shown in Fig. 6.6. In this formulation the Volterra-PDM subsystems capture the essential endogenous nonlinear dynamics be- tween the two neurons, while the residuals represent exogenous disturbances that drive the CLPP system. It should be noted that e3 and e1 are tertiary signals, with 1 indicating a false negative and -1 indicating a false positive in the model predic- tions. Physiologically, the -1 values are viewed as exogenous inhibitory signals which prevent an output spike. Since in this case the exogenous inputs are the open-loop residuals, the measured signals (CA3m and CA1m) are by denition equivalent to the recorded signals CA3p and CA1p. 6.4.3 Stimulation with Random Poisson Inputs The operation of the CLPP model can be stimulated with any chosen external disturbances, and not just the empirical residuals (as done above), to examine its response characteristics. Here, the spectral characteristics of the CLPP model were examined through stimulation with two independent random broadband (Poisson) disturbances. The resulting CLPP intra-loop spike trains (shown in Fig. 6.7) were found to exhibit theta band oscillations, which were present in our recorded CA3 spike train and are known to be prevalent in the rodent hippocampus (Buzs aki, 2002; Colgin, 2013). The thresholds of the CLPP subsystems were chosen as to minimize theta resonances in CA1 while maintaining them in CA3, as was observed 157 10 20 30 40 50 e3 Hz 10 20 30 40 50 CA3p 10 20 30 40 50 CA3m 10 20 30 40 50 CA3 10 20 30 40 50 e1 Hz 10 20 30 40 50 CA1p 10 20 30 40 50 CA1m 10 20 30 40 50 CA1 CA3→CA1 CA1→CA3 + + Figure 6.6: CLPP System driven by residual input. Each inset shows the given signal in the frequency domain (below) and time domain (above). The time domain rasterplot is a representative sample of the signal spanning 6 seconds. Disturbance signals e1 and e3 are tertiary signals with positive spikes in black and negative spikes in green. Their corresponding spectra are below in the same color. Each frequency-domain plot shows the Fourier transform of the signal, which was then low-passed through a 20 Hz lter and then smoothed to ease visibility. in real data (see section 6.4.4). It should be emphasized that these resonant modes emerge purely from the nonlinear dynamical nature of our model and not through any of the inputs, which are broadband. To distinguish whether the emergent resonant modes arise from any of the in- dividual subsystems or from the closed-loop conguration of our model (e.g. the specic connectivity of the two subsystems) the open-loop version of our model 158 10 20 30 40 50 e3 10 20 30 40 50 CA3p 10 20 30 40 50 CA3m 10 20 30 40 50 CA3 10 20 30 40 50 e1 10 20 30 40 50 CA1p 10 20 30 40 50 CA1m 10 20 30 40 50 CA1 CA3→CA1 CA1→CA3 + + Figure 6.7: CLPP Model driven by broadband inputs. Layout is the same as Fig. 6.6 except that here the exogenous signals e3 and e1 were binary rather than tertiary. (Fig. 6.2) was stimulated using the same broadband spike trains. The results, shown in Fig. 6.8, show that while the open-loop model can produce very weak theta oscillations, only the full closed loop model can produce the strong theta, and to a lower extend delta, oscillations shown in actual CA3 recordings. The fact that strong theta and delta oscillations are an emergent characteristic of the closed-loop model of the hippocampus, and not the open-loop model, represents a key nding of our study and constitutes a means of validating the CLPP model and its potential utility. 159 5 10 15 20 25 30 e3 5 10 15 20 25 30 e1 5 10 15 20 25 30 Open Loop CA3p 5 10 15 20 25 30 Open Loop CA1p 5 10 15 20 25 30 Closed Loop CA3p 5 10 15 20 25 30 Closed Loop CA1p 5 10 15 20 25 30 True CA3 5 10 15 20 25 30 True CA1 (A) (B) (C) (D) (E) (F) (G) (H) Figure 6.8: Comparison of Open-Loop and Closed-Loop systems driven by identical broad- band inputs e3 and e1. Note robust theta rhythms only arise in the closed-loop case. In order to examine how the specic characteristics of the resonant modes emerge from the Volterra-PDM subsystems, simple barebones synthetic subsystems were analyzed (see methods). A CLPP system was constructed where both subsystems were identical and consisted only of a single PDM and a threshold trigger (without an ANF). The single PDM, shown in Fig. 6.9a, consisted of a cosine function of frequency, f 0 . When this system was stimulated with Poisson spike trains, the CLPP output signals (CA3m,CA1m) were found to have resonant modes exactly at the PDM frequencyf 0 (Fig. 6.9b). Furthermore, if the PDM had no resonant peak, there were no corresponding resonant peaks in the CLPP output signals (results not shown). These results, which were obtained over several values of f 0 (Fig. 6.9c), show that the emergent resonant modes are intimately linked to the PDM spectral peaks, which represent the unique dynamical properties of each subsystem. It should be noted that a well established result of linear systems theory states 160 that the output of a linear system in the frequency domain is the product of the system transfer function and the input in the frequency domain ( Y (w) = H(w) X(w) ). Our result extends the essence of this result to spike train systems where the output goes through a highly nonlinear thresholding operation (Marmarelis, Citron, and Vivo, 1986). Empirically, the main dierence of our point-process system and linear systems is the addition of harmonics caused by the thresholding operation. In the synthetic cosine system, these harmonics are odd-integer multiples off 0 and can be seen in Fig. 6.9b,c. It should be noted that such harmonics have been observed in electrophysiological experiments (Leung and Yu, 1998). 5 10 15 20 25 5 10 15 20 25 30 PDM Dominant Frequency f 0 CLPP Resonant Modes vs PDM Frequency CA3p RM 0 0.2 0.4 0.6 0.8 1 −1 −0.5 0 0.5 1 5Hz Cosine PDM Time (ms) 0 10 20 30 40 0 0.1 0.2 0.3 0.4 0.5 Frequency (Hz) 5Hz CLPP Output Spectogram (A) (B) (C) Figure 6.9: Analysis of synthetic system showing relationship between PDMs and resonant modes. In this synthetic system, each subsystem is composed of a single cosine PDM of freq f 0 =5Hz, shown in (A). The only nonlinearity in each subsystem is the threshold trigger (i.e. no ANFs). (B) Shows the CLPP output signal CA1m. Note the resonant mode at 5Hz (marked by red line) matches f 0 exactly. (C) A plot mapping the CLPP output resonant mode with the PDM spectral peak (f 0 ). Each vertical column represents the FFT of the CLPP output at the given PDM dominant frequency f 0 . Notice that despite the strong nonlinearity of the threshold trigger, the CLPP resonant mode corresponds exactly to the PDM spectral peak. Also note the presence of harmonics at 2*f 0 161 6.4.4 Aect of Trigger Threshold on Resonant Modes Several researchers have pointed out that the baseline and threshold potential in individual neurons and neuronal populations are not static but rather both dynam- ical, varying with the neurons past spiking activity, and nonstationary, varying over time due to a variety of external factors (Lu et al., 2012). Furthermore, it has been shown that the baseline potential is signicant for the development of spontaneous cellular oscillations such as theta (Fricker, Verheugen, and Miles, 1999). In our sub- system models, the baseline potential and threshold potential are both encapsulated in the threshold trigger. The CLPP output resonant mode power was calculated for identical broadband inputs over several values of the threshold trigger parameters of both subsystems, as shown in Fig. 6.10. Threshold values were presented as per- centage deviations from the reference threshold value, which was dened in section 6.3.3. It can be seen that the threshold value can either facilitate or inhibit the emergent theta resonant mode in either subsystem. Furthermore, the same thresh- old value can aect CA3 and CA1 dierently. This modulating eect of thresholds on resonant modes is signicant as it could help account for the fact that in actual data, the theta mode is present only in CA3, not CA1. To more accurately study the eects of the threshold trigger on the emergent resonant modes, we analyzed a simple barebones synthetic system identical to that of the previous section. The only dierence was that the single PDM was the sum of two cosine waves of dierent frequencies f 1 and f 2 , as shown if Fig. 6.11a (time- domain) and 6.11b (frequency domain). The synthetic CLPP system was stimulated with identical broadband signals over several values of the threshold trigger. For each simulation, the power in thef 1 = 2Hz andf 2 = 7Hz frequencies was calculated in the output signals. It was observed that each simulation had only one dominant resonant mode which was determined by the threshold trigger value (Fig. 6.11c,d). 162 −20 −15 −10 −5 0 5 10 −20 −15 −10 −5 0 5 10 15 20 TT−31 CA3p Theta Power TT−13 −5 0 5 10 15 20 30 <21 <25.4 −20 −15 −10 −5 0 5 10 −20 −15 −10 −5 0 5 10 15 20 CA1p Theta Power TT−31 TT−13 0 5 10 20 25 30 <15.2 <5.99 (A) (B) Figure 6.10: Eect modifying threshold triggers has on emergent theta resonant modes of CA1p (left) and CA3p (right). X and y axis measure the percent-change of thresholds from the default threshold (marked by red dot), which is calculated by the methods described previ- ously. Z-axis (color) measures the percent change in the strength of the theta resonant mode as compared to the default case. Each point represents the theta band power in a simulation conducted with the threshold values in the x,y axis and broadband inputs. Identical broadband inputs were used in each simulation. Notice that dierent threshold combinations favor a stronger theta in either CA3 or CA1, for example by lowering the CA1!CA3 threshold by 10% and raising the CA3!CA1 threshold by 2% (marked by green dot), theta only appears in CA3, as in real data. Thus when the threshold was set to 4,f 1 was dominant whilef 2 was almost entirely unobserved (Fig. 6.11e), but when the threshold was set to 5, the roles were reversed withf 2 dominant and f1 unobserved (Fig. 6.11f). This synthetic study conrms the role of the threshold in tandem with the PDMs to determine the characteristics of the output resonant modes. This unique role of the threshold in determining which resonant modes will emerge in the system has no correlate in linear systems theory. 6.4.5 Stimulation Testing An advantage of closed-loop modeling relative to open-loop modeling is that it may provide a more accurate prediction of the system response to external perturbation due to its incorporation of feedback components (Marmarelis et al., 2013a). Un- derstanding the hippocampal response to stimulation is vital for designing eective 163 0 0.2 0.4 0.6 0.8 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 Time (ms) System PDM in Time Domain 0 5 10 15 0 0.2 0.4 0.6 0.8 1 1.2 1.4 System PDM in Frequency Domain Frequency (Hz) 0 2 4 6 8 10 0 1 2 3 4 5 6 7 8 9 10 CLPP Resonant Modes vs Threshold Threshold CA3p RM 0 2 4 6 8 10 TT: 10 TT: 9 TT: 8 TT: 7 TT: 6 TT: 5 TT: 4 TT: 3 TT: 2 TT: 1 Time (s) SpikeTrain Rasterplots 0 2 4 6 8 10 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 CLPP Output for TT=3 (Frequency Domain) 0 2 4 6 8 10 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 CLPP Output for TT=7 (Frequency Domain) (A) (B) (C) (D) (E) (F) Figure 6.11: Eect threshold has on synthetic cosine system. Each subsystem was composed of a single PDM which was the sum of two cosine waves of dierent frequency. (A) Shows the PDM in the time domain, while (B) shows the PDM in the frequency domain. The only nonlinearity in each subsystem is the threshold trigger (i.e. no ANFs). (C) Eect of changing the threshold trigger on the resonant mode peak power. Plot is the same as Fig. 6.9C. Notice that the threshold determines which resonant mode will emerge in the CLPP output signals. (D) shows the evolution of the spike train rasterplots as the threshold is changed. (E) and (F) show the CLPP outputs in the frequency domain when the threshold is 3 and 7, respectively. neurostimulation patterns for seizure abatement in the emerging eld of deep-brain stimulation for epilepsy (see section 6.5.4). The response characteristics of the CLPP model for two random narrowband (RNB) disturbances of various oscillation fre- quencies were examined. Each RNB disturbance was constructed according to the procedure dened in section 6.3.4. It should be noted that while the RNB signals had dierent frequencies of oscillation, their mean ring rate (MFR) was the same - i.e. they had the same amount of spikes. This is essential, since the pulsecount of periodic, or xed-interval, stimuli is directly proportional to their frequency. Thus, it is impossible to disentangle whether a given stimulus performs better due to its frequency or due to its spikecount. 164 The MFR of the intra-loop CA1 activity is plotted in Fig. 6.12a for various oscillation frequencies of the two RNB disturbances (x-axis is for e3 and y-axis is for e1). Increase of CA1 activity is observed when either e3 or e1 were stimulated in the theta/alpha range (6-14 Hz). Highest increase is seen for the combination of theta stimulation in e3 and alpha stimulation in e1. Most intriguing is the nding that the combination of low-gamma stimulation (28-30 Hz) in both disturbances, as well as high-delta stimulation (4-5 Hz) in e1, results in signicant suppression of CA1 activity (Fig. 6.12b). 5 10 15 20 25 30 35 5 10 15 20 25 30 35 RM−e1 MFR of CA3p (Hz) RM−e3 10 15 20 25 <9.28 10 20 30 0 0.05 0.1 e3. MFR=16.2Hz 10 20 30 0 0.05 0.1 e1. MFR=16.3Hz 10 20 30 0 0.05 0.1 CA3p. MFR=4.42Hz 10 20 30 0 0.05 0.1 CA1p. MFR=3.75Hz (A) (B) Figure 6.12: (A) CA3p output MFR where e3 & e1 are RNB inputs of various frequencies. X and y axis show the narrowband frequency of e3 and e1 respectively. The x and y value of 0 corresponds to a Poisson input. Note that when the CLPP system is driven by RNB theta inputs (around 8 Hz) system output will greatly increase. When the system is driven by low frequencies or 28 Hz, system output will greatly decrease. (B) example of CLPP driven by two RNB inputs centered at 28 Hz. Note that subsystem output MFR is far less than input MFR. In order to further disentangle the contributions of the individual subsystems and the closed-loop conguration to the CLPP frequency response, a single-input closed-loop system was constructed which had only one disturbance signal: e3 (Fig. 6.13a). e3 was chosen over e1 since presumably much more exogenous inputs enter the system from CA1 to CA3, via the EC, rather than the direct pathway from CA3 to CA1. The threshold of the single-input systems was set such that when it is stimulated with the recorded CA3 signal, the output CA1 MFR is equivalent to the 165 recorded CA1 MFR. Identical RNB disturbances where then used to stimulate both the single-input closed-loop system and the open loop CA3!CA1 system (Fig. 6.2). The output MFR for each RNB frequency, shown in Fig. 6.13b, can be understood as point-process transfer functions, analogous to the transfer function studied in linear systems. The dierences between the responses' of the two systems quantita- tively measures how the CA1!CA3 feedback connectivity strengthens/ attenuates various input frequencies. For example, while both open-loop and closed-loop sys- tems respond strongly to theta and suppress activity above 18 Hz, the closed-loop system strongly amplies the open-loop theta response. This helps explain why a full closed-loop model is needed to replicate the strong theta oscillations seen in empirical data (Fig. 6.8) Furthermore, the resonant theta frequency is raised from 4 Hz in the open-loop case to 8 Hz in the closed-loop case. Several groups have shown experimental evidence that random Poisson stimula- tion can be more ecacious for seizure abatement than periodic stimulation (Wyck- huys et al., 2010; Van Nieuwenhuyse et al., 2014; Buel et al., 2014). Here, the CLPP system was used as a testbed to assess the ecacy of oscillatory vs random stimulation. Namely, the system was stimulated with oscillatory RNB spiketrains and broadband/random Poisson spiketrains of increasing MFRs. To allow fair com- parison, for each frequency, the total spikecount of the RNB trains matched the spikecount of the Poisson spiketrains. Thus, unlike in Fig 6.12 where the spikecount of the RNB was kept constant, here both the frequency and spikecount of the RNB trains progressively increased. To maintain simplicity, only CA3 was stimulated, as in the single-input CLPP system of Fig. 6.13a. As shown in Fig. 6.14, the response of increasing the MFR of Poisson trains was simple: more input spikes meant more output spikes. The response of the RNB train, however was more complex. Once again the theta resonance peak can be seen, along with inhibition from 15-30 Hz. 166 Figure 6.13: (A) Schematic of single-input closed-loop model from CA3!CA1. (B) Open- loop (OL) and Single-Input closed loop (SiCL) output MFR responses to RNB inputs of varying frequencies. These results suggest that oscillatory vs random is not the correct question to ask, as the answer depends will vary depending on which frequency is used. Rather, one should ask what is the optimal frequency of stimulation. It should be noted that these results for RNB stimulus cannot be compared with Fig. 6.12,6.13, since as already mentioned, there the MFR was kept constant, while here it is progressively increasing to match the Poisson MFR. 167 Figure 6.14: System output, as measured by MFR of CA1, when stimulated with random and RNB spiketrains of various frequencies. Each trial was conducted 10 times. Lines and shaded region show mean standard deviation. 6.5 Discussion This paper present a methodology for closed-loop modeling of the hippocampal CA3 and CA1 regions. The model is composed of two PDM/ANF subsystems followed by a threshold trigger as described in our previous publications (Marmarelis, 2004; Marmarelis et al., 2013b). The PDMs can be thought of as an ecient basis of functions to describe the dynamical transformation between any two signals, while the ANFs describe the static nonlinearity which 'links' the PDM output with the 168 output spike train. The PDM/ANF methodology for modeling nonlinear systems has shown itself to be a highly robust and exible method which lends itself to physiological interpretation. Stimulation of the closed-loop model showed the emer- gence of theta oscillations, a phenomena not seen in the open loop models. Also, simulation of the closed-loop model with RNB stimulation suggests that our model can be useful in optimizing neurostimulation patterns to suppress seizures. 6.5.1 Closed-Loop Modeling Closed-loop modeling is to be used whenever two signals bidirectionally in uence each other. Although the data used in this study was from the CA3 and CA1 re- gions of the hippocampus, in principle the model can be used for any bidirectionally causal point-process signals. A similar model based on continuous signals was previ- ously used to model cerebral hemodynamics (Marmarelis et al., 2013a). There, the response of the system to perturbations of blood pressure and CO 2 was compared in open-loop and closed-loop mode. Here, closed-loop modeling was used due to the complex nested-loop nature of hippocampal anatomy and several experimental studies which have conrmed that electrical activity can reenter and reverberate through the hippocampal-entorhinal formation (see Fig. 6.1 and section 6.5.2. Traditional model validation criteria such as those measuring predictive power cannot be applied to the closed-loop model since it does not aim to predict an output spike train as open-loop models usually do; rather it aims to describe the behavior of the system under various external stimuli. We suggest that the emergence of theta resonances in the closed-loop model and not in the open-loop models provides validation and utility to the closed-loop model (see Fig. 6.8). One approach to rigourously validate the closed-loop model would be to stimulate the system and compare the system response with the open-loop and closed-loop model predictions 169 such as those shown in Fig. 6.13 and 6.8. Such data, however, was not available in this study. It is important to clarify the distinct roles of open-loop and closed-loop modeling. Open-loop models, such as those derived previously for the hippocampus (Song et al., 2007; Zanos et al., 2008; Marmarelis et al., 2013b), aim to describe the functional transformation of activity from CA3 to CA1, or vice-versa. However, open-loop modeling will be misleading when describing phenomena which involve the entire closed-loop system. A notable example of this is the response of the system to external perturbations as these perturbations will not only aect the target area but will reverberate around the entire loop. This is why the open- loop model was not able to predict the emergence of theta resonances in response to external broadband stimulation. Knowing how the hippocampus responds to external stimulation is particularly important for the emerging eld of deep-brain stimulation which aims to use precisely such stimulation to abate seizures, which are characterized by hypersynchronous resonant activity (see section 6.5.4). It should be noted that closed-loop modeling is dierent from the more com- monly used autoregressive modeling such as done in Song et al. (2007), Eikenberry and Marmarelis (2013), and Valenza et al. (2013). In autoregressive modeling, the output time-series past has an eect on the output present. In closed-loop mod- eling, the output time-series past has an eect on the input present. In practice, this subtle dierence means that during simulations, the input to a given subsystem must continuously be reevaluated as it evolves through the simulation. In future work it would be benecial to develop a nested-loop model which incorporates both closed-loop eects that model hippocampal reentrance and autoregressive eects that model the neural refractory period and afterhyperpolarization, a feature ab- sent from our current model (Spruston and McBain, 2007). 170 6.5.2 Hippocampal Reentrance The closed-loop model is based on the working hypothesis that CA1 can causally in uence CA3. Anatomically, this may arise through backprojecting interneurons from CA1 to CA3 (Sik et al., 1994), or through the hippocampal-entorhinal loop (Fig. 6.1a). Deadwyler et al. (1975) was the rst to show that population spikes induced in CA3 spread to the EC and reentered CA3 through the Dendate Gyrus. Since then, several studies have used local eld potential (LFP) recordings to con- rm reentrance and to elucidate the multiple simultaneous anatomical pathways by which it may occur. It has been shown that reentrant activity may leave the hippocampus either directly from CA1!EC or from CA1!Subiculum!EC (Finch et al., 1986; Tamamaki and Nojyo, 1995; Craig and Commins, 2005). Further- more, activity may reenter the hippocampus either through the Dendate Gyrus or CA3 via the perforant path (Stringer and Lothman, 1992; Wu, Canning, and Le- ung, 1998; Kloosterman, Haeften, and Silva, 2004) or directly from EC!CA1 via the temporoammonic path (Yeckel and Berger, 1990; Jones, 1993; Bartesaghi and Gessi, 2003). Optical imaging studies have also clearly shown that information can reenter and reverberate around the hippocampal-entorhinal loop (Iijima et al., 1996; Ezrokhi et al., 2001). An advantage of the data-driven nonparametric modeling ap- proach used here is that no a priori assumptions are made about the system under study, and thus reentrance can be modeled without complete knowledge of its un- derlying pathways and mechanisms. Furthermore, the model will not change with future physiological discoveries (Marmarelis, 2004; Song, Marmarelis, and Berger, 2009; Song et al., 2009b). Most studies on reentrance have analyzed LFP activity induced by electrical stimulation. In a previous paper, we have used a large dataset of neurons to demon- strate predictive power (ie Granger causality) from CA1 to CA3 on the single-neuron 171 level in the nonpathological unperturbed brain. There it was found that of the 144 neuron pairs examined, 96 ( 67%) showed CA1!CA3 causality. Furthermore, it was found that the number of bidirectionally connected neurons was signicantly higher than would be expected by chance, suggesting the CA3 and CA1 pyramidal cells tend to be wired topographically in a bidirectional/loop fashion. This nding has also been conrmed experimentally in both anatomical and physiological studies (Tamamaki and Nojyo, 1995; Buzs aki, 1989). Although much evidence points to true causality from CA1!CA3 on the network (LFP) level, and the previous paper showed Granger causality on the single neuron level, the extent of true causality on the single neuron level has yet to be established due to possibility of unobserved inputs such as the septum in uencing both CA3 and CA1 regions (Colom, 2006). Thus, the results of this study must be regarded with some caution until experiments will be performed which elucidate the relative in uence of truly causal connections and unobserved inputs on the CA1!CA3 predictive relationship. 6.5.3 Theta Emergence Hippocampal theta rhythms have been extensively studied and implicated in neural coding, learning, and memory (for excellent reviews, see Buzs aki (2002), Axmacher et al. (2006), Colgin (2013), and Buzs aki and Moser (2013)). The mechanisms by which network theta oscillations emerge in the hippocampus involve complex interactions occurring simultaneously on many hierarchal levels. On a cellular level, EC stellate cells and hippocampal pyramidal cells have been shown to have inherent membrane resonances in the theta range (Leung and Yu, 1998; Pike et al., 2000; Haas and White, 2002; Schreiber et al., 2004). On a regional level, CA1, CA3, and the EC have been shown to generate theta oscillations endogenously in-vivo (Alonso and Garcia-Austt, 1987; Kocsis, Bragin, and Buzs aki, 1999; Goutagny, Jackson, 172 and Williams, 2009). Although cellular membrane resonances cannot by themselves lead to theta oscillations, they have been theorized to strongly in uence frequency of oscillation in networks of interconnected pyramidal cells and interneurons such as those found in the above 'pacemaker' areas (Hutcheon and Yarom, 2000). Finally, on the interregional level, the above pacemakers are interconnected in a complex nested loop and receive inputs from the medial septum (Fig. 6.1a), which has been shown to be critical for the emergence of hippocampal theta (Winson, 1978; Colom, 2006). In the present model we have shown that neuronal theta oscillations emerge through the closed-loop modeling of the hippocampal-entorhinal formation; further- more, these rhythms are absent in the open-loop conguration. Although the model was based on recordings of CA3 and CA1, the nonparametric data-driven nature of the PDM modeling approach implies that several hierarchal levels of interaction will be incorporated into the PDMs. For example, CA3!CA1 PDMs incorporate both CA1 neuronal integration (a cellular mechanism) and potential CA1 pyramidal- interneuron interactions (a regional mechanism). Because it is not yet clear how much the PDMs are in uenced by unobserved inputs (see section 6.5.2), the cur- rent results must be regarded with caution. At the very least, we have shown that although open-loop systems may show a slight preference for theta, closed-loop mod- eling and bidirectional connectivity is necessary for full neural oscillations to emerge (see Fig. 6.8). This is well supported by experiments which have shown CA1 to be an endogenous theta generator but have not shown any theta resonances in re- sponde to white noise stimulation (Jahnsen and Karnup, 1994; Goutagny, Jackson, and Williams, 2009). Furthermore, we have provided strong evidence to support the hypothesis that the hippocampal-Entorhinal loop is involved in generating and modulating hippocampal-entorhinal theta rhythms. However, the full extent of this 173 involvement can only be claried once the causality question is resolved. 6.5.4 Epilepsy and Seizure Abatement Epilepsy is a neurological disorder characterized by chronic seizures which aects 1- 2% of the US population (Begley et al., 2000). Although the mechanisms of seizure development are still not clear, several animal studies have linked hippocampal ictal activity with uncontrolled oscillations along the hippocampal-entorhinal loop (Pare, Llinas, et al., 1992; Stringer and Lothman, 1992; Nagao, Alonso, and Avoli, 1996). Interestingly, a study by Barbarosie and Avoli (1997) showed that 1 Hz stimulation of CA1 actually decreased seizure frequency along the hippocampal-entorhinal loop. This nding is also re ected in our results where it was found that<2 Hz stimulation decreased output activity (see Fig. 6.12). Later a study by Bragin et al. (1997) showed that in-vivo ictal activity could emerge independently in both the EC and hippocampus even after the two were lesioned. This supported the notion that seizure rhythmicity was due to a complex coupling of nested oscillators rather than just a single closed loop oscillator. Although the role of the hippocampal-entorhinal loop in human temporal lobe epilepsy (TLE) remains unknown, these animal studies suggest that it is intimately involved in initiating and sustaining seizures and that perturbation of this loop with electrical stimulation may abate seizure onset. Deep brain stimulation (DBS) is a promising new alternative to traditional epilepsy treatments such as antiepileptic drugs and resective surgery which both have major drawbacks: drugs have a 30% nonresponder rate and strong side-eects, while surgery isn't applicable for all patients, has acute and long-term risks, and has a large remission rate (Brodie and Dichter, 1996; Engel et al., 2003). Respon- sive cortical focal stimulation has recently been approved in the US and scheduled thalamic stimulation has previously been approved in Europe (Ben-Menachem and 174 Krauss, 2014; Heck et al., 2014; Fisher and Velasco, 2014). However the mechanisms by which DBS works are still not clear and few quantitative models have been de- veloped (Durand and Bikson, 2001; Mina et al., 2013). The above studies suggest that a better understanding of hippocampal-entorhinal loop dynamics may help elu- cidate the mechanisms by which DBS abates temporal lobe seizures. Furthermore, a computational model of these dynamics may not only serve to elucidate these mechanisms, but also assist in designing ecient neurostimulation patterns, a pro- cess currently done by physicians using a brute-force approach. Such a framework has recently been applied for designing ecient DBS patterns for epilepsy (Taylor et al., 2015) and Parkinson's disease (Holt and Neto, 2014; Grill, Brocker, and Kent, 2014). However, in Parkinson's disease the barriers to clinical translation are much lower since stimulation ecacy can be assessed intermediately by observing the change in patient tremor (Brocker et al., 2013). In this paper, we propose that the closed-loop nonlinear nonparametric model of hippocampal-entorhinal loop dynamics may be used as a working model to study the eects of DBS on hippocampal seizures. The eects of DBS frequency was analyzed by perturbing the closed-loop model with random narrowband (RNB) spike trains across a wide range of frequencies. It was found that while certain frequencies such as high theta/alpha (4-12 Hz) dramatically increased hippocampal output, other frequencies such as delta (<2 Hz) and low gamma (30Hz) signicantly reduced output (see Fig. 6.12). Furthermore, it was found that depending on the oscillation frequency, the advocated RNB trains may be more ecacious than random (Poisson) trains. Although our model is still in the proof-of-concept stage, it may already help explain some unresolved phenomena seen in experimental studies. First, it is well known that some frequencies initiate seizures while others suppress them. For example in the rodent thalamus, low-frequency stimulation (LFS) at 8 175 Hz induces seizures, while high-frequency stimulation (HFS) at 100 Hz suppresses them (Mirski et al., 1997); conversely, in the human caudate nucleus, 4-6 Hz LFS suppresses seizures, while 50-100 Hz HFS stimulation induces them (Chkhenkeli and Chkhenkeli, 1997). Furthermore, it has been shown that vastly dierent stimulation frequencies may suppress seizures in the same system. For example, Kinoshita et al. (2005) showed that both 1 Hz and 50 Hz stimulation had anticonvulsant eects in TLE. Our results suggest that these seemingly disparate multiphasic eects of neurostimulation arise from the closed-loop dynamical interaction between the various regions of the hippocampal-entorhinal circuit, and in such a circuit it is expected to see multiple resonant modes and multiple inhibitory zones. Thus, we suggest that the closed-loop modeling methodology advocated here may be used to understand these complex interactions and obtain ecient DBS patterns to abate seizures. However, we emphasize that while the presented paradigm of model-based neu- rostimulation is promising, the specic results should not be taken overly literally. For example, although in our model 30 Hz gamma stimulation was found to be optimal, we cannot yet be certain whether this is an inherent feature of rodent hippocampal circuitry, or was specic to the particular circuitry of the given ro- dent during the behavioral task from which the data was obtained. Furthermore, our model says nothing of how stimulation of any other area such as the thalamus would aect the hippocampus. Nonetheless, we believe the model-based approach, if carefully applied, can answer all these questions. In the current study, MFR was used as a measure of seizure activity. In the future, we hope to expand our model by incorporating multiple units and LFP recordings. This would allow our model to utilize much more realistic indicators of seizure activity than MFR such as population synchrony and eld potential os- 176 cillations. Also, we hope to provide experimental validation to the computational results obtained here. 177 Chapter 7 Designing Patient-Specic Optimal Neurostimulation Patterns for Seizures from Human Single Unit Hippocampal Data Abstract Neurostimulation is a promising therapy for abating epileptic seizures. However, it is extremely dicult to identify optimal stimulation patterns experimentally. In this study we use nonlinear statistical modeling to reconstruct the unique connectivity and interneuronal dynamics of 24 neurons recorded from a human hippocampus. Spontaneous seizure-like activity is induced in-silico in the reconstructed neuronal network. This network is then used as a testbed to design and validate a wide range 178 of neurostimulation patterns. It was found that the commonly used synchronized periodic trains were not able to permanently abate seizures at any frequency. A sim- ulated annealing global optimization algorithm was then used to identify an optimal stimulation pattern which successfully abated 100% of seizures. Finally, in a fully responsive, or "closed-loop" neurostimulation paradigm, the optimal stimulation successfully prevented the network from entering the seizure state. We propose that the framework presented here for algorithmically identifying patient-specic neu- rostimulation patterns can greatly increase the ecacy of neurostimulation devices for seizures. 7.1 Introduction Epilepsy is a neurological disorder characterized by chronic seizures which aects 1-2% of the US population (Begley et al., 2000). Standard treatments include antiepileptic drugs and resective surgery. However, both have major drawbacks. Up to 30% of patients do not respond to drugs; of those who do, many suer seri- ous side-eects such as nausea, dizziness, drowsiness, and weight-gain (Brodie and Dichter, 1996). Furthermore, Surgery is not an option for many patients, and when it is, there is a large remission rate within 1-2 years (Engel et al., 2003). In recent years, neurostimulation has emerged as a promising approach to reduc- ing seizures. In 2013, the FDA approved the Neuropace RNS system, the rst device for responsive cortical neurostimulation for epilepsy (Sun, Morrell, and Wharen Jr, 2008). However, thus far results have shown that neurostimulation provides only palliative relief from seizures rather than a full cure. For example, the Neuropace device has provided a 54% decrease in median seizure frequency with 9% of pa- tients being seizure free after two years (Heck et al., 2014). While these results are impressive, especially considering they were obtained on the most dicult patient 179 category, they are far from perfect. Much research has shown that the ecacy of neurostimulation could be increased by carefully designing the temporal pattern of stimulation pulses. Frequency of stimulation (Chkhenkeli and Chkhenkeli, 1997; Cordeiro et al., 2013), periodicity (Wyckhuys et al., 2010; Buel et al., 2014), and, in the case of multiple electrodes, synchronicity (Nelson et al., 2011; Van Nieuwenhuyse et al., 2014) have all been shown to in uence the success of neurostimulation. Furthermore, many have ar- gued that stimulation must be custom tailored to the unique seizure topology and dynamics of each particular patient (Holt and Neto, 2014; Mina et al., 2013; Tay- lor et al., 2015). However, designing optimal neurostimulation patterns is extremely challenging in animal models and in human patients. Researchers cannot order seizures "on-demand" to test a wide range of stimulus patterns. Oftentimes, physi- cians must wait months before learning if a particular stimulus works. Furthermore, researchers can only stimulate each seizure once, and cannot go "back in time" to see how the seizure would have evolved with no stimulation or with dierent stim- ulation. Due to these diculties the Neuropace device recommends that physicians keep stimulus frequency xed at 200 Hz and if that fails to adjust the stimulus cur- rent. Nonetheless, there is an increasing feeling in the eld that a more principled approach to stimulation design is needed (Nagaraj et al., 2015). In past work we have developed a closed-loop model of the rodent hippocam- pus and used this model to identify the optimal frequency of stimulation needed to reduce network output (Sandler et al., Under Review). However, the above model was limited to 2 reciprocally connected neurons and thus important features of epilepsy such as population synchrony could not be studied. Here we use 24 neurons recorded from the hippocampus of a human temporal lobe epilepsy (TLE) patient to reconstruct the patient's distinct neuronal connectivity and causal dy- 180 namics. Spontaneous seizure activity is then initiated in the reconstructed neuronal network. Finally, this model is used as an in-silico testbed for designing and testing ecient neurostimulation patterns. The optimal stimulus, obtained via a global optimization algorithm, was found to abate 100% of seizures, and signicantly out- performed any other traditional stimulation types. We believe that such a patient- specic algorithmic approach to neurostimulation design can signicantly increase the ecacy of neurostimulation devices for epilepsy. 7.2 Methods 7.2.1 Experimental Setup and Preprocessing Human Subjects and Surgery All procedures were reviewed and approved by the Institutional Review Board of Wake Forest University, in accordance with National Institutes of Health guidelines. All surgical procedures were performed at Wake Forest Baptist Medical Center. Postoperative monitoring and all neurocognitive experiments were performed at the Comprehensive Epilepsy Center at Wake Forest Baptist Medical Center. One adult subject underwent surgical implantation of FDA- approved hippocam- pal electrodes with shaft electrodes capable of single-unit recording and eld poten- tial recording (Ad-Tech Medical Instrumentation Corporation, Racine, WI) for lo- calization of seizures. Inclusion in this study was voluntary and consented separately from the surgical procedure. Prospective study participants underwent appropriate clinical epilepsy screening evaluations. Preoperative planning and intraoperative placement of depth electrodes was performed using a frameless Brainlab Cranial Navigation System (BrainLab North America, Westchester, IL) to plan and guide electrode entry points, electrode trajectories and target points within the CA3 and 181 CA1 subelds of each hippocampus. Entry points were created using either cranial burr-holes or bone aps as dictated by clinical needs. Two electrodes were placed into each hippocampus, one anteriorly positioned in the head of the hippocampus and a second in the posterior body. Intraoperative monitoring conrmed both sin- gle unit and eld activity from all electrodes. Electrodes were subsequently passed through and secured to the scalp. Electrode localization was conrmed using post- operative MRI. Neurocognitive experiments were performed on post-implantation days 3-7 depending on clinical needs and indication for seizure localization. Elec- trodes were explanted after seizures were localized. In addition to the standard localization protocol, these patients participated in approved neurocognitive tasks while single-unit neuronal ensemble activity was recording using the microelectrodes contained within the implanted FDA-approved electrode while performing approved neurocognitive tasks. In addition to this stan- dard practice, neuronal ensemble recordings will be made from subject hippocampi continually at rest and during neuropsychological testing. Subjects will undergo no additional electrode implantation nor additional monitoring phase. During the study period, subjects will be evaluated for safety by physical and neurological ex- aminations, and appropriate laboratory and radiographic examination. Continuous video will also be recorded which is standard for all patients undergoing EMU pro- cedures. Recording from Hippocampus 10 minutes of continuous recordings were used to estimate all models. Single unit neural activity was isolated and recorded using Plexon MAP electrophysiological recording systems. Spikes were discretized using a 2 ms bin. 182 7.2.2 Modeling and Analysis Model Conguration A probabilistic model was used to predict the ring probability of a given output CA3 neuron based on its own spiking history and the past and present spiking activity of all other functionally connected CA3 neurons. Thus the probability that a particular neuron, y(t) will re at time t is expressed by the probability, ^ y(t): ^ y(t) =Pr y(t) = 1jx 1 (t):::x N (t);y(t1) =H h x 1 (t):::x N (t);y(t1) i (7.1) wherefx n (t)g re ect the N eectively connected spiketrains from CA3, re ects the nite memory of the system which ranges from 0 M, and H[] re ects the mathematical model which is used to describe the dynamical transformation fromfx n (t)g!y(t). The generalized linear modeling (GLM) framework was used wherebyH[] was decomposed into a linearized function of the inputs, (t), followed by a static nonlinearity, here chosen to be the probit link function (Truccolo et al., 2005; Song et al., 2007): ^ y(t) = (t); = 1 p 2 2 Z x 1 e 1 2 ( (t) ) 2 (7.2) (t), the linearized component of the GLM, takes the form of a nonparametric multiple-input autoregressive model which describes the dynamical transformation between input and output spike trains. It consists of a feedforward component, re ecting the eect of the N input cells on the output cell, and a linear feed- back/autoregressive component re ecting the subthreshold and suprathreshold ef- 183 fects the output cell has on itself. Thus, the output is calculated as: (t) =k 0 + N X n=1 F [x n (t);k n ] + M+1 X =1 F [y(t);k AR ] (7.3) whereF [x n (t);k n ] models the feedforward eects of inputx n (t),F [y(t);k AR ] models feedback eects, and k 0 , the constant oset term, models the baseline potential. While in the past feedforward eects were modeled to be either linear (Sandler et al., 2014) or nonlinear (Sandler et al., 2015; Song et al., 2007), in this study, using the sparse group selection algorithm (see section 7.2.2), it is possible to determine which inputs are best linked to the output via a linear kernel and which inputs are best linked via a nonlinear kernel, taking the form of a 2nd order quadratic Volterra kernel (see Fig. 7.1b) (Marmarelis, 2004; Rajan and Bialek, 2013). In past studies, feedforward eects were xed to be either linear (Sandler et al., 2014) or nonlinear (Sandler et al., 2015; Song et al., 2007), and feedback eects were xed to be linear (Song et al., 2007). In this study, using the sparse group selection algorithm (see section 7.2.2), it is possible to determine whether particular inputs and feedbackback eects are best modeled by either a linear or nonlinear kernel (see Fig. 7.1b). In this study, linear refers to convolution with a linear lter, k (1) (), while nonlinear refers to convolution with a quadratic (2nd order Volterra) lter, k (2) ( 1 ; 2 ) (Marmarelis, 2004; Rajan and Bialek, 2013). Mathematically, these operations are respectively dened by Eq.7.4a,b: F [x(t);k (1) ()] = M X =0 k (1) ()x(t) (7.4a) F [x(t);k (2) ( 1 ; 2 )] = M X 1 =1 M X 2 =1 k (2) ( 1 ; 2 )x(t 1 )x(t 2 ) (7.4b) 184 It was found that a memory of 100 ms was sucient to model the dynamical eects of most neurons, and thusM was xed to 50 (100 ms/2 ms binwidth). In order to reduce the amount of model parameters and thereby increase parameter stability, we applied the Laguerre expansion technique (LET) to expand the feedforward and feedback lters overL Laguerre basis functions (Marmarelis, 2004). L = 6 Laguerre basis functions were used. Correspondingly, the amount of parameters in linear kernels was reduced fromM toL (savings of 44 parameters) and in 2nd order kernels from to M(M + 1)=2 to L(L + 1)=2 (savings of 1254 parameters). The Laguerre parameter was xed at 0.542 to re ect this system memory (Marmarelis, 2004). Model Estimation Presumably, only a small portion of the total recorded neurons,R casually in uence any given neuron in the reconstructed neuronal network (RNN). This motivates the central task of identifying which neurons are eectively connected and which are not (Bullmore and Sporns, 2009; Fallani et al., 2014). Most methods which aim to estimate eective connectivity adopt a Granger causality approach whereby neuron A is eectively connected to neuron B only if it can help predict when neuron B will spike (Krumin and Shoham, 2010; Kim et al., 2011; Sandler et al., 2014; Zhou et al., 2014). Here a penalized group regression approach was adopted which implicitly maximizes predictive power while immensely improving computationally eciency over most explicit Granger causality approaches which usually rely on stepwise input selection (Song et al., 2013; Robinson, Song, and Berger, 2015). To proceed with penalized group regression, Eq. 7.2 is rst recast in matrix form: ^ y = () = (Vc) (7.5) where ^ y and are the vectors consisting of ^ y(t) and (t) for 1 t T , V is the 185 design matrix consisting of the convolved inputs and, for quadratic kernels, their cross products (Marmarelis, 2004), and c is the vector of model parameters to be estimated. In the 24 neuron RNN, there are 300,000 observations and 649 unknown parameters. The input parameters are divided into 2R groups consisting of 2 groups for every putative input: one group for theL 1st order kernel parameters and another for the L(L + 1)=2 2nd order kernel parameters. The objective is now to nd the optimal parameter vector,c which minimizes the cost function composed of the sum of the negative log-loss likelihood and the group regularization term, P (c): C(c;y;V ) = T X t=1 y(t)log^ y(t) + (1y(t))log(1 ^ y(t)) + 2R X g=1 P (c g ) (7.6) wherec g is the group parameter vector containing only the parameters within group g. The function of the regularization term is to automatically set to 0 any parameter groups which are not found to signicantly in uence the output - thus implicitly estimating sparse functional connectivity. Here, the group minimax concave penalty (MCP) regularizer was chosen over the more conventional group LASSO because (1) it induces much less regularization based parameter shrinkages (biases) than the latter (2) it leads to much sparser solutions than the latter (Breheny and Huang, 2014; Zhang, 2010). The MCP regularizer is dened as: P (c g ;; ) = 8 > > < > > : kc g k kcgk 2 2 kc g k 1 2 2 kc g k> (7.7) where determines the strength of regularization and , which was xed at 3, determines the range over which MCP regularization is applied (Breheny and Huang, 2014). The group coordinate descent algorithm outlined in Breheny and Huang 186 (2014) was used to ndc . The optimal value was selected using a warm-start regularization path ap- proach using 90 logarithmically spaced values. At each iteration, the Pearson correlation, was computed between the recorded spiketrain, y(t), and the esti- mated spike probabilities, ^ y(t), on a testing set consisting of 20% of data randomly selected fromV . was used since (1) it was previously found to be a robust metric of similarity between spiketrains and continuous signals (Sandler et al., 2014) and (2) it led to sparser solutions than the more commonly used cross-entropy error. Finally, the optimal was selected as the largest which achieved > 99% of the max value. The regularization path can be seen in SF 7.7. Since any regularization will necessarily bias obtained parameters to varying degrees, all parameters of nonsparse groups were reestimated without sparse groups and without the regularization term. Note that due to computational eciency and simplicity the initial search for uses a logit link function (Breheny and Huang, 2014), while the nal reestimation was done using the probit link of Eq. 7.2. All computations were done in Matlab using custom code available upon request. A standard 3.2Ghz, 6-core desktop computer was able to estimate the 24-neuron RNN in approximately 3 hours. Model Validation To avoid overtting, Monte Carlo style simulations were used to select those models which represent signicant causal connections between input and output neurons and do not just t noise (Sandler et al., 2014). The following procedure was used: in each run the real input was divided into 40 blocks and these blocks were randomly permuted with respect to the output. A model was then generated between the permuted inputs and the real output, and the Pearson correlation coecient, i , 187 was obtained as a metric of performance. T = 40 such simulations were conducted for each output and a set of performance metrics, f i g T i , was obtained. Then, using Fisher's transformation, we tested the hypothesis, H 0 , that was within the population off i g. If this hypothesis could be rejected at the 99.99% signicance level, the model was deemed signicant. The very conservative threshold (P < :0001) was used due to the large amount of comparisons being made. Two metrics were used to evaluate the goodness-of-t of the estimated models by comparing the estimated continuous output, ^ y(t) with the true binary output y(t). Receiver Operator characteristic (ROC) curves plot the true positive rate against the false positive rate over the putative range of threshold values for the continuous output,y(t) (SF. 7.7c, Zanos et al. (2008)). The second metric used was the discrete KS test (Haslinger, Pipa, and Brown, 2010; Song et al., 2013) which compares the ISI distribution of the time rescaled probabilistic estimates with that of a homogenous Poisson process (SF. 7.7d). All model assessment metrics were evaluated on the testing set. Simulation & Clustering The simulated neuronal outputs, ~ y n (t) are dened as an inhomogeneous Bernoulli process with spiking determined by: ~ y n (t) = 8 > > < > > : 1 (~ n (t);)>u 0 (~ n (t);)u (7.8) wheref~ n (t)g are the simulated ring probabilities based on past and present sim- ulated spike activity and u is a standard uniform random number. Equivalently, the neuronal output can be viewed as being generated by a prethreshold signal, composed of the sum of a deterministic component (Eq. 7.4) and Gaussian white 188 noise (GWN) of variance 2 , followed by a xed threshold of 0 which is implicit in probit link formulation (Berger et al., 2012): ~ y n (t) = 8 > > < > > : 1 ~ n (t) +N (0; 2 )> 0 0 ~ n (t) +N (0; 2 ) 0 (7.9) A clustering algorithm was used to identify the distinct stable dynamical states which emerged in the RNN output (Sasaki, Matsuki, and Ikegaya, 2007; Burns et al., 2014). To apply the clustering algorithm, the instantaneous MFR was rst computed for each neuron using a 100 ms moving average lter. 5 principal components (PCs) were then extracted from the 24 instantaneous MFR vectors (Fig. 7.2c). These 5 PCs were then clustered using a standard k-means algorithm. Finally, for each state, dominant subnetworks of neurons were found by identifying the most signicant neurons within the cluster (Fig. 7.2e). Stimulation Optimization The RNN framework allows binary external stimulation, s(n;t) to be applied to neuron (or 'electrode') n at time t by superimposing a spike onto that neuron at that time. Essentially, Eq. 7.8 is modied to be: ~ y n (t) = 8 > > < > > : 1 ( n (t);)>u or s(n;t) = 1 0 ( n (t);)u (7.10) After seizure and control states were identied, a two-stage simulated annealing (SA) algorithm was used to identify the optimal spatiotemporal stimulation pattern, s (n;t), which would most quickly and reliably move the network from the seizure state to the control state (Kirkpatrick, Vecchi, et al., 1983). 189 Multiple SA streams were run in parallel with 2 dierent mode of stimulation: periodic spiketrains (PTs) and random (Poisson) spiketrains (RTs). Each electrode could take on 8 possible values:fOFF, 5 Hz, 20 Hz, 60 Hz, 100 Hz, 140 Hz, 180 Hz, 220 Hzg. The electrode parameter determined the periodic stimulation frequency for FITs and the Poisson rate for RITs. Note that while for PTs, each parameter vector determines a unique spatiotemporal pattern, for RTs, each paramter vector only species the ring probability, and could correspond to an almost innite amount of dierent spatiotemporal patterns. To further reduce the amount of parameters, the electrodes for neurons which were functionally disconnected from the RNN were kept o. The SA algorithm was run on a standard exponential cooling schedule with 120 global iterations with temperatures logarithmically spaced between T start = 10 and T end =:01. Each global iteration had 80 local iterations where the temperature was kept constant. At the end of each global iteration, the parameter was reset to the minimum of the global iteration (Henderson, Jacobson, and Johnson, 2003). In every SA iteration, the network was initialized in the seizure state. The neu- rostimulation pattern, s(n;t), was designed based on the current parameter values, and applied for 250ms. Since the seizure state most obviously corresponded with high ring rates, the SA algorithm aimed to attain the stimulation pattern which most lowered the network MFR in the 100 ms after stimulation ended. Thus the cost function to be minimized was the ratio of stimulus-induced MFR to the control MFR (i.e. the MFR when no stimulation was applied). Note that the lowest value the cost function could take was 0, and when it was > 1, the stimulation actually intensied the seizure. At each iteration the same stimulus was applied 3 times, with dierent random number seeds, and the average MFR ratio over the 3 runs was used as the SA cost function. A sample optimization path can be seen in Fig. 190 7.9b. The SA algorithm was run twice using dierent denitions for neighboring states. In both rounds one electrode was chosen to randomly transition to a neighboring parameter value. In the rst round, each frequency transitioned either to the upper or lower frequency, or turned o with equal probability. If an already OFF electrode was selected, it transitioned to a random frequency. This was done to encourage sparsity and to identify the subset of electrodes to more carefully optimize in the second round. In the second round, the OFF electrodes from the rst round were not adjusted and the remaining electrodes transitioned either to the higher or lower frequency with equal probability (see SF. 7.9)a. As the SA algorithm does not explicitly penalize nonsparse solutions, a stepwise pruning algorithm was used to remove super uous electrodes. During each cycle, the algorithm individually iterated though all E "on" electrodes and calculated the cost function if the selected electrode was "turned o". At the end of each cycle the electrode which caused the greatest decrease in the cost function was turned o, and the algorithm would continue to the next cycle and consider only the E 1 remaining electrodes. If no electrodes were found whose removal decreased the cost function, than the pruning algorithm stopped and no further electrodes were turned o. It was found that only 1 electrode was turned o by the pruning algorithm. 7.3 Results 7.3.1 Reconstructed Neuronal Network The study aims to design a realistic testbed for developing and screening ecient neurostimulation patterns for seizure abatement. This testbed, which we have dubbed a reconstructed neuronal network (RNN), is estimated from single unit 191 activity in area CA3 of a human TLE patient undergoing monitoring for resective surgery (see supp. methods, SF. 7.6). All procedures were reviewed and approved by the Institutional Review Board of Wake Forest University, in accordance with Na- tional Institutes of Health guidelines. In 10 minutes of recording (SF. 7.6), R = 24 distinct units were identied. The aim of the RNN is to use the observed spiking activity to reconstruct in-silico the distinct eective connectivity of the 24 recorded neurons. In other words, for each neuron the RNN attempts to answer which of the other R 1 neurons casually in uence it, and what is the dynamical nature of that in uence. Thus, the RNN diers from most articial neuronal networks which tend to have stereotyped connections between neurons and whose parameters are only loosely based on physiological data. Furthermore, the RNN is distinct from much recent work using graph theory to analyze large amounts of simultaneously recorded neuronal signals, since these studies tend to focus on functional connec- tivity, or undirected statistical associations between neuronal activity, rather than directed causal connections, or eective connectivity. (Bullmore and Sporns, 2009; Yae et al., 2015). In our model, the ring probability of each neuron at timet, ^ y(t), was determined by its own past spiking activity and the past and present spiking activity of all other N connected CA3 neurons, fx n (t)g within a nite memory of M = 100ms and modeled using a generalized linear model (GLM) with a probit link (Song et al., 2007): ^ y(t) = (t); = k 0 |{z} baseline + N X n=1 F [x n (t);k n ] | {z } interneuronal +F [y(t);k AR ] | {z } feedback ; (7.11) where (;) is the probit link function dened by the cumulative normal distribu- tion, with set to 1. (t) is the linear component of the GLM which incorporates: 192 (1) a constant oset, k 0 which determines the baseline potential, (2) a feedback or autoregressive component which describes how the neurons past spiking history in uences its present spiking, and (3) N interneuronal components which describe how the past and present spiking activity of other casually connected neurons in- uence the current spiking of the output neuron. F [x(t);k] denotes either linear convolution if k is a linear lter, or quadratic convolution if k is a nonlinear (2nd order Volterra) lter (see Eq. ??. The complexity of each lter k is determined by the group regularization algorithm used for model tting. The feedback component, characterized by the lterk AR (), can be intuitively thought of as the afterhyperpo- tential (AHP) (Spruston and McBain, 2007) and encapsulates intracellular processes such as the absolute and relative refractory period, slow potassium conductances, and I h conductances. The interneuronal components, characterized by the set of input-output ltersfk n g, can be intuited as the waveform of the EPSP from then th input neuron onto the output neuron. A nonlinear lter is potentially included to describe interactions between two input pulses, such as paired pulse facilitation and depression (Song, Marmarelis, and Berger, 2009; Sandler et al., 2015). It should be noted that the model is a "blackbox", or entirely based on data, and thus makes no a priori assumptions of the nature of the feedback and interneuronal dynamics. Therefore, the estimated lters include the previously listed phenomena as well as more indirect/nonlinear processes such as dendritic integration, spike generation, active membrane conductances, and feedforward interneuronal inhibition (thereby allowing the lters between two pyramidal cells to be inhibitory). All GLM parameters, which implicitly describe both connectivity and causal dynamics, were t simultaneously for each neuron using group regularization and a coordinate descent algorithm. It should be noted that because parameters were estimated from spontaneous data rather than from direct perturbations of the net- 193 work, all parameter estimates may be biased by unobserved inputs (see discussion). A Monte Carlo style shuing approach was used to insure all obtained models had signicant predictive power and were not simply overtting. The obtained connec- tivity graph is shown in Fig. 7.1a. The optimization regularization path is shown in Supp. Fig. 7.7. Further metrics quantifying the predictive power of the estimated models, including ROC plots and the KS-test are shown in Supp. Fig. 7.7. 18/24 neuronal models and 22.8% of all possible connections were found to be signicant. The remaining 6 neurons were functionally isolated from the network: they nei- ther in uenced any other neurons or were in uenced by them. Of the signicant connections, a much larger number than expected were found to be bidirectionally connected (71.43%,P <. Fig. 7.1b). All neurons had an average of 5.3 inputs and outputs. Furthermore, there was a positive correlation between number of inputs and number of outputs, suggesting that ??? (Fig. 7.1c) 1 . A sample system is shown in Fig. 7.1e for neuron 22 which is casually in u- enced by neuronsf2; 8; 14g. Several features of the system can be interpreted from the lters. Note that neurons 8 and 14 are connected linearly while neuron 2 is connected with a quadratic lter, thus implying it exerts some form of short term potentiation. Also note that neuron 14 exerts an entirely excitatory in uence on neuron 22, while the eect of neuron 8 oscillates between excitation and inhibition. Finally, the feedback kernel is composed of initial refractory inhibition, followed by oscillatory bursting activity. Interestingly it oscillates at 5 Hz, squarely in the theta range, which has been implicated extensively in memory tasks in the hippocampus (Buzsaki, 2006; Sandler et al., 2014). All obtained lters are shown in Supp. Fig. 7.10. 1 small-world? 194 Figure 7.1: (A) Graph of all identied eective connections. Dashed lines indicate unidirec- tional connections, while solid lines indicate bidirectional connections. Note that some (like 23) neurons are eectively disconnected from the from the population. (B) Barplot showing % of total possible connections and the % of those which are bidirectional. (C) Positive correlation of the # of outputs a given neuron has vs the # of inputs it has. Regression data is given in the gure for best t line. Red lines shown means of x and y data. (D) (E) Sample three input system of neuron #22 (enclosed in red box in A). Input spiketrains are convolved with either a linear or quadratic lter and then summed with Gaussian noise and feedback eects to generate the prethreshold sum. They are then put through a threshold of 0 to generate a binary output. 7.3.2 Seizure Initiation and Classication Once the eective connectivity and dynamics were estimated, the RNN was allowed to run without perturbation and stochastically generate simulated hippocampal CA3 activity,f~ y n (t)g for all 24 neurons (Pillow et al., 2008). As expected the RNN, which was estimated from normal (nonictal) spiking activity of a TLE patient, generated physiological ring rates with very low levels of synchronization (Fig. 7.2a). In order to induce seizure dynamics, two modication were performed. First, the 'in-silico' 195 neuronal membrane potential was raised by increasing the baseline ring probability parameterk 0 . This mimics the common practice of inducing seizures experimentally by pharmacologically raising the membrane baseline potential (Fricker, Verheugen, and Miles, 1999; Avoli et al., 2002). Second, the level of stochastic noise driving the network was reduced by lowering . To intuitively understand this modication, note that in the extreme case of = 0, the network is entirely deterministic and generates either no activity at all oscillates in a xed limit cycle; alternatively, in the other extreme of =1, the network generates completely random (Poisson) ring. Thus, intuitively, lowering sigma increases population control over the neurons and tends to promote the persistent oscillations which characterize seizures. It was found that raising the baseline by B = 30% relative to the threshold and lowering to .725 was sucient to generate spontaneously emerging realistic seizures lasting on anywhere between a few seconds to over a minute as seen in real human data (Fig. 7.2b; Bower et al. (2012) and Truccolo et al. (2014)). In order to gain more intuition about the network dynamics, principal compo- nents (PCs) were extracted from the network activity based on instantaneous MFR. Fig. 7.2c shows the network trajectory through time in PC space. Finally, a clus- tering algorithm was used to identify the distinct stable dynamical states which emerged in the RNN spiking activity (Fig. 7.2d-f; Sasaki, Matsuki, and Ikegaya (2007) and Burns et al. (2014)). As can be seen, under the selectedf;Bg parame- ters, the RNN jumped between only 2 stable states: normal and seizure. However, it should be noted that under dierentf;Bg parameters, occasionally > 2 sta- ble states emerged (Mazzucato, Fontanini, and La Camera, 2015). The dominant subnetwork of neurons which comprised the seizure cluster is shown in Fig. 7.2e. These 6 neurons are responsible for most of the spiking activity within the seizure state, and their reciprocal connectivity is presumably responsible for maintaining 196 the seizure dynamics. Many neurons maintained their regular ring rate, and 2 neurons even reduced their ring rate. These observations match the heterogeneity of neurons in recorded human seizures (Bower et al., 2012). Interestingly, all of the neurons within the seizure subnetwork had lowering ring rates in actual recordings, suggesting they may be principal cells. Finally, the cluster state of the network was identied at each moment in time (Fig. 7.2b,bottom). As can be seen, this method is able to reliably detect the when the RNN enters and leaves the seizure state, thus eectively making it a prototype seizure detection algorithm (Mormann et al., 2007; Cook et al., 2013; Nagaraj et al., 2015). Figure 7.2: (A) 4 minutes of simulated ring of RNN in physiological conditions. (B) Top: Simulated ring of RNN after modications to induce seizures. Notice the RNN sponta- neously enters and leaves the seizure state. Bottom: the network cluster state through time (see D). (C) Trajectory of RNN activity in (B) within PC space. Color indicates time. (D) Results of clustering algorithm applied to PC trajectory. Red dots indicate cluster centers. (E) The connectivity between the subnetwork of neurons which were found to dominate the seizure cluster (yellow). Bigger circles indicate bigger MFRs during seizures. (F) Additional metrics characterizing the two clusters, including proportion of time, MFR, and Fano factor in each state. Note that MFR was scaled to have a maximum of 1 to promote visualization. 197 7.3.3 Identifying Optimal Neurostimulation for Seizure Abatement Once the eective connectivity of the human TLE patient was estimated, and re- alistic seizure activity simulated, we aimed to identify a spatiotemporal pattern of electric stimulation which could reliably and eciently induce the network to leave the seizure state. At rst glance, this is a paradoxical task: we want to lower net- work spiking activity by applying external spikes to the network. However, several experimental studies have shown that this could be accomplished using precisely designed patterned stimulation (Durand and Bikson, 2001; Heck et al., 2014). Our working assumption was that there exist 24 electrodes which could each stimulate a single neuron without aecting any of the others. A "pulse" in an electrode at a given time would elicit a single contemporaneous spike in the as- sociated neuron at that time. The experimental implications of this assumptions will be discussed later. Computationally, however, this introduces a highly com- plex optimization problem. If we only consider whether a given electrode will be on or o, there are 2 24 , or over 16 million possibilities. If any of the 24 could take on an arbitrary pattern of spikes over 250ms, this number would increase to 2 3000 . To make the problem more formidable, only two modes of stimulation were considered: periodic trains (PTs) having a xed frequency of stimulation, and random, or Poisson, trains (RTs) with dierent MFRs (see Fig. 7.4b). Thus, ev- ery electrode was considered a parameter which could take on 8 values, including fOFF; 5Hz; 20Hz; 60Hz; 100Hz; 140Hz; 180Hz; 220Hzg. This allowed a total of 2 8 24 possible stimulation types. The stimulation length was xed to 250 ms and a two-stage simulated annealing algorithm was used to nd the optimal parameters which would make the network leave the seizure state most quickly (see Supp. methods). Then, in order to promote electrode sparsity, a pruning algorithm was used to "turn o" all electrodes which 198 were deemed super uous. The optimal stimulation parameters and spatiotemporal pattern are shown in Fig. 7.3a,b. It was found that only 4/24 ( 17%) electrodes needed to be turned on. Of those, 2 electrodes had frequencies of 180 Hz, and the remaining two had frequencies of 100 Hz and 220 Hz. This conrms experimen- tal evidence showing that high frequency stimulation (HFS) is optimal for abating seizures (Durand and Bikson, 2001). Interestingly, 2 of the selected electrodes (22 and 24) stimulated the epileptic subnetwork (Fig. 7.2e), while the other two stimu- lated outside the subnetwork, suggesting that direct stimulation of the seizure focus itself may not be the most eective route. By initializing the RNN simulations under identical seizure conditions and using identical sequences of random numbers, one could compare how a seizure would have evolved under dierent types of applied stimulation. Fig. 7.2c,d show how a network initialized in the seizure state evolves when no stimulation is applied (control) and when the optimal stimulation in Fig. 7.2b is applied. Additionally, Fig. 7.2e,f shows the population MFR and PC trajectory in both simulations. As can be seen from these gures, the optimal stimulation is able induce the network to leave the seizure state in under 250ms, and more importantly the network stays in the nonictal state after the 250 ms stimulation ended. Our working premise has assumed that precisely designed independent, or un- synchronized, stimulation across multiple sites could improve responsive neurostim- ulation. While some work has supported this hypothesis (Nelson et al., 2011), most DBS studies, due to either experimental or theoretical consideration (Durand and Bikson, 2001), have only looked at single site stimulation where an electric pulse presumably stimulates a large population of neurons simultaneously (Sun and Mor- rell, 2014). Furthermore, most studies have used PTs rather than RTs despite a few studies indicating that RTs may be superior to PTs (Wyckhuys et al., 2010; Van 199 Figure 7.3: (A) Directed connectivity graph from Fig. 7.1a, where the optimal electrode frequencies are indicated by the color of the circles surrounding the neurons. No circles indicates that neuron is not to be stimulated. (B) Rasterplot of optimal spatiotemporal pattern from the SA algorithm. Rasterplots of evolution of a seizure when no stimulation is applied is shown in (C), and when optimal stimulation is applied in (D). Cluster state is shown below both rasterplots (yellow=seizure, blue=normal). (E) Distance from the seizure state cluster center is shown for both control and stimulation runs. Notice that stimulation induces the network to rapidly move away from the seizure cluster center. This can be seen more clearly in (F) which shows the trajectory of both runs in PC space (see Fig. 7.2c. Red lines in (B-E) indicate the beginning and end of stimulation. Nieuwenhuyse et al., 2014). In order to compare synchronized PT and RT stimu- lation with independent multi-electrode stimulation we rst attempted to nd the optimal stimulation frequency/rate for PT/RT stimulation. This was done by de- livering identical patterns of stimulation to all 24 neurons simultaneously for 250ms. The optimal frequency was found by sweeping from 5 Hz to 220 Hz in 50 Monte- Carlo style trials. Once again, in each trial, the RNN was initialized in the seizure state, and identical random numbers were used for each frequency of stimulation 200 in order to allow a fair comparison of the stimulation frequencies under equivalent conditions (which notably is impossible in real life). The results are shown in Fig. 7.4b. Neither PTs or RTs, of any frequency, were found to signicantly help in ending seizures. In fact, at many frequencies they actually exacerbated the length of seizures. Interestingly however, PTs and RTs at high frequencies (>180 Hz) did temporarily move the network away from the seizure zone (SF. ??). However, as soon as the stimulation was turned o, the seizure continued. It should be empha- sized however, that these results cannot be generalized beyond the particular patient from whom this data was estimated from, and high frequency stimulation has been shown to be ecacious in a large number of patients (Heck et al., 2014). Finally, the optimal stimulation was compared with other unsynchronized multi- electrode stimulation patterns to insure that the acquired simulated annealing so- lution is indeed a signicant local, if not global, minima. In each simulation, the optimal stimulation was applied along with no stimulation (NON), and various al- ternative types of stimulation patterns including: (1) 200-Hz synchronized periodic stimulation (PT), (2) 200-Hz synchronized random stimulation (RT; Fig. 7.4a,b), (3) random unsynchronized multi-electrode (RM) stimulation having the same amount of selected electrodes and total bursts as the SA solution, and (4) stimulation using the same electrodes as the SA solution, but with mixed frequencies (MF). The re- sults are shown in Fig 7.4c,d. Again, it can be seen that both types of synchronized stimulation failed to provide good results. Once again, it can be seen that synchro- nized stimulation cannot permanently move the network out of the seizure state. Also, while the alternative 2 unsynchronized sham stimulation type (RM and MF) were able to abort 20% and 5%, respectively, they were far inferior to the optimal SA solution which was able to abort 100% of the seizures, thereby conrming the ecacy of the SA algorithm to identify optimal stimulation patterns. 201 Figure 7.4: (A) Examples of optimal 90Hz PT (top) and 85Hz RT applied equally to all 24 neurons. (B) The optimal PT and RT frequencies were found by comparing seizure length over various frequencies over 50 trials. Black lines indicate seizure length in control (no stimulation) conditions. Line and shading show meanSEM. (C) Comparison of perfor- mance of 6 types of stimulation (see text) over 50 trials. Stimulation was applied for the rst 500ms (vertical red line). Each row shows the network cluster over time (see Fig.2b,d). As can be seen, only the optimal SA stimulation pattern was able to move the network from the seizure cluster (yellow) to the normal cluster (blue). (D) MUST CHANGE THIS TO AND B TO REFLECT A BETTER METRIC, LIKE % OF SEIZURES ABORTED! 7.3.4 Responsive Neurostimulation In order to assess the feasibility of model-based responsive neurostimulation, the optimal neurostimulation pattern was delivered in "real-time" as soon as a seizure was detected. Causal clustering allowed real-time seizure detection by identifying when the network shifted from the normal to seizure cluster (Fig. 2,3). A simple control strategy was employed whereby the optimal 250 ms neurostimulation pattern (Fig. 3b) was applied as soon as the network entered the seizure state. At the end of the stimulation, if the network was still in the seizure state, another round of stimulation was immediately applied; otherwise, the stimulation therapy ended. Notably, this is the same control algorithm currently used in the Neuropace RNS R 202 device (RNS R System User Manual). Two Simulations were performed with and without responsive neurostimulation (Fig. 5). Both simulations were under epilepticf;Bg parameters and were per- formed using identical initial conditions and random number generators. As can be seen, in the control case, 4 spontaneous seizures emerges lasting between 5 and 34 seconds. The responsive neurostimulation algorithm was able to detect all these seizures (and additional seizures which emerged while the network was in the seizure state). Furthermore, in most cases a single round of stimulation was able to prevent the network from going into a prolonged seizure state. In one case, at 72 seconds, the rst stimulus was unsuccessful eliminating the seizure, so a second stimulus was applied thereafter and successfully stopped the seizure. Figure 7.5: (A) 2 minutes of spontaneous network activity under epileptic conditions where 2 seizures spontaneously emerged. (B) identical network activity in (A), but with responsive neurostimulation, whereby the optimal 250 stimulus was delivered at the times indicated by the red lines. As can be seen, the stimulation was able to avert a prolonged seizure. 7.4 Discussion In this study single neuron activity from human hippocampus was used to develop a reconstructed neuronal network (RNN) which replicates in-silico the distinctive 203 connectivity and causal dynamics of the recorded 24 neurons (Fig. 1). the RNN was estimated using a nonparametric/phenomenological approach based entirely on recorded data and which makes few a priori assumptions about the biophysical nature of the network dynamics (Pillow et al., 2008). The spiking probability of each neuron was estimated using a realistic model incorporating the output neu- ron's past spiking history and the spiking history of all connected neurons. Group regularization allowed for the ecient and compact estimation of connectivity and model complexity (i.e. whether neuronal interactions are best described by a linear or nonlinear lter). 7.4.1 RNN for Neurostimulation Design After the RNN was estimated, seizure dynamics were induced by raising membrane potential and isolating the network from external noise (Fig. 2), both features which have been implicated in initiating physiological seizures (Fricker, Verheugen, and Miles, 1999; Wendling et al., 2003; Warren et al., 2010). Finally, a simulated annealing algorithm was used to design an optimal stimulation pattern to induce the network to leave the seizure state (Fig. 3). The optimal stimulation was found to abate 100% of seizures and was successfully used in a responsive stimulation paradigm to prevent seizures from developing in the RNN (Fig. 4-5). These results lead us to hypothesize that (1) the unique nature of every patient's seizure focus proscribes any single neurostimulation pattern from being optimal in every patient (2) the distinctive nature of the seizure focus can be exploited to algorithmically develop ecient patient-specic neurostimulation patterns. A conceptually similar approach has been used in a recent study where a neural mass model of the thalamocortical network estimated from patient data was used to explain why particular frequencies of stimulation were successful to abate seizures 204 while others were not (Mina et al., 2013). Additionally, such a customized/algorithmic approach has already begun to be applied to develop neurostimulation patterns for Parkinson's Disease (PD) (Holt and Neto, 2014; Grill, Brocker, and Kent, 2014). However, in PD, one has near instantaneous feedback of the stimulation by assessing its eects on the patient's tremor (Brocker et al., 2013). The challenges are much greater in epilepsy where physician must oftentimes wait several months before they can access the quality of a particular stimulation design due to the infrequency of seizures. Furthermore, physicians cannot "go back in time" to see how a particu- lar seizure would have evolved had a dierent stimulation pattern been applied or had no stimulation been applied at all. The latter is particularly important since responsive neurostimulation aims to perturb the network in the preictal state and thus prevent the seizure from ever occurring; currently, however, devices such as the Neuropace suer from a very high false-positive rate. This means that the lack of a seizure following stimulation cannot be used as indicative of its success since in most cases no seizure would have developed regardless of the stimulation pattern. Due to these diculties, the Neuropace manual recommends that physicians use a 200 Hz periodic stimulus, and if it is unsuccessful to increase the current amplitude (RNS R System User Manual). This is despite the fact that the device allows two leads to be independently programmed with a wide range of complex stimuli and a frequency range of 1-333 Hz (Sun, Morrell, and Wharen Jr, 2008). We believe the bottleneck in the performance of devices such as the Neuropace, which currently reduces seizures by an impressive but far from perfect 54% (Heck et al., 2014), is not the hardware, but rather the physicians inability to successfully identify optimal stimulation parameters. In this study, model based in-silico neurostimulation optimization is presented as a solution to this vital issue. In this paradigm, a patient-specic model is used as 205 a testbed or hypothesis engine for the design and validation of optimal neurostim- ulation. This paradigm provides solutions to many of the experimental issues of neurostimulation design: one may obtain seizures "on-demand" by initiating the network in the seizure state; furthermore, by using identical sequences of random numbers, one can "go back in time" and observe how the seizure would have evolved under dierent applied stimuli. Using this testbed we gained many insights into neurostimulation and generated many testable predictions. It was observed that synchronized random (Poisson) stimulation provides slight benets over periodic stimulation - an observation previously observed in the literature both experimen- tally and in computational models (Wyckhuys et al., 2010; Buel et al., 2014). However, neither of these stimulus styles provided the benets that random unsyn- chronized stimulation over multiple sites provided, suggesting the need to conduct more experiments exploiting multiple electrodes for stimulation (Cook et al., 2013; Van Nieuwenhuyse et al., 2014). Furthermore, it was observed that the optimal stim- ulation disproportionably targeted neurons outside of the epileptic subnetwork (i.e. focus). However, our most important nding was that optimized stimulation over multiple electrodes can signicantly outperform any of the previously mentioned stimulus styles by exploiting each patient's unique network topology and dynamics. Most importantly the optimal neurostimulation pattern identied here can be ex- perimentally validated by applying it to the patient for which it was estimated on. This ability to experimentally validate our model predictions is lacking in many of the computational studies exploring neurostimulation. 7.4.2 Modeling Methodology and Limitations While the current framework provided very strong computational results, several limitations need to be addressed before it can be applied experimentally. The 206 network dynamics were estimated from spontaneous/observed data and predictive power was used to determine connectivity. This Granger-causality approach is bi- ased by unobserved inputs, and in this case, every neuron within CA3 and every neuron which inputs to CA3 (such as those from the entorhinal cortex) are po- tential unobserved inputs. Furthermore, our model of electrical stimulation, which assumes a single electrode can illicit a spike in a single neuron, is overly simplistic. Any stimulation will aect at least dozens of surrounding cells (Wei and Grill, 2005; Desai et al., 2014); furthermore, suprathreshold stimulation may kindle seizures (Racine, 1972) and thus subthreshold stimulation which only increases a neuron's ring probability without guaranteeing a spike is more realistic. Both of these issues can simultaneously be overcome by actively perturbing the network using sequential stimulus pulses across multiple electrodes. Ecient algorithms are already being de- veloped for how to optimally design such experiments (Lepage, Ching, and Kramer, 2013; Kim et al., 2014) The model presented here relied on single unit activity. In practice, it may be more realistic to apply the presented framework for identifying optimal stimulation to other electrophysiological signals such as ECoG. In this case, each of the specic steps would be modied, while the overall framework would remain the same. For example, in the simulated annealing algorithm the current cost function of MFR would need to be substituted for a metric which can be applied to continuous signals, such as high frequency oscillations. Furthermore, due to the diculty of recording single units during human seizures, and the diculty of estimating reliable models from such short data records, the present work synthetically induced a seizure. In future work, estimating network dynamics from actual seizure data may lead to better results. Most importantly, there is a need to validate the obtained results experimentally. We imagine that the advocated framework will need to go through 207 several iterations of experimental renement until the strong computational results achieved here can be matched in actual animal models or human patients. From a computational perspective, several improvements can be made for stim- ulation design and control. While the simulated annealing algorithm considered a very large space of stimulation possibilities (2 176 total), several stimulation patterns were not considered such as those which intermixed periodic and random pulse trains and completely arbitrary pulse trains (Grill Jr and Brocker, 2014). Furthermore, results may potentially be improved by incorporating the relative phases between dierent pulse trains into the algorithm. Also, a relatively simple control strat- egy based on the Neuropace device was employed. The advantage of this strategy was that the stimulation was independent of the seizure specics. In the future more sophisticated control algorithms may be employed which emit dierent stimu- lation patterns based on the quality and progression of the specic seizure (Ching, Brown, and Kramer, 2012; Zalay and Bardakjian, 2013; Kalitzin et al., 2014; Ehrens, Sritharan, and Sarma, 2015). Overall, while many improvements can be made in the specics, we believe that the overall framework presented here has the potential to signicantly increase the eectiveness of neurostimulation for epilepsy. 7.4.3 Vision Our speculative and perhaps overly optimistic vision is that in the future epileptic patients will be implanted with stimulation devices consisting of multiple electrodes which are capable of both recording and stimulating (Ryapolova-Webb et al., 2014) and can be independently programmed. Upon implantation, an automatic stim- ulation algorithm will perturb the network to establish safe current levels and to map eective connectivity between the observed areas. Machine learning algorithms will then program the initial stimulation parameters. As is currently done in the 208 Neuropace (RNS R System User Manual), the device will automatically record all detected seizure activity and this data will be uploaded daily to a computer. Then, this data and patient input will be used oine to analyze the success of yesterday's stimulation. Finally, a reinforcement learning paradigm (Gosavi, 2014), will be used to adjust parameters for the next day. While admittedly, such a task may seem in- credibly dicult to realize we believe that the growth of machine learning in the last decade has made this more realistic to accomplish than ever before. Furthermore, this goal may be much more accessible than other promising treatments for epilepsy since the hardware is already there. All that needs to be done is to improve the software. 7.5 Supplementary Figures 209 Figure 7.6: Probe placement superimposed on an MRI scan showing the hippocampal forma- tion. Each probe consists of 24 microelectrodes and 4 macroelectrodes. 210 Figure 7.7: (A,B) Plots show # of parameters selected (A) and testing set correlation, , (B) for dierent values of the regularization parameter, . Each plot shows a line for each of the 24 neurons in the RNN. Notice that as 1= is increased (and thus regularization is weakened), more parameters enter the model until we have a full model. Dots show the optimal selected for each neuron. (C) ROC plots for each of the 24 neurons, showing model predictive power. Notice that each of the lines are above the dashed blue line (TPR=FPR) which represents a model with no predictive power. (D) vertical KS Plots for each of the 24 neurons, along with normalized 95% condence bounds (Song et al., 2013). As can be seen, most models fall within the bounds. 211 Figure 7.8: All kernels for the 24 neuron RNN. Each box shows the linear or quadratic kernel for each RNN. Grey boxes indicate no eective connectivity between those neurons. Autoregressive lters are along the diagonal. 212 Figure 7.9: (A) Schematic for how neighbors are selected in the rst (top) and second (bottom) round of simulated annealing. (B) Top shows evolution of electrode parameters during the second round of simulated annealing. Notice that some electrodes are consistently kept o as they were not selected during the rst round. Bottom shows the associated cost function for each parameter value. (C) Final costs for 6 parallel runs of the SA algorithm using dierent modes of stimulation: PT,RNB, and RT. Vertical red line shows that the best results were achieved for PT stimulation and these parameters were the selected ones. 213 Figure 7.10: (A) Synchronized 200 Hz periodic stimulus. (B) Network output. Notice that the seizures temporarily stops during stimulation; however, the seizure returns almost immediately once stimulation is turned o. (C) The same plot in principal-component space. 214 References [1] Adelson, Edward H and James R Bergen (1985). \Spatiotemporal energy models for the perception of motion". In: Journal of the Optical Society of America A 2.2, pp. 284{299. [2] Aertsen, AMHJ and PIM Johannesma (1981). \The spectro-temporal recep- tive eld". In: Biological cybernetics 42.2, pp. 133{143. [3] Ahmed, Omar J and Mayank R Mehta (2009). \The hippocampal rate code: anatomy, physiology and theory". In: Trends in neurosciences 32.6, pp. 329{ 338. [4] Alfa, Attahiru Sule and Marcel F Neuts (1995). \Modelling vehicular trac using the discrete time Markovian arrival process". In: Transportation Science 29.2, pp. 109{117. [5] Aljade, Johnatan et al. (2013). \Spike Triggered Covariance in Strongly Correlated Gaussian Stimuli". In: PLoS computational biology 9.9, e1003206. [6] Alonso, A and E Garcia-Austt (1987). \Neuronal sources of theta rhythm in the entorhinal cortex of the rat". In: Experimental brain research 67.3, pp. 493{501. [7] Amit, Daniel J (1995). \The Hebbian paradigm reintegrated: Local reverber- ations as internal representations". In: BEHAVIORAL AND BRAIN SCI- ENCES 18, pp. 617{657. 215 [8] Anderson, William S et al. (2009). \Phase-dependent stimulation eects on bursting activity in a neural network cortical simulation". In: Epilepsy re- search 84.1, pp. 42{55. [9] Arcas, Blaise Ag uera y, Adrienne L Fairhall, and William Bialek (2001). \What can a single neuron compute?" In: Advances in Neural Information Processing Systems 13: Proceedings of the 2000 Conference. Vol. 13. MIT Press, p. 75. [10] | (2003). \Computation in a single neuron: Hodgkin and Huxley revis- ited". In: Neural Computation 15.8, pp. 1715{1749. [11] Avoli, Massimo et al. (2002). \Network and pharmacological mechanisms leading to epileptiform synchronization in the limbic system in vitro". In: Progress in neurobiology 68.3, pp. 167{207. [12] Axmacher, Nikolai et al. (2006). \Memory formation by neuronal synchro- nization". In: Brain research reviews 52.1, pp. 170{182. [13] Barbarosie, Michaela and Massimo Avoli (1997). \CA3-driven hippocampal- entorhinal loop controls rather than sustains in vitro limbic seizures". In: The Journal of neuroscience 17.23, pp. 9308{9314. [14] Barbieri, Riccardo et al. (2005). \A point-process model of human heartbeat intervals: new denitions of heart rate and heart rate variability". In: Amer- ican Journal of Physiology-Heart and Circulatory Physiology 288.1, H424{ H435. [15] Bartesaghi, R, M Migliore, and T Gessi (2006). \Input{output relations in the entorhinal cortex{dentate{hippocampal system: evidence for a non-linear transfer of signals". In: Neuroscience 142.1, pp. 247{265. 216 [16] Bartesaghi, Renata and Tiziana Gessi (2003). \Activation of perforant path neurons to eld CA1 by hippocampal projections". In: Hippocampus 13.2, pp. 235{249. [17] Bartesaghi, Renata, Tiziana Gessi, and Michele Migliore (1995). \Input- output relations in the entorhinal-hippocampal-entorhinal loop: entorhinal cortex and dentate gyrus". In: Hippocampus 5.5, pp. 440{451. [18] Begley, Charles E et al. (2000). \The Cost of Epilepsy in the United States: An Estimate from Population-Based Clinical and Survey Data". In: Epilepsia 41.3, pp. 342{351. [19] Ben-Menachem, Elinor and Gregory L Krauss (2014). \Epilepsy: Respon- sive neurostimulation [mdash] modulating the epileptic brain". In: Nature Reviews Neurology. [20] Berger, Theodore W et al. (2005). \Restoring lost cognitive function". In: Engineering in Medicine and Biology Magazine, IEEE 24.5, pp. 30{44. [21] Berger, Theodore W et al. (2012). \A hippocampal cognitive prosthesis: multi-input, multi-output nonlinear modeling and VLSI implementation". In: Neural Systems and Rehabilitation Engineering, IEEE Transactions on 20.2, pp. 198{211. [22] Berry, Stephen D and Richard F Thompson (1978). \Prediction of learn- ing rate from the hippocampal electroencephalogram". In: Science 200.4347, pp. 1298{1300. [23] Bertram, Edward H (2013). \Neuronal circuits in epilepsy: Do they matter?" In: Experimental neurology 244, pp. 67{74. [24] Betz, WJ (1970). \Depression of transmitter release at the neuromuscular junction of the frog". In: The Journal of Physiology 206.3, p. 629. 217 [25] Bialek, William and Rob R van Steveninck (2005). \Features and dimensions: Motion estimation in y vision". In: arXiv preprint q-bio/0505003. [26] Bibbig, Andrea et al. (2007). \Beta rhythms (15{20 Hz) generated by non- reciprocal communication in hippocampus". In: Journal of neurophysiology 97.4, pp. 2812{2823. [27] Bishop, Christopher M et al. (2006). Pattern recognition and machine learn- ing. Vol. 1. springer New York. [28] Boido, Davide et al. (2014). \Network Dynamics During the Progression of Seizure-Like Events in the Hippocampal{Parahippocampal Regions". In: Cerebral Cortex 24.1, pp. 163{173. [29] Bower, Mark R et al. (2012). \Spatiotemporal neuronal correlates of seizure generation in focal epilepsy". In: Epilepsia 53.5, pp. 807{816. [30] Bragin, A et al. (1997). \Epileptic afterdischarge in the hippocampal{entorhinal system: current source density and unit studies". In: Neuroscience 76.4, pp. 1187{1203. [31] Breheny, Patrick and Jian Huang (2014). \Group descent algorithms for non- convex penalized linear and logistic regression models with grouped predic- tors". In: Statistics and Computing, pp. 1{15. [32] Brocker, David T et al. (2013). \Improved ecacy of temporally non-regular deep brain stimulation in Parkinson's disease". In: Experimental neurology 239, pp. 60{67. [33] Brodie, Martin J. and Marc A. Dichter (1996). \Antiepileptic Drugs". In: New England Journal of Medicine 334.3, pp. 168{175. [34] Buel, Ine et al. (2014). \The eect of high and low frequency cortical stim- ulation with a xed or a poisson distributed interpulse interval on corti- 218 cal excitability in rats". In: International journal of neural systems 24.02, p. 1430005. [35] Bullmore, Ed and Olaf Sporns (2009). \Complex brain networks: graph the- oretical analysis of structural and functional systems". In: Nature Reviews Neuroscience 10.3, pp. 186{198. [36] Burns, Samuel P et al. (2014). \Network dynamics of the brain and in uence of the epileptic seizure onset zone". In: Proceedings of the National Academy of Sciences 111.49, E5321{E5330. [37] Buzs aki, Gy (1989). \Two-stage model of memory trace formation: a role for noisy brain states". In: Neuroscience 31.3, pp. 551{570. [38] Buzs aki, Gy orgy (1996). \The hippocampo-neocortical dialogue". In: Cere- bral Cortex 6.2, pp. 81{92. [39] | (2002). \Theta oscillations in the hippocampus". In: Neuron 33.3, pp. 325{340. [40] Buzsaki, Gyorgy (2006). Rhythms of the Brain. Oxford University Press. [41] Buzs aki, Gy orgy, Costas A Anastassiou, and Christof Koch (2012). \The origin of extracellular elds and currentsEEG, ECoG, LFP and spikes". In: Nature Reviews Neuroscience 13.6, pp. 407{420. [42] Buzs aki, Gy orgy and Edvard I Moser (2013). \Memory, navigation and theta rhythm in the hippocampal-entorhinal system". In: Nature neuroscience 16.2, pp. 130{138. [43] Buzs aki, Gy orgy and Xiao-Jing Wang (2012). \Mechanisms of gamma oscil- lations". In: Annual review of neuroscience 35, pp. 203{225. [44] Buzs aki, Gy orgy et al. (1991). \Emergence and propagation of interictal spikes in the subcortically denervated hippocampus". In: Hippocampus 1.2, pp. 163{180. 219 [45] Cantrell, Donald R et al. (2010). \Non-centered spike-triggered covariance analysis reveals neurotrophin-3 as a developmental regulator of receptive eld properties of ON-OFF retinal ganglion cells". In: PLoS computational biology 6.10, e1000967. [46] Caruana, Rich and Alexandru Niculescu-Mizil (2004). \Data mining in metric space: an empirical analysis of supervised learning performance criteria". In: Proceedings of the tenth ACM SIGKDD international conference on Knowl- edge discovery and data mining. ACM, pp. 69{78. [47] Chichilnisky, EJ (2001). \A simple white noise analysis of neuronal light responses". In: Network: Computation in Neural Systems 12.2, pp. 199{213. [48] Ching, ShiNung, Emery N Brown, and Mark A Kramer (2012). \Distributed control in a mean-eld cortical network model: implications for seizure sup- pression". In: Physical Review E 86.2, p. 021920. [49] Chkhenkeli, SA and IS Chkhenkeli (1997). \Eects of therapeutic stimula- tion of nucleus caudatus on epileptic electrical activity of brain in patients with intractable epilepsy". In: Stereotactic and functional neurosurgery 69.1- 4, pp. 221{224. [50] Chrobak, James J, Robert W Stackman, and Thomas J Walsh (1989). \In- traseptal administration of muscimol produces dose-dependent memory im- pairments in the rat". In: Behavioral and neural biology 52.3, pp. 357{369. [51] Citron, MC and VZ Marmarelis (1987). \Applications of minimum-order Wiener modeling to retinal ganglion cell spatiotemporal dynamics". In: Bio- logical cybernetics 57.4-5, pp. 241{247. [52] Colgin, Laura Lee (2013). \Mechanisms and functions of theta rhythms". In: Annual review of neuroscience 36, pp. 295{312. 220 [53] Colom, Luis V (2006). \Septal networks: relevance to theta rhythm, epilepsy and Alzheimer's disease". In: Journal of neurochemistry 96.3, pp. 609{623. [54] Cook, Mark J et al. (2013). \Prediction of seizure likelihood with a long-term, implanted seizure advisory system in patients with drug-resistant epilepsy: a rst-in-man study". In: The Lancet Neurology 12.6, pp. 563{571. [55] Cordeiro, Joacir G et al. (2013). \Modulation of excitability by continuous low-and high-frequency stimulation in fully hippocampal kindled rats". In: Epilepsy research 107.3, pp. 224{230. [56] Craig, Sarah and Sean Commins (2005). \Interaction between paired-pulse facilitation and long-term potentiation in the projection from hippocampal area CA1 to the entorhinal cortex". In: Neuroscience research 53.2, pp. 140{ 146. [57] Crick, Francis and Christof Koch (1990). \Towards a neurobiological the- ory of consciousness". In: Seminars in the Neurosciences. Vol. 2. Saunders Scientic Publications, pp. 263{275. [58] Csicsvari, Jozsef et al. (2003). \Mechanisms of gamma oscillations in the hippocampus of the behaving rat". In: Neuron 37.2, pp. 311{322. [59] Dan, Yang, Gidon Felsen, and Jon Touryan (2009). Extracellular recording from cells in cat primary visual cortex. CRCNS.org. English. [60] Dayan, Peter and Laurence F Abbott (2001). Theoretical neuroscience. Cam- bridge, MA: MIT Press. [61] De Boer, Egbert and Paul Kuyper (1968). \Triggered correlation". In: Biomed- ical Engineering, IEEE Transactions on 3, pp. 169{179. [62] Deadwyler, Sam A et al. (1975). \Physiological studies of the reciprocal con- nections between the hippocampus and entorhinal cortex". In: Experimental neurology 49.1, pp. 35{57. 221 [63] Desai, Sharanya Arcot et al. (2014). \Deep brain stimulation macroelectrodes compared to multiple microelectrodes in rat hippocampus". In: Frontiers in neuroengineering 7. [64] Di, Giovanna J et al. (2010). \Development of a closed-loop neural prosthesis for vestibular disorders". In: Journal of Automatic Control 20.1, pp. 27{32. [65] Dittman, Jeremy S, Anatol C Kreitzer, and Wade G Regehr (2000). \Inter- play between facilitation, depression, and residual calcium at three presynap- tic terminals". In: The Journal of Neuroscience 20.4, pp. 1374{1385. [66] Doose, Hermann and Wolfgang K Baier (1988). \Theta rhythms in the EEG: a genetic trait in childhood epilepsy". In: Brain and Development 10.6, pp. 347{354. [67] Durand, Dominique M and Marom Bikson (2001). \Suppression and control of epileptiform activity by electrical stimulation: a review". In: Proceedings of the IEEE 89.7, pp. 1065{1082. [68] Eggermont, Jos J (1993). \Wiener and Volterra analyses applied to the au- ditory system". In: Hearing research 66.2, pp. 177{201. [69] Ehrens, Daniel, Duluxan Sritharan, and Sridevi V Sarma (2015). \Closed- loop control of a fragile network: application to seizure-like dynamics of an epilepsy model". In: Frontiers in neuroscience 9. [70] Eikenberry, Steen E and Vasilis Z Marmarelis (2013). \A nonlinear autore- gressive Volterra model of the Hodgkin{Huxley equations". In: Journal of computational neuroscience 34.1, pp. 163{183. [71] Emerson, Robert C et al. (1987). \Nonlinear directionally selective subunits in complex cells of cat striate cortex". In: J Neurophysiol 58.1, pp. 33{65. [72] Engel, Jerome et al. (2003). \Practice parameter: Temporal lobe and lo- calized neocortical resections for epilepsy Report of the Quality Standards 222 Subcommittee of the American Academy of Neurology, in Association with the American Epilepsy Society and the American Association of Neurological Surgeons". In: Neurology 60.4, pp. 538{547. [73] Ezrokhi, VL et al. (2001). \Reverberation of Excitation in Living Hippocam- pal Formation{Entorhinal Cortex Slices from Rats. Optical Recording". In: Neuroscience and behavioral physiology 31.1, pp. 31{37. [74] Fairhall, Adrienne L et al. (2006). \Selectivity for multiple stimulus features in retinal ganglion cells". In: Journal of neurophysiology 96.5, pp. 2724{2738. [75] Fallani, Fabrizio De Vico et al. (2014). \Graph analysis of functional brain networks: practical issues in translational neuroscience". In: Philosophical Transactions of the Royal Society B: Biological Sciences 369.1653, p. 20130521. [76] Fell, Juergen et al. (2003). \Rhinal{hippocampal theta coherence during declarative memory formation: interaction with gamma synchronization?" In: European Journal of Neuroscience 17.5, pp. 1082{1088. [77] Fell, J urgen et al. (2001). \Human memory formation is accompanied by rhinal{hippocampal coupling and decoupling". In: Nature neuroscience 4.12, pp. 1259{1264. [78] Finch, David M et al. (1986). \Neurophysiology of limbic system pathways in the rat: projections from the subicular complex and hippocampus to the entorhinal cortex". In: Brain research 397.2, pp. 205{213. [79] Fisch, Bruce J (2003). \Interictal Epileptiform Activity: Diagnostic and Be- havioral Implications: 2002 ACNS Presidential Address". In: Journal of clin- ical neurophysiology 20.3, pp. 155{162. [80] Fisher, Robert S and Ana Luisa Velasco (2014). \Electrical brain stimulation for epilepsy". In: Nature Reviews Neurology. 223 [81] Fitzgerald, Jerey D et al. (2011). \Second order dimensionality reduction using minimum and maximum mutual information models". In: PLoS com- putational biology 7.10, e1002249. [82] Fournier, Julien et al. (2014). \Hidden Complexity of Synaptic Receptive Fields in Cat V1". In: The Journal of Neuroscience 34.16, pp. 5515{5528. [83] Fox, Jessica L, Adrienne L Fairhall, and Thomas L Daniel (2010). \Encoding properties of haltere neurons enable motion feature detection in a biologi- cal gyroscope". In: Proceedings of the National Academy of Sciences 107.8, pp. 3840{3845. [84] Fricker, Desdemona, Jos AH Verheugen, and Richard Miles (1999). \Cell- attached measurements of the ring threshold of rat hippocampal neurones". In: The Journal of Physiology 517.3, pp. 791{804. [85] Frost, Victor S and Benjamin Melamed (1994). \Trac modeling for telecom- munications networks". In: Communications Magazine, IEEE 32.3, pp. 70{ 81. [86] Fuster, Joaquin M (2000). \Cortical dynamics of memory". In: International Journal of Psychophysiology 35.2, pp. 155{164. [87] Givens, Bennet (1996). \Stimulus-evoked resetting of the dentate theta rhythm: relation to working memory". In: Neuroreport 8.1, pp. 159{163. [88] Gloveli, Tengis et al. (1997). \Frequency-dependent information ow from the entorhinal cortex to the hippocampus". In: Journal of neurophysiology 78.6, pp. 3444{3449. [89] Gonzalez-Sulser, Alfredo et al. (2011). \The 4-aminopyridine< i> in vitro</i> epilepsy model analyzed with a perforated multi-electrode array". In: Neu- ropharmacology 60.7, pp. 1142{1153. 224 [90] Gosavi, Abhijit (2014). Simulation-based optimization: parametric optimiza- tion techniques and reinforcement learning. Vol. 55. Springer. [91] Goutagny, Romain, Jesse Jackson, and Sylvain Williams (2009). \Self-generated theta oscillations in the hippocampus." In: Nature neuroscience 12.12. [92] Green, John D and Arnaldo A Arduini (1954). \Hippocampal electrical ac- tivity in arousal." In: Journal of neurophysiology. [93] Grill Jr, Warren M and David T Brocker (2014). Non-regular electrical stim- ulation patterns designed with a cost function for treating neurological disor- ders. US Patent 8,923,981. [94] Grill, Warren M, David T Brocker, and Alexander R Kent (2014). Devices, systems and methods for deep brain stimulation parameters. US Patent 20,140,350,634. [95] Haas, Julie S and John A White (2002). \Frequency selectivity of layer II stellate cells in the medial entorhinal cortex". In: Journal of neurophysiology 88.5, pp. 2422{2429. [96] Hafting, Torkel et al. (2005). \Microstructure of a spatial map in the entorhi- nal cortex". In: Nature 436.7052, pp. 801{806. [97] Hampson, Robert E, John D Simeral, and Sam A Deadwyler (1999). \Dis- tribution of spatial and nonspatial information in dorsal hippocampus". In: Nature 402.6762, pp. 610{614. [98] Hampson, Robert E et al. (2012a). \Closing the loop for memory prosthesis: Detecting the role of hippocampal neural ensembles using nonlinear models". In: Neural Systems and Rehabilitation Engineering, IEEE Transactions on 20.4, pp. 510{525. [99] Hampson, Robert E et al. (2012b). \Facilitation and restoration of cogni- tive function in primate prefrontal cortex by a neuroprosthesis that uti- 225 lizes minicolumn-specic neural ring". In: Journal of neural engineering 9.5, p. 056012. [100] Hanley, James A and Barbara J McNeil (1982). \The meaning and use of the area under a receiver operating characteristic (ROC) curve". In: Radiology 143.1, pp. 29{36. [101] Haslinger, Robert, Gordon Pipa, and Emery Brown (2010). \Discrete time rescaling theorem: determining goodness of t for discrete time statistical models of neural spiking". In: Neural computation 22.10, pp. 2477{2506. [102] Hebb, Donald Olding (1949). The organization of behavior: A neuropsycho- logical theory. Psychology Press. [103] Heck, Christianne N et al. (2014). \Two-year seizure reduction in adults with medically intractable partial onset epilepsy treated with responsive neu- rostimulation: Final results of the RNS System Pivotal trial". In: Epilepsia 55.3, pp. 432{441. [104] Henderson, Darrall, Sheldon H Jacobson, and Alan W Johnson (2003). \The theory and practice of simulated annealing". In: Handbook of metaheuristics. Springer, pp. 287{319. [105] Higgins, James J. (2003). Intoruction to Modern Nonparametric Statistics. [106] Hjorth-Simonsen, A (1971). \Hippocampal eerents to the ipsilateral en- torhinal area: An experimental study in the rat". In: Journal of Comparative Neurology 142.4, pp. 417{437. [107] Hochberg, Leigh R et al. (2006). \Neuronal ensemble control of prosthetic devices by a human with tetraplegia". In: Nature 442.7099, pp. 164{171. [108] Holt, Abbey B and Theoden I Neto (2014). \Origins and suppression of oscillations in a computational model of Parkinsons disease". In: Journal of computational neuroscience 37.3, pp. 505{521. 226 [109] Huerta, Patricio T and John E Lisman (1995). \Bidirectional synaptic plas- ticity induced by a single burst during cholinergic theta oscillation in CA1 in vitro". In: Neuron 15.5, pp. 1053{1063. [110] Hutcheon, Bruce and Yosef Yarom (2000). \Resonance, oscillation and the intrinsic frequency preferences of neurons". In: Trends in neurosciences 23.5, pp. 216{222. [111] Iijima, Toshio et al. (1996). \Entorhinal-hippocampal interactions revealed by real-time imaging". In: Science 272.5265, pp. 1176{1179. [112] Jahnsen, Henrik and Sergei Karnup (1994). \A spectral analysis of the inte- gration of articial synaptic potentials in mammalian central neurons". In: Brain research 666.1, pp. 9{20. [113] Jeerys, JGR et al. (2012). \Limbic Network Synchronization and Tempo- ral Lobe Epilepsy". In: Jasper's Basic Mechanisms of the Epilepsies. 4th. National Center for Biotechnology Information (US), Bethesda (MD). url: http://europepmc.org/abstract/MED/22787650. [114] Jing, Xingjian et al. (2012). \Understanding neuronal systems in movement control using Wiener/Volterra kernels: A dominant feature analysis". In: Journal of neuroscience methods 203.1, pp. 220{232. [115] Johnson, Luke R, Joseph LeDoux, and Valerie Doyere (2009). \Hebbian rever- berations in emotional memory micro circuits". In: Frontiers in neuroscience 3, p. 27. [116] Jones, Roland SG (1993). \Entorhinal-hippocampal connections: a specula- tive view of their function". In: Trends in neurosciences 16.2, pp. 58{64. [117] Kaardal, Joel et al. (2013). \Identifying functional bases for multidimensional neural computations". In: Neural computation 25.7, pp. 1870{1890. 227 [118] Kahana, Michael J, David Seelig, and Joseph R Madsen (2001). \Theta re- turns". In: Current opinion in neurobiology 11.6, pp. 739{744. [119] Kalitzin, Stiliyan et al. (2014). \Multiple oscillatory states in models of col- lective neuronal dynamics". In: International journal of neural systems 24.06, p. 1450020. [120] Kamondi, Anita et al. (1998). \Theta oscillations in somata and dendrites of hippocampal pyramidal cells in vivo: Activity-dependent phase-precession of action potentials". In: Hippocampus 8.3, pp. 244{261. [121] Kang, Eunji E et al. (2010). \Transformation of neuronal modes associated with low-Mg2+/high-K+ conditions in an in vitro model of epilepsy". In: Journal of biological physics 36.1, pp. 95{107. [122] Kaplan, Stephen, Martin Sonntag, and Eric Chown (1991). \Tracing recur- rent activity in cognitive elements (TRACE): A model of temporal dynamics in a cell assembly". In: Connection Science 3.2, pp. 179{206. [123] Katz, Bo and R Miledi (1968). \The role of calcium in neuromuscular facili- tation". In: The Journal of physiology 195.2, pp. 481{492. [124] Kim, Anmo J, Aurel A Lazar, and Yevgeniy B Slutskiy (2011). \System identication of Drosophila olfactory sensory neurons". In: Journal of com- putational neuroscience 30.1, pp. 143{161. [125] Kim, Sanggyun et al. (2011). \A Granger causality measure for point process models of ensemble neural spiking activity". In: PLoS computational biology 7.3, e1001110. [126] Kim, Woojae et al. (2014). \A hierarchical adaptive approach to optimal experimental design". In: Neural computation. 228 [127] Kinoshita, Masako et al. (2005). \Electric cortical stimulation suppresses epileptic and background activities in neocortical epilepsy and mesial tem- poral lobe epilepsy". In: Clinical neurophysiology 116.6, pp. 1291{1299. [128] Kirkpatrick, Scott, MP Vecchi, et al. (1983). \Optimization by simmulated annealing". In: science 220.4598, pp. 671{680. [129] Kloosterman, Fabian, Theo van Haeften, and Fernando H Lopes da Silva (2004). \Two reentrant pathways in the hippocampal-entorhinal system". In: Hippocampus 14.8, pp. 1026{1039. [130] Kocsis, Bernat, Anatol Bragin, and Gy orgy Buzs aki (1999). \Interdepen- dence of multiple theta generators in the hippocampus: a partial coherence analysis". In: The Journal of neuroscience 19.14, pp. 6200{6212. [131] Koh, Taiho and Edward J Powers (1985). \Second-order Volterra ltering and its application to nonlinear system identication". In: Acoustics, Speech and Signal Processing, IEEE Transactions on 33.6, pp. 1445{1455. [132] Korenberg, MJ, SB Bruder, and PJ Mcllroy (1988). \Exact orthogonal ker- nel estimation from nite data records: Extending Weiner's identication of nonlinear systems". In: Annals of biomedical engineering 16.2, pp. 201{214. [133] Korenberg, Michael J and Ian W Hunter (1990). \The identication of non- linear biological systems: Wiener kernel approaches". In: Annals of biomedical engineering 18.6, pp. 629{654. [134] Krausz, Howard I (1975). \Identication of nonlinear systems using random impulse train inputs". In: Biological Cybernetics 19.4, pp. 217{230. [135] Krumin, Michael and Shy Shoham (2010). \Multivariate autoregressive mod- eling and granger causality analysis of multiple spike trains". In: Computa- tional intelligence and neuroscience 2010, p. 10. 229 [136] Lansner, A and E Frans en (1992). \Modelling Hebbian cell assemblies com- prised of cortical neurons". In: Network: Computation in Neural Systems 3.2, pp. 105{119. [137] Lee, YW and M Schetzen (1965). \Measurement of the Wiener Kernels of a Non-linear System by Cross-correlation". In: International Journal of Control 2.3, pp. 237{254. [138] Lepage, Kyle Q, ShiNung Ching, and Mark A Kramer (2013). \Inferring evoked brain connectivity through adaptive perturbation". In: Journal of computational neuroscience 34.2, pp. 303{318. [139] Leung, L Stan and Hui-Wen Yu (1998). \Theta-frequency resonance in hip- pocampal CA1 neurons in vitro demonstrated by sinusoidal current injec- tion". In: Journal of neurophysiology 79.3, pp. 1592{1596. [140] Li, X-G et al. (1994). \The hippocampal CA3 network: an in vivo intracellular labeling study". In: Journal of Comparative Neurology 339.2, pp. 181{208. [141] Liley, AW and KAK North (1953). \An electrical investigation of eects of repetitive stimulation on mammalian neuromuscular junction". In: J Neuro- physiol 16.5, pp. 509{527. [142] Loeb, Gerald E (1990). \Cochlear prosthetics". In: Annual review of neuro- science 13.1, pp. 357{371. [143] Loeb, Gerald E et al. (2001). \BION< sup></sup> system for distributed neu- ral prosthetic interfaces". In: Medical engineering & physics 23.1, pp. 9{18. [144] London, Michael and Michael H ausser (2005). \Dendritic computation". In: Annu. Rev. Neurosci. 28, pp. 503{532. [145] Lopantsev, Valeri and Massimo Avoli (1998). \Participation of GABAA- mediated inhibition in ictallike discharges in the rat entorhinal cortex". In: Journal of neurophysiology 79.1, pp. 352{360. 230 [146] Lu, Ude, Dong Song, and Theodore W Berger (2011). \Nonlinear dynamic modeling of synaptically driven single hippocampal neuron intracellular ac- tivity". In: Biomedical Engineering, IEEE Transactions on 58.5, pp. 1303{ 1313. [147] Lu, Ude et al. (2012). \Nonlinear Dynamic Modeling of Neuron Action Po- tential Threshold During Synaptically Driven Broadband Intracellular Ac- tivity". In: Biomedical Engineering, IEEE Transactions on 59.3, pp. 706{ 716. [148] Mann, Edward O and Ole Paulsen (2005). \Mechanisms underlying gamma (40Hz) network oscillations in the hippocampusa mini-review". In: Progress in biophysics and molecular biology 87.1, pp. 67{76. [149] Marmarelis, Panos Z and Vasilis Z Marmarelis (1978). \Analysis of physio- logical systems: The white-noise approach". In: [150] Marmarelis, Panos Z and KI Naka (1973). \Nonlinear analysis and synthesis of receptive-eld responses in the catsh retina. I. Horizontal cell leads to ganglion cell chain." In: Journal of Neurophysiology 36.4, pp. 605{618. [151] Marmarelis, Panos Z and Ken-Ichi Naka (1974). \Identication of multi- input biological systems". In: Biomedical Engineering, IEEE Transactions on 2, pp. 88{101. [152] Marmarelis, VZ and TW Berger (2005). \General methodology for nonlinear modeling of neural systems with Poisson point-process inputs". In: Mathe- matical biosciences 196.1, pp. 1{13. [153] Marmarelis, VZ, MC Citron, and CP Vivo (1986). \Minimum-order Wiener modelling of spike-output systems". In: Biological cybernetics 54.2, pp. 115{ 123. 231 [154] Marmarelis, VZ and ME Orme (1993). \Modeling of neural systems by use of neuronal modes". In: Biomedical Engineering, IEEE Transactions on 40.11, pp. 1149{1158. [155] Marmarelis, VZ, DC Shin, and R Zhang (2012). \Linear and nonlinear mod- eling of cerebral ow autoregulation using Principal Dynamic Modes". In: The open biomedical engineering journal 6, p. 42. [156] Marmarelis, VZ et al. (2012). \Design of optimal stimulation patterns for neuronal ensembles based on Volterra-type hierarchical modeling". In: Jour- nal of neural engineering 9.6, p. 066003. [157] Marmarelis, VZ et al. (2013a). \Closed-loop dynamic modeling of cerebral hemodynamics". In: Annals of biomedical engineering 41.5, pp. 1029{1048. [158] Marmarelis, Vasilis Z (1993). \Identication of nonlinear biological systems using Laguerre expansions of kernels". In: Annals of biomedical engineering 21.6, pp. 573{589. [159] | (1994). \On kernel estimation using non-Gaussian and/or non-white input data". In: Advanced Methods of Physiological System Modeling. Springer, pp. 229{242. [160] | (1997). \Modeling methology for nonlinear physiological systems". In: Annals of biomedical engineering 25.2, pp. 239{251. [161] | (2004). Nonlinear dynamic modeling of physiological systems. Wiley- Interscience. [162] Marmarelis, Vasilis Z, Mikko Juusola, and Andrew S French (1999). \Princi- pal dynamic mode analysis of nonlinear transduction in a spider mechanore- ceptor". In: Annals of biomedical engineering 27.3, pp. 391{402. [163] Marmarelis, Vasilis Z, Theodoros P Zanos, and Theodore W Berger (2009). \Boolean modeling of neural systems with point-process inputs and outputs. 232 Part I: Theory and simulations". In: Annals of biomedical engineering 37.8, pp. 1654{1667. [164] Marmarelis, Vasilis Z et al. (1999). \Nonlinear analysis of renal autoreg- ulation in rats using principal dynamic modes". In: Annals of biomedical engineering 27.1, pp. 23{31. [165] Marmarelis, Vasilis Z et al. (2013b). \Nonlinear modeling of dynamic in- teractions within neuronal ensembles using Principal Dynamic Modes". In: Journal of computational neuroscience 34.1, pp. 73{87. [166] | (2014). \On parsing the neural code in the prefrontal cortex of pri- mates using principal dynamic modes". In: Journal of computational neuro- science 36.3, pp. 321{337. [167] Mazzucato, Luca, Alfredo Fontanini, and Giancarlo La Camera (2015). \Dy- namics of Multistable States during Ongoing and Evoked Cortical Activity". In: The Journal of Neuroscience 35.21, pp. 8214{8231. [168] McCartney, Holly et al. (2004). \Theta reset produces optimal conditions for long-term potentiation". In: Hippocampus 14.6, pp. 684{687. [169] Medvedev, AV (2001). \Temporal binding at gamma frequencies in the brain: paving the way to epilepsy?" In: Australasian Physics & Engineering Sciences in Medicine 24.1, pp. 37{48. [170] Mina, Faten et al. (2013). \Modulation of epileptic activity by deep brain stimulation: a model-based study of frequency-dependent eects". In: Fron- tiers in computational neuroscience 7. [171] Mirski, Marek A et al. (1997). \Anticonvulsant eect of anterior thalamic high frequency electrical stimulation in the rat". In: Epilepsy research 28.2, pp. 89{100. 233 [172] Mitsis, GD and VZ Marmarelis (2002). \Modeling of nonlinear physiolog- ical systems with fast and slow dynamics. I. Methodology". In: Annals of biomedical engineering 30.2, pp. 272{281. [173] Mitsis, GD et al. (2002). \Modeling of nonlinear physiological systems with fast and slow dynamics. II. Application to cerebral autoregulation". In: An- nals of biomedical engineering 30.4, pp. 555{565. [174] Mizuseki, Kenji et al. (2009). \Theta oscillations provide temporal windows for local circuit computation in the entorhinal-hippocampal loop". In: Neuron 64.2, pp. 267{280. [175] Mormann, Florian et al. (2007). \Seizure prediction: the long and winding road". In: Brain 130.2, pp. 314{333. [176] Nagao, T, A Alonso, and M Avoli (1996). \Epileptiform activity induced by pilocarpine in the rat hippocampal-entorhinal slice preparation". In: Neuro- science 72.2, pp. 399{408. [177] Nagaraj, Vivek et al. (2015). \Future of seizure prediction and intervention: closing the loop". In: Journal of Clinical Neurophysiology 32.3, pp. 194{206. [178] Naylor, David (2002). \Changes in Nonlinear Signal Processing in Rat Hip- pocampus Associated with Loss of Paired-Pulse Inhibition or Epileptogene- sis". In: Epilepsia 43.s5, pp. 188{193. [179] Nelson, Timothy S et al. (2011). \Exploring the tolerability of spatiotempo- rally complex electrical stimulation paradigms". In: Epilepsy research 96.3, pp. 267{275. [180] NeuroPace, Inc. RNS R System User Manual. Accessed: 2015-06-30. [181] Nyhus, Erika and Tim Curran (2010). \Functional role of gamma and theta oscillations in episodic memory". In: Neuroscience & Biobehavioral Reviews 34.7, pp. 1023{1035. 234 [182] O'Keefe, John and Jonathan Dostrovsky (1971). \The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat". In: Brain research 34.1, pp. 171{175. [183] Ogura, H (1985). \Estimation of Wiener kernels of a nonlinear system and a fast algorithm using digital Laguerre lters". In: 15th NIBB Conference, Okazaki, Japan, pp. 14{62. [184] Olvera-Cort es, E, M Cervantes, and I Gonzalez-Burgos (2002). \Place-learning, but not cue-learning training, modies the hippocampal theta rhythm in rats". In: Brain research bulletin 58.3, pp. 261{270. [185] Pacia, Steven V and John S Ebersole (1997). \Intracranial EEG substrates of scalp ictal patterns from temporal lobe foci". In: Epilepsia 38.6, pp. 642{654. [186] Paninski, Liam (2003). \Convergence properties of three spike-triggered anal- ysis techniques". In: Network: Computation in Neural Systems 14.3, pp. 437{ 464. [187] Pare, D, R Llinas, et al. (1992). \Role of the hippocampal-entorhinal loop in temporal lobe epilepsy: extra-and intracellular study in the isolated guinea pig brain in vitro". In: The Journal of neuroscience 12.5, pp. 1867{1881. [188] Park, Il M et al. (2013). \Spectral methods for neural characterization using generalized quadratic models". In: Advances in Neural Information Process- ing Systems, pp. 2454{2462. [189] Peneld, Wilder and Herbert Jasper (1954). \Epilepsy and the functional anatomy of the human brain." In: [190] Petersen, Rasmus S et al. (2008). \Diverse and temporally precise kinetic feature selectivity in the VPm thalamic nucleus". In: Neuron 60.5, pp. 890{ 903. 235 [191] Pike, Fenella G et al. (2000). \Distinct frequency preferences of dierent types of rat hippocampal neurones in response to oscillatory input currents". In: The Journal of Physiology 529.1, pp. 205{213. [192] Pillow, Jonathan W and Il M Park (2011). \Bayesian spike-triggered co- variance analysis". In: Advances in neural information processing systems, pp. 1692{1700. [193] Pillow, Jonathan W and Eero P Simoncelli (2006). \Dimensionality reduction in neural models: an information-theoretic generalization of spike-triggered average and covariance analysis". In: Journal of vision 6.4, p. 9. [194] Pillow, Jonathan W et al. (2008). \Spatio-temporal correlations and visual signalling in a complete neuronal population". In: Nature 454.7207, pp. 995{ 999. [195] Racine, Ronald J (1972). \Modication of seizure activity by electrical stim- ulation: I. After-discharge threshold". In: Electroencephalography and clinical neurophysiology 32.3, pp. 269{279. [196] Rajan, Kanaka and William Bialek (2013). \Maximally informative stimulus energies in the analysis of neural responses to natural signals". In: PloS one 8.11, e71959. [197] Rapela, Joaqu n, Jerry M Mendel, and Norberto M Grzywacz (2006). \Esti- mating nonlinear receptive elds from natural images". In: Journal of Vision 6.4, p. 11. [198] Rieke, Fred et al. (1999). Spikes: Exploring the Neural Code. MIT press. [199] Robinson, Brian S, Dong Song, and Theodore W Berger (2015). \Estimation of a large-scale generalized Volterra for neural ensembles with group lasso and local coordinate descent". In: Engineering in Medicine and Biology Society (EMBC), 2015 36th Annual International Conference of the IEEE. IEEE. 236 [200] Rossum, Mark CW van (2001). \A novel spike distance". In: Neural Compu- tation 13.4, pp. 751{763. [201] Rotstein, Horacio G et al. (2005). \Slow and Fast Inhibition and an H-Current Interact to Create a Theta Rhythm in a Model of CA1 Interneuron Network". In: J Neurophysiol 94, pp. 1509{1518. [202] Rust, Nicole C et al. (2005). \Spatiotemporal elements of macaque v1 recep- tive elds". In: Neuron 46.6, pp. 945{956. [203] Ryapolova-Webb, Elena et al. (2014). \Chronic cortical and electromyographic recordings from a fully implantable device: preclinical experience in a non- human primate". In: Journal of neural engineering 11.1, p. 016009. [204] Saillet, Sandrine et al. (2009). \Manipulating the epileptic brain using stimu- lation: a review of experimental and clinical studies." In: Epileptic Disorders. [205] Saleem, Aman B, Holger G Krapp, and Simon R Schultz (2008). \Receptive eld characterization by spike-triggered independent component analysis". In: Journal of vision 8.13, p. 2. [206] Samengo, In es and Tim Gollisch (2013). \Spike-triggered covariance: geomet- ric proof, symmetry properties, and extension beyond Gaussian stimuli". In: Journal of computational neuroscience 34.1, pp. 137{161. [207] Sandler, R et al. (2013). \Closed-loop modeling of the hippocampus and design of neurostimulation patterns for suppressing seizures". In: Neural En- gineering (NER), 2013 6th International IEEE/EMBS Conference on. IEEE, pp. 1143{1146. [208] Sandler, Roman A and Vasilis Z. Marmarelis (in press). \Understanding Spike Triggered Covariance using Wiener Theory for Receptive Field Identica- tion". In: Journal of Vision. 237 [209] Sandler, Roman A et al. (2014). \Model-based asessment of an in-vivo predic- tive relationship from CA1 to CA3 in the rodent hippocampus". In: Journal of computational neuroscience, pp. 1{15. [210] Sandler, Roman A et al. (2015). \System identication of point-process neu- ral systems using Probability Based Volterra kernels". In: Journal of neuro- science methods 240, pp. 179{192. [211] Sandler, Roman A et al. (Under Review). \Closed-Loop Hippocampal Mod- eling and the Design of Neurostimulation Patterns for Suppressing Seizures". In: Journal of Neural Engineering. [212] Sargolini, Francesca et al. (2006). \Conjunctive representation of position, direction, and velocity in entorhinal cortex". In: Science 312.5774, pp. 758{ 762. [213] Sasaki, Takuya, Norio Matsuki, and Yuji Ikegaya (2007). \Metastability of active CA3 networks". In: The Journal of Neuroscience 27.3, pp. 517{528. [214] Scharfman, Helen E (2007). \The CA3 backprojection to the dentate gyrus". In: Progress in brain research 163, pp. 627{637. [215] Schreiber, Susanne et al. (2004). \Subthreshold resonance explains the frequency- dependent integration of periodic as well as random stimuli in the entorhinal cortex". In: J Neurophysiol 92.1. [216] Schwartz, Odelia and Eero P Simoncelli (2001). \Natural signal statistics and sensory gain control". In: Nature neuroscience 4.8, pp. 819{825. [217] Schwartz, Odelia et al. (2006). \Spike-triggered neural characterization". In: Journal of Vision 6.4, p. 13. [218] Scoville, William Beecher and Brenda Milner (1957). \Loss of recent memory after bilateral hippocampal lesions". In: Journal of neurology, neurosurgery, and psychiatry 20.1, p. 11. 238 [219] Sharpee, Tatyana, Nicole C Rust, and William Bialek (2004). \Analyzing neural responses to natural signals: maximally informative dimensions". In: Neural computation 16.2, pp. 223{250. [220] Sik, Attila et al. (1994). \Inhibitory CA1-CA3-hilar region feedback in the hippocampus". In: Science 265.5179, pp. 1722{1724. [221] Slee, Sean J et al. (2005). \Two-dimensional time coding in the auditory brainstem". In: The Journal of Neuroscience 25.43, pp. 9978{9988. [222] S oderstr om, Torsten (1981). \Identication of stochastic linear systems in presence of input noise". In: Automatica 17.5, pp. 713{725. [223] Song, Dong, Vasilis Z Marmarelis, and Theodore W Berger (2009). \Para- metric and non-parametric modeling of short-term synaptic plasticity. Part I: Computational study". In: Journal of computational neuroscience 26.1, pp. 1{19. [224] Song, Dong et al. (2007). \Nonlinear dynamic modeling of spike train trans- formations for hippocampal-cortical prostheses". In: Biomedical Engineering, IEEE Transactions on 54.6, pp. 1053{1066. [225] | (2009a). \Nonlinear modeling of neural population dynamics for hip- pocampal prostheses". In: Neural Networks 22.9, pp. 1340{1351. [226] Song, Dong et al. (2009b). \Parametric and non-parametric modeling of short-term synaptic plasticity. Part II: Experimental study". In: Journal of computational neuroscience 26.1, pp. 21{37. [227] Song, Dong et al. (2013). \Identication of sparse neural functional connec- tivity using penalized likelihood estimation and basis functions". In: Journal of computational neuroscience 35.3, pp. 335{357. [228] Srensen, Knud Erik and Michael T Shipley (1979). \Projections from the subiculum to the deep layers of the lpsilateral presubicular and entorhinal cor- 239 tices in the guinea pig". In: Journal of Comparative Neurology 188.2, pp. 313{ 333. [229] Spruston, N and C McBain (2007). \Structural and functional properties of hippocampal neurons". In: The hippocampus book, pp. 133{201. [230] Staubli, U and Fang Bo Xu (1995). \Eects of 5-HT3 receptor antagonism on hippocampal theta rhythm, memory, and LTP induction in the freely moving rat". In: The Journal of neuroscience 15.3, pp. 2445{2452. [231] Steveninck, R.R de Ruyter van and W Bialek (1988). \Real-time performance of a movement-sensitive neuron in the blow y visual system: coding and information transfer in short spike sequences". In: Proceedings of the Royal society of London. Series B. Biological sciences 234.1277, pp. 379{414. [232] Stringer, Janet L and Eric W Lothman (1992). \Reverberatory seizure dis- charges in hippocampal-parahippocampal circuits". In: Experimental neurol- ogy 116.2, pp. 198{203. [233] Sun, Felice T and Martha J Morrell (2014). \Closed-loop Neurostimulation: The Clinical Experience". In: Neurotherapeutics, pp. 1{11. [234] Sun, Felice T, Martha J Morrell, and Robert E Wharen Jr (2008). \Respon- sive cortical stimulation for the treatment of epilepsy". In: Neurotherapeutics 5.1, pp. 68{74. [235] Swanson, LW and WM Cowan (1977). \An autoradiographic study of the organization of the eeret connections of the hippocampal formation in the rat". In: Journal of Comparative Neurology 172.1, pp. 49{84. [236] Swanson, LW, JM Wyss, and WM Cowan (1978). \An autoradiographic study of the organization of intrahippocampal association pathways in the rat". In: Journal of Comparative Neurology 181.4, pp. 681{715. 240 [237] Tamamaki, Nobuaki and Yoshiaki Nojyo (1995). \Preservation of topography in the connections between the subiculum, eld CA1, and the entorhinal cortex in rats". In: Journal of Comparative Neurology 353.3, pp. 379{390. [238] Taylor, Peter Neal et al. (2015). \Optimal control based seizure abatement using patient derived connectivity". In: Frontiers in Neuroscience 9, p. 202. [239] Tesche, CD and J Karhu (2000). \Theta oscillations index human hippocam- pal activation during a working memory task". In: Proceedings of the National Academy of Sciences 97.2, pp. 919{924. [240] Theunissen, Fr ed eric E et al. (2001). \Estimating spatio-temporal receptive elds of auditory and visual neurons from their responses to natural stimuli". In: Network: Computation in Neural Systems 12.3, pp. 289{316. [241] Tibshirani, Robert (1996). \Regression shrinkage and selection via the lasso". In: Journal of the Royal Statistical Society. Series B (Methodological), pp. 267{ 288. [242] Touryan, Jon, Gidon Felsen, and Yang Dan (2005). \Spatial structure of complex cell receptive elds measured with natural images". In: Neuron 45.5, pp. 781{791. [243] Truccolo, Wilson et al. (2005). \A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate eects". In: Journal of neurophysiology 93.2, pp. 1074{1089. [244] Truccolo, Wilson et al. (2014). \Neuronal ensemble synchrony during human focal seizures". In: The Journal of Neuroscience 34.30, pp. 9927{9944. [245] Valenza, Gaetano et al. (2013). \Point-process nonlinear models with La- guerre and Volterra expansions: instantaneous assessment of heartbeat dy- namics". In: 241 [246] Van Nieuwenhuyse, B et al. (2014). \In Search of Optimal DBS Paradigms to Treat Epilepsy: Bilateral Versus Unilateral Hippocampal Stimulation in a Rat Model for Temporal Lobe Epilepsy". In: Brain stimulation. [247] Victor, Jonathan D and Keith P Purpura (1997). \Metric-space analysis of spike trains: theory, algorithms and application". In: Network: computation in neural systems 8.2, pp. 127{164. [248] Wang, Xiao-Jing (2001). \Synaptic reverberation underlying mnemonic per- sistent activity". In: Trends in neurosciences 24.8, pp. 455{463. [249] Warren, Christopher P et al. (2010). \Synchrony in normal and focal epileptic brain: the seizure onset zone is functionally disconnected". In: Journal of neurophysiology 104.6, pp. 3530{3539. [250] Wei, Xuefeng F and Warren M Grill (2005). \Current density distributions, eld distributions and impedance analysis of segmented deep brain stimula- tion electrodes". In: Journal of neural engineering 2.4, p. 139. [251] Weiland, James D, Alice K Cho, and Mark S Humayun (2011). \Retinal pros- theses: current clinical results and future needs". In: Ophthalmology 118.11, pp. 2227{2237. [252] Wendling, F et al. (2002). \Epileptic fast activity can be explained by a model of impaired GABAergic dendritic inhibition". In: European Journal of Neuroscience 15.9, pp. 1499{1508. [253] Wendling, Fabrice (2008). \Computational models of epileptic activity: a bridge between observation and pathophysiological interpretation". In: [254] Wendling, Fabrice et al. (2003). \Epileptic fast intracerebral EEG activity: evidence for spatial decorrelation at seizure onset". In: Brain 126.6, pp. 1449{ 1459. 242 [255] West, MJ, LHJG Slomianka, and H J G Gundersen (1991). \Unbiased stere- ological estimation of the total number of neurons in the subdivisions of the rat hippocampus using the optical fractionator". In: The Anatomical Record 231.4, pp. 482{497. [256] Westbrook, GL (2000). \Seizures and epilepsy". In: Principles of neural sci- ence 4, pp. 910{935. [257] Westwick, David T and Robert E Kearney (2003). Identication of nonlinear physiological systems. Vol. 7. John Wiley & Sons. [258] Whittington, Miles A and Roger D Traub (2003). \< i> Interneuron Diversity series</i>: Inhibitory interneurons and network oscillations< i> in vitro</i>". In: Trends in neurosciences 26.12, pp. 676{682. [259] Wiebe, Sherman P and Ursula V St aubli (2001). \Recognition memory cor- relates of hippocampal theta cells". In: The Journal of Neuroscience 21.11, pp. 3955{3967. [260] Wiener, Norbert (1966). \Nonlinear problems in random theory". In: Non- linear Problems in Random Theory, by Norbert Wiener, pp. 142. ISBN 0-262- 73012-X. Cambridge, Massachusetts, USA: The MIT Press, August 1966.(Pa- per) 1. [261] Winson, Jonathan (1978). \Loss of hippocampal theta rhythm results in spa- tial memory decit in the rat". In: Science 201.4351, pp. 160{163. [262] Wolpaw, Jonathan R and Dennis J McFarland (2004). \Control of a two- dimensional movement signal by a noninvasive brain-computer interface in humans". In: Proceedings of the National Academy of Sciences of the United States of America 101.51, pp. 17849{17854. 243 [263] Wolters, Gezinus and Antonino Raone (2008). \Coherence and recurrency: Maintenance, control and integration in working memory". In: Cognitive pro- cessing 9.1, pp. 1{17. [264] Wu, Kun, Kevin J Canning, and L Stan Leung (1998). \Functional inter- connections between CA3 and the dentate gyrus revealed by current source density analysis". In: Hippocampus 8.3, pp. 217{230. [265] Wyckhuys, Tine et al. (2010). \Suppression of hippocampal epileptic seizures in the kainate rat by Poisson distributed stimulation". In: Epilepsia 51.11, pp. 2297{2304. [266] Yae, Robert B et al. (2015). \Physiology of functional and eective networks in epilepsy". In: Clinical Neurophysiology 126.2, pp. 227{236. [267] Yeckel, Mark F and Theodore W Berger (1990). \Feedforward excitation of the hippocampus by aerents from the entorhinal cortex: redenition of the role of the trisynaptic pathway." In: Proceedings of the National Academy of Sciences 87.15, pp. 5832{5836. [268] Zalay, Osbert C and Berj L Bardakjian (2013). \Synthesis of high-complexity rhythmic signals for closed-loop electrical neuromodulation". In: Neural Net- works 42, pp. 62{73. [269] Zanos, Theodoros P et al. (2008). \Nonlinear modeling of causal interre- lationships in neuronal ensembles". In: Neural Systems and Rehabilitation Engineering, IEEE Transactions on 16.4, pp. 336{352. [270] Zhang, Cun-Hui (2010). \Nearly unbiased variable selection under minimax concave penalty". In: The Annals of Statistics, pp. 894{942. [271] Zhong, Yuru et al. (2004). \Nonlinear analysis of the separate contributions of autonomic nervous systems to heart rate variability using principal dynamic 244 modes". In: Biomedical Engineering, IEEE Transactions on 51.2, pp. 255{ 262. [272] Zhou, Douglas et al. (2014). \Granger causality network reconstruction of conductance-based integrate-and-re neuronal systems". In: PloS one 9.2, e87636. [273] Zosimovskii, VA and VA Korshunov (2012). \Excitation Waves Returning to the Hippocampus via the Entorhinal Cortex Can Reactivate Populations of Trained Field CA1 Neurons during Deep Sleep". In: Neuroscience and Behavioral Physiology 42.2, pp. 133{143. [274] Zosimovskii, VA, VA Korshunov, and VA Markevich (2008). \Conditions re- quired for the appearance of double responses in hippocampal eld CA1 to application of single stimuli to Sh aer collaterals in freely moving rats". In: Neuroscience and behavioral physiology 38.3, pp. 313{321. 245
Abstract (if available)
Abstract
Responsive Deep Brain Stimulation (DBS) is a promising alternative to traditional treatments for epilepsy such as respective surgery and antiepileptic medication. However, an open issue with DBS is identifying the optimal temporal patterns of stimulation to abate seizures. Identification of these patterns is made difficult by the lack of a controllable experimental framework by which to test them. Namely, the experimenter is unable to obtain consistent seizures 'on demand’, he can only apply a single pattern of stimulation to each seizure, and he is unable to know how the seizure would have evolved had no stimulation been applied. Furthermore, these issues become significantly magnified when dealing with humans. To overcome these issues, we have developed a data-derived in-silico hippocampal network to function as a testbed for designing optimal DBS spatiotemporal patterns. Each network (Fig. 1a) is obtained by estimating the causal and dynamic (effective) connectivity between neurons recorded from the CA3 and CA1 regions of rodent hippocampus. Causal connectivity was estimated using Granger-causality like stepwise input selection and interneuronal dynamics were modeled by a nonlinear Volterra model estimated using the generalized linear modeling (GLM) framework [1].
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Experimental and computational models for seizure prediction
PDF
Functional consequences of network architecture in rat hippocampus: a computational study
PDF
Nonlinear modeling of causal interrelationships in neuronal ensembles: an application to the rat hippocampus
PDF
Decoding memory from spatio-temporal patterns of neuronal spikes
PDF
Simulating electrical stimulation and recording in a multi-scale model of the hippocampus
PDF
A million-plus neuron model of the hippocampal dentate gyrus: role of topography, inhibitory interneurons, and excitatory associational circuitry in determining spatio-temporal dynamics of granul...
PDF
Brain connectivity in epilepsy
PDF
Nonlinear dynamical modeling of single neurons and its application to analysis of long-term potentiation (LTP)
PDF
Geometric and dynamical modeling of multiscale neural population activity
PDF
Applications of graph theory to brain connectivity analysis
PDF
Therapeutic electrical stimulation strategies for neuroregeneration and neuroprotection of retinal neurons
PDF
Functional connectivity analysis and network identification in the human brain
PDF
Data-driven image analysis, modeling, synthesis and anomaly localization techniques
PDF
AI-driven experimental design for learning of process parameter models for robotic processing applications
PDF
Graph-based models and transforms for signal/data processing with applications to video coding
PDF
Insights from a computational approach to natural killer cell activation
PDF
From matching to querying: A unified framework for ontology integration
PDF
Reconstruction and estimation of shear flows using physics-based and data-driven models
PDF
Metasurfaces in 3D applications: multiscale stereolithography and inverse design of diffractive optical elements for structured light
PDF
Compression of signal on graphs with the application to image and video coding
Asset Metadata
Creator
Sandler, Roman A.
(author)
Core Title
Data-driven modeling of the hippocampus & the design of neurostimulation patterns to abate seizures
School
Viterbi School of Engineering
Degree
Doctor of Philosophy
Degree Program
Biomedical Engineering
Publication Date
08/28/2015
Defense Date
06/30/2015
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
CA1,CA3,DBS,deep brain stimulation,entorhinal cortex,epilepsy,hippocampus,kernels,neuroprosthetics,Neuroscience,nonlinear,OAI-PMH Harvest,spike triggered covariance,STC,system identification,Volterra,Wiener
Format
application/pdf
(imt)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Marmarelis, Vasilis Z. (
committee chair
), Khoo, Michael C. K. (
committee member
), Shanechi, Maryam M. (
committee member
), Song, Dong (
committee member
)
Creator Email
rsandler00@gmail.com
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c40-172991
Unique identifier
UC11275945
Identifier
etd-SandlerRom-3859.pdf (filename),usctheses-c40-172991 (legacy record id)
Legacy Identifier
etd-SandlerRom-3859.pdf
Dmrecord
172991
Document Type
Dissertation
Format
application/pdf (imt)
Rights
Sandler, Roman A.
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
CA1
CA3
DBS
deep brain stimulation
entorhinal cortex
epilepsy
hippocampus
kernels
neuroprosthetics
nonlinear
spike triggered covariance
STC
system identification
Volterra
Wiener