Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Power-efficient biomimetic neural circuits
(USC Thesis Other)
Power-efficient biomimetic neural circuits
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
Power-Ecient Biomimetic Neural Circuits Pezhman Mamdouh Department of Electrical Engineering University of Southern California A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulllment of the Requirements for the Degree Doctor of Philosophy (ELECTRICAL ENGINEERING) December 2019 Abstract Pezhman Mamdouh, (Ph.D., Electrical Engineering) Power-Ecient Biomimetic Neural Circuits Thesis directed by Prof. Alice C. Parker ii Acknowledgements I would like to express my highest gratitude to my advisor Dr. Alice C. Parker for her help and guidance and my colleagues in BioRC group. iii Contents Chapter 1 Introduction 1 1.1 Problem Statement and Motivation . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Baseline Neuron model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Hypothesis Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 Organization of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Biological Background and Related Work 8 2.1 Biological Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.1.1 Dendritic Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.1.2 Dendritic Arbor Circuit Designs . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.2 Astrocytes Role in Regulation of Brain Energy . . . . . . . . . . . . . . . . . . . . . 13 3 Towards Low-power Dendrites 16 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.2 A Low-power Sub-threshold Neuron through Gating Dendritic Arbor . . . . . . . . . 18 3.2.1 The Wake-up Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.2.2 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.2.3 Proposed Neuron Functional Results . . . . . . . . . . . . . . . . . . . . . . . 22 3.2.4 Power Consumption Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.3 Low-power Non-linear Dendritic Computation . . . . . . . . . . . . . . . . . . . . . . 26 iv 3.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.3.2 Adder Building Blocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.3.3 EPSP Blocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.3.4 Control Unit Blocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.3.5 Simulation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.3.6 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.3.7 Non-linearity analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.3.8 Power Consumption Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 4 Charge-based Summation 36 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 4.2 A Low-Power Switched-Capacitor Dendritic Arbor . . . . . . . . . . . . . . . . . . . 37 4.2.1 Dendritic Arbor Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.2.2 Switch Phase Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4.3.1 Simulation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4.3.2 Signal Integration and Propagation . . . . . . . . . . . . . . . . . . . . . . . . 42 4.3.3 Parametric Variation on Switched-Capacitor Dendrite . . . . . . . . . . . . . 44 4.3.4 Non-linear Switched-Capacitor Dendritic Arbor . . . . . . . . . . . . . . . . . 50 4.3.5 Power Consumption Evaluation and Comparison . . . . . . . . . . . . . . . . 52 5 Modeling Astrocyte Intervention in Energy Regulation of the Brain 54 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 5.2 Modeling Astro-Neuron Lactate Shuttle . . . . . . . . . . . . . . . . . . . . . . . . . 54 5.2.1 Modied Axon Hillock Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . 56 5.2.2 Experimental Results for Modifed BioRC Axon Hillock Circuit . . . . . . . . 58 5.2.3 Inter-domain Astrocyte Process for Modeling the Astro-Neural Lactate Shuttle 61 v 6 Pattern Detection Using Dendritic Computation 66 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 6.2 Neuron Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 6.3 Dendrite Branch Circuits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 6.4 Dendritic Trunk Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 6.5 Axonhillock Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 6.6 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 Bibliography 72 vi Tables Table 3.1 Simulation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.2 Power Consumption Analysis of the Neuron . . . . . . . . . . . . . . . . . . . . . . . 24 3.3 Simulation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.4 EPSP Characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.1 Simulation Setup for the Switched-Capacitor Dendritic Arbor . . . . . . . . . . . . . 41 4.2 Power Consumption Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 5.1 Modied Axon Hillock Circuit Conguration . . . . . . . . . . . . . . . . . . . . . . 58 5.2 Power Consumption Components for Axon Hillock Circuit . . . . . . . . . . . . . . . 60 5.3 Inter-Spike Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 5.4 Power Consumption Analysis of Astro-Neural Shuttle Module . . . . . . . . . . . . . 64 6.1 Inter-spike Delays of the Neuron Detecting the Patterns . . . . . . . . . . . . . . . . 70 vii Figures Figure 1.1 BioRC Neuron Structure [11] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 BioRC Dendrite Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1 Non-linear Dendritic Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2 Methods for Processing PSPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.3 Astrocyte-Neuron Lactate Shuttle . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.1 The Neuron Structure for the Low-power Experiments . . . . . . . . . . . . . . . . . 18 3.2 The Dendritic Arbor for Our Experiments . . . . . . . . . . . . . . . . . . . . . . . . 19 3.3 Threshold-Check Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.4 Wake-up Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.5 Wake-up Circuit Output Pulse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.6 Proposed Neuron Functional Result . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.7 Activity-dependent Power Consumption Evaluation . . . . . . . . . . . . . . . . . . . 25 3.8 Adder Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.9 EPSP Block . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.10 Adder Control Unit(CU) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.11 Adder's Linear Functional Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.12 Adder's Non-linear Functional Result . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.13 Functional Result of the Adder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 viii 3.14 Power Consumption Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.1 Switched-Capacitor Dendritic Arbor Structure . . . . . . . . . . . . . . . . . . . . . 37 4.2 Congurations of the Proposed Dendritic Arbor . . . . . . . . . . . . . . . . . . . . . 38 4.3 Switch Phase Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.4 Conguration of Two-stage Dendrite . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 4.5 Dendritic Arbor Functional Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 4.6 Signal Propagation in a 5-stage Arbor . . . . . . . . . . . . . . . . . . . . . . . . . . 43 4.7 Post-Synaptic Potentials Produced by Synapse Circuits as Input . . . . . . . . . . . 44 4.8 Axial Resistance Variations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 4.9 Leakage Channel Conductance Variation, Leakage Transistors Dimention Variations 47 4.10 Leakage Channel Conductance Variation, Leakage Voltage (V LK ) Variations . . . . 48 4.11 Sampling Frequency Variations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.12 Non-linear Switched-Capacitor Dendritic Arbor . . . . . . . . . . . . . . . . . . . . . 50 4.13 Neuron's Response Enhanced With And Without Dendritic Spikes . . . . . . . . . . 51 4.14 Power Consumption Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 5.1 Astro-Neuron Lactate Shuttle Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 5.2 Axon Hillock Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 5.3 Experimental Results on the Modied Axon Hillock Circuit . . . . . . . . . . . . . . 59 5.4 Modulation of Frequency of Spiking vs. Astrocyte Input . . . . . . . . . . . . . . . . 59 5.5 Instantaneous Power Consumption for the Axon Hillock Circuit . . . . . . . . . . . . 60 5.6 BioRC Modied Neuron Enhanced with an Astro-Neuron Lactate Shuttle . . . . . . 61 5.7 Circuit Simulation Results for the Astro-Neural Lactate Shuttle Model . . . . . . . . 63 5.8 Instantaneous Power Consumption Components for Astro-Neural Shuttle Circuit Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 5.9 Power Consumption Components for Astro-Neural Shuttle Circuit Model . . . . . . 65 ix 6.1 Block Diagram of the Neuron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 6.2 Dendrite Branch Circuits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 6.3 Dendritic trunk Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 6.4 Axon Hillock Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 6.5 Experimental Results of a Neuron Detecting Two Patterns . . . . . . . . . . . . . . . 71 x Chapter 1 Introduction 1.1 Problem Statement and Motivation Neuromorphic designs modeling how the human's brain processes massive amount of information in parallel through the integration of memory and processing elements have gained signicant attention due to their superior performance in a specic domain of applications compared to Von-Nueman based digital processors. Such designs are being used in applications such as prosthetic devices [1{5], cognitive sensory applications [6{11], and biomimetic approaches used for bio-inspired computing [11{14]. Analog neuromorphic circuit design approach utilizes physical characteristics of silicon devices mimicking neurons exhibiting behaviors close to that of biological neurons. Such neurons are computationally more powerful than those ignoring biological complexities while consuming fewer hardware resources [15]. One of the highest hurdles in constructing neuromorphic circuits is to reduce power consumption while not oversimplifying neural models. Power consumption plays a crucial role in the overall eciency of a neuromorphic design due to the following reasons: Scalability: One of the barriers of scaling up a neural network is due to the limited power budget of the CMOS chips while scaling up the size and the number of neurons of the design imposes a higher leakage density and power consumption. Although neuromorphic engineers can use sub-threshold implementation for the circuits, the number of neurons and their input synapses are limited compared to the brain, even in 1 large scale analog neuromorphic designs [16, 17]. This situation gets worse when deep sub-micron technologies are targeted for fabrication as the leakage component in power consumption becomes dominant in deep sub-micron technologies [18]. Unreachable power critical devices: There are applications where the critical device is not reachable in order to exchange the power supply when it loses power such as a robot lunched on the Mars in order to perform image processing or a wireless sensor network spread in a jungle to detect re where the communication is point to point and a node failure might isolate the network and result in network failure. Undesirable/high-cost exchangeability of the power supply: Prosthetic devices are an example of such domain of applications where the device is expected to last for a very long time after surgery in the patient's body; otherwise, the patient must undergo the same surgery again which is highly undesirable and of a high cost. Reliability: High power consumption results in higher junction temperature which in uences device wear out and breaking down of gate oxide as well as metal failure caused by electro-migration phenomena [19]. The main goal of the BioRC project [20] in the last decade has been designing circuits that mimic the cognition, learning, and memory capabilities of the brain; power eciency had always been a concern and none of the circuits were designed for low-power consumption. The main focus of this thesis is designing circuits in order to capture biological neural complexities while maintaining ultra-low power consumption. 1.1.1 Baseline Neuron model The BioRC neuron model shown in Fig. 1.1 has been used as our baseline model. In the distal branches of the dendritic arbor, 80% of synapses are excitatory and 20% are inhibitory [21]. Hence, the ratio of 4:1 for excitatory and inhibitory synapses has been used although that can certainly be 2 subject to alteration as required per application. Upon pre-synaptic neural ring, synapses produce Post-Synaptic Potentials (PSPs). The dendritic arbor is where the integration of PSPs takes place. Integration of PSPs on the same branches is non-linear depending on their magnitudes. Small PSPs in terms of magnitude sum linearly, medium PSPs sum super-linearly, and large PSPs sum sub-linearly. The processed PSPs on dierent branches of the arbor gets attenuated as they travel towards the soma and get summed linearly. At the soma, if the potential (i.e.,V Soma ) reaches a threshold, the Axon Hillock circuit produces pattern(s) of Action Potentials (APs). Figure 1.1: BioRC Neuron Structure [11] The main component of power consumption in the BioRC neurons is the dendritic arbor [22{24] due to non-linear integration of PSPs, making BioRC computations closer to that of biological processes as compared to many current neuromorphic designs which ignore this nonlinear behavior for simplicity while some neural functions such as dendritic spikes heavily depend on it. Our goal is to preserve the complexities of the BioRC neuron model while optimizing it for low-power consumption. Aiming for low-power consumption through circuit-level optimization while not oversimplifying capabilities of this module (i.e., dendritic arbor) makes the design challenging and complicated. Mimicking such complex computation capability while maintaining low-power consumption has been the goal of this research and to this end, we have re-designed the dendritic arbor circuit that is believed to be highly evolved in cortical cognition for low-power consumption. 3 1.2 Hypothesis Statement Integration of PSPs in the state of art dendritic arbor circuits is carried out through summation of voltages [23{25] or currents [26,27] by providing current through active devices. All such approaches suer from leakage paths in the circuit, detrimental to the power eciency of the circuit. Inspired by the biological process of charge integration as a result of neurotransmitters release upon pre-synaptic activity, we hypothesize that sampling PSPs on capacitors as dendritic nodes once a pre-synaptic activity is detected and serializing them to perform the summation will be a promising approach to saving power. We hypothesize that saving power using this approach is gained not only through deploying passive elements to play the main role in summation as opposed to current approaches using active devices, but also through providing the opportunity to gate the power of the arbor while there is no activity at input synapses of the dendrite. This way the power consumption of a neuron will be controlled as a function of input synaptic activity. One of the key points in neural processing is how the synaptic inputs of a neuron are encoded to pattern(s) of APs at the output; a process in which the role of dendritic computation cannot be neglected [28]. Locations of synapses and magnitudes of the PSPs on the dendrite aect the process of the current integration and result in nonlinear processing of PSPs [11,28]. On the BioRC project, the spatial and temporal non-linear integration of post-synaptic potentials forces the distributed computation that mimics the biological neuron, requiring more complex addition than required for less complex dendritic arbors. The BioRC project has traditionally performed such distributed computations using classic voltage adders [11,23,24] along with additional specialized circuitry [25] to generate dendritic spikes under special circumstances. A classic voltage adder circuit has contributed signicantly to power consumption in the BioRC neurons mainly because of considering the capability of adding both excitatory and inhibitory PSPs resulting non-zero V gs on its input transistors at resting potential (i.e., 0.0v), as shown in Fig. 1.2. We hypothesize that separating the EPSP domain from the IPSP domain using current mode addition eliminates the non-zero V gs problem while facilitating inter-branch 4 (a) Classic Voltage Adder used in BioRC Project [23] (b) BioRC Dendritic Adders Conguration [23] Figure 1.2: BioRC Dendrite Implementation dendritic current integration, as no adders will be required for summation of the inter-branch currents. We hypothesize that this way the power will be saved due to the elimination of non-zero V gs imposed at resting potential on input transistors in intra-branch processes and also due to the removal of inter-branch adders. Neuroscience research in the last decade has shed light on how astrocytes play important roles in neurotransmission, homeostatic processes, and energy management [29{31]. Astrocyte processes en-wrap synapses and can communicate bi-directionally with neurons. Astro-Neuronal shuttle refers to a process where the high activity is detected by the astrocytes at the neuron's synapses signaling another energy substrate other than glucose is required for the ongoing neural process. Consequently, astrocytes produce and release lactate to be used by the neurons as supplementary fuel upon high energy demand by neurons regulating them for spiking at a higher frequency rate as required [29]. We hypothesize that mimicking the astrocyte's metabolic role in order to detect activity in a neural network and adjusting the frequency of spiking accordingly not only aects the power consumption as a dynamic power management technique but also makes each neuron more powerful in terms of computational power in encoding synaptic information. 5 1.3 Organization of the Thesis The main focus of this thesis proposal is to re-design and optimize BioRC neural circuits for low-power consumption. The main component of power consumption in BioRC neurons has been the dendritic arbor, the module believed to be highly evolved in cognition, learning, and memory. Hence, the overall eciency of a neuromorphic design is highly impacted by this module. There has been a long history behind the modeling of dendrites in the circuits reviewed in Chapter 2. In Chapter 3, we introduce the steps we took to minimize the power consumption of the main component of the power in the BioRC neuron, i.e., the dendritic arbor. We introduce a low-power neuron we designed through gating the dendritic arbor utilizing asynchronous neural communications and sub-threshold implementation. Asynchronous event-driven communication of the neurons can be exploited to gate the power of the dendrite module when there is no neural activity; hence, reducing the leakage density of the dendritic arbor, which is the dominant factor of the neurons' power consumption. We introduce the power-ecient intra-branch dendritic adder we have designed to emulate dendritic computations within the branches of the dendritic arbor in Chapter 3. The addition within the branches of the dendrites is non-linear depending on the magnitude of post-synaptic potentials. This non-linear addition triggers the generation of dendritic spikes known to be a trigger for synaptic long-term potentiation { a mechanism highly involved in learning and memory [11,32]. The power eciency of the design is not only due to newly introduced structure, but also power gating of inactive current paths in the circuit as well as sub-threshold implementation. In Chapter 4, we have introduced our switched-capacitor dendritic circuit model. The key point of this model is deploying capacitors as dendritic nodes for sampling PSPs on them and serializing them to perform summation through charge transfer. The circuit is an event-driven bi-phase circuit sequenced between sampling and adding phases once a pre-synaptic activity is detected. By varying introduced parameters in the dendritic model, dierent spatiotemporal characteristics of the dendrites can be captured. Utilizing the event-driven sequencing and power 6 gating, power consumption becomes a function of pre-synaptic activity in this model. In Chapter 5, we have shown the role of astrocytes in regulating the frequency of spiking and have correlated that to power consumption. Astrocytes have been re-designed for monitoring synaptic activity and feeding it to the axon hillock so that its spiking frequency can be regulated. We have shown how increasing neural activity in adjacent domains can modulate the frequency of spiking and have evaluated how power consumption is regulated. Finally, in Chapter 6, we combine the ecient circuits we have designed to construct an application of dendritic computations and the roles it can play in overall neural encoding. The goal is to demonstrate the complex functions that can be exhibited by a single neuron in order to distinguish two pieces of knowledge. 7 Chapter 2 Biological Background and Related Work 2.1 Biological Background The main focus of this thesis is on designing low-power neural circuits complying with the BioRC neuron model introduced as the baseline model in Chapter 1, taking into account dendritic computation characteristics as well as the astrocyte role's in the regulation of energy. In this chapter, we will introduce the relevant biological background to dendritic computation and astrocyte's dynamics in terms of the brain's energy substrate regulation. The rest of the chapter is organized as follows: Dendritic computation and most relevant dendritic arbor circuits are introduced in Sections 2.1.1 and 2.1.2, respectively. Section 2.2 is dedicated to astrocyte's metabolic role in regulating the brain's energy substrate. 2.1.1 Dendritic Computation Biological neurons perform extensive nonlinear computations over time and space within each neuron that in uence neural ring, computations that are believed to be important to cognition, learning, and memory. Research in the last two decades has shed light on how complex each neuron in the network is and how they contribute to the entire neural computation [28,33,34]. Specically, dendritic computations play crucial roles in neural behaviors (including learning, cognition, and episodic memory [11,32,35]) that cannot be neglected. The dendritic arbor, where intra-neuron computation takes place, is a large spanning-tree process enabling a neuron to receive stimuli from thousands of other neurons in dierent areas of 8 a neural network. Information sent to the neuron results in Post-Synaptic Potentials (PSPs) received on the branches of the dendritic arbor through synaptic chemical communications. (PSPs) are spatially and temporally integrated and the resultant membrane potential is propagated, with attenuation, all the way to the soma. Once the membrane voltage at the axon hillock reaches a threshold, an Action Potential (AP) is generated. This threshold can vary depending on the slope of the membrane potential. Although dendrites are the infrastructure for electrical signaling within a neuron, their passive and active properties highly aect the whole process of neural computation. Neurons receive thousands of synaptic inputs at the dendrites and compress the result into an AP or pattern of APs at the axon hillock as the output. Dendrites can behave like cable and delay lines [36, 37]. Therefore, as a PSP travels from the dendrite to the soma, it gets attenuated, as modeled by Wilfrid Rall's equation [37]. As a result, the spatial location of a synapse aects the relative strength of an input required to re a neuron. Also, for a neuron to pay attention to a set of inputs, the temporal delays of PSPs as they travel towards the soma need to be considered [37]. This means that farther PSPs need be activated rst and closer ones later in order to maximize the combined eect. The question is why a neuron can have thousands of inputs while each could get ltered out due to attenuation; this question is partially answered by the existence of dendritic spikes that result from a small number of active inputs to cause neural spiking. Dendrites are capable of generating spikes by themselves if a cluster of inputs co-exists in time and space, meaning they are no longer passive elements of the computation. Locations of synapses and magnitudes of the PSPs on the dendrite aect the process of the current integration and result in nonlinear processing of PSPs [11, 25, 28, 33, 34, 38]. Therefore, how each neuron processes dendritic PSPs plays a crucial role in the entire neural activity. Location of a synapse and magnitude of the PSPs at the dendrite aect the process of the integration of current [28, 33, 34] and result in nonlinear processing of PSPs as demonstrated in Fig. 2.1a. PSPs on dierent branches sum linearly; however, summation on the same branch depends on the magnitude of the input PSPs. Small magnitudes sum linearly, medium magnitudes 9 sum super-linearly, and large magnitudes sum sub-linearly. Fig. 2.1c shows non-linear summation of two EPSP on the same branch when both the EPSPs are tied and varied together. Shunting inhibition also refers to the super-linear inhibition eect of an inhibitory PSP on excitatory PSPs existing on the same branch. Although inhibition occurs linearly between branches, an inhibitory input within a branch can veto excitatory inputs that are upstream, as shown in Fig. 2.1b [11,28]. (a) Excitatory Addition [11] (b) Shunting Inhibition [11] (c) Dendritic Summation of EPSPs on the Same Branch [33] Figure 2.1: Non-linear Dendritic Computation 10 A central goal of the BioRC project [20] is to mimic such distributed, nonlinear computations. This mimicry imposes constraints on the implementations of computations for performing summation of potentials, resulting in more complex circuitry than that found in many neuromorphic circuits, since the BioRC computations are not only nonlinear but also depend on the position and arrival time of the spikes resulting in post-synaptic potentials within each neuron. Several circuits that implement dendritic spikes have been reported on the BioRC project [11, 24, 25, 38], along with extensive use of distributed dendritic computations in many circuit examples [11], demonstrating its value in neuromorphic computations, while none have been low-power designed. 2.1.2 Dendritic Arbor Circuit Designs There has been a long history of modeling passive properties of dendrites in circuits complying with Wilfrid Rall's mathematical model [37]. An articial passive dendritic arbor using silicon resistors and capacitors was initially introduced in [36] taking into account temporal and spatial properties of the input synaptic signals. Later, the idea of using switched capacitors as the axial resistors in the dendritic arbor and conguring the resistance through the clock frequency was introduced [39]. Back and forth propagation of somatic spikes in the dendrites has been modeled [40] while using transconductance ampliers as the leakage channels. A 2D re-congurable dendritic arbor, where each node is enhanced with a diuser, a leak transistor and active channels capturing both passive and active properties of the dendrites has been introduced [26]. Later, Field Programmable Analog Array implementation of a dendritic arbor focusing on the passive properties of the dendrites while providing a exible infrastructure for the design of dendritic arbors was introduced [27]. Non-linear behavior exhibited by dendrites evoked by spatio-temporal synaptic input patterns has been captured and implemented in an aVLSI chip introduced [41]. Prior research projects [23{27] have used two main approaches for summation of the PSPs in the dendrite: 1. current-mode addition [26, 27], and 2. voltage-mode addition [23{25]. In current 11 mode addition, synaptic potentials are converted into current and fed into the dendritic arbor nodes to be summed. The main drawback of current mode approaches is the leakage current going through the circuit from the supply voltage to the leakage channels as shown in Fig. 2.2a. These designs [26,27,40] are referred to as transconductance-based models throughout this thesis. In voltage mode addition, PSPs in each segment are summed up and the resultant potential is propagated in the arbor. The most well-known structures used as the voltage adder are the current mirror based adder used in [23,24] and average and amplify circuit used in [11,25] as shown in Fig. 2.2c and Fig. 2.2b, respectively. Both designs suer from the leakage current going through the circuit due to the existence of paths between VDD and VSS even if the input remains at the resting potential (i.e., 0v). (a) Transconductance-Based Models [26,27] (b) Average and Amplify Method used in [11,25] (c) Current Mirror Adder [23,24] Figure 2.2: Methods for Processing PSPs 12 Following our research plan regarding dendritic arbor design, we began by examining PSP summation and implementing a variety of low-power circuits. We have designed and introduced a low-power dendritic adder for processing of PSPs in Chapter 3 that not only captures non- linear behavior of dendrites with respect to PSPs magnitudes but also maintains ultra-low power consumption. In Chapter 4, we introduce a sub-threshold voltage-mode switched-capacitor circuit deploying passive elements to carry out signal addition in the dendritic arbor through charge addition along with power gating, while taking into account passive properties of the dendrites for signal propagation. 2.2 Astrocytes Role in Regulation of Brain Energy The brain consumes high levels of energy during neurotransmission provided by glucose as the main source. It receives glucose from the blood-brain barrier (i.e., BBB) through glucose transporter 1 (i.e., GLUT1) metabolizing it in order to synthesize neurotransmitters that are required for it to function properly [29]. Several studies have suggested that excitatory glutamate- mediated signaling processes constitute as much as 80% of the energy consumed in the grey matter [42{44]. However, other metabolic resources can be used as alternatives supplementing the glucose upon high energy demand. Lactate and ketones are examples that can be used during continuous physical activity, and extended periods of starvation, respectively [29]. Astrocytes, a type of glial cell in the brain, have been found to have important roles in uptake and release of neurotransmitters, homeostatic processes, and energy management [29{31]. Astrocyte processes enwrap synapses and can communicate bi-directionally with neurons. These cells play a crucial role in regulating brain energy substrate when a complementary source other than glucose is required during high neuronal activity. Sensing glutamate at the synapses upon high neuronal activity is the initiator of the Astrocyte-Neuron Lactate Shuttle (i.e., ANLS) { the process by which astrocytes produce lactate upon detection of the high neuronal activity to be used as a complementary source of energy by neurons when the energy requirements of the brain are 13 high [29]. The Astrocyte-neuron lactate shuttle process has been demonstrated in Fig. 2.3a Glutamate released in the synaptic cleft is taken up by astrocytes through excitatory amino acid transporters (i.e., EAATs) along with 3 Na+ ions. These Na+ ions cause Na+/K+ ATPase to consume ATP triggering glucose utilization and uptake in the astrocytes. The glucose is transferred by endothelial cells to astrocytes from blood vessels through glucose transporters (i.e., GLUT1). Afterwards, the lactate is derived from the glucose via pyruvate by the isoenzyme of lactate dehydrogenase (i.e., LDH5). Finally, the lactate is transported out of the astrocyte and absorbed by the neurons through monocarboxylate transporters (i.e., MCT). Lactate absorbed and metabolized by neurons results in increased ATP levels, promoting depolarization of the membrane and increasing neuronal activity. A group of neurons known as lactate-sensors consumes lactate as a complementary source of energy for spontaneous activity, such as Orexin neurons [45]. Orexin neurons are good candidates to correlate brain activity and energy demands based on the roles they play such as igniting food intake, stimulation of glucose production, and projecting arousal-related cells for wakefulness. The ring rate of such neurons is signicantly aected by the lactate concentration believed to be produced by astrocytes upon neural activity [45] { supporting evidence for the astrocyte-neuron shuttle hypothesis. The excitability of the Orexin neurons upon presence and absence of lactate has been investigated [45] through patch-clamp recording on the hypothalamic brain slices of male Sprague Dawley rats. The slices were initially incubated with only 0.5mM glucose and a current ramp changing from 0 to 100pA was applied in order to measure the ring frequency as demonstrated in Fig. 2.3b labeled as LE (i.e., Low-Energy). Then the same experiment was repeated on slices incubated with 0.5mM glucose and 5mM lactate and a signicant increase in the frequency of spiking was observed as shown in Fig. 2.3b labeled as HE (i.e., High-Energy). Such an experiment suggests not only does low availability of energy substrate reduce basal ring of Orexin neurons upon stimulation, but also makes them less sensitive to further stimulation 14 (a) Astrocyte-Neuron Lactate Shuttle Process [29] (b) Eect of Lactate on Excitability of Orixin Neurons [45] Figure 2.3: Astrocyte-Neuron Lactate Shuttle resulting in saturation of the frequency of spiking. In Chapter 5, we have modeled Astro-neural lactate shuttle through modication of prior BioRC circuits so that the frequency of spiking can be regulated by astrocytes through sensing inter-domain synaptic activity. Under low synaptic activity, the frequency of spiking will be low and the circuit will be congured for low power consumption. Upon detection of synaptic activity by astrocytes, the frequency of spiking increases and the circuit will be congured for high power consumption mode. 15 Chapter 3 Towards Low-power Dendrites 3.1 Introduction In this Chapter, we introduce the steps we took to minimize the power consumption of the main component of the power in the BioRC neuron, i.e., the dendritic arbor. In Section 3.2, we introduce the low-power neuron we designed through gating the dendritic arbor utilizing asynchronous neural communications and sub-threshold implementation. Sub-threshold design is a key methodology for ultra-low power design and has been attractive to the neuromorphic community in the past decade, since the exponential current-voltage characteristic of CMOS devices in this mode of operation enables modeling a diverse range of neuro-modulating signals [46{48]. Operating in the sub-threshold region, the current owing through a MOS device is inversely and exponentially dependent on the threshold voltage (V t ) of the transistor [18]. Power gating is a low-power circuit design technique used to decrease the sub-threshold leakage current during the idle time of a circuit. To do so, a header or footer high voltage threshold (HVT) transistor is added to the circuit [18]. As a result, the sub-threshold leakage of a transistor decreases exponentially as a function of V t chosen for the HVT transistor during the idle time of the circuit. Asynchronous event-driven communication of the neurons within a neural network provides the opportunity of activating the dendritic arbor of a neuron whenever there is an input PSP to be processed and gating its power when there is no input PSP. This property can be exploited for reducing the leakage density of the dendritic arbor, which is the dominant factor of the neurons' 16 power consumption as validated by our simulation results in Section 3.2.4. In Section 3.2, we focus on the power reduction aspect and apply it to the circuits introduced in the BioRC project [22,23,25]; however, the technique can be applied to any analog circuit modeling neurons within a neural network. In Section 3.3, we introduce a power-ecient biomimetic intra-branch dendritic adder to emulate dendritic computations within the branches of the dendritic arbor. There were two concerns associated with power consumption of BioRC dendrite design as introduced in Chapter 1, Section 1.2 : 1. non-zero V gs imposed on inputs resulting in a leakage current going through the circuit even when the inputs are at resting potential (i.e., 0v), and 2. extra adders required to perform inter-dendritic branch potentials. The goal of the proposed design in Section 3.3, is to address the power consumption issues of prior BioRC dendritic arbor designs [23] as well as capturing nonlinear responses to intra-branch stimuli. Our proposed current-mode adder maintains low-power consumption through elimination of non-zero V gs issue introduced in Section 1.2 as well as sub-threshold implementation. Current- mode addition here facilitates inter-branch current summation such that they can be summed up without requiring extra adders. 17 3.2 A Low-power Sub-threshold Neuron through Gating Dendritic Arbor The neuron structure used here is shown in Fig. 3.1. The wake-up circuit observes prior neurons' activities and all the action potentials (AP) from prior neurons are fed into it. Once there is activity, this circuit keeps the header HVT PMOS transistor active so that the current can be passed through the dendritic arbor as well as the threshold-check circuit that basically converts the dendritic current to voltage and amplies it. Consequently, the header HVT PMOS transistor gates the leakage current of the dendritic arbor and threshold-check circuits while no input is applied to the neuron. Input post synaptic potentials (PSP) are small in terms of magnitude and cannot be used for power gating of a dendritic arbor and threshold-check circuit. However, each PSP is triggered by the action potential of a prior neuron, which is large enough in terms of magnitude for controlling the header PMOS transistor, but the duration of AP is around 10 times smaller than that of PSPs. As a result, Prior APs are fed into a wake-up circuit that increases the duration of the AP and adjusts it to that of PSPs. The wake-up circuit is demonstrated and discussed in Section 3.2.1. Figure 3.1: The Neuron Structure for the Low-power Experiments The structure of the adder used for processing the PSPs in the dendritic arbor is shown in Fig. 3.2. At the output node, the dendritic current is generated through subtraction of inhibition current from the sum of all excitatory currents. HVT transistors cannot be used for functional transistors in the dendritic adder as dendritic arbor transistors deal with small changing PSPs and an HVT transistor can lter those signals out. However, the header HVT transistor is controlled 18 Figure 3.2: The Dendritic Arbor for Our Experiments (a) Threshold Check Circuit (b) Comparator Circuit Figure 3.3: Threshold-Check Structure with a wake-up signal that is large enough to trigger it. The axon hillock circuit is a two-stage circuit including: 1. threshold-check stage, and 2. spike circuit stage. The threshold-check circuit is shown in Fig. 3.3a. In the rst stage of the threshold-check circuit, dendritic output current is converted to a voltage through a diode-connected transistor. The comparator in the threshold-check circuit is the well-known dierential amplier shown in Fig. 3.3b. The spike and the synapse circuits used here are based on prior research[11, 23, 25]. The spike circuit deals with large signals in terms of amplitude enabling the use of HVT transistors, in order to reduce the leakage current in the spike circuit. 19 Figure 3.4: Wake-up Circuit 3.2.1 The Wake-up Circuit The wake-up circuit is shown in Fig. 3.4. The inputs to this circuit are action potentials of the prior neurons. This circuit has two modes of operation: 1. quiescent mode, and 2. wake- up mode. The quiescent mode is the steady-state response of the circuit when none of the prior neurons re. In this case, M p pulls the input voltage of the rst inverter up to VDD through its leakage current and the wake-up output is VDD. The wake-up mode is the case when any of the prior neurons re (i.e., any of the prior neurons AP goes high). In this case, the voltage at the input of the rst inverter falls to zero through AP pull-down transistors (i.e., M1-5), and the wake-up circuit output falls to zero. The voltage at the input of the rst inverter gradually increases through the M p leakage current and once this voltage reaches the threshold of the switching for the inverter, it switches and also makes the second inverter switch as well. As a result, the wake-up circuit output goes high eventually. High Voltage Threshold (HVT) transistors have been used as the AP pull-down transistors (i.e., M1-5) as well as ones used in the inverters in order to reduce their leakage currents when they are idle. The latency of charging up the equivalent capacitance at the input node of the inverter through the M p leakage current determines the wake-up output pulse duration. The output pulse duration is a function of two variables: 1. size of M p , and 2. V b . These parameters have been set such that timing of the wake-up circuit matches with that of PSPs as discussed in Section 3.2.2.1. 20 Table 3.1: Simulation Setup Parameter Value Simulator HSPICE Transistor Model PTM Technology 45nm CMOS VDD 350mV Vss -350mV NMOS Nominal V th 467mV PMOS Nominal V th 411mV High Threshold Voltage 600mV 3.2.2 Experimental Setup The experiments were conducted using HSPICE as the circuit simulator and 45-nm PTM [49] as the transistor model. Table 3.1 shows simulation parameters for the example neuron. The circuit was implemented in the sub-threshold operating mode. For the HVT transistors, the threshold voltage was set to 600mV. For AP and PSP characterization, the amplitudes of APs were considered to be 300mV and 10 times larger than that of PSPs (i.e., 30mV). The duration of an AP was set to be 1s and the duration of PSPs were considered to be 10 times longer (i.e., 10s) than that of AP. For the threshold-check circuit in the axon hillock part, the threshold voltage (i.e., Vthreshold in Fig. 3.3a) is set such that the neuron res with more than 2 PSPs with 30mV amplitude. All the introduced neuron's parameters can be subject to alteration based on a specic neuromorphic application. The synapse circuits have been congured to achieve the noted timings for the PSPs. The Spike circuit in both the baseline neuron and the gated neuron has been implemented using HVT transistors. 21 Figure 3.5: Wake-up Circuit Output Pulse 3.2.2.1 Wake-up Circuit Conguration The key characteristic of the wake-up circuit is its output pulse duration that needs to be set based on the maximum duration of PSPs. V b and sizing of the M p transistor in the wake-up circuit determine the duration of its output pulse. We have xed the size of M p to minimum size and changed theV b in order to achieve dierent pulse widths for the wake-up circuit demonstrated in Fig. 3.5. In Fig. 3.5, V b has been varied from 260mV to 360mV with 20mV steps resulting in output pulse width changing from 1s to 10s approximately. Throughout the rest of the experiments in the paper,V b was set to 360mV andM p 's size was set to minimum in order to achieve the wake-up pulse of 10s. 3.2.3 Proposed Neuron Functional Results The proposed neuron functional results was shown in Fig. 3.6 based on the parameter setup introduced in Section 4.3.1. The synapses have been congured to generate EPSPs and IPSP (i.e., EPSP1 through EPSP4 and IPSP1 shown in Fig. 3.6a. The wake-up circuit wakes the dendritic arbor and threshold-check circuit whenever there is a PSP to be processed through the wake-up signal which is active low. The output of the comparator (Comp) is basically the amplied version of the PSP summation and is shown in Fig. 3.6c. The neuron exhibits shunting inhibition behavior when its IPSP gets activated. Here the inhibitory signal cancels out the eects of the excitatory 22 inputs in the fth time slot (i.e., from 60s to 70s) as shown in Fig. 3.6c. AP is the action potential of the neuron and psp out is its synaptic response to the AP as shown in Fig. 3.7a. As it can be observed, the neuron only res when more than two EPSPs are active at a time and there is no inhibition. (a) EPSP and IPSP Inputs (b) Comparator Response (c) Wake-up Circuit Response (d) AP and PSP Responses Figure 3.6: Proposed Neuron Functional Result 23 3.2.4 Power Consumption Analysis The proposed neuron has been compared with the baseline neuron (i.e., a neuron with exactly the same conguration but without applying power gating) in terms of power consumption in this section. The power consumption of the dierent components in the introduced architecture and baseline is shown in Table 3.2. The idle power is the power consumed while all input PSPs are at the resting potential (i.e., 0.0V). The active power in Table 3.2 refers to the case when all the input PSPs become active once with maximum amplitude (i.e., 30mV) at the same time during a time interval. Table 3.2: Power Consumption Analysis of the Neuron Component Proposed Baseline Active Power (W) Idle Power (W) Active Power (W) Idle Power (W) A Synapse 2.06E-10 1.67E-10 2.05E-10 1.67E-10 Wake-up Circuit 2.47E-10 1.89E-10 N/A N/A Dendritic Arbor 9.12E-09 1.59E-09 1.12E-08 1E-08 Comparator 1.06E-09 4.83E-11 1.44E-09 1.1E-09 Axon Hillock 1.73E-10 5.02E-11 1.77E-10 5.01E-11 Total 1.18E-08 2.88E-09 1.40E-08 1.24E-08 The most power-hungry part of the introduced neuron is the dendritic arbor dominating the power consumption of the neuron as shown in Table 3.2. The power consumption of the wake-up circuit is 0.25nW and constitutes 2% of the whole neuron's power. The eciency of the proposed neuron in terms of power reduction depends on the portion of time the circuits (i.e., dendritic arbor adder and threshold voltage checker) are gated over the total time of operation. As a result, we evaluate the neuron's power consumption as a function of the activity factor (declared as), and dene the activity factor as the portion of time the circuits are not gated over the total time of operation. We have varied the from 0 to 1 with 0.1 steps through generation of appropriate input spike patterns as shown in Fig. 3.7a and evaluated the power consumption as a function of during each interval. Each spike is applied to all the synapses of the neuron and produces PSPs with the 24 (a) Input Spike Patterns for Power Consumption Evaluation (b) Total Power Consumption Comparison Figure 3.7: Activity-dependent Power Consumption Evaluation duration of 10s. Spikes are separated from each other by a timing gap of 10s within each interval of as shown in Fig. 3.7a. The total power consumption of the proposed and baseline neurons as a function of activity factor are demonstrated in Fig. 3.7b. The best case of power reduction is the idle mode ( = 0), where the power consumption is reduced by 83% as demonstrated in Fig. 3.7b. As the activity increases, the power consumption of the proposed neuron increases. On the average, the power consumption is reduced by 33.6% as compared to the baseline case when the activity factor varies from 0 to 1 with 0.1 steps. When = 1 (i.e., the circuits are always turned on) the power consumption of the proposed neuron is still less than that of baseline neuron. The reason is that the header HVT transistors in the proposed neuron act as a current source limiting the current drawn by the neuron's gated circuits. However, in the baseline case, the current drawn from the supply voltages is exponentially variable and dependent on the V gs imposed on the PMOS current mirrors in both the dendritic arbor and the threshold-check circuits. 25 3.3 Low-power Non-linear Dendritic Computation 3.3.1 Introduction In this section, a power-ecient biomimetic intra-branch dendritic adder is introduced to emulate dendritic computations within the branches of the dendritic arbor. The goal of the proposed design in this section is to address the power consumption issues of prior BioRC dendritic arbor designs [23] as well as capturing nonlinear responses to intra-branch stimuli. 3.3.2 Adder Building Blocks The structure of the adder is demonstrated in Fig. 3.8. The adder consists of four main parts: 1. EPSP blocks, 2. sink to source current mirror (M1-M2), 3. source to sink current mirror (M3-M4), 4. IPSP block. The adder should only exhibit non-linear behavior when at-least two EPSPs are active at a time. This is dictated by a Non-Linear Enable (NLE) signal produced by the external circuit used for coincidence detection in the BioRC project [11,25]. Based on neuroscience research [21], 80% of synapses are excitatory and 20% are inhibitory in the distal branches of the dendritic arbor. Hence, the ratio of 4:1 for EPSP and IPSP blocks has been used although that is certainly subject to alteration as required per application. I E is the sum of excitatory currents generated by EPSP blocks. M1-M2 pair is a current mirror used to copy and source the sum of excitatory currents to the output node. M3-M4 copies and sinks inhibitory current generated by the IPSP block from the output node. Separating the EPSP domain from the IPSP domain dictates zeroV gs on the input transistors during the idle mode, eliminating extra leakage current caused by non-zero V gs in the prior designs [23,24]. Operating in the sub-threshold region, if V ds > 4U t , the current owing in a transistor is shown in (3.1), whereU t andV t are the thermal voltage (U t =26mV at 300K) and threshold voltage of a transistor, respectively. Parameters in (3.2), are as follows: 0 is carrier mobility, C ox is the capacitance for the gate oxide, W and L are channel width and length respectively, andn = 1+ C d Cox is known as the sub-threshold slope factor, where C d is depletion capacitance [50]. 26 Figure 3.8: Adder Structure I sub =I 0 :e ( Vgs(M1)V t nU t ) ;ifV ds > 4U t (3.1) I 0 = 0 C ox W L (n 1)U 2 t (3.2) V gs (M1) =V gs (M2);I(M2) = W (M2)=L(M2) W (M1)=L(M1) :I E (3.3) I output =I E I inh = 4 X j=1 I exc;j I inh (3.4) AsV gs remains the same on both M1 and M2, the current going through M2 is derived based on (3.3). If M1 and M2 are identical transistors, the current going through both transistors will be the same. The same explanations apply to the M3-M4 current mirror. Finally, the output current is derived from the subtraction of IPSP from EPSP currents as shown in (3.4). EPSP and IPSP are small signals with positive and negative magnitude, respectively. Vdd and Vss are supply voltages for the EPSP and IPSP domains that are considered to be positive and negative, respectively. 3.3.3 EPSP Blocks EPSP Blocks deal with nonlinear dendritic current generation based on the magnitude of input EPSPs upon coincidence detection (i.e., when two or more than two EPSPs are active). An EPSP circuit block diagram is shown in Fig. 3.9. Each EPSP block is enhanced with a control 27 Figure 3.9: EPSP Block unit (CU) that triggers super-linear or sub-linear current generation based on the magnitude of its input EPSP while two or more EPSPs are activated at the same time dictated by the NLE signal; otherwise, it performs linear current generation. Biologically, small magnitude EPSPs sum linearly, medium magnitude EPSPs sum super-linearly, and large EPSPs in terms of magnitude sum sub- linearly. In order to have only one of the afore-mentioned currents on I out of an EPSP block, high threshold voltage (HVT) transistors are considered as the footers to cut o the leakage current in the other two branches, such that only one out of three branches gets activated as dictated by the CU. The channel widths of the footer transistors is considered to be relatively larger than that of MLin, MSup, and MSub so that the source voltages of MLin, MSup, and MSub are close to ground. Imposing the same V gs on MLin, MSup, and MSub, the current of I out for each EPSP block will be derived based on (3.5). Lin, Sup, and Sub signals are binary variables used for illustrating the activation of a branches within EPSP block, where only one is activated at a time. The current of super-linear or sub-linear branches will be derived based on the relative sizing of their functional transistors (i.e., MSup, and MSub) as compared to that of the linear branch (i.e., MLin). Vb on Md has been adjusted such that no current drop occurs at the edges of the super-linear and sub-linear regions and its eect is captured by I Sup (Max) (i.e., the maximum current at the edge of the super-linear and sub-linear regions) in (3.5). 28 Figure 3.10: Adder Control Unit(CU) I out = Lin +Sup: W Sup =L Sup W Lin =L Lin +Sub: W Sup =L Sup W Lin =L Lin :I Lin (V epsp ) +Sub: I Sup (Max) (3.5) 3.3.4 Control Unit Blocks The Control Unit (CU) shown in Fig. 3.10 manages which branch of the EPSP block should be activated at a time. The comparators are well-known dierential pairs as used in [24]. If EPSP's voltage is greater thanV ref , the comparator's output will be pulled up indicating EPSP's magnitude is greater than V ref . When NLE signal is low (i.e., No overlapping EPSPs) the comparators will be deactivated and their outputs will be low. As a result, only the Lin signal will be high and Sup and Sub will be low. NLEb is also an inverted version of the NLE signal produced by an inverter in the CU. Once the NLE signal gets activated (i.e., one or more than one EPSPs are active), as long as EPSP's magnitude is smaller than V ref (Medium), only Lin signal is high and others are low. When the EPSP's magnitude is betweenV ref (Medium) andV ref (Large), both Lin and Sub signals would be low and as a result, Sup signal, which is the output of a CMOS NOR gate, goes high. When the EPSP's magnitude is greater than V ref (Large), Sub signal goes high and the other two signals remain at 0. The glitch lter removes the glitches from the Sup signal by delaying the Sub signal such that its rise and fall times match with that of Lin signal, avoiding Sup signal going 29 Table 3.3: Simulation Setup Parameter Value Simulator HSPICE Technology 45nm CMOS Vdd 300mV Vss {300mV Nominal V th 427mV High V th 700mV Table 3.4: EPSP Characterization Type V epsp Range Small V epsp < 60mV Medium 60mV<V epsp < 120mV Large V epsp > 120mV high erroneously. 3.3.5 Simulation Setup The adder circuit has been implemented in HSPICE using 45-nm PTM [49] as the transistor model. The Simulation parameters are shown in Table 3.3. The EPSP magnitude characterization has been demonstrated in Table 3.4. EPSPs have been modeled with trapezoidal piecewise linear (PWL) input pulses with 10ms duration. The rise time and fall times for EPSPs are 2ms and 5ms, respectively. For IPSPs, the duration has been considered the same as that of EPSPs. Timing and magnitude parameters of PSPs are exible to change based on a specic neuromorphic application. 3.3.6 Simulation Results Functional results of the proposed adder have been demonstrated in the waveforms of Fig. 3.11 and Fig. 3.12. As the rst evaluation, the linear functionality (i.e., Lin=Vdd, Sup=0, and Sub=0 xed in the CU) of the adder has been veried in Fig. 3.11. Four EPSP inputs (i.e., EPSP1 through EPSP4) and one inhibitory signal (i.e., IPSP1) have been assigned values as shown in Fig. 3.11. The adder's output current is demonstrated as I out . In the fth slot (i.e., 60ms to 70ms), the inhibitory signal cancels out the eect of the excitatory inputs as demonstrated. Finally, in the sixth slot (i.e., 75ms to 87ms), the summation current for two overlapping EPSPs has been demonstrated. In order to demonstrate the nonlinear behavior of the adder, we tied two excitatory inputs 30 of the adder to the EPSP signal shown in Fig. 3.12, while the other two excitatory and inhibitory blocks were deactivated, and the controlling signals (i.e. Lin, Sup, and Sub) for each EPSP block have been assigned to values manually, based on the operating mode of interest. In this evaluation, Vb input in the EPSP blocks of the adder has been tied to zero to demonstrate the sub-linear function in terms of magnitude compared to linear operating mode. As shown in Fig. 3.12, the adder performs sub-linear, linear, and super-linear addition in the rst, second, and third slot, respectively. Figure 3.11: Adder's Linear Functional Result 3.3.7 Non-linearity analysis To demonstrate the nonlinear behavior of the adder, we tied two excitatory inputs together varying from 0 to 300mV, while the other two excitatory and inhibitory blocks were deactivated, and measured the adder's output current. In this evaluation, Vb in the Fig. 3.9 has been adjusted such that no current drop occurs at the boundary of the sub-linear and super-linear region. As it 31 Figure 3.12: Adder's Non-linear Functional Result can be seen in Fig. 3.13, the adder exhibits linear behavior when input EPSP is small (i.e., when V epsp < 60mV ), super-linear behavior for the medium EPSPs (i.e., when 60mv<V epsp < 120mV ), and the output current almost saturates for the large EPSPs (i.e., whenV epsp > 120mV ), complying with the neuroscience model. For linear operation region, the regression line has been depicted and formulated in Fig. 3.13. The R-squared error metric indicates 0.2% error in the linear region of operation. The nonlinear behavior exhibited by the adder is on average +33% and {23% in super and sublinear areas of operation, respectively. Figure 3.13: Functional Result of the Adder 32 3.3.8 Power Consumption Analysis For the power consumption evaluation, we compared the proposed adder with the non-linear voltage adder introduced in [24] that mimics intra-branch dendritic summation as a function of PSP's magnitude. Another design considered for comparison with the proposed adder is the classic current mirror voltage adder [23] as introduced in Fig. 1.1 from which the design in [24] is derived. We refer to the classic current mirror used in [23] and the non-linear voltage adder introduced in [24] as BaselineL and BaselineNL, respectively, in the rest of the section. We compare to these adders because they have been designed to be used in BioRC circuits with distributed computations. For power consumption comparison, we congure all the designs for two cases: 1. Both inputs are tied together varying from 0 to 300mV (i.e., adding two EPSPs). 2. The adders are fed to one IPSP input varying from 0 to {100mV, while the other input remains at resting potential (i.e., 0V). In our experiments, the power consumption is evaluated based on the assumption that each PSP is being triggered by a single spike. The power consumption of the Baseline designs, as well as the proposed adder for Case 1, is demonstrated in Fig. 3.14a. Both the baselineL and BaselineNL cases are PMOS-based designs. As a result, the worst case of power consumption for the baseline cases is when the input's magnitude is smallest, since it imposes higher V gs on the input transistors resulting in higher leakage current. The graph for the power consumption of the BaselineNL's case is initially above the BaselineL case due to the overhead caused by controlling units for determining the mode of operation (i.e., Linear, Super-linear, or Sub-linear). The current increases as the adder enters super and sub-linear modes of operation and so does the power consumption. In the BaselineL case, as the magnitude of EPSP increases, V gs on the input transistors decreases, which results in the reduction of leakage current. As a result, the power consumption decreases as well. In the proposed adder case, increasing the input EPSPs magnitude results in increasing the power in the linear and super-linear region, since higher leakage current will go through the adder circuit. In the sub-linear region, the power drops initially and then increases as input EPSP's 33 magnitude increases. The reason is that the channel length of the functional transistor in charge of sub-linear addition is increased to exhibit sub-linear behavior. As a result, short channel eects (i.e., V t roll-o eect and Drain Induced Barrier Lowering a.k.a. DIBL eect) in the sub-linear region decrease resulting in decreasing leakage power initially. On average, the proposed adder results in 3X and 7.42X power reduction as compared to the BaselineL and BaselineNL cases, respectively, as shown in Fig. 3.14b. For the second evaluation, the designs are fed with an IPSP input varying from 0 to {100mV, while the other input remains at the resting potential (i.e., 0V). The power consumption for this case is demonstrated in Fig. 3.14c. The BaselineNL's power consumption is a bit more than that of BaselineL case due to the overhead of controlling units. The sub-threshold current of a transistor is exponentially dependent on the V gs as shown in (3.1). Therefore, increasing the V gs results in the exponential increase of sub-threshold current. Consequently, the power consumption goes up exponentially. On average, the power consumption of the proposed adder is 12.78X and 13.7X less than that of the BaselineL and BaselineNL cases, respectively as shown in Fig. 3.14d. 34 (a) Power Consumption for Case 1 (b) Average Power Consumption for Case 1 (c) Power Consumption for Case 2 (d) Average Power Consumption for Case 2 Figure 3.14: Power Consumption Evaluation 35 Chapter 4 Charge-based Summation 4.1 Introduction State of art neuromorphic dendritic arbor designs perform PSP summation through adding voltages or currents using active devices. We propose and introduce a switched-capacitor dendritic arbor circuit model where PSPs are summed up based on the charges stored on the dendritic nodes implemented by passive capacitors. Using this approach, addition of PSPs are carried out in two phases: 1. sampling PSPs on the dendritic nodes upon pre-synaptic activity and 2. serializing nodes so that sampled PSPs gets summed up across capacitors. This approach targets elimination of sub-threshold leakage current going through active devices in the state of art designs aiming for power consumption minimization. Our model makes use of asynchronous communications of neurons in order to initiate sequencing the circuit between sampling and adding phases. This is achieved through sensing pre-synaptic activity and keeping the arbor awake for the duration of the PSP. Not only does the proposed approach avoid unnecessary oscillations between circuit's phases aecting power consumption of the module dramatically, but also provides the opportunity of power gating the arbor in order to minimize the leakage current through it in the idle mode. 36 4.2 A Low-Power Switched-Capacitor Dendritic Arbor Here, we introduce a sub-threshold voltage-mode switched-capacitor circuit deploying passive elements to carry out signal addition in the dendritic arbor along with power gating, while taking into account passive properties of the dendrites for signal propagation. The circuit is a 2-phase circuit and carries out addition by sampling PSPs on the capacitors in the nodes of the arbor, and then serializing them so the sampled PSP voltages can be summed. In this section, we focus on linear and passive properties of the dendrites. 4.2.1 Dendritic Arbor Structure The structure of the dendritic arbor is shown in Fig. 4.1. Using this architecture, each neuron is enhanced with a switch-phase circuit in order to sequence the sampling and addition modes of operation. and b (i.e., the inverted version of ) are the signals used for switching between the circuit's phases. The switch phase module receives the asynchronous action potentials (AP) of the prior (pre-synaptic) neurons and wakes up if any prior neuron res, initiating sampling and charge transfer in the arbor. It stays awake as long as there is a PSP to be processed and goes to sleep after a delay equal to the longest possible PSP. The structure of this module is discussed in Section 4.2.2. Figure 4.1: Switched-Capacitor Dendritic Arbor Structure The proposed dendritic arbor consists mainly of dendritic nodes connected to each other 37 through isolating transistors. The circuit has two phases of operation: 1. sampling phase and 2. addition phase. In order to decouple the phases of the circuit from each other, HVT (High Threshold Voltage) transistors have been used as the sampling and isolating transistors so their leakage current does not interfere with each other. M LK transistors are used for modeling the leakage channels in the segments of the dendritic arbor. When is low, the circuit is congured for sampling (Fig. 4.2a), assuming ideal switches for sampling and isolating transistors. During this phase, each capacitor in a dendritic node is charged based on its corresponding input PSP through sampling transistors connected to its terminals, while isolating transistors disconnect the dendritic nodes from each other. When is high, the circuit is congured so that all capacitors are serialized while being disconnected from the PSP sources (Fig. 4.2b). As a result, the output voltage in this phase is derived through summation of the voltages across each capacitor as shown in Fig. 4.2b, assuming ideal switches and capacitors. The leakage channels deplete the charge from each node based on the time constant dictated by the channel resistance of leakage transistors (i.e., M LK ) and the equivalent parasitic capacitance at the end of a dendritic segment. (a) Sampling Phase (b) Adding Phase (c) Charge Transfer Conguration Figure 4.2: Congurations of the Proposed Dendritic Arbor 38 Figure 4.3: Switch Phase Circuit Charge transfer occurs in the addition phase of the circuit resulting in signal propagation along the dendritic arbor. During this phase, the behavior of the capacitors in the arbor nodes (i.e., C j ) can be approximated with a resistor whose value is T 2:C j , whereT is the period of the signal. As a result, the structure of the dendritic arbor in terms of charge transfer during the addition phase is as shown in Fig. 4.2c. TheR ax (j) is the axial resistance of thej th segment of the dendritic arbor and is equivalent to resistance of two serialized resistances: R(C j ): the equivalent resistance of C j during adding phase, and R CH (M I;j ): the channel resistances of the isolating transistor. R CH (M LK ) and C eq (j) also correspond to the channel resistance of the leakage transistor and the equivalent parasitic capacitance at the end of the j th segment of the dendritic arbor, respectively. 4.2.2 Switch Phase Circuit Asynchronous event-driven communication of neurons provides the opportunity to power gate and initiate sampling only when there is activity at the inputs of a neuron in the network, avoiding unnecessary oscillations. The structure of the switch phase circuit, shown in Fig. 4.3, consists of two parts: the wake-up circuit and ring oscillator. The wake-up circuit acts like a mono stable circuit and is triggered by a pre-synaptic neuron's action potentials. In the steady state, the interior output node of the wake-up circuit is pulled up by the M P sub-threshold leakage current and therefore, the output of the wake-up circuit is 0v. This stops the ring oscillator from oscillating 39 as the feedback is broken by the NAND gate holding its output to VDD. Once there is activity in a prior neuron, the interior output node (V int ) of the wake-up circuit falls to 0v and its output is pulled up to VDD. Therefore, feedback in the oscillator is started and the output of the oscillator keeps oscillating as long as the wake-up circuit's output remains at VDD. A transmission gate has been added to the b path and has been sized so the delay of the generation of and b is identical. M P 's sub-threshold leakage current gradually charges the interior output node (V int ) of the wake-up circuit and after a delay of T trig , V int is pulled up to VDD and consequently, its output falls to 0. At this point, the oscillator no longer oscillates, as the feedback path is broken by the NAND gate holding its output at VDD. T trig is the output pulse width of the wake-up circuit and is a function of the M P channel conductance (i.e., controlled by V ref as well as its channel width and length) and equivalent capacitance at the interior output node of the wake-up circuit. T trig should to be adjusted based on the longest possible PSP in the network. In order to minimize the leakage current of this module, the footer AP transistors in the circuit as well as all the logic gates used in the module have been implemented using HVT transistors. 4.3 Experimental Results In this section, we demonstrate diverse spatial and temporal patterns the dendrite model introduced in Section 4.2.1 can exhibit while consuming signicant lower power compared to state of the art dendrites. In order to demonstrate the behavior of the proposed dendritic arbor, rst we show the signal integration in case of both excitatory and inhibitory PSPs in Section 4.3.2 as well as how the signal is propagated through the arbor. In Section 4.3.3, we perform parametric varation on the key parameters of the model to show various spatio-temporal patterns it can exhibit as a part of the BioRC neuron. In Section 4.3.4, we enhance the dendrite model with dendritic spikes, that corresponds to super-linear responses of dendrites while multiple adjacent inputs co-exist at the same time. Finally, we perform comprehensive power analysis on the introduced switched-capacitor based dednritic arbor and compare it to the state of the art dendrites in Section 4.3.5. 40 Table 4.1: Simulation Setup for the Switched-Capacitor Dendritic Arbor PARAMETER VALUE Supply Voltage (VDD) 300mV Nominal V th 467mV High Threshold Voltage 600mV Dendritic Node Capacitance (C j ) 10fF Sampling and Isolating Transistors Dimensions W=180nm, L=45nm Leakage Transistor Dimensions W=90nm, L=45nm V LK : Leakage Transistor Bias 0V T trig : Wake-up Circuit Pulse Width 9.96s T : Sampling Signal Period 0.93s 4.3.1 Simulation Setup Simulation experiments have been conducted using the HSPICE circuit simulator and 45nm CMOS [49] as the transistor model. Table 4.1 shows simulation parameters for the circuits used in the model. All the circuits are implemented in the sub-threshold operating mode. For AP and PSP characterization, the amplitudes of APs are 300mV, 10 times larger than that of PSPs (i.e., 30mV). The duration of an AP has been set to be 1s and the durations of PSPs are adjusted to be 10 times longer (i.e., 10s) than that of the AP in the worst case. The wake-up circuit parameters have been congured to achieve T trig = 9:96s based on the considered worst case duration of PSPs (i.e., 10s). T (i.e., sampling signal's period) is set to 0.93s through sizing all the gates in order to achieve symmetric rise and fall delays. 41 (a) Two-stage Dendrite (b) Node1 in Excitation Mode (c) Node1 in Inhibition Mode Figure 4.4: Conguration of Two-stage Dendrite 4.3.2 Signal Integration and Propagation In order to demonstrate the behavior of the proposed dendritic arbor, a two-segment dendrite as shown in Fig. 4.4a is used and synthesized based on the parameters introduced in Table 4.1. In this experiment, PSP signals have been modeled using piece-wise linear pulses with rise and fall times of 2s and 5s, respectively. Signal addition on two adjacent nodes of a two-segment arbor is shown in Fig. 4.5. The rst segment synapse is congured in excitatory (E1) and then inhibitory (I1) modes as shown in Fig. 4.4c and Fig. 4.4b, respectively. The second segment is xed to be in excitatory (E2) mode. As shown, once there is a PSP to be processed, the sampling is initiated and is then stopped after 9.96s. Based on the parameters set for the arbor nodes, the attenuation is 8mV per stage. In order to demonstrate the signal propagation in the arbor, a 5-stage arbor is synthesized and fed to a switch phase circuit with a 0.5s period, while the conductance of the leakage channels has been decreased through setting V LK ={100mv. A PSP with a magnitude of 30mV is fed to the rst stage of the arbor and signal propagation has been probed at dierent nodes as shown in Fig. 4.6. As it can be observed, the PSP is attenuated based on the RC time constant that is present in each stage as explained in Section 4.2.1. 42 Figure 4.5: Dendritic Arbor Functional Results Figure 4.6: Signal Propagation in a 5-stage Arbor 43 4.3.3 Parametric Variation on Switched-Capacitor Dendrite In this Section, we perform parametric variations on the introduced switched-capacitor dendrite to show the diverse spatio-temporal patterns it can exhibit as a part of the BioRC neuron. As a result, we implemented a 4-stage dendrite and included that in the neuron model introduced in Fig. 1.1 and observed the V Soma . The input PSPs are shown in Fig. 4.7, which is used consistently among all the experiments in this Section. Charge transfer characteristics are aected by the key parameters of the model introduced in Section 4.2.1 aecting axial resistance of dendritic nodes as well as the conductance of the leakage channels. Frequency of sampling also plays a key role in the accuracy of captured PSPs in terms of shape. In Section 4.3.3.1, we show the circuit response to key parametric variations with respect to axial resistance, leakage channel conductance, and frequency of sampling. In all the experiments, all the transistor sizings are kept the same as Table 4.1 unless explicitly mentioned for parameters of interest in each Section. Figure 4.7: Post-Synaptic Potentials Produced by Synapse Circuits as Input 44 4.3.3.1 Parametric Variation Experimental Results In order to vary axial resistance of dendritic nodes, we xed the channel width of isolating transistors as W M iso = 5L min while varying their channel lengths as L M iso = 6KL min , where K2f1; 2; 3; 4g and L min =45nm. As it can be observed in Fig. 4.8a, higher values of K corresponds to higher axial resistance and as a result, the maximum amplitude of summed PSPs at V Soma is reduced as shown in the Fig. 4.8b. Leakage channel conductance determines the time constant at which the charges damp at each dendritic node and is aected by dimensions of leakage transistor (i.e.,M LK ) as well as leakage bias voltage (i.e.,V LK ). Therefore, we performed two separate experiments to vary Leakage channel conductance. First, we xed the leakage bias voltage to 50mV (i.e., V LK = 50mV ) while varying the dimensions of the leakage transistor as W M LK = 5W min and L M LK = 6KL min where K 2f1; 2; 3; 4g and L min =45nm. As it's shown in Fig.4.10, higher values for L M LK result in charges lasting longer on the nodes and increase the overall amplitude of summed PSPs, since the conductance of the leakage channel reduces in this case. Also, we varied the leakage bias voltage (i.e., V LK ) to vary leakage channel conductance. As it's shown in Fig. 4.10b, the higher values of V LK increases the conductance of the leakage channels and charges damp faster. Frequency of sampling (i.e., 1=T ) is another key parameter in the model aecting the accuracy of restored PSPs on the capacitors in each dendrite node. As it's shown in Fig. 4.11, higher frequency of sampling results in less distortion on restored summed PSPs; however, since there is less chance for sampling transistors to provide the current on the dendrite capacitors, the amplitude will be lower. 45 (a) Channel Length Variations for Isolating Transistors (b) Peak Voltage at Soma Figure 4.8: Axial Resistance Variations 46 (a) Channel Length Variations for Leakage Channel Transistors (b) Peak Voltage at Soma Figure 4.9: Leakage Channel Conductance Variation, Leakage Transistors Dimention Variations 47 (a) Leakage Voltage (V LK ) Variations (b) Peak Voltage at Soma Figure 4.10: Leakage Channel Conductance Variation, Leakage Voltage (V LK ) Variations 48 (a) Sampling Frequency Variations, T =250nS (b) Sampling Frequency Variations, T =500nS (c) Sampling Frequency Variations, T =750nS (d) Sampling Frequency Variations, T =1S Figure 4.11: Sampling Frequency Variations 49 4.3.4 Non-linear Switched-Capacitor Dendritic Arbor In this section, we enhance the introduced switched-capacitor dendritic arbor with dendritic spikes. A dendritic spike gets generated when a cluster of inputs co-exist in time and space and this results in a super-linear dendritic response. Fig. 4.12 demonstrates details of non-linear switched- capacitor dendritic arbor. The coincidence signal detects the existence of multiple adjacent inputs existing at the same on the same branch of the dendrite [11]. For the gain stage, a non-inverting amplier has been designed as demonstrated in Fig. 4.12b. The second stage is a PMOS-based common-source amplier that is activated when the rst stage input goes low; hence, reducing the static leakage power in the idle mode. (a) Non-linear Dendritic Branch (b) Gain Stage Figure 4.12: Non-linear Switched-Capacitor Dendritic Arbor Simulation results shown in Fig. 4.13 correspond to the BioRC neuron's response to input PSPs in Fig. 4.13a once with a linear dendrite and then with non-linear dendrite introduced in Sections 4.2.1 and 4.3.4, respectively. Non-linear response is triggered if two or more than two 50 adjacent synapses are active at a time. The neuron is adjusted to re when more than two input PSPs are active at a time as it can be observed in Fig. 4.13d. However, as it's shown in Fig. 4.13c and Fig. 4.13e, the non-linear response triggered by two synapses being active at the same time is strong enough to cause the neuron to re. (a) Non-linear Dendritic Branch (b) V SOMA in Neuron without Dendritic Spikes (c) V SOMA in Neuron with Dendritic Spikes (d) AP in Neuron without Dendritic Spikes (e) AP in Neuron with Dendritic Spikes Figure 4.13: Neuron's Response Enhanced With And Without Dendritic Spikes 51 Table 4.2: Power Consumption Analysis Design Power Consumption Per Node P active (W) P idle (W) Current Mirror Based [23,24] 7.11E-09 8.95E-09 Average and Amplify Method [25] 1.46E-09 1.09E-09 Transconductance-Based [26,27] 1.84E-10 1.72E-10 Proposed Dendritic Arbor 1.53E-10 6.56E-11 4.3.5 Power Consumption Evaluation and Comparison In order to evaluate and compare the power consumption with other designs, a two-segment dendrite is used and synthesized for each state of the art design without non-linear dendritic spiking feature referred to as a node in Table 4.2. The proposed dendritic arbor has been compared with baseline voltage adders [23, 25] as well as transconductance-based models [26, 27] as introduced in Section 2.1.2. For the average and amplify method, a dierential amplier in the sub-threshold region of operation has been used and designed to exhibit voltage gain of two (i.e., A v =2). The current mirror voltage adder [23] has also been implemented in the sub-threshold operational mode. For the transconductance-based model a simple pull up NMOS transistor has been used and fed to the PSP in a segment to generate the post-synaptic current. Power consumption per node for each design is shown in Table 4.2. P active is the active power and corresponds to the power consumption of a design when adding two excitatory PSPs. P idle is the idle power and corresponds to the case when both inputs remain at resting potential (i.e., 0v). In the proposed dendritic arbor, signal addition is achieved through serializing passive elements rather than charging the output capacitance through the current provided by active devices and the power supply. Therefore, as the simulation results demonstrate, the proposed design reduces the power consumption (both the active and idle power) by an order of magnitude as compared to the average and amplify design used in [25] and current mirror based design [23]. The P idle is greater than P active for the current mirror based design due to the fact that it's a PMOS based design and a larger V gs will be imposed on its input transistors while the input remains at the resting potential. 52 The active power consumption of the proposed design is reduced by 17% compared to transconductance-based designs [26, 27], while its idle power consumption is 2.6 times less than that of those designs as shown in Table 4.2. Fig. 4.14 shows the total power consumption dened in (4.1) as a function of activity factor (i.e., , dened as active time of the arbor over the total time of operation for the arbor). As it can be observed, transconductance-based designs tend be more leaky and consume sub-threshold leakage power even if the input remains at the resting potential. However, this issue is mitigated in the proposed design with power gating. P total =:P active + (1):P idle (4.1) Figure 4.14: Power Consumption Analysis 53 Chapter 5 Modeling Astrocyte Intervention in Energy Regulation of the Brain 5.1 Introduction In this chapter, we model the astrocyte's metabolic role that has a signicant eect on the brain's energy substrate regulation [29], as introduced in Section 2.2. In Section 5.2, we propose and introduce required modications to the BioRC neuron model in order to include astro-neuron lactate shuttle property such that frequency of spiking gets tuned by astrocytes based on inter- domain synaptic activity, as introduced in Section 2.2. In Section 5.2.2, we evaluate frequency modulation of a spiking circuit (i.e., axon hillock) as a result of astrocyte input and analyze its power consumption. In Section 5.2.3, we show how increasing neural activity in adjacent domains can increase the frequency of spiking and evaluate how the astro-neural lactate shuttle aects the power consumption of domains. 5.2 Modeling Astro-Neuron Lactate Shuttle Fig. 5.1 demonstrates the high-level block diagram of neurons in a domain interconnected with astrocytes used for modeling the astro-neuron lactate shuttle. The BioRC Synapse circuit [11] contains 2 parts connected at the synaptic cleft: 1. neurotransmitter side, and 2. receptor side. The synaptic activity in the astro-neuron shuttle hypothesis is detected by the release of glutamate as the neurotransmitter in the synaptic clefts of the neurons. The astrocyte module in Fig. 5.1 basically monitors activities of the neurons in the domain through summing up all the synaptic cleft voltages as the measure of activity and feeds it to the BioRC axon hillock circuit [11, 25] in 54 order to modulate the frequency of the spiking based on neural activity. The frequency of the axon hillock circuit gets tuned based on the astrocyte input fed to it. The Astrocyte circuit details are introduced in Section 5.2.3. Modications required to the BioRC axon hillock circuit for frequency modulation have been introduced in Section 5.2.1. Any dendritic model introduced in the prior chapters can be used as the dendritic arbor for the neurons based on the application of interest; however, we used the charge-based dendritic arbor we designed and elaborated in Chapter 4. Figure 5.1: Astro-Neuron Lactate Shuttle Model 55 5.2.1 Modied Axon Hillock Circuit The BioRC axon hillock circuit [11, 25] is shown in Fig. 5.2a. The input stage connected to the V soma is a common-source amplier followed by an inverter. This circuit has 2 operating modes: 1. quiescent and 2. spiking. In the quiescent mode (i.e., when V soma is small), amplier output would be high and X11 pulls down the output (AP=0). Also, the inverter output (A) would be low (A=0). Assuming we have had a previous spike, X4 would have been activated and node B would be pulled down (B=0). This makes node C go high (C=Vdd) and the Na+ channel controlled by X8 will be deactivated. When the spike is gone (i.e., AP=0) andV soma is small (resulting in A=0) node B would be pulled up through the (X5, X6) chain. Pulling node B up causes node D to be pulled down and as a result, the K+ channel controlled by X7 would be deactivated. Once V soma becomes high enough, a sequence of events happen in the circuit to generate a spike. Amplier output drops to 0 (X11 deactivates) and A would be pulled up (A=Vdd). Node B was pulled up in the quiescent mode; therefore, (X1, X2) chain would be turned on resulting in C=0. As a result, X8 turns on (Na+ channel opens) and AP will be pulled up (AP=Vdd). When AP reaches Vdd, X4 turns on and node B falls to 0 making node C high (B=0, and C=Vdd). This deactivates X8 and the Na+ channel closes. In addition, when B goes to 0, D rises to Vdd through X10 and X7 (K+ channel) gets activated pulling down the AP to 0. The axon hillock circuit can be used for two modes of spiking: 1. single spike and 2. burst mode [11]. The existence of transistor X5 forces single spike mode, meaning the AP would not go high again, since as long as V Soma and as a result node A are high, node B cannot be pulled up so that node C would be pulled down and AP goes high. In order to produce a new spike in this mode, V Soma (and consequently node A) needs to be pulled down. Removing X5 and maintaining X6 and X4 as an inverter result in burst mode, meaning the AP contiguously spikes, since as long as V Soma is high enough (meaning A is high as well), node B and C values can be updated. We have added a feedback path as shown in Fig. 5.2b in order to control the frequency 56 (a) BioRC Axon Hillock Circuit [11,25] (b) Modied Axon Hillock Circuit Figure 5.2: Axon Hillock Circuit 57 of spiking. This feedback path is in charge of resetting node A such that the circuit can be initialized to produce spikes continuously as long as V Soma is high enough. The time between two consecutive spikes will be determined by the delays of the inverters in the feedback loop. The delays of the inverters are controlled through Astro andAstrob knobs generated by the astrocyte module controlling the rise and fall times of the inverter. The supply voltage of the feed-back inverters (i.e., VDDH) needs to be set such that the axon hillock circuit will be capable of producing an appropriate range of spiking frequency. Since the axon hillock circuit is implemented in the sub- threshold region of operation, the supply voltages of the inverters are likely to be higher than the supply voltage of the axon hillock circuit. V ref1 andV ref2 are bias voltages such that the feedback loop remains active and the neuron can still perform continuous spiking even in the case of low synaptic activity. 5.2.2 Experimental Results for Modifed BioRC Axon Hillock Circuit 5.2.2.1 Frequency Adaptation We have implemented the modied axon hillock circuit introduced in Section 5.2.1 with the synapse as output load, with the parameters introduced in Table 5.1. The circuit has been implemented in the sub-threshold region of operation. Astro and Atrob signals are input ramp signals as demonstrated in Fig. 5.3. They basically control the delay of the feedback loop in the axon hillock circuit through adjusting rise and fall times of the inverters. V Soma is set to Vdd such that the circuit keeps spiking. As it can be seen, higher astrocyte input (i.e., Astro) results in a faster frequency of spiking. Fig. 5.4 shows the modulation of the frequency of spiking versus the astrocyte input. The frequency of spiking ranges from 613KHz to 1.8MHz as the astrocyte input ramps up to 300mV from 0V. Table 5.1: Modied Axon Hillock Circuit Conguration PARAMETER VDD VDDH V ref1 V ref2 VALUE 350mV 500mV 300mV 150mV 58 Figure 5.3: Experimental Results on the Modied Axon Hillock Circuit Figure 5.4: Modulation of Frequency of Spiking vs. Astrocyte Input 59 Table 5.2: Power Consumption Components for Axon Hillock Circuit Static Power (nW) 0.09495 Peak Power (nW) 21.4 5.2.2.2 Power Consumption Evaluation The instantaneous power consumption of the axon hillock circuit implemented in Section 5.2.2 has been demonstrated in Fig. 5.5. In this experiment, V Soma is held at VDD to keep the neuron ring and we have applied a ramp input changing from 0 to 300mV to V astro as shown in Fig. 5.3. As the frequency of spiking increases as a result of astrocyte input ramping up, power consumption increases. Table 5.2 shows static and peak power consumption of the module. The static power corresponds to the case where V Soma and V astro are at resting potential(i.e., 0V). Figure 5.5: Instantaneous Power Consumption for the Axon Hillock Circuit 60 5.2.3 Inter-domain Astrocyte Process for Modeling the Astro-Neural Lactate Shuttle We have implemented a modied version of the BioRC neuron enhanced with astro-neural shuttle demonstrated in Fig. 5.6. For the sake of simplicity, we only consider two domains and label them as low-energy and high-energy. The only new component is the astrocyte module responsible for detecting activity in the high-energy domain and tuning the frequency of the low-energy domain. Since this module is in charge of integrating neurotransmitters released in the synaptic clefts of the high-energy domain, we used the same charge-based adder introduced in Chapter 4. For the inverting amplier, the same common-source amplier used in [11] is tuned and used. Figure 5.6: BioRC Modied Neuron Enhanced with an Astro-Neuron Lactate Shuttle 61 Table 5.3: Inter-Spike Delays Inter-spike delay (nS) Before Warm-up After Warm-up N1.AP N/A (Single-Spike) 0.89nS N2.AP 2.23nS 0.89nS 5.2.3.1 Experimental Results for Astro-Neural Shuttle Circuit Model The low-energy and high-energy domains are fed with pre-synaptic action potential inputs at frequencies of 33.3KHz and 100KHz, respectively. As it can be observed in Fig. 5.7, the rst set of post-synaptic potentials (i.e.,N1:SYN i where i2f1; 2; 3; 4g from 0 to 10S) caused by pre- synaptic action potential inputs fed to the low-energy domain can only enforce the neuron to re a single spike. When the high-energy domain starts warming up and ring (i.e., from 10S to 30S), the neurotransmitters are integrated at Astro signal. The rst set of post-synaptic potentials in the high-energy domain (i.e., N2:SYN i where i2f1; 2; 3; 4g from 10S to 20S) enforces the neuron to re two consecutive spikes with an inter-spike delay of 2.23nS. However, as the domain gets warmed up during the second set of inputs (i.e., from 20S to 30S), a train of spikes is produced with an average inter-spike delay of 0.94nS. After the high-energy domain is warmed up, we feed the second set of inputs (i.e., from 30S to 40S) to the low-energy domain. As it is demonstrated, this time, a train of spikes with an inter-spike delay of 0.89nS is being produced rather than a single spike. This shows how the astrocytes can intervene in energy regulation, providing a higher frequency of spiking as needed in a domain when the neural activity increases in an adjacent domain. 62 Figure 5.7: Circuit Simulation Results for the Astro-Neural Lactate Shuttle Model 63 5.2.3.2 Power Consumption Evaluation for the Astro-Neural Lactate Shuttle Circuit Model In this section, we evaluate the power consumption of the circuit model introduced for the BioRC Astro-Neuron Shuttle in Section 5.2.3 through analyzing the instantaneous, static, peak, and average power consumption of the modules. The instantaneous power consumption of the main modules of the models (i.e., N1, N2, and Astrocyte modules) is demonstrated in Fig. 5.9. The instantaneous power consumption for N1 module (i.e., low-energy domain) demonstrates how the astrocyte intervention modulates power consumption. The average power consumption of N1 module increases by 2.04 times after the warm-up phase (i.e., from 30S to 40S) compared to before warm-up (i.e., from 0S to 10S). This is due to astrocyte intervention as the activity is increased in the adjacent high-energy domain resulting in reducing inter-spike delay in the low- energy domain from 10S (i.e., single spike mode) to 0.89nS (train of spikes). The static power corresponds to the case when inputs are at resting potential(i.e., 0v). As it can be observed, in Fig. 5.9a, the static power of the domains (i.e., high-energy and low-energy domains) are almost the same, while the static power consumption of the astrocyte is 67% less than average of the domains. The peak power consumption corresponds to the case when all pre-synaptic action potentials are active in the domain and the neuron is ring. As it can be observed in Fig. 5.9b the peak power consumptions of the domains are one order of magnitude higher than that of the astrocyte module. The average power consumption of the astrocyte module is 4 times less than that of the domains in the overall experiment (i.e., from 0s to 60s) as demonstrated in the Fig. 5.9c. Table 5.4: Power Consumption Analysis of Astro-Neural Shuttle Module Power Component N1 N2 Astrocyte Static Power (nW) 0.944 0.97 0.572 Peak Power (nW) 16.6 20.3 1.12 Average Power (nW) 2.66 3.28 0.733 64 (a) Instantaneous Power for N1 Module (b) Instantaneous Power for N2 Module (c) Instantaneous Power for Astrocyte Module (d) Instantaneous Power of the Whole Module Figure 5.8: Instantaneous Power Consumption Components for Astro-Neural Shuttle Circuit Model (a) Static Power Consumption (b) Peak Power Consumption (c) Average Power Consumption Figure 5.9: Power Consumption Components for Astro-Neural Shuttle Circuit Model 65 Chapter 6 Pattern Detection Using Dendritic Computation 6.1 Introduction Biological neurons perform extensive nonlinear computations over time and space within each neuron that in uence overall neural encoding. The essence of nonlinear dendritic computations cannot be neglected as some neural behaviors depend heavily on these functions [11, 28]. As a central goal of BioRC project, circuits have been designed to mimic such distributed, non-linear computations [11]. In this chapter, we put together the ecient circuits we have designed to elaborate an application of dendritic computations and the roles it can play in overall neural encoding. The goal is to demonstrate the complex functions that can be exhibited by a single neuron in order to distinguish two pieces of knowledge. 6.2 Neuron Structure The high-level block diagram of the neuron is shown in Fig. 6.1. The patterns are assumed to be detected edges belonging to characters2f+;Xg given as pre-synaptic inputs to the neuron on two dierent branches of the dendritic arbor. The neuron is designed to re two dierent frequencies on activation of a specic branch of the dendrite. Four adjacent synapses being active at the same time on a branch triggers a dendritic spike that is large enough to cause the neuron to re. The neuron activates if at least one branch is active and the voltage at soma modulates the frequency of spiking. 66 Figure 6.1: Block Diagram of the Neuron 6.3 Dendrite Branch Circuits The dendritic arbor used for processing of input PSPs is demonstrated in Fig. 6.3 and it mainly consists of three parts: 1. synapse integration (Adder), 2. coincidence detection, and 3. non-linear computation. The same charge-based adder introduced in Chapter 3 has been deployed for the integration of the PSPs. If all four pre-synaptic inputs (i.e., AP1 to AP4) are activated at a time, the coincidence detection circuit activates and results in a dendritic spike to be propagated down the dendritic trunk. The circuit used for coincidence detection is shown in Fig. 6.3. The pull-up networks of the rst stage are tuned such that charging time of the input nodes of the second stage (i.e., NOR circuit) is adjusted the same as the duration of a PSP. 67 Figure 6.2: Dendrite Branch Circuits Figure 6.3: Dendritic trunk Circuit 68 6.4 Dendritic Trunk Circuit The dendritic trunk structure is shown in Fig. 6.3. This module is in charge of summing up the voltages produced by the branches with attenuation. The further the input from the soma, the higher the attenuation. The output of this module is used for modulation of the frequency of the axon hillock circuit. 6.5 Axonhillock Circuit The axon hillock circuit is shown in Fig. 6.5 as introduced and designed in Chapter 5. The spiking initiates if the spikeEn signal goes high, which is the result of ORing two coincidence signals produced by the branches, meaning if at least one branch is active the neuron should re. The frequency of spiking is determined through the charge accumulated at soma (i.e., the end of dendritic trunk). Since branch1 is further from the soma it incurs higher attenuation compared to branch2 which is closer to soma. As a result, a lower frequency of spiking is expected when only branch1 is active. Figure 6.4: Axon Hillock Circuit 69 6.6 Experimental Results The simulation results for the neuron introduced in Section 6.2 are shown in Fig. 6.5. The PSP signals on a branch are labeled as PSP bi j where i2f1; 2g and j2f1; 2; 3; 4g. Voltages at the outputs of the branches are labeled Branch i out where i2f1; 2g. Branch1 and Branch2 are associated with pattern + and X, respectively. The voltage on the soma (i.e., V Soma ) modulates the frequency of spiking. As it can be observed, branch2 incurs less attenuation since it's closer to the soma and as a result, the neuron res at a higher rate. The inter-spike delay characteristics of the two patterns are shown in Table. 6.1. The neuron res 1.42 times faster on average detecting character X compared to the detection of character +. Table 6.1: Inter-spike Delays of the Neuron Detecting the Patterns Delays Data for pattern + Data for Pattern X min 2.28S 1.94S average 2.945S 2.07S max 3.61S 2.21S 70 Figure 6.5: Experimental Results of a Neuron Detecting Two Patterns 71 Bibliography [1] R. Jung, S. V. Jung, and B. Srimattirumalaparle, \Neuromorphic controlled powered orthotic and prosthetic system," July 2 2014. US Patent App. 14/322,326. [2] A. K. Thota, S. Kuntaegowdanahalli, J. Orbay, A. K. Starosciak, J. J. Abbas, K. W. Horch, and R. Jung, \A multi-lead multi-electrode system for neural-interface enabled advanced prostheses," in Biomedical Engineering Conference (SBEC), 2013 29th Southern, pp. 109{110, IEEE, 2013. [3] S. S. Kuntaegowdanahalli, R. Jung, J. J. Abbas, and K. Horch, \Fitting system for a neural enabled limb prosthesis system," Mar. 17 2014. US Patent App. 14/216,826. [4] S. Shah, H. Toreyin, O. T. Inan, and J. Hasler, \Recongurable analog classier for knee-joint rehabilitation," in 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 4784{4787, Aug 2016. [5] S. Shah, C. N. Teague, O. T. Inan, and J. Hasler, \A proof-of-concept classier for acoustic signals from the knee joint on a fpaa," in SENSORS, 2016 IEEE, pp. 1{3, IEEE, 2016. [6] J. K. Kim, P. Knag, T. Chen, and Z. Zhang, \A 640m pixel/s 3.65mw sparse event-driven neuromorphic object recognition processor with on-chip learning," in 2015 Symposium on VLSI Circuits (VLSI Circuits), pp. C50{C51, June 2015. [7] G. Cauwenberghs, \Reverse engineering the cognitive brain," Proceedings of the National Academy of Sciences, vol. 110, no. 39, pp. 15512{15513, 2013. [8] E. Chicca, F. Stefanini, C. Bartolozzi, and G. Indiveri, \Neuromorphic electronic circuits for building autonomous cognitive systems," Proceedings of the IEEE, vol. 102, no. 9, pp. 1367{ 1388, 2014. [9] G. Indiveri, E. Chicca, and R. J. Douglas, \Articial cognitive systems: From vlsi networks of spiking neurons to neuromorphic cognition," Cognitive Computation, vol. 1, no. 2, pp. 119{ 127, 2009. [10] L. A. Pastur-Romay, F. Cedr on, A. Pazos, and A. B. Porto-Pazos, \Deep articial neural networks and neuromorphic chips for big data analysis: pharmaceutical and bioinformatics applications," International journal of molecular sciences, vol. 17, no. 8, p. 1313, 2016. [11] C.-C. Hsu, Dendritic Computation and Plasticity in Neuromorphic Circuits. PhD thesis, University of Southern California, 2014. 72 [12] S. Ramakrishnan, A. Basu, L. K. Chiu, J. Hasler, D. Anderson, and S. Brink, \Speech processing on a recongurable analog platform," IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 22, no. 2, pp. 430{433, 2014. [13] S. Koziol, S. Brink, and J. Hasler, \A neuromorphic approach to path planning using a recongurable neuron array ic," IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 22, no. 12, pp. 2724{2737, 2014. [14] S. Barzegarjalali, Demonstrating the role of multiple memory mechanisms in learning patterns using neuromorphic circuits. PhD thesis, University of Southern California, 2016. [15] S. Furber, \Large-scale neuromorphic computing systems," Journal of neural engineering, vol. 13, no. 5, p. 051001, 2016. [16] J. Schemmel, D. Briiderle, A. Griibl, M. Hock, K. Meier, and S. Millner, \A wafer-scale neuromorphic hardware system for large-scale neural modeling," in Circuits and systems (ISCAS), proceedings of 2010 IEEE international symposium on, pp. 1947{1950, IEEE, 2010. [17] B. V. Benjamin, P. Gao, E. McQuinn, S. Choudhary, A. R. Chandrasekaran, J.-M. Bussat, R. Alvarez-Icaza, J. V. Arthur, P. A. Merolla, and K. Boahen, \Neurogrid: A mixed-analog- digital multichip system for large-scale neural simulations," Proceedings of the IEEE, vol. 102, no. 5, pp. 699{716, 2014. [18] J. Rabaey, Low power design essentials. Springer Science & Business Media, 2009. [19] A. Vassighi and M. Sachdev, Thermal and Power Management of Integrated Circuits (Series on Integrated Circuits and Systems). Secaucus, NJ, USA: Springer-Verlag New York, Inc., 2006. [20] \Bio-mimetic real-time cortex project." [21] M. Megas, Z. Emri, T. Freund, and A. Gulyas, \Total number and distribution of inhibitory and excitatory synapses on hippocampal CA1 pyramidal cells," Neuroscience, vol. 102, no. 3, pp. 527{540, 2001. [22] A. C. Parker, J. Joshi, W. Dweik, J. Koita, N. A. D. Singh, and O. Ayo, \A cmos implementation of neural dendritic computations with cost analysis," [23] A. C. Parker, J. Joshi, C.-C. Hsu, and N. A. D. Singh, \A carbon nanotube implementation of temporal and spatial dendritic computations," in 2008 51st Midwest Symposium on Circuits and Systems, pp. 818{821, IEEE, 2008. [24] X. Zhou, Y. Guo, A. C. Parker, C.-C. Hsu, and J. Choma, \Biomimetic non-linear CMOS adder for neuromorphic circuits," in 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), 2013. [25] C.-C. Hsu and A. C. Parker, \Dynamic spike threshold and nonlinear dendritic computation for coincidence detection in neuromorphic circuits," in 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 461{464, IEEE, 2014. [26] E. Farquhar, D. Abramson, and P. Hasler, \A recongurable bidirectional active 2 dimensional dendrite model," in IEEE International Symposium on Circuits and Systems, pp. 313{316, 2004. 73 [27] S. Nease, S. George, P. Hasler, S. Koziol, and S. Brink, \Modeling and implementation of voltage-mode CMOS dendrites on a recongurable analog platform," IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS, vol. 6, no. 1, pp. 76{84, 2012. [28] M. London and M. H ausser, \Dendritic computation," Annu. Rev. Neurosci., vol. 28, pp. 503{ 532, 2005. [29] M. B elanger, I. Allaman, and P. J. Magistretti, \Brain energy metabolism: focus on astrocyte- neuron metabolic cooperation," Cell metabolism, vol. 14, no. 6, pp. 724{738, 2011. [30] Y. Irizarry-Valle and A. C. Parker, \An astrocyte neuromorphic circuit that in uences neuronal phase synchrony," IEEE transactions on biomedical circuits and systems, vol. 9, no. 2, pp. 175{ 187, 2015. [31] Y. Irrizary-Valle, Modeling Astrocyte-Neural Interactions in CMOS Neuromorphic Circuits. PhD thesis, University of Southern California, 2015. [32] S. Remy and N. Spruston, \Dendritic spikes induce single-burst long-term potentiation," Proceedings of the National Academy of Sciences, vol. 104, no. 43, pp. 17192{17197, 2007. [33] P. Poirazi, T. Brannon, and B. W. Mel, \Arithmetic of subthreshold synaptic summation in a model CA1 pyramidal cell," Neuron, vol. 37, no. 6, pp. 977{987, 2003. [34] A. Polsky, B. W. Mel, and J. Schiller, \Computational subunits in thin dendrites of pyramidal cells," Nature neuroscience, vol. 7, no. 6, pp. 621{627, 2004. [35] K. M. Hynna and K. Boahen, \Nonlinear in uence of t-channels in an in silico relay neuron," IEEE Trans. Biomed. Engineering, vol. 56, no. 6, pp. 1734{1743, 2009. [36] J. G. Elias, \Articial dendritic trees," Neural Computation, vol. 5, no. 4, pp. 648{664, 1993. [37] W. Rall, \Theoretical signicance of dendritic trees for neuronal input-output relations," Neural theory and modeling, vol. 7397, 1964. [38] C.-C. Hsu, A. C. Parker, and J. Joshi, \Dendritic computations, dendritic spiking and dendritic plasticity in nanoelectronic neurons," in 2010 53rd IEEE International Midwest Symposium on Circuits and Systems, pp. 89{92, IEEE, 2010. [39] J. G. Elias, D. P. M. Northmore, and S. M. Meshreki, \Wide-range variable dynamics using switched capacitor neuromorphs," in Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS '94, vol. 6, pp. 451{454 vol.6, May 1994. [40] C. Rasche and R. J. Douglas, \Forward-and backpropagation in a silicon dendrite," IEEE Transactions on Neural Networks, vol. 12, no. 2, pp. 386{393, 2001. [41] Y. Wang and S. C. Liu, \A two-dimensional congurable active silicon dendritic neuron array," IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 58, pp. 2159{2171, Sept 2011. 74 [42] N. R. Sibson, J. Shen, G. F. Mason, D. L. Rothman, K. L. Behar, and R. G. Shulman, \Functional energy metabolism: In vivo 13c-nmr spectroscopy evidence for coupling of cerebral glucose consumption and gl utamatergic neuronal activity," Developmental neuroscience, vol. 20, no. 4-5, pp. 321{330, 1998. [43] F. Hyder, A. B. Patel, A. Gjedde, D. L. Rothman, K. L. Behar, and R. G. Shulman, \Neuronal{ glial glucose oxidation and glutamatergic{gabaergic function," Journal of Cerebral Blood Flow & Metabolism, vol. 26, no. 7, pp. 865{877, 2006. [44] D. Attwell and S. B. Laughlin, \An energy budget for signaling in the grey matter of the brain," Journal of Cerebral Blood Flow & Metabolism, vol. 21, no. 10, pp. 1133{1145, 2001. [45] M. P. Parsons and M. Hirasawa, \Atp-sensitive potassium channel-mediated lactate eect on orexin neurons: implications for brain energetics during arousal," Journal of Neuroscience, vol. 30, no. 24, pp. 8061{8070, 2010. [46] V. Rangan, A. Ghosh, V. Aparin, and G. Cauwenberghs, \A subthreshold avlsi implementation of the izhikevich simple neuron model," in 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, pp. 4164{4167, Aug 2010. [47] A. S. Demirkol and S. Ozoguz, \A low power vlsi implementation of the izhikevich neuron model," in 2011 IEEE 9th International New Circuits and systems conference, pp. 169{172, June 2011. [48] G. Indiveri, B. Linares-Barranco, T. Hamilton, A. van Schaik, R. Etienne-Cummings, T. Delbruck, S.-C. Liu, P. Dudek, P. H a iger, S. Renaud, J. Schemmel, G. Cauwenberghs, J. Arthur, K. Hynna, F. Folowosele, S. SA IGHI, T. Serrano-Gotarredona, J. Wijekoon, Y. Wang, and K. Boahen, \Neuromorphic silicon neuron circuits," Frontiers in Neuroscience, vol. 5, p. 73, 2011. [49] \Predictive transistor model." [50] A. Wang, B. H. Calhoun, and A. P. Chandrakasan, Sub-threshold design for ultra low-power systems, vol. 95. Springer, 2006. 75
Abstract (if available)
Abstract
Neuromorphic designs modeling how the human brain processes massive amounts of information in parallel through the integration of memory and processing elements have gained significant attention being used in applications such as cognitive sensory applications and biomimetic approaches used for bio-inspired computing. Analog neuromorphic circuit design approach utilizes physical characteristics of silicon devices mimicking neurons exhibiting behaviors close to that of biological neurons. Such neurons are computationally more powerful than those ignoring biological complexities while consuming fewer hardware resources. One of the highest hurdles in constructing neuromorphic circuits is to reduce power consumption while not oversimplifying neural models. ❧ The main focus of this thesis is designing circuits to capture biological neural complexities while maintaining ultra-low power consumption. The main goal of the BioRC project in the last decade has been designing circuits that mimic the cognition, learning, and memory capabilities of the brain while power efficiency had always been a concern. Dendritic arbor where Post Synaptic Potentials are processed has been the most power-hungry component of the neurons due to non-linear computations happening over time and space. We have re-designed this module deploying a switched capacitor circuit model where once a pre-neural activity is detected, the integration occurs and the power is gated while in the idle mode. Also, we modeled astrocyte's metabolic role that has a significant effect on the brain’s energy substrate regulation. We introduced required modifications to the BioRC neuron model to include Astro-neuron Lactate Shuttle property such that the frequency of spiking gets tuned by astrocytes based on inter-domain synaptic activities and evaluated how the Astro-neural Lactate Shuttle affects the power consumption of the domains.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Modeling astrocyte-neural interactions in CMOS neuromorphic circuits
PDF
Dynamic neuronal encoding in neuromorphic circuits
PDF
Astrocyte-mediated plasticity and repair in CMOS neuromorphic circuits
PDF
Circuit design with nano electronic devices for biomimetic neuromorphic systems
PDF
A biomimetic approach to non-linear signal processing in ultra low power analog circuits
PDF
Dendritic computation and plasticity in neuromorphic circuits
PDF
Plasticity in CMOS neuromorphic circuits
PDF
Emulating variability in the behavior of artificial central neurons
PDF
Power efficient design of SRAM arrays and optimal design of signal and power distribution networks in VLSI circuits
PDF
Variation-aware circuit and chip level power optimization in digital VLSI systems
PDF
Energy efficient design and provisioning of hardware resources in modern computing systems
PDF
In-situ digital power measurement technique using circuit analysis
PDF
Neuromorphic motion sensing circuits in a silicon retina
PDF
Analog and mixed-signal parameter synthesis using machine learning and time-based circuit architectures
PDF
Biosensing and biomimetic electronics
PDF
Subthreshold circuit design for ultra-low-power sensors
PDF
Towards efficient edge intelligence with in-sensor and neuromorphic computing: algorithm-hardware co-design
PDF
Power optimization of asynchronous pipelines using conditioning and reconditioning based on a three-valued logic model
PDF
Energy aware integrated circuits for communication and biomedical applications
PDF
An asynchronous resilient circuit template and automated design flow
Asset Metadata
Creator
Mamdouh, Pezhman
(author)
Core Title
Power-efficient biomimetic neural circuits
School
Viterbi School of Engineering
Degree
Doctor of Philosophy
Degree Program
Electrical Engineering
Publication Date
10/28/2019
Defense Date
08/20/2019
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
astrocytes,astro-neuron lactate shuttle,biomimetic neural circuits,dendritic computation,low power neural circuits,neuromorphic circuits,OAI-PMH Harvest
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Parker, Alice (
committee chair
), Beerel, Peter (
committee member
), Nakano, Aiichiro (
committee member
)
Creator Email
mamdouh@usc.edu,pezhman.mamdouh@gmail.com
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c89-227574
Unique identifier
UC11674710
Identifier
etd-MamdouhPez-7885.pdf (filename),usctheses-c89-227574 (legacy record id)
Legacy Identifier
etd-MamdouhPez-7885.pdf
Dmrecord
227574
Document Type
Dissertation
Rights
Mamdouh, Pezhman
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
astrocytes
astro-neuron lactate shuttle
biomimetic neural circuits
dendritic computation
low power neural circuits
neuromorphic circuits