University of Southern California Dissertations and Theses

Computational tools for large-scale analysis of brain function and structure

(USC Thesis Other)

Computational tools for large-scale analysis of brain function and structure

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Computational Tools
for
Large-Scale Analysis
of
Brain Function and Structure
by
Anna Nadtochiy
A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(COMPUTATIONAL BIOLOGY AND BIOINFORMATICS)
December 2023
Copyright 2024 Anna Nadtochiy

Table of Contents
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Chapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Large-Scale Studies of Synaptic Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Large-Scale Studies of Neuronal Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Biological Foundations for Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.1 Neurons and Synapses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.2 Excitation and Inhibition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.3 Diversity of Inhibitory Neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.4 Functional and Structural Synaptic Plasticity . . . . . . . . . . . . . . . . . . . . . 8
1.3.5 Model Organisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3.6 Advantages of zebrafish in large-scale brain studies . . . . . . . . . . . . . . . . . . 10
1.3.7 Classical Conditioning and Fear Conditioning in Memory Research . . . . . . . . . 10
1.3.8 Numerosity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.4 Thesis Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Chapter 2: Detection of Regional Changes in 3D Synaptic Distributions Using Support Vector
Machines and Leave-One-Out Technique . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.1 Synopsis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2 Design and Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2.1 SVM-Based Analysis of Synaptic Point Clouds . . . . . . . . . . . . . . . . . . . . . 18
2.3 Usage Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3.1 Changes in Synapse Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3.2 Changes in Synaptic Strength . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.4 Application: Synaptic Changes in Excitatory Synapses Following Fear Conditioning . . . . 24
2.4.1 Experimental Design and Data Collection . . . . . . . . . . . . . . . . . . . . . . . 25
2.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.5 Limitations and Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.5.1 Limitations in Synapse Validation and Assumptions . . . . . . . . . . . . . . . . . 46
2.5.2 Synapse Pairing and Intensity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.5.3 Refining Analytical Approach for Brain Mapping . . . . . . . . . . . . . . . . . . . 47
2.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
2.7 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.8 Code Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
ii

Chapter 3: Flexible Computational Pipeline for Cell-Type-Aware Analysis of Synaptic Changes . . 51
3.1 Synopsis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.2 Design and Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.2.1 Modular Tasks and Configurable Pipeline . . . . . . . . . . . . . . . . . . . . . . . 54
3.2.2 Pipeline Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.3 Usage Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.3.1 Cell-Type Specific Synaptic Connectivity . . . . . . . . . . . . . . . . . . . . . . . . 58
3.3.2 Time-Dependent Synaptic Connectivity Patterns . . . . . . . . . . . . . . . . . . . 60
3.3.3 Time-Dependent & Cell-Type Specific Synaptic Connectivity Patterns . . . . . . . 62
3.4 Application:
Synaptic Changes in Inhibitory Synapses Following Fear Conditioning . . . . . . . . . . . 63
3.4.1 Experimental Design and Data Collection . . . . . . . . . . . . . . . . . . . . . . . 63
3.4.2 Configuring the Pipeline for Cell-Type-Specific Synaptic Change Detection . . . . 67
3.4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.5 Limitations and Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.5.1 Training Data Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.5.2 Introducing Human-in-the-Loop for Quality Control . . . . . . . . . . . . . . . . . 71
3.5.3 Integrating Model Training Control and Continuous Updates . . . . . . . . . . . . 72
3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.7 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.8 Code Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Chapter 4: VoDEx: a Python Library for Time Annotation and Management of Volumetric
Functional Imaging Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.1 Synopsis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.2 Design and Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.2.1 Modules and Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.2.2 Pipeline: Data Mapping and Querying . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.3 Usage Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.3.1 High-Throughput assays on Calcium Data Processing . . . . . . . . . . . . . . . . 82
4.3.2 Individual Neuron Activity on Calcium Data Processing . . . . . . . . . . . . . . . 82
4.3.3 Transitions between behaviors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.3.4 Code examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.4 Application: The Study of Numerosity in Zebrafish Larvae . . . . . . . . . . . . . . . . . . 86
4.4.1 Overview of the Experiment and Data Acquisition . . . . . . . . . . . . . . . . . . 86
4.4.2 Dataset Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.4.3 Visual Numerosity Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.4.4 Data Processing and Analysis of a Single Fish Sample . . . . . . . . . . . . . . . . . 89
4.4.5 VoDEx utility for Numerosity Study Data Processing . . . . . . . . . . . . . . . . . 90
4.4.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
4.5 Limitations and Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
4.5.1 Expansion for Image Format Support . . . . . . . . . . . . . . . . . . . . . . . . . . 98
4.5.2 Enabling User-Created Database Queries . . . . . . . . . . . . . . . . . . . . . . . . 98
4.5.3 Enhancing Experiment Sharing Capabilities . . . . . . . . . . . . . . . . . . . . . . 99
4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
4.7 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
4.8 Code Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
iii

Chapter 5: Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
iv

List of Figures
2.1 An example of a separable problem in a 2 dimensional space. . . . . . . . . . . . . . . . . . 18
2.2 Tail Flick Conditioning (TFC), a fear conditioning paradigm for larval zebrafish. . . . . . . 26
2.3 Imaging excitatory synapses in larval zebrafish. . . . . . . . . . . . . . . . . . . . . . . . . 27
2.4 Characterization of PSD-95.FingR expression in 14-16 dpf zebrafish larvae. . . . . . . . . . 28
2.5 An algorithm for semi-automated identification of synapses from raw SPIM data. . . . . . 30
2.6 Detecting excitatory synapse change following TFC in individual fish. . . . . . . . . . . . . 32
2.7 Feature based registration onto a template zebrafish brain. . . . . . . . . . . . . . . . . . . 33
2.8 Absence of Region-Wide Synaptic Changes following TFC in larval zebrafish. . . . . . . . 37
2.9 Synapse changes with TFC in larval zebrafish. . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.10 Regional differences in synapse formation in the pallium of learner fish. . . . . . . . . . . . 40
2.11 Gained and lost vs. subthreshold synapses. . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.12 Neuronal activation within the anterolateral pallium in response to the CS in learner fish
and to the US in naïve fish. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.13 Anatomical correlation of regions of increased neuronal activity and synaptic gain
following TFC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.1 Schematic representation illustrating the data flow through individual tasks. . . . . . . . . 56
3.2 Workflow for Cell-Type Specific Synaptic Connectivity Analysis. . . . . . . . . . . . . . . . 59
3.3 Workflow for Time-Dependent Synaptic Connectivity Analysis. . . . . . . . . . . . . . . . 60
3.4 Schematic Overview of the Fear Conditioning Experiment and Analysis Focusing on
Inhibitory Synapses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
v

3.5 Imaging inhibitory synapses in larval zebrafish. . . . . . . . . . . . . . . . . . . . . . . . . 65
3.6 Immunofluorescence staining allows to distinguish between glia cells and inhibitory and
excitatory neurons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.7 Workflow for Cell-Type Specific Synaptic Change Detection. . . . . . . . . . . . . . . . . . 69
3.8 Cell-Type-Specific Synaptic Changes Following Fear Conditioning Identified by the Pipeline. 69
4.1 Vodex Experiment Database Schema. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.2 Illustration of the VoDEx pipeline applied to a Toy Dataset. . . . . . . . . . . . . . . . . . . 81
4.3 Visual stimuli used in the numerosity stimuli presentation. . . . . . . . . . . . . . . . . . . 88
4.4 Pseudo-random stimulus cycle for numerosity response isolation in zebrafish larvae. . . . 89
4.5 Segmentation of numerosity-tuned cells. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.6 Using Whole-brain functional imaging to find neural substrate of zebrafish numerosity
capability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
vi

Abstract
The last decade has witnessed a transformative shift in the field of neuroscience, largely fueled by advancements in fluorescence imaging tools and labeling techniques. This shift toward studying synaptic
changes at the level of individual synapses and neural activity at single-cell resolution across the entire
brain has generated increasingly complex data sets. This thesis introduces three contributions that address the methodological challenges accompanying these advancements. The first contribution is an analytical framework employing Support Vector Machines (SVMs), offering a paradigmatic departure from
traditional aggregate synaptic studies. This approach allows for the detection of subregional synaptic
changes that would otherwise remain obscure. The second contribution is a computational pipeline explicitly designed for the scalable analysis of synaptic changes. It is engineered to accommodate evolving
computational methods and unique experimental paradigms emerging in this growing field. Lastly, the
thesis describes VoDEx, a specialized tool for managing complex 3D data sets generated by functional
volumetric imaging. VoDEx streamlines the integration of experimental timelines and behavioral events
with acquired imaging data, thereby improving the accuracy and reproducibility of functional analyses.
Together, these computational tools represent significant advancements in the large-scale analysis of brain
function and structure, offering the adaptability and precision required for future neuroscientific research.
vii

Chapter 1
Introduction
The forthcoming introductory sections, Large-Scale Studies of Synaptic Changes and Large-Scale Studies
of Neuronal Activity lay the groundwork for the overarching theme of this thesis, "Computational Tools
for Large-Scale Analysis of Brain Function and Structure". In the context of this work, "function" refers
specifically to neuronal activity, and "structure" refers to synaptic organization, and by "large scale," we are
addressing studies that scrutinize tens of thousands of neurons or synapses in a single experimental setup.
Building on advances in imaging and labeling techniques, both sections set the stage for the computational
approaches presented later in this thesis.
1.1 Large-Scale Studies of Synaptic Changes
Recent advances in imaging and labeling techniques have expanded our ability to record synapses across
extensive brain regions, moving beyond individual cells and small volumes. In 2018, Zhu et al. conducted a
remarkable study (Zhu et al., 2018), constructing a comprehensive three-dimensional synapse map, at subsynaptic resolution, throughout the entire mouse brain which revealed diverse excitatory synapse types
distributed throughout various brain regions. Subsequent studies expanded on this approach to construct
the Mouse Lifespan Synaptome Atlas (Cizeron et al., 2020) that maps how the composition of synapses
in the brain changes from development to adulthood and old age. However, a significant limitation of
1

these studies is the reliance on imaging methods that require the animal to be sacrificed, which limits their
utility for studies of synaptic changes caused by experiences and memories, as they can not track the fate
of individual synapses over time.
Understanding the details of changes related to experience, learning, and memory requires longitudinal
studies within the same subject, which places limitations on imaging and labeling techniques. Imaging
methods must not disrupt the normal flow of the processes under study. For example, high phototoxicity
(harmful effects due to intense exposure to light) or prolonged restraint could disrupt the learning process.
Similarly, labeling techniques must be carefully designed to not alter the normal functioning of synapses.
During the past 15 years, substantial progress has been made in both these domains, with the last five
years witnessing a notable upsurge in extensive synaptic studies.
Fluorescence microscopy is a promising imaging technique that can facilitate large-scale longitudinal
imaging of synapses in living subjects. Traditional confocal imaging offers high resolution to resolve
individual synapses and can be used when labels are sufficiently stable (not prone to photobleaching),
while recently emerging light-sheet microscopy (Huisken et al., 2004; Keller et al., 2008) can be used for
less stable labels or applications that require low phototoxicity.
The two main challenges in synaptic labeling are maintaining normal synaptic function and resolving
individual synapses in the images. Transgenic labeling, a prevalent technique involving the genetic fusion
of target proteins with fluorescent markers, addresses these issues (Bosch et al., 2014; J. L. Chen et al., 2012;
Meyer et al., 2014; Villa et al., 2016). Notably, specific transgenic lines show minimal impact on synaptic
function, with protein levels comparable to endogenous expressions (Callahan et al., 2019). Additionally,
a modified gene trap method can create transgenic lines that express proteins at natural levels (Trinh
et al., 2011). However, concerns remain about possible overexpression that disrupts synaptic function
(El-Husseini et al., 2000). Furthermore, dense labeling of all synapses in a region by transgenic methods
complicates the identification of individual synapses.
2

In recent years, a variety of specialized genetic tools have been developed to map functional synapses
and label both excitatory and inhibitory types without altering protein levels. Fibronectin intrabodies
generated with mRNA display (FingRs) have been developed to target the principal structural proteins of
excitatory and inhibitory synapses, specifically PSD95 and Gephyrin (Gross et al., 2013). These FingRs bind
to their respective target proteins and function as intracellular antibodies in both fixed and live neurons.
When these FingRs are fused to GFP, they enable visualization of excitatory or inhibitory synapses. To
minimize background signal, a transcriptional control system was developed, ensuring the expression level
of FingRs closely matches that of the target protein. This system incorporates the KRAB(A) transcription
repressor fused to a Zinc-finger (ZF) binding domain (Margolin et al., 1994; Witzgall et al., 1994), attached
to the FingR, and ZF binding sites are inserted upstream of the FingR expression promoter. In this setup,
when FingRs bind to target proteins in dendrites, the ZF-KRAB(A) complex is hindered from entering
the nucleus to turn off transcription, allowing continuous FingR expression as long as unbound target
proteins are present. Conversely, if all target proteins are bound, unbound ZF-KRAB(A) moves to the
nucleus, halting transcription. This dynamic ensures that the FingR’s expression level is tightly regulated
to match its target, enhancing the clarity of synaptic visualization and allowing the intensity of FingR
labels to serve as a proxy for synaptic strength. This approach has been effectively used in various studies
for investigating synaptic modifications (Gross et al., 2013; Kannan et al., 2016; Kwon et al., 2018; Sinnen
et al., 2017; Walker et al., 2017).
GFP Reconstitution Across Synaptic Partners (GRASP) is another popular technique for synaptic labeling (Feinberg et al., 2008; J. Kim et al., 2011), utilizing two non-fluorescent fragments of Green Fluorescent Protein (GFP) expressed in separate neurons, which reassemble into a functional GFP molecule
at the synapse. This method has been refined over time, with notable advancements like the enhanced
GRASP (eGRASP) technique and dual-eGRASP, which increases signal intensity, allows for the reconstitution of cyan or yellow fluorescent proteins, and expands this functionality, enabling visualization of two
3

different presynaptic populations converging on a single postsynaptic neuron (Choi et al. [Choi], 2018).
Importantly, these techniques do not alter synaptic strength, ensuring that observed synaptic dynamics
are physiological rather than artifacts of the labeling process. These developments have significantly enhanced the ability to study and visualize synaptic connections in live organisms, offering a detailed and
dynamic view of synaptic formations and interactions. Additional labels are reviewed in (J.-E. Choi et al.,
2020).
The accurate detection of both large-scale and local synaptic changes, along with the development
of innovative methods to interpret them, is critically important in neuroscience. Despite the surge in
studies that use fluorescence imaging to observe synaptic changes, the primary focus of current analysis
tools remains on detecting or segmenting synapses in images (Iascone et al., 2020; Kulikov et al., 2019),
leaving broader challenges unaddressed. These include not only identifying synapses but also classifying
and quantifying their changes, such as synapse formation and elimination, or changes in synaptic strength.
Moreover, it is essential to analyze these changes in the context of the specific neuron types involved and to
identify the dominant mechanisms operating across large brain regions, which may encompass hundreds
of neurons and thousands of synapses.
To address the limitations in existing analytical tools, this thesis contributes to understanding and
interpreting large-scale synaptic changes. Chapter 2, Detection of Regional Changes in 3D Synaptic Distributions Using Support Vector Machines and Leave-One-Out Technique, introduces a novel method to
detect large regional changes in the 3D distribution of synapses at different time points. This method
opens avenues for a better understanding of learning, memory, and other cognitive processes. Chapter
3, Flexible Computational Pipeline for Cell-Type-Aware Analysis of Synaptic Changes, presents a flexible
analytical pipeline specifically tailored to detect changes in synapses from fluorescent microscopy data.
This pipeline is designed to be adaptable, recognizing the unique requirements of individual experiments
in a rapidly evolving field. By providing these innovative tools and approaches, these two projects aim
4

to fill the current gap in methodologies for large-scale synaptic analysis and advance our capabilities in
neuroscience research.
1.2 Large-Scale Studies of Neuronal Activity
The development of advanced fluorescent imaging tools and labeling techniques in the last decade has
been equally transformative in the field of neuronal activity as in synaptic studies. Parallel to the shift
toward imaging individual synapses on a large scale, the studies of neural activity experienced a similar
transition to studying neuronal activity at the single-cell resolution across the whole brain, generating
increasingly complex 3D datasets. This transition was primarily driven by the development of bright
genetically encoded calcium indicators (GECIs) such as GCaMP (Baird et al., 1999; T.-W. Chen et al., 2013;
Miyawaki et al., 1997; Nakai et al., 2001). GECIs, including GCaMP, function by changing brightness in
response to variations in intracellular calcium levels. Calcium concentration within neurons increases
when the neurons are active, making calcium a reliable indicator of neuronal activity. Large number of
GECI studies have employed two-photon microscopy, a technique known for its deep tissue penetration
and reduced phototoxicity, allowing for detailed imaging of neuronal activity within intact tissue (Denk et
al., 1990). Coupled with fluorescent imaging techniques such as light sheet imaging (Huisken et al., 2004;
Keller et al., 2008) and light field microscopy (Cong et al., 2017; Levoy et al., 2006; Madaan et al., 2021;
Prevedel et al., 2014; Truong et al., 2020), which offer fast imaging speeds, high resolution, and minimal
phototoxicity, GECIs have facilitated large-scale 3D studies of neuronal dynamics in model organisms such
as zebrafish at single-cell resolution (Ahrens et al., 2012, 2013; Dunn et al., 2016; Portugues et al., 2014).
With the continued improvement of imaging techniques, the field is starting to tackle complex behavioral
assays. Current technologies now allow for single-cell whole-brain imaging of freely behaving animals or
animals engaged in a virtual reality environment (Marques et al., 2019; Yang et al., 2022). This results in a
substantial increase in both the size and complexity of the generated datasets.
5

Considerable efforts are being made to adapt existing analytical tools to these new forms of data.
The primary objective is signal extraction, specifically isolating the signal of individual neurons from the
overall image. Many algorithms and toolboxes originally designed for simpler two-dimensional calcium
imaging datasets (Giovannucci et al., 2019a; Pachitariu et al., 2016) are being adapted to accommodate the
new 3D datasets. However, the task of adapting these tools to 3D remains nontrivial, especially given the
large amounts of data. This amplifies the need for robust data handling solutions tailored for 3D datasets.
The growing complexity of experiments and the 3D nature of new datasets require robust data management and annotation approaches. Currently, no user-friendly solutions are designed to manage these
complexities, leaving a gap in the existing analytical toolset. Emerging standardized formats such as Neurodata Without Borders (Rübel et al., 2019) and Brain Imaging Data Structure (Gorgolewski et al., 2016) are
steps towards bridging this gap, enabling the storage of annotations alongside imaging data and providing
tools for their comprehensive processing. However, these formats have their limitations, particularly in
terms of user-friendliness. Their inherent complexity and the substantial time investment required for
their implementation can be daunting for both novice and experienced users. Significant efforts are being
made to make these formats more accessible to a broader audience, but adopting these standards often
requires substantial commitment. As a result, many researchers still rely on manual data management and
annotation, or turn to custom scripting solutions, approaches that are not scalable and prone to human
error.
To meet these challenges, Chapter 4 of this thesis, VoDEx: a Python Library for Time Annotation and
Management of Volumetric Functional Imaging Data, describes an open-source Python package specifically designed to address the issue of handling volumetric data and complex annotations. This package
offers a user-friendly and reliable solution that aims to fill the current gap in analytical tools, enabling
researchers to process and interpret increasingly complex and large-scale datasets with confidence. Additionally, it provides capabilities for adapting existing 2D tools to the requirements of 3D data.
6

1.3 Biological Foundations for Applications
1.3.1 Neurons and Synapses
Neurons are the fundamental building blocks of the nervous system, responsible for transmitting and processing information through electrical and chemical signals. They consist of a soma (cell body), dendrites
that receive signals from other neurons, and an axon that transmits signals to other neurons or effector
cells. Synapses are specialized connections between neurons where information is transmitted from one
neuron to another. The neuron that transmits the signal is called the presynaptic neuron, while the neuron
that receives the signal is called the postsynaptic neuron.
1.3.2 Excitation and Inhibition
Neurons can be excitatory, increasing the probability of activation of the postsynaptic neuron, or inhibitory, suppressing this probability. Excitation and inhibition are two fundamental processes in neural
circuits that regulate the flow of information (Caroni et al., 2012). Most neurons in the brain are excitatory;
they propagate activity through neural circuits. Inhibitory neurons play a modulatory role: stabilizing
neuronal networks, preventing excessive excitation, and synchronizing the firing patterns of a large collection of cells, essential for rhythmic activities tied to cognition and behavior (Buzsáki and Wang, 2012;
Klausberger and Somogyi, 2008). A balance between excitation and inhibition is crucial to maintaining the
proper functioning of neural circuits and ensuring the stability of neuronal activity (Flores and Mandez,
2014).
1.3.3 Diversity of Inhibitory Neurons
Inhibitory neurons play a critical role in a variety of physiological processes, including learning and memory (Barron, 2021). They display a large diversity of morphologies, activity patterns, and chemical characteristics (Ascoli et al., 2008; Kubota, 2014; Markram et al., 2004; Tasic et al., 2016; Zeisel et al., 2015).
7

The three main subtypes are parvalbumin neurons (PV), somatostatin neurons (SOM), and vasoactive intestinal polypeptide neurons (VIP), classified according to the characteristic proteins they express (Rudy
et al., 2010). Each type plays a unique role in modulating neuronal activity (Fu et al., 2014; Gentet et al.,
2012; S.-H. Lee and Dan, 2012; S. Lee et al., 2013; Lovett-Barron et al., 2012; Pi et al., 2013; Wilson et al.,
2012; Zhang et al., 2014). PV neurons offer fast inhibition of neighboring excitatory neurons (Rudy and
McBain, 2001). SOM neurons provide a delayed response to stimuli (Ma et al., 2010) and primarily target
distal dendrites (Markram et al., 2004). VIP neurons inhibit other inhibitory neurons, ultimately reducing
the overall inhibition of excitatory neurons.
Some inhibitory neurons exclusively target dendrites, while others focus on the perisomatic region,
the region around the soma of the neuron (Freund and Katona, 2007; Maccaferri, 2004). Such perisomatictargeting inhibitory neurons are able to shape the input-output relationship of their target neurons.
1.3.4 Functional and Structural Synaptic Plasticity
Synaptic plasticity refers to the ability of synapses to undergo changes in their strength and structure in
response to activity and experience. It plays a critical role in learning and memory processes. Functional
plasticity refers to changes in synaptic connection strength, such as long-term potentiation (LTP) or longterm depression (LTD), which are associated with strengthening or weakening of synaptic transmission,
respectively (Bliss and Lømo, 1973; Ito, 1989; Lømo, 1966; Malenka and Bear, 2004; Siegelbaum and Kandel, 1991). Structural plasticity, on the other hand, involves changes in the physical structure of synapses,
including the formation, elimination, and remodeling of synaptic connections. Both functional and structural plasticity contribute to adaptive changes in neural circuits that underlie learning and memory (Fauth
and Tetzlaff, 2016; Perez-Alvarez et al., 2014)
Synaptic plasticity in excitatory synapses has been shown to follow the Hebbian rule: the strengthening of the synapse occurs as a result of coactivation of the pre- and postsynaptic neurons, often paraphrased
8

as "fire together - wire together" (Hebb, 1949). These changes are then followed by homeostatic plasticity,
a mechanism that aims to maintain a steady level of input to neurons and maintain the stability of neural
circuits (H.-K. Lee and Kirkwood, 2019). Structural plasticity is less studied but can be similarly categorized into Hebbian structural plasticity, which increases or decreases the number of synapses based on
levels of neuronal activity, and homeostatic structural plasticity, which counterbalances these changes by
adding and removing synapses (Fauth and Tetzlaff, 2016). The relationship between synaptic and structural plasticity, as well as the interactions between Hebbian and homeostatic plasticity, have not been fully
studied. The mechanisms governing inhibitory synapses are even more complex and less studied. While
the Hebian rule generally does not apply to inhibitory synapses, synaptic plasticity in inhibitory synapses
has been found to depend on activation of the postsynaptic neuron (Flores and Mandez, 2014).
1.3.5 Model Organisms
Model organisms play a vital role in advancing our understanding of biological processes, including the
complexities of the brain. They provide valuable tools for studying both the structure and function of neuronal circuits (Friedrich et al., 2013). Conservation of genetic pathways and cellular physiology between
species, especially across vertebrates, including humans, allows researchers to investigate the brain, study
disease mechanisms, and test potential therapies in an ethical and controlled manner.
A notable advantage of using model organisms, particularly those with smaller brains, is the ability
to record a substantial proportion of the brain at single-cell or single-synapse resolution. This allows one
to study brain activity and synaptic changes on a large scale without losing details, minimizing the likelihood of overlooking critical insights. Recent advances in optical imaging techniques have significantly
improved both temporal and spatial resolution and coverage, sparking the expectation that the integration
of large-scale imaging with the compact brains of model organisms, along with tools such as optogenetics,
9

computational methodologies, and connectomics, may expedite the discovery of overarching principles
governing the brain (Ahrens and Engert, 2015).
1.3.6 Advantages of zebrafish in large-scale brain studies
The zebrafish has shown great promise as an ideal model organism for large-scale brain studies. Zebrafish
offer several advantages, including low cost, transparency, and accessibility to sophisticated genetic manipulations. These characteristics make them ideal for quantitative analyses of circuit structure and function
(Friedrich et al., 2013).
Zebrafish, because of their optical transparency, offer a unique benefit for fluorescence imaging studies. Zebrafish enable imaging of live and behaving animals at the level of individual neurons and down to
individual synapses. Additionally, zebrafish permits long-term imaging, enabling the analysis of brain activity involving nearly all of its 100,000 neurons (Ahrens et al., 2012) and longitudinal imaging of individual
synapses (Appelbaum et al., 2010) during a complete sleep/wake cycle. Such studies provide insights that
cannot be obtained by comparing brain activity and synaptic changes in different samples, and no other
animal model currently offers opportunities to study such phenomena on a brain-wide scale.
1.3.7 Classical Conditioning and Fear Conditioning in Memory Research
Classical conditioning is a fundamental form of associative learning in which an initially neutral conditioned stimulus (CS) becomes effective in producing a response when paired with a strong unconditioned
stimulus (US). This phenomenon was first systematically studied by Ivan Pavlov in the early 1900s through
his experiments in which dogs were conditioned to salivate in response to the ringing of a bell (CS), previously paired with food (US).
Fear conditioning, a robust model for studying associative learning and memory formation, is a specific
application of this principle. In fear conditioning, a neutral CS, such as a tone, is paired with an aversive
10

US, like an electric foot-shock in rodents. This pairing forms a fear memory, which makes the CS capable
of eliciting a fear response. The fear conditioning paradigm involves three phases: habituation, acquisition
training, and testing. Habituation consists of repeated presentation of CS alone to minimize the fear response to the CS. During the training, the CS predicts an aversive event (US), linking them and forming a
memory. Testing assesses the strength of fear memory by observing the behavior or physiological changes
in response to CS after the training.
Fear conditioning serves as a valuable tool for studying the underlying neurobiology of learning and
memory, offering a controlled framework for investigating neural circuits. Although the synaptic changes
underlying fear conditioning have been extensively studied, much remains to be fully understood. Many
studies focus on the amygdala brain region due to its critical role in fear conditioning and emotional
learning in general (Davis, 1994). For instance, excitatory synapses in the amygdala play a critical role in
the consolidation of fear memory, with long-term potentiation (LTP) at these synapses being necessary
for the establishment of fear memory (Kandel et al., 2021). Inhibitory circuits also play a role in both
fear-memory acquisition and extinction (Krabbe et al., 2018; Singh and Topolnik, 2023). In the lateral
subdivision of the central amygdala (CeL), fear conditioning causes increased synaptic potentiation onto
somatostatin-positive inhibitory neurons, and preventing this potentiation has been shown to impair the
formation of fear memories (Li et al., 2013). Finally, extinction of fear memories might require two distinct
modifications of synapses: reversing previously acquired synaptic changes in excitatory synapses and
inducing plasticity in inhibitory synapses. Both modifications aim to suppress conditioned fear responses
but through different mechanisms. This implies that extinction involves both the reversal of pre-acquired
memories, akin to ’unlearning,’ and the formation of new synaptic connections, reflecting ’new learning’
about contingency (C. Lee et al., 2023; Maren, 2015).
Teleost fish, including species like zebrafish, goldfish, sticklebacks, medaka, and guppies, demonstrate
reliable classical conditioning in a wide range of experiments, making them valuable models for studying
11

fear memory formation. The pallium in these species, functionally analogous to the mammalian amygdala
and hippocampus, is critical for emotional learning and memory, as supported by lesion studies (Salas et
al., 2006). Although various teleost models are employed in neuroscience research, zebrafish stand out due
to their optical transparency. This unique feature of zebrafish allows for detailed, real-time observation of
neural activity and synaptic alterations in living, behaving fish during fear conditioning experiments.
1.3.8 Numerosity
The study of numerosity, or the ability to perceive and discriminate between different quantities, serves as
a fundamental aspect of cognitive neuroscience. This skill has ecological relevance across various species,
aiding in essential behaviors such as foraging, social interactions, and predator avoidance. In many species,
even those that have evolved independently, numerosity is represented by the same mechanism, an approximate number system (ANS), which underscores its biological significance and adaptive value (Nieder,
2018). Despite its widespread, experimentally-verified occurrence in the animal kingdom, from mammals
and birds to reptiles and fish, the neural mechanisms underlying numerosity remain largely unexplored,
especially in non-mammalian species (Dehaene and Changeux, 1993; Hyde and Spelke, 2011; Katz et al.,
2016).
Recent findings in zebrafish research (Messina et al., 2022) have revealed that the telencephalon and
the thalamus play crucial roles in numerosity discrimination in this species. This discovery is particularly
significant, as it aligns with similar findings in more complex animals (Kobylkov et al., 2022), including
humans and non-human primates (Nieder, 2020), where homologous brain regions are involved in numerosity processing. This opens the possibility to use zebrafish as an animal model to study the neural
correlates of numerosity in vertebrates.
12

Studying the neural basis of numerosity is essential for understanding numerical cognition. A deeper
understanding of how the brain represents and processes numerical information would enable the investigation of the relationship between numerosity and continuous magnitudes, the validation of specialized
mechanisms for numerosity perception, the development of computational models of numerical cognition, and the improvement of behavioral essays of numerosity perception. Moreover, understanding the
neural basis of numerosity has potential applications in addressing cognitive deficits, informing educational strategies, and even in the development of artificial intelligence systems designed to mimic natural
numerical cognition.
1.4 Thesis Overview
This thesis addresses critical aspects of large-scale analysis of brain function and structure. Chapter 1,
Introduction, lays the groundwork for the overarching themes of this thesis, and provides an introduction to the concepts of large-scale studies in neuroscience. It highlights recent advances in imaging and
labeling techniques that have paved the way for the computational approaches presented in subsequent
chapters and provides the necessary biological background. Chapter 2, Detection of Regional Changes
in 3D Synaptic Distributions Using Support Vector Machines and Leave-One-Out Technique, describes an
innovative technique for detecting regional changes in 3D synaptic distributions, while Chapter 3, Flexible
Computational Pipeline for Cell-Type-Aware Analysis of Synaptic Changes, presents a flexible computational pipeline to detect and analyze synaptic changes. Finally, Chapter 4, VoDEx: a Python Library for
Time Annotation and Management of Volumetric Functional Imaging Data, provides a Python library for
efficient time annotation and volumetric functional imaging data management. Collectively, these contributions enhance our understanding of brain function and structure, enabling more sophisticated and
comprehensive neuroscientific research.
13

Chapter 2
Detection of Regional Changes in 3D Synaptic Distributions Using
Support Vector Machines and Leave-One-Out Technique
2.1 Synopsis
The primary objective of this project is to characterize structural changes in excitatory synapses within
the pallium brain region of larval zebrafish following fear conditioning. Understanding the mechanisms
of memory formation is key to decoding how neural circuits undergo adaptive changes and associative
fear conditioning serves as a valuable model to investigate these alterations, particularly at the synaptic
level. Given the prevalence of excitatory neurons in the nervous system and their dominant role in the
transmission of information through neural networks, exploring changes in excitatory synapses is central
to grasping the fundamental mechanisms underlying memory formation. The findings from this study
have been published in Dempsey, W. P., Du, Z., Nadtochiy, A., Smith, C. D., Czajkowski, K., Andreev,
A., Robson, D. N., Li, J. M., Applebaum, S., Truong, T. V., Kesselman, C., Fraser, S. E., & Arnold, D. B.
(2022). Regional synapse gain and loss accompany memory formation in larval zebrafish. Proceedings of
the National Academy of Sciences, 119(3). https://doi.org/10.1073/pnas.2107661119
A distinctive aspect of this project is the unprecedented nature of the data collected. The larval zebrafish model enables the acquisition of 3D images that capture tens of thousands of synapses throughout
14

the entire pallium. Existing research methods typically use transcranial microscopy (Lai et al., 2012, 2018;
Xu et al., 2019), electron microscopy (Ostroff et al., 2010), and electrophysiological techniques (Clem and
Huganir, 2010; W. B. Kim and Cho, 2017; Nabavi et al., 2014; Namburi et al., 2015) to study synaptic changes
in small areas of the brain or focus on specific cells of interest, for example, comparing synaptic changes
that occur in engram cells versus nonengram cells (Choi, 2018; C. Lee et al., 2023; Roy et al., 2017; Ryan et
al., 2015). In contrast, we do not have preconceived knowledge of the specific regions or synaptic changes
to anticipate. This study allows synaptic changes to reveal the regions undergoing significant changes,
rather than relying on predefined areas or cells.
The unique challenges posed by the sparse synaptic labeling, coupled with the low success rate of our
fear conditioning protocol, required an innovative analytical approach. Our synaptic labeling technique
marks only about 0.5% of cells in each fish, making the evaluation of local synaptic changes unreliable.
For example, when using a sample size of 10 fish, many 10x10 µm super-voxels contain synapses from
only 2 or 3 fish. Furthermore, the cells are labeled at random, which means that the same super-voxel
across different fish samples could contain synapses from diverse neuronal populations. Consequently,
without a large sample size, the observed difference in local synaptic changes between different groups
could be largely attributed to labeling variability. Furthermore, our protocol has a low success rate in
inducing a fear response in fish (around 10%) indicating that collecting enough samples would require
years of consistent experimentation. These experimental constraints necessitated an innovative approach
to analytical methods.
In response to these challenges, we employed Support Vector Machine algorithms (SVM) as our primary analytical approach. To our knowledge, this marks the first application of SVM in the context of
synaptic data and synaptic changes. We also repurposed the leave-one-out (LOO) technique, traditionally
used for model evaluation, to assess the biological relevance of observed synaptic changes.
15

Using our novel approach, we identified key synaptic changes in the ventrolateral pallium related to
the formation of fear memories. Specifically, we observed a significant increase in the number of excitatory
synapses after fear conditioning. This finding contributes to the ongoing debate about the relative importance of synapse numbers versus synaptic strength. Moreover, neurons in this region that were activated
during fear training began responding to a previously neutral stimulus post-conditioning. These results
reinforce the idea that the observed synaptic changes in the ventrolateral pallium are directly tied to the
formation of associative fear memories.
The subsequent sections elaborate on the broader capabilities and applications of our novel analytical
approach. We discuss its design and implementation, its potential utility in various research contexts,
and, specifically, its application to the study of excitatory synaptic changes following the formation of fear
memories. It is important to note that the synaptic study presented here is part of a multi-lab collaboration
that covers diverse research aspects such as biological techniques, data management, and optical design.
2.2 Design and Implementation
In the design of our analytical method, we represent synaptic changes as three-dimensional point clouds.
Each point is classified into one of two categories, the nature of which depends on the specific synaptic
changes under investigation. For studies focusing on changes in the number of synapses, Class 1 represents
the emergence of a new synapse, while Class 2 indicates the loss of a synapse. Similarly, in studies that
examine synaptic strength, Class 1 signifies an increase in synaptic strength of an individual synapse
and Class 2 represents a decrease. These points act as spatial markers within the brain, forming threedimensional point clouds that identify the locations of synaptic changes. Our method is designed to detect
regions of significant synaptic changes in various experimental groups based on these point clouds.
Building on our analytical design, we formulate a straightforward hypothesis: If significant biological
changes occur, they should manifest as a consensus within experimental groups regarding the locations of
16

Class 1 and Class 2 synaptic changes. To investigate this hypothesis, we employ Support Vector Machines
(SVMs) combined with a leave-one-out strategy. SVMs with a linear kernel are particularly suited for
their robustness in dealing with overlapping point-cloud data, and the leave-one-out strategy is central
to our approach, as it allows us to examine synaptic changes systematically. Following the principles of
leave-one-out cross-validation, in which one sample is omitted at each iteration, we generate multiple
SVM hyperplanes, omitting one subject in the group at each step. We use the coordinates (x, y, z) of each
synapse as features for these hyperplanes. These hyperplanes effectively partition the three-dimensional
space, distinguishing regions associated with Class 1 and Class 2 synaptic changes. This is done based on
the remaining aggregate point cloud for a group after one subject has been omitted.
We propose two key conditions for assessing the biological relevance of synaptic changes: Consistency
in SVM Hyperplane Orientation and Consistency in Synaptic Changes within SVM-Identified Regions, as
outlined below. The rest of this section details the implementation of SVM and the adaptation of LOOCV
tailored to our specific application.
• Consistency in SVM Hyperplane Orientation: Within each experimental group, the orientation
of the SVM hyperplanes that separate the two classes should exhibit consistency across all leaveone-out samples. Such consistency would indicate that these hyperplanes effectively capture the
region of characteristic synaptic changes for the group as a whole.
• Consistency in Synaptic Changes within SVM-Identified Regions: Should significant biological changes occur, each leave-one-out sample must display similar proportions of Class 1 and Class
2 points on either side of the separating SVM hyperplane. This consistency signifies similar synaptic
changes within regions the SVM determines, further supporting the concept of biologically relevant
alterations.
17

2.2.1 SVM-Based Analysis of Synaptic Point Clouds
This section introduces the methodology for large-scale analysis of synaptic changes using Support Vector
Machines (SVMs). We specifically address the application of linear kernel SVMs to assess the biological
significance of synaptic changes within distinct groups, utilizing a leave-one-out (LOO) approach. This
section also outlines potential scenarios for inter-group comparison of synaptic changes, summarizing the
capabilities and limitations of our method.
2.2.1.a Overview of Support Vector Machine (SVM)
The Support Vector Machine (SVM) is one of the most successful methods in the field of supervised learning, developed by Cortes and Vapnik in 1995 (Cortes and Vapnik, 1995). It is designed for binary classification tasks, where the goal is to categorize data points into one of two classes. SVM operates by finding a
decision boundary or a hyperplane that best separates the data points of the two classes. This boundary is
chosen to maximize the margin between the classes, ensuring that it is as far as possible from the closest
data points of each class, known as support vectors, Figure 2.1 (Cortes and Vapnik, 1995).
Figure 2.1: An example of a separable problem in a 2 dimensional space. The support vectors, marked
with grey squares, define the margin of largest separation between the two classes.
Note. From “Support-Vector Networks,” by C. Cortes and V. Vapnik, 1995, Machine Leaming, 20, 273-297 (https:
//doi.org/10.1007/bf00994018).
18

2.2.1.b Linear Kernel SVM
Within the realm of SVMs, various kernel functions can be applied to transform the data into a suitable
format for classification (Brereton and Lloyd, 2010). One such implementation is the Linear Kernel SVM,
which is particularly effective in scenarios where the data exhibits high class overlap. The linear kernel
assumes that the data is linearly separable, which means that it can be divided by a hyperplane in the
original feature space. This assumption simplifies the computation and is particularly useful for reducing
the variance when dealing with overlapping data.
2.2.1.c SVM with LOO for Subject-Level Synaptic Changes Within Groups
In this section, we address the limitations of using cumulative point clouds with support vector machines
(SVMs) for assessing subject-level synaptic changes and propose a leave-one-out (LOO) strategy as a solution. Consider a study involving multiple groups, each composed of several subjects. In each subject,
synaptic changes are depicted as a 3D point cloud, divided into two distinct classes. The overarching
question concerns differences in the spatial distribution of synaptic changes between groups. A straightforward approach might involve applying SVMs to the aggregate point clouds of each group to identify
regions corresponding to the two classes. However, this raises a key issue: while the SVM-generated regions may align with the group’s overall synaptic changes, they may not adequately capture the changes
within each subject. As such, relying solely on SVMs and aggregate point clouds proves to be insufficient.
Addressing the issue of aggregate point clouds, we consider an alternative in which the group-derived
SVM hyperplane is applied to individual subjects. One might propose using the hyperplane generated
from the group’s cumulative point cloud to segregate synaptic changes within each subject. However,
this strategy runs into the same problem as before: since the hyperplane was derived from a cumulative
data set that includes all subjects, it effectively introduces an overlap between the training and test sets,
undermining the validity of the analysis.
19

To overcome these limitations, we implement a leave-one-out technique tailored for individual subjects
within a group. Specifically, for each subject within a group, we generate a unique hyperplane using a
cumulative point cloud that excludes the data specific to that subject. This approach produces hyperplanes
for each subject without any knowledge of the distribution of synaptic changes within the subject. Our
fundamental expectation is that if all subjects within a group collectively ’agree’ on the location of class 1
and class 2 changes, the individual hyperplanes should consistently segregate the same two regions within
the brain for each LOO sample. Furthermore, the representation of class 1 and class 2 changes within these
regions should exhibit a consistent trend across all subjects.
2.2.1.d Scenarios for Inter-Group Comparison of Synaptic Changes
After analyzing synaptic changes within groups, we next consider how to compare these changes across
multiple groups. We outline three plausible scenarios based on our framework: homogeneous intra-group
changes, quantitative differences in homogeneous changes, and heterogeneous intra-group changes, outlined below. We can apply our method to compare synaptic changes within and across groups by evaluating
these scenarios.
• Homogeneous Intra-group Changes: In some groups, all subjects may manifest similar synaptic changes. Here, the identified brain regions are likely to be consistent within the group. One
can compare the spatial orientation of these regions between groups. For instance, one group may
delineate changes along a left-right axis, while another does so along an up-down axis.
• Quantitative Differences in Homogeneous Changes: Even if multiple groups isolate similar
brain regions , the magnitude of these synaptic changes may vary between groups. This allows for
a comparative assessment of the degree of these changes across groups.
20

• Heterogeneous Intra-group Changes: Some groups may not show consensus among subjects
regarding synaptic changes. In such cases, the spatial distribution of changes would appear random
or heterogeneous. This lack of uniformity is also noteworthy and may indicate a more complex
underlying neural process that warrants further investigation.
2.2.1.e Limitations of the Approach
While our method provides valuable information on large-scale synaptic changes, the coarse-grained nature of our method imposes limitations on its practical applications. Our data constraints require a coarsegrained approach: sparse synaptic labeling (1% of cells per fish) and the low learning rate in our fear
conditioning protocol make it impractical to obtain large sample sizes for finer analyzes. Therefore, our
method focuses on large-scale coarse-grained changes, dividing the entire region into only two categories
based on the SVM hyperplane. As such, it is not equipped to detect small-scale localized synaptic changes,
a limitation that is particularly relevant when studying inhibitory synapses, as discussed in Chapter 4, or
when synaptic changes are diffusely distributed throughout the region.
2.2.1.f Practical Implementation and Summary of the Procedure
In the practical implementation of our SVM-based approach, we utilized Matlab to run a Support Vector
Machine (SVM) algorithm. To segregate distinct brain regions based on a 3D point cloud, where each
point represents one of two classes of synaptic changes of interest we used a SVM with a linear kernel.
This classifier produced a hyperplane that optimally separates the two classes of synaptic changes based on
their 3D spatial distribution. To maintain class balance during hyperplane calculations, random sampling
was applied throughout the pallium volume when the number of Class 1 synapses differed from Class 2
synapses. This hyperplane defined by SVM effectively partitions the pallium into regions with different
fractional representations of class 1 and class 2 changes. To avoid bias, we use a leave-one-out approach
21

for each sample within each group. Specifically, SVM training used point clouds from all subjects in a
particular group, omitting synapses belonging to the individual subject for which the hyperplane was
being determined. This process was iterated until unique hyperplanes were computed for each sample.
Following the calculation of the hyperplane, characteristics such as the fraction of class 1 and class 2
changes were estimated on each side of the hyperplane for each sample. Importantly, these hyperplanes
do not carry information regarding the synaptic distribution of the fish to which they were applied.
2.3 Usage Scenarios
Our methodology offers a versatile framework for investigating synaptic plasticity, both structural and
functional, focusing specifically on changes in synapse number and synaptic strength, as both are crucial
for understanding comprehensive synaptic changes induced by memory formation. The following scenarios illustrate the applicability of our approach in the context of an experiment where the brain region is
imaged at two distinct time points to capture synaptic changes. These scenarios differ in their definition
of class 1 and class 2 for synaptic changes, as well as in the estimation of the cost of misclassification for
individual samples.
2.3.1 Changes in Synapse Number
This scenario aims to map the changes in the number of synapses. Using ’lost’ synapses as class 1 and
’gained’ synapses as class 2, our method identifies subregional changes in synapse numbers between two
distinct time points. In this context, ’lost’ synapses refer to those present at the first time point of imaging
but absent at the second time point. Conversely, ’gained’ synapses were absent initially but appear at the
second time point.
22

2.3.2 Changes in Synaptic Strength
This scenario considers changes in synaptic strength as indicative of functional synaptic plasticity. Utilizing the intensity of the imaging label as a proxy for synaptic strength, our method categorizes synapses
into those with increased and decreased synaptic strength based on changes in intensity levels. Unlike conventional SVM, which assumes equal importance for each point, in this scenario we use sample-weighted
SVM. The absolute values of the intensity changes serve as sample weights, allowing us to factor in the
significance of larger changes in synaptic strength.
23

2.4 Application: Synaptic Changes in Excitatory Synapses Following
Fear Conditioning
This section investigates changes in excitatory synapses that underlie associative learning, using fear conditioning in zebrafish as a model. Utilizing our analysis framework, we identified subregion-specific synaptic changes most prominent in a subset of fish categorized as learners, a group of fish that underwent fear
conditioning training and consistently demonstrated an escape response when presented with the conditioned stimulus (CS) during testing. These findings strongly suggest that these synaptic alterations are
closely tied to the formation of associative fear memories.
This chapter gives an overview of the following aspects of this study:
• Experimental Design and Data Collection: This section outlines the key components of the
experimental framework that lay the groundwork for our large-scale synaptic and neuronal analysis.
It includes the steps conducted by our collaborators: the tail flick conditioning paradigm (TFC), data
acquisition, individual sample data preprocessing, immunostaining of pERK and photoconversion
experiments.
• Subegion-Specific Synaptic Changes: Our method revealed significant differences in synaptic
gain within specific brain regions among the learner group. This affirms the involvement of these
subregions in associative learning. Contrary to expectation, we only detected negligible changes
in regional synaptic strength. This suggests that the formation of new synapses is the primary
mechanism underlying fear memory formation.
• Correlation with Neuronal Activity: We found that regions with increased synaptic gain also
showed elevated neuronal activity during the fear memory formation and retrieval in the learner
group, as indicated by heightened levels of pERK, a surrogate marker of neuronal activity. This
points to a spatial correlation between structural and functional changes.
24

This real-world application demonstrates the utility of our methodology in uncovering synaptic changes
within the context of large-scale analysis and provides insights into the underlying mechanisms of associative fear memory formation. These findings have been published in the Proceedings of the National
Academy of Sciences, further validating the significance and impact of our work (Dempsey et al., 2022).
2.4.1 Experimental Design and Data Collection
This section details the methodologies employed in our study, covering the experimental framework, data
acquisition, and data processing stages. First, we discuss the Tail-Flick Conditioning paradigm used as
a learning model in zebrafish. Subsequent subsections focus on the techniques employed for detecting
synapses and methods to identify lost and gained synapses. Additionally, the consolidation of synaptic
data across different zebrafish samples is discussed, followed by the protocols used to capture neuronal
activity through immunostaining.
2.4.1.a Tail-Flick Conditioning as a Learning Paradigm in Zebrafish
In this study, we utilize Tail-Flick Conditioning (TFC) as a specialized learning paradigm for zebrafish,
an adaptation of classical fear conditioning. The TFC process is broken down into habituation, training,
and testing phases. During habituation, fish are repeatedly exposed to an LED light, serving as a neutral
stimulus. In the training phase, this LED light is followed by infrared laser heating, acting as the unconditioned stimulus (see Figure 2.2). After training, during the testing phase, fish are categorized based on
their behavioral response to the LED light: those that consistently attempt to escape by flicking their tails
are labeled as Learners (L), while those that do not respond are termed Nonlearners (NL). Fish responding
3 or 4 out of 5 times are classified as Partial Learners (PL). The total time for the TFC protocol is ≈3 hr.
Fish are given a 1 hr rest between the first imaging timepoint and TFC and another 1 hr rest between TFC
and the second imaging timepoint. The full time between imaging timepoints for a fish trained with TFC
25

is ≈5 hr, figure 2.3 A. To assess the potential synaptic effects beyond the learning paradigm, three control
groups are used: fish exposed only to the infrared laser (US), only to the LED light (CS), and those not
exposed to any stimulus (NS). These control groups capture synaptic changes due to repeated exposure to
threat, repeated exposure to unnatural LED light, and natural brain development over the training period,
respectively.
Figure 2.2: Tail Flick Conditioning (TFC), a fear conditioning paradigm for larval zebrafish. (A) The head
of the zebrafish undergoing TFC is encased in agarose, leaving the tail free to move. (B) During training,
fish are exposed to both LED light (CS) and laser heating (US). In response to the US, all fish flicked their
tails vigorously (black bars, showing the last training rounds for three learners (L) and three nonlearner
fish (NL)). (C) During the testing phase of TFC, the fish is exposed to the CS alone. Learners respond
immediately, nonlearners do not.
Figure adapted from Dempsey et al., 2022
2.4.1.b Live imaging of excitatory synapses using PSD-95.FingR and SPIM
In order to identify and quantify synaptic changes in the zebrafish pallium that occur as a result of learning,
we employ live imaging using a specialized probe, PSD-95.FingR. This recombinant intrabody, fused to
GFP, selectively targets PSD-95, a crucial component of excitatory synapses. Live imaging takes place both
before and after the training period to capture structural changes underlying memory formation (Figures
2.3 A). The probe serves two purposes: It indicates the number of synapses by labeling the postsynaptic
sites and serves as a readout for synaptic strength. Specifically, synapse strength correlates with synapse
size; larger synapses have more PSD-95, thus recruiting more PSD-95.FingR (Figure 2.4 B), making them
26

Figure 2.3: Imaging excitatory synapses in larval zebrafish. (A) Schematic of TFC/SPIM imaging experiments. The zebrafish is rotated by θ = 25 deg from the fluorescence detection axis (green arrow) to avoid
illuminating the eyes with the incident light sheet (blue arrow). (B) Dorsal view of a 15 dpf larval zebrafish
head with pallium region imaged by SPIM outlined with dashed white line. (C) Maximum intensity projection of a stack of SPIM images of PSD-95.FingR (red outlined region in B) showing synaptic puncta and a
bright nucleus (N). (D) Map of synapses identified from D. Gray scale intensity reflects total PSD-95.FingR
labeling for each punctum. (Scale bar in B, 100 µm; C and D, 10 µm.)
Figure adapted from Dempsey et al., 2022
appear brighter in imaging. Importantly, PSD-95.FingR is designed to sparsely label approximately 0.5% of
excitatory synapses in each subject (Figure 2.4 A), allowing us to resolve individual synapses (Figure 2.3
C, D). Finally, the use of this probe does not interfere with the normal structure or function of synapses,
ensuring an accurate representation of synaptic changes during learning processes (Gross et al., 2013),
Figure 2.4 C, D.
To capture high-quality live images of PSD-95.FingR-GFP-labeled synapses, we adapted our 2 Photon
Selective Plane Illumination Microscopy (SPIM) system specifically for live zebrafish larval imaging. The
PSD-95.FingR-GFP label appeared dim in live imaging because its brightness is correlated with the levels
of the native PSD-95 protein (Gross et al., 2013). This presented a challenge for effective live imaging and
required us to optimize the imaging system. Our customizations markedly improved contrast and imaging
27

Figure 2.4: Characterization of PSD-95.FingR expression in 14-16 dpf zebrafish larvae. (A) Mosaic transgenic zebrafish expressing PSD-95.FingR-GFP and exogenous PSD-95-TagRFP in pallial neurons. Left:
PSD-95.FingR-GFP-labeled synapses and nuclei (N). Middle: Synapses labeled with PSD-95-TagRFP. Right:
Merge showing co-labeling of individual synapses. (B) Intensity of PSD-95.FingR-GFP vs. PSD-95-TagRFP
labeling is highly correlated (R = 0.9) for colabeled puncta (N = 3 fish). (C) Flick ratios (fraction of time that
the tail is engaged in flicking during initial presentation of the CS) during testing following TFC for mosaic
transgenic fish expressing PSD-95.FingR-GFP (Inj, N = 128 fish) vs. uninjected fish (UI, N = 115) are not
significantly different (p > 0.05 Mann-Whitney). (D) Fractions of superlative learners (L) and nonlearners
(NL), which flicked their tails 5 times or 0 times, respectively, in response to 5 CSs were similar in UI vs. Inj.
There were a higher percentage of partial learners (PL), which flicked 3 or 4 times, and a lower percentage
of weak learners (WL), which flicked 1 or 2 times, in Inj. vs. UI
Figure adapted from Dempsey et al., 2022
28

speed and allowed for effective visualization of labeled synapses without compromising normal biology
(Truong et al., 2011).
2.4.1.c Synapse Detection in SPIM Images
We employed a two-step annotation process to identify and quantify synapses in the acquired SPIM images. A dedicated computer algorithm initially located 3D intensity peaks in denoised image stacks and
labeled them as candidate synapse locations (Figure 2.5). Subsequently, trained annotators evaluated these
candidate locations, either validating them as true synapses or discarding them as false positives. This
approach resulted in 3D point clouds, with individual points indicating excitatory synapses and each retaining the corresponding intensity of the label. These values served as a proxy for synaptic strength in
the subsequent analysis.
29

Figure 2.5: An algorithm for semi-automated identification of synapses from raw SPIM data. (A) Axial
and lateral resolution of the custom-built SPIM microscope. Top: An individual optical section from the
center of a 170 nm bead in lateral (left) and axial (right) orientations. Colormap indicates normalized intensity. Bottom: Average lateral (dashed line) and axial (solid line) distribution of N = 23 beads embedded
in agarose. After Gaussian fitting, FWHM (resolution limit) was approximated laterally as 0.45 +/- 0.01
µm and axially as 1.28 +/- 0.059 µm (mean +/- SEM). (B) Top: An individual optical section from a raw
SPIM image. Middle: A single line (dashed line in the top image) from the center of the optical section
above in expanded form. Bottom: An intensity profile of the dashed line from the image above showing
local maxima of intensity within a 1.3 µm sliding window (red arrows). Peaks represent potential synapses
within the line of intensity data, but many are fluctuations in background noise. (C) Images corresponding
to those in B with 2.16 µm Gaussian blur applied to remove maxima that are due to high frequency fluctuations in background autofluorescence. (D) Maxima identified from C are applied to the original image to
mark candidate synapses, which are then classified as true or false positives by blinded observers. Centroid
position and intensity information for the identified synapses (labeled with a red crosshair in the image)
are extracted. The red area under the curve represents the “core intensity” of the synapse, which is a
measure of the amount of PSD-95 at the synapse and of synaptic strength. Note that this one-dimensional
representation of the three-dimensional algorithm used in the study was presented here for simplicity.
Scale bars in A, 2 µm; D, 10 µm;
Figure adapted from Dempsey et al., 2022
30

2.4.1.d Identifying Lost, Gained and Unchenged Synapses
To detect changes in synaptic point clouds, we classified synapses into the ones that were present both
before and after the training and synapses lost or gained post-training. Initially, landmark-based registration aligned the point clouds using the positions of nuclei in the images. Subsequent to registration,
synapses were paired using their Euclidean distances; a synapse in the pre-training image was considered
paired with a synapse in the post-training image if their separation was less than 4 um, an estimated distance a synapse can move naturally over the time course of the training, Figure 2.6 A, B, C. This process
categorized synapses into three distinct point clouds: paired synapses, lost synapses, and gained synapses.
A summary of the processing pipeline can be found in Figure 2.6 D, E. We then studied the change in
intensity between the paired synapses to investigate alterations in synaptic strength, while the lost and
gained synapses were studied to examine changes in the number of synapses.
31

Figure 2.6: Detecting excitatory synapse change following TFC in individual fish. (A) A single neuron
from a mosaic transgenic a 14-16 dpf zebrafish expressing membrane-targeted Dendra2 that was photoconverted with a combination of 488 nm and 730 nm light, and imaged twice with a 5 hour interval between
images (see Section 2.4.1.g). This interval corresponds to the period of time between SPIM images obtained during a TFC experiment. Individual spine-like protrusions present in both timepoint 1 (white) and
timepoint 2 (magenta) were selected by hand by a trained observer. The images were registered to each
other using the un-photoconverted Dendra2 channel data. (B) Distances separating the tips of the same
spine in the aligned images, exemplified in A, were measured and used to generate a histogram. Data in
histogram represents 189 synapse pairs from 4 zebrafish. Sampled spines tend to move no more than 4 um
in five hours. (C) The number of lost and gained spines, measured by comparing the images at timepoint
1 and timepoint 2 of the fish referred to in A and B. (D) Schematic of the processing pipeline. The pipeline
allowed us to identify lost and gained synapses following TFC and the change in label intensity of the
synapses that were present both before and after (black arrows in the paired point cloud). (E) Point clouds
identified in a single learner fish: synapses before (white) and after TFC (magenta) after alignment, but
before pairing; and lost (cyan) and gained (yellow) point clouds after pairing. (Scale bar A 10 um, E 20 um.)
Figure adapted from Dempsey et al., 2022
32

2.4.1.e Synaptic Data Consolidation across Fish Samples
To consolidate spatial data of synaptic changes across multiple subjects of zebrafish, each individual larval
image undergoes a two-stage registration process for alignment with a multi-resolution template brain
image, Figure 2.7. The template brain was created by extending the image beyond the pallium and imaging
the fish at different resolutions. Initially, three gross anatomical landmarks are identified, including the
rostral tip of the pineal gland, the bottom center of the left anterior neuromast near the olfactory pit, and
the anterior point of the dorsal sulcus division. An affine transformation is applied to these landmarks to
bring each image into a common reference frame. Additionally, the plane that best defines the dorsal sulcus
(the "midplane") is identified manually, and a rigid midplane alignment is performed if necessary, ensuring
accurate positioning in the reference coordinate system. This approach provides a unified "canonical"
zebrafish frame of reference for subsequent analysis and visualization of experimental results.
Figure 2.7: Feature based registration onto a template zebrafish brain. (A) Single optical sections from
3 fish (cyan, magenta, yellow) before registration show a lack of alignment. Dorsal view of the pallium
region (red highlighted region in Insert). (B) Same sections as in A are aligned following registration using
anatomical landmarks (see methods). Scale bars, 50 µm.
Figure adapted from Dempsey et al., 2022
33

2.4.1.f Capturing Neuronal Activity through Immunostaining of pERK
To capture neuronal activity in zebrafish larvae after fear conditioning training (TFC) and in response to
unconditioned stimulus (US) and conditioned stimulus (CS), we employed a specific antibody staining technique. Fish were fixed with paraformaldehyde to halt biological activity and preserve tissue. This fixation
occurred shortly after the last stimulus was presented, capturing a "snapshot" of cellular activity. Subsequently, the fixed samples underwent a series of preparatory treatments: washing, pigment bleaching,
and permeabilization for antibody staining. Phosphorylated Extracellular signal-regulated Kinase (pERK)
served as our readout of neural activity. As a part of the MAPK/ERK pathway, pERK undergoes rapid
phosphorylation changes in response to cellular stimulation, including neural activation. Therefore, pERK
levels provide a proxy for the degree of neuronal activity within specific brain regions at the moment of
fixation. Anti-pERK antibodies coupled with AlexaFluor488-conjugated secondary antibodies facilitated
the visualization of pERK concentrations throughout the pallium. This protocol allowed us to visualize
neuronal activity under varying conditions (CS, US, or NS), both in naive fish and those subjected to TFC,
providing critical insights into the neural mechanisms underlying learned behaviors.
2.4.1.g Photoconversion experiments
To visualize and track dendrites of specific neurons, we used a technique called photoconversion. Photoconversion involves changing the color of a fluorescent protein, Dendra2, within these neurons. Dendra2
is a photoswitchable fluorescent protein, capable of irreversible photoconversion from green to red in response to intense blue-light irradiation at 460-500 nm. The Dendra2 protein is genetically engineered to
be present in neurons by inserting it into the fish DNA, using the β-actin2 promoter, which controls the
expression of this protein. For photoconversion, we first used a confocal microscope to locate the neurons
of interest. Then, a conversion filter plate was placed in the light path just before the objective of the
confocal microscope and 488 nm + 730 nm laser light was focused onto a small portion of the neuron cell
34

body of interest. By repeatedly exposing this area to light, we successfully changed the color of Dendra2
in that specific neuron. This process allows us to clearly see the dendrites and axons of the targeted cells
(Figure 2.13 B), and to assess the movement of spines in the pallium during a 5-hour period (Figure 2.6 A).
35

2.4.2 Results
This section reports on synaptic and neural changes following Fear Conditioning in larval zebrafish. Our
analyses found no changes in overall synaptic strength or number of synapses within the pallium. However, we identified specific sub-regional gain in number of synapses in the ventrolateral pallium among
the learner group. Additionally, we observed a spatial correlation between increased neural activity and
synapse gain, suggesting that fear conditioning induces localized synaptic changes that could be crucial
for memory formation. The details follow in the subsections below.
2.4.2.a Absence of Region-Wide Synaptic Changes
Our findings indicate that fear conditioning training (TFC) has no substantial impact on overall synaptic
strength and number of synapses within the pallium :
• No statistically significant changes were observed when comparing the total number of synapses in
the pallium before and after TFC for L, PL, and control groups (CS only, US only, and NS). A slight
reduction (10%) in synapse numbers was detected in the NL group, Figure2.8.A
• The median intensity of PSD-95.FingR-GFP label of synapses before and after TFC was not significantly different for any of the groups, Figure 2.8.B.
36

Figure 2.8: Absence of Region-Wide Synaptic Changes following TFC in larval zebrafish. (A) Total number
of synapses before (t1) and after (t2) TFC for individual fish (P > 0.05 for L, PL, US, NS, and CS; ∗P < 0.05
for NL, Wilcoxon test). (B) Average total intensity of the PSD-95.FingR label associated with individual
synapses that are present both before (t1) and after (t2) TFC does not change significantly for L, PL, NL,
US, NS and CS fish (p > 0.05, Wilcoxon).
Figure adapted from Dempsey et al., 2022
We extended our analysis by aligning the synaptic point clouds from different groups of fish onto a
common zebrafish brain template (2.4.1.e). Cumulative distributions (CDs) of lost, gained, and persistent
synapses were generated for each group. Consistent with the aforementioned results, our analysis revealed
no significant shifts in the rate of loss, gain, or persistent synapses for L, PL, NL, and control fish, Figure
2.9 D, E, F. In summary, these results indicate that TFC does not induce region-wide synaptic changes
in the pallium, necessitating further investigation into sub-regional changes for understanding memory
formation.
2.4.2.b Significant Synapse Gain in the Ventrolateral Pallium
Employing the methodology described in 2.3.1, we observed a relatively consistent separation of the pallium into ventrolateral and dorsomedial regions across individual hyperplanes in the learner group, Figure
37

Figure 2.9: Synapse changes with TFC in larval zebrafish. (A) Dorsal view of all identified excitatory
synapses in the left pallium (highlighted in inset) of a learner fish before (white) and after (magenta) TFC
alignment of the two synapse images was accomplished by manually identifying and then computationally
aligning the labeled nuclei (not shown in images). (B) Schematic of algorithm to identify synapses lost or
gained following TFC. All synapses were grouped into pairs, one from before TFC and the second from
after, such that the distance between the pair is minimized. Each synapse could only be part of a single
pair. All pairs separated by a distance greater than or equal to 4 µm are considered different synapses
(i.e., lost if from before TFC or gained if from after). (C) Dorsal view of lost (cyan) and gained (yellow)
synapses following TFC from the learner fish shown in B reveals more lost synapses medially and more
gained synapses laterally. (D–F) Synapse turnover in fish before and after TFC did not show significant
variation between different categories of fish (P > 0.05, Kruskal–Wallis test) except for PL and NS (loss
fraction and unchanged fraction, P < 0.05, Dunn’s multiple comparison test). (Scale bars, 20 µm.) n = 11
L, 6 PL, 11 NL, 11 US, 11 NS, and 11 CS fish.
Figure adapted from Dempsey et al., 2022
2.10 B. In contrast, hyperplanes in nonlearners and control fish exhibited greater variability. Notably,
the fractional gain in the number of synapses was approximately 30% higher in the ventrolateral pallium
among the learner group, as shown in Figure 2.10 C. This absence of a consistent pattern of changes in
synapse number among nonlearners and control fish supports the idea that the primary mechanism of
synaptic plasticity in response to fear conditioning is the change in the number of synapses specifically
localized to the ventrolateral pallium.
38

Figure 2.10
39

Figure 2.10: Regional differences in synapse formation in the pallium of learner fish. A) Left: Map of lost
(cyan) or gained (yellow) synapses in the left pallium following TFC for superlative learner (L) fish in dorsal
view (highlighted in inset, midline to the right). Results from each fish were registered onto a canonical
fish prior to pooling. Middle: Coronal view of pooled L results shows a decision boundary plane (SVM
hyperplane, red) that optimally divides the left pallium into two regions of differential synapse change:
dorsomedially, synapse loss predominates; ventrolaterally synapse gain predominates (region highlighted
in inset; midline to the left). Right: Slightly tilted 3D coronal view of SVM hyperplane for L fish (red)
shown against the outline of the canonical zebrafish brain (gray; M: midplane). (B) SVM hyperplanes
(which separate areas of predominant synapse loss from predominant synapse gain) calculated using the
leave-one-out method for each combination of n-1 fish (green, see 2.2.1.c), average SVM hyperplane shown
in red. SVM hyperplanes generated for L fish show the least variation among those calculated for the six
groups. Individual fish hyperplanes which separate areas of predominant synapse loss from predominant
synapse gain (green) and an average hyperplane per group (red) for learners, nonlearners, and the control
groups. Hyperplanes generated for learners fish show the least variation among those calculated for the six
groups. (C) Synapse fractional gain leave one out analysis for each group reveals that learners have gained
significantly more (≈30%) synapses ventrolaterally vs. dorsomedially relative to the SVM hyperplane (∗∗∗
p < 0.005, Wilcoxon). PL, NL, and control groups do not show a significant difference in synapse gain
between the two sides of the SVM hyperplane (p > 0.05, PL; p > 0.1, NL, controls, Wilcoxon). (D) The ratio
of PSD-95.FingR-GFP fluorescence intensity per synapse before and after TFC (Iafter/Ibefore) is negligibly
different in medial vs. lateral regions in L (≈1% difference, ∗ p < 0.05, Wilcoxon), in US (< 4%, ∗ p < 0.05,
Wilcoxon), and in PL, NL, CS, NS (< 4%, p > 0.25, Wilcoxon). (Scale bar, 20 µm.)
Figure adapted from Dempsey et al., 2022
2.4.2.c Sub-Regional difference in synaptic strength is negligibly small
By employing the methodology detailed in 2.3.2 to divide the pallium into two sub-regions that showed the
maximum difference in changes in synaptic strength, we calculated the ratio of synaptic intensity before
and after training (Iafter/Ibefore) for the paired synapses. The median intensity differences across these
sub-regions were notably small: approximately 1% for the learner group, less than 4% for the nonlearner
and US control groups, and up to 5% for the CS and NS control groups. These findings, further detailed in
Figure 2.10 D, confirm that the variance in synaptic strength among the pallium sub-regions is negligible.
2.4.2.d Gained and lost vs. subthreshold synapses.
To ascertain whether the identified gained and lost synapses were not simply dim synapses undergoing
minor intensity fluctuations around the detection threshold, we analyzed intensity distributions. Specifically, we examine the intensity values at t1 (pre-TFC) for lost and unchanged synapses, and at t2 (post-TFC)
40

for gained and unchanged synapses. This analysis included data from 61 fish (11 learners, 11 nonlearners,
6 US-only, 11 CS-only and 11 NS groups).
After adjusting synapse intensities for camera offset and bleaching, we determined the synapse-specific
signal-over-local-background (SBR) by dividing the peak intensity pixel of each synapse by the 10th percentile intensity pixel within a 5x5 pixel neighborhood. SBR values were then represented in histograms
comparing lost versus unchanged synapses before, and gained versus unchanged synapses after TFC (Figure 2.11 A, B). Additionally, we plotted the cumulative density function (CDF) of peak intensity values for
these comparisons (Figure 2.11 D, E).
To further investigate, we simulated intensity changes in the entire population of unchanged synapses.
This simulation aimed to identify the extent of intensity reduction or increase required to replicate the
CDF of truly lost or gained synapses. In Figure 2.11 F, we reduced the intensity of each ’unchanged before’ synapse by different fractions (10%, 50%, or 80%) and identified those that would fall below the SBR
threshold, thus being classified as ’lost’. Figure 2.11 G illustrates a parallel process for ’unchanged after’
synapses, where intensity reduction resulted in corresponding fold increases in intensity (1.1, 2, and 5-fold,
respectively).
This simulation indicates that significant changes in synapse intensity are necessary to match the
experimental intensity distributions of gained and lost synapses, suggesting that these are not merely dim
synapses crossing the detection threshold.
41

Figure 2.11: Gained and lost vs. subthreshold synapses. (A) Distributions of intensities at t1 of unchanged
synapses (present at both t1 and t2) vs. synapses that were subsequently lost, are similar. SBR refers to the
ratio of total measured intensity of a synapse vs. local background, suggesting that synapses of all intensities at t1 can contribute to the lost population. Thus, it is unlikely that lost synapses are dim synapses
that have undergone a modest loss in intensity (B) Distributions of SBR of unchanged synapses at t2 vs.
gained synapses are similar. Thus, it is unlikely that gained synapses are subthreshold synapses that have
undergone a modest gain in intensity. (C) Distributions of the number of synapses segmented from the
same image before and after a 10% reduction in image intensity. The image stacks were independently
segmented. The intensity reduction resulted in 33 lost synapses and 27 gained synapses (∼ 2% of the 1323
synapses present in the original image) with ∼98% of synapses are segmented in both images. (D) Cumulative distributions of intensity corresponding to distributions in A are also similar for lost and unchanged
synapses. (E) Cumulative distributions of intensity corresponding to histograms in B are also similar for
gained and unchanged synapses. (F) Cumulative distributions of intensities at t1 for lost synapses, as predicted by a simulation where synaptic intensities of the unchanged synapse population were diminished
by different percentages. An 80% reduction in intensities produces a cumulative distribution (blue) similar
to that of the histogram derived experimentally from lost synapses (cyan). This suggests that only a very
large loss of intensity could cause the experimentally observed synapse loss. (G) Cumulative distributions
for gained synapses predicted by a simulation where the original intensities of tp2 unchanged synapses
have been reduced to model an intensity of a synapse at tp1 being magnified by different factors. A five-fold
increase in intensities produces a cumulative distribution (blue) similar to that of the histogram derived
experimentally from gained synapses (yellow). This suggests that only a very large increase in intensity
would be needed for synapses present below the subthreshold SBR at t1 to form the gained synapses observed experimentally. Scale bar 5 um.
Figure adapted from Dempsey et al., 2022
42

2.4.2.e Spatial Correlation Between Increased Neural Activity and Synapse Gain
Our findings demonstrate a spatial correlation between regions manifesting elevated neural activity during
memory retrieval and those exhibiting increased synapse formation. Employing pERK as an indicator of
neuronal activity, we observed substantial differences in the activity in the pallial region between learner
(L) and non-learner (NL) zebrafish following exposure to the conditioned stimulus (CS), as illustrated in Figure 2.12. Interestingly, naive fish subjected solely to the unconditioned stimulus (US) exhibited increased
activity in the same region, implying that learning can induce neural circuit modifications, causing a formerly neutral stimulus to evoke an aversive response. Notably, synapse gains are localized in the same
regions that display heightened activity in response to CS in learners (L) and partial learners (PL). Given
that pERK mainly labels neuronal cell bodies and synapses are located primarily on dendrites, we investigated whether the dendrites of the cells whose cell bodies are localized in the pERK-identified region,
coinciding with the area of synapse gain. In vivo visualization of individual dendrites revealed that approximately 90% of these structures reside within the area of increased synapse formation, as indicated in
Figure 2.13, B. Together, this evidence suggests that the anterolateral pallial region contains neurons that
respond to aversive stimuli in naïve fish, and these neurons become activated by neutral stimuli after the
generation of new excitatory synapses after TFC.
43

Figure 2.12: Neuronal activation within the anterolateral pallium in response to the CS in learner fish
and to the US in naïve fish. (A) Intense immunostaining of pERK in the pallium (magenta highlighted
region, Inset) of an L fish exposed to 5 CSs following TFC. The strong signal in an anterolateral region
(yellow outline) of this optical section reveals regional neuronal activation. Relatively less immunostaining
is present in the medial pallium (cyan outline). (B) An NL fish shows a lack of pERK staining in the
anterolateral region (yellow outline) after exposure to 5 CSs in this equivalent optical section. (C) A naïve
fish reveals strong pERK staining in the same anterolateral region (yellow outline) after exposure to 10
USs. Equivalent optical section to those in A and B. (D) A naïve fish exposed to 10 CSs does not show
concentrated pERK labeling in the anterolateral region (yellow outline). Optical section equivalent to those
in A–C. (E) A naïve fish not exposed to a CS or US (NS) does not show concentrated pERK labeling in the
anterolateral region (yellow outline). Optical section equivalent to those in A–D. (F) L and US-exposed
naïve subjects show a significantly higher lateral:medial pERK intensity ratio compared to NL and naïve
untreated subjects (*P < 0.02, ***P < 0.005, n = 5 fish per group, Kruskal–Wallis multiple comparison test).
White dashed lines mark the border of the pallium (midline = M) in A–E. (Scale bar for A–E, 20 µm.)
Figure adapted from Dempsey et al., 2022
44

Figure 2.13: Anatomical correlation of regions of increased neuronal activity and synaptic gain following
TFC. (A) Coronal (Left), dorsomedial (Middle), and ventrolateral (Right) views of the region of increased
pERK labeling in the left pallium of learner fish (see inset cartoon) exposed to CS. pERK staining is located
mainly on the ventrolateral side of the DB plane, where synapse gain predominates in learners. (B) Five
individually photoconverted cells, whose cell bodies lie within the pERK staining area of the pallium shown
in A, have processes that are predominantly (~90%) on the ventrolateral side of the DB plane. Coronal
(Left), dorsomedial (Middle), and ventrolateral (Right) views. Magenta asterisks in left and right subpanels
indicate the position of cell bodies. (Scale bar 30 µm)
Figure adapted from Dempsey et al., 2022
-
45

2.5 Limitations and Future Directions
2.5.1 Limitations in Synapse Validation and Assumptions
Our detection and analysis methods do not allow for gold-standard validation of synapses such as those
obtained with electron microscopy. Therefore, our conclusions rest on the assumption that our methods,
validated by correlative and suggestive evidence, accurately identify excitatory synapses. In addition,
although PSD-95 is a well-established marker for synaptic strength, particularly in mammalian systems
(El-Husseini et al., 2002), we cannot rule out the possibility that there are instances in which changes in
levels of PSD-95 do not reflect changes in synaptic strength.
2.5.2 Synapse Pairing and Intensity Analysis
The methodology of our study, which is based on colocalization to identify the same synapses before and
after fear conditioning, presents certain limitations. Specifically, a minor misalignment of the images before and after conditioning can lead to incorrect synapse pairs. This issue is particularly problematic in
densely packed synaptic regions, where one incorrect pairing can trigger a cascade of errors, affecting
subsequent pairings. This effect is constrained by the maximum pairing radius we have set (4 um), which
means that our overall conclusions about the number of lost and gained synapses in a given region remain unaffected. However, this cascading error can substantially distort our interpretation of changes in
synaptic intensity, leading to potentially misleading conclusions about synaptic dynamics.
To illustrate this challenge, consider a simplified scenario with three synapses having initial intensities of 5, 10, and 15 arbitrary units (a.u.). If each of these synapses experiences an increase in intensity of
20%, their respective intensities should rise to 6, 12, and 18 a.u. However, if misalignment leads to incorrect
pairings, such as matching the first synapse (initially 5 a.u.) with the third post change (18 a.u.), the second
46

with the first (10 to 6 a.u.), and the third with the second (15 to 12 a.u.), our analysis would incorrectly suggest that two synapses decreased in intensity while one increased and would measure an average increase
in intensity of approximately 67% in the region. Therefore, the perceived variance is dramatically altered
and the actual increase in uniform intensity is obscured (mean I1/I1 ratio increased from 1.2 to approximately 1.7, and standard deviation increased from 0 to approximately 1.4). To measure the impact of the
misalignment on our analysis, we propose running simulations with actual data by performing random
pairings within the original 4 um radius and evaluating their effects on both mean intensity and intensity
variation.
To address this limitation, future research could implement more sophisticated methods for synapse
pairing. A potential approach involves considering additional features beyond the position of the synapse.
For example, incorporating information about the synapse shape and local neighborhood characteristics,
possibly encoded using an autoencoder applied to a small area surrounding each synapse, might enhance
the accuracy of synapse pairing. By refining our pairing methodology, we can improve the reliability of
our interpretations and gain a more accurate understanding of synaptic changes.
2.5.3 Refining Analytical Approach for Brain Mapping
The use of linear SVM hyperplanes does not capture the anatomy of the brain sub-regions. Future iterations
of the method could explore the use of other coordinate systems, such as spherical instead of Cartesian,
to better match the curved architecture of the brain. Expanding the feature set beyond just the spatial
coordinates (x, y, z) may provide a more comprehensive understanding of synaptic alterations. These
modifications could further increase the specificity and robustness of the method for different brain regions
and behavioral paradigms.
47

2.6 Conclusion
We have pioneered the application of Support Vector Machines (SVMs) in analyzing large-scale synaptic
changes. Using a novel analytical framework that incorporates SVMs and the leave-one-out technique, we
introduced two criteria to assess the biological relevance of synaptic changes: consistency in SVM hyperplane orientation and alterations within SVM-identified regions. This allowed us to detect sub-regional
synapse changes that would otherwise remain obscure.
A key finding from our analysis is an increase in excitatory synapses in the ventrolateral pallium. This
synaptic gain was most prominent in the learner group and was accompanied by an increase in neuronal
activity in the same brain region, supporting the link between synaptic changes and the formation of
associative fear memory. Interestingly, while there were significant regional variations in synapse number,
the average labeling intensity of PSD-95 at synapses before and after learning showed only minor changes.
Previous studies using post hoc labeling have found that hippocampal engram cells have increased
spine density relative to nonengram neurons following contextual fear conditioning (CFC) in mice (Choi,
2018; Roy et al., 2017; Ryan et al., 2015). Although these studies did not directly measure synapse change,
their results are nonetheless consistent with our conclusion that memory formation is associated with
region- and cell-specific increases in synapse formation. This interpretation is also consistent with recent
work showing that selectivity of neurons in the ferret visual cortex arises from the most common visual
inputs rather than from the strongest inputs (Scholl et al., 2020). In contrast, our observation that PSD95.FingR labeling of synapses changed minimally with learning, even in regions in which dramatic gain and
loss of synapses occurred, would appear to be at odds with previous studies. In particular, a recent imaging
study found that spines on CA1 engram cells have approximately twice the volume of spines in nonengram
cells following CFC (Choi, 2018). This apparent discrepancy could be explained by several differences
between our study and previous ones. Specifically, earlier studies depended on post hoc comparisons of
synapses on different cells, whereas we directly compare the same synapses at two separate time points. In
48

addition, the memories studied here were formed in the brains of a different species (fish versus mice), at a
different age (larvae versus adults), in a different brain region (pallium versus hippocampus), in a different
cell type (pallial neurons versus CA1 neurons), and as a result of a different learning paradigm (TFC versus
CFC) compared with previous studies. Furthermore, increases in synaptic strength could have occurred in
a subset of sparsely distributed pallial cells, causing these increases to be obscured by noise. They could
also have been transient and disappeared prior to the second imaging. In addition, synaptic strength could
have changed as a result of presynaptic mechanisms our paradigm could have missed. Finally, nonsynaptic
changes that would have been difficult to detect with our system, such as increases in neuronal excitability
or myelination, could have contributed to learning (Pan et al., 2020; Yiu et al., 2014).
Our results, using pERK labeling to identify regions of increased neuronal activity, showed that cells
within a discrete region in the anterolateral pallium respond intensely to both the US in naïve fish and the
CS in learner fish. This anterolateral region is thus active both during memory formation and retrieval
(Josselyn et al., 2015; Tonegawa et al., 2015). Importantly, this region overlaps with the area in which intense synapse formation occurs in learners. Our pERK results corroborate those of a recent study in which
Ca2+ activity in the mouse amygdala imaged before and after fear conditioning showed that the pattern of
cellular responses to the CS following learning became similar to responses to the US in naïve mice (Grewe
et al., 2017). These anterolateral pallial cells may be homologous to aversive cells found in the basolateral
amygdala in mice (Beyeler et al., 2018; Namburi et al., 2015). However, unlike aversive amygdalar cells
in the mouse, which are distributed in a salt-and-pepper pattern, those in the anterolateral pallium of the
larval zebrafish appear to be tightly clustered, making them easier to visualize and suggesting that they
might be amenable to investigations into how neurons encode fear.
Our methodology represents a significant departure from traditional approaches to synaptic analysis.
Instead of relying on prior knowledge of the expected location of synaptic changes, we allowed the data
49

to reveal the regions that undergo significant changes. The application of computational techniques like
SVMs opens the door to a more nuanced and data-driven understanding of structural changes in the brain.
2.7 Acknowledgments
This chapter reflects the collaborative effort of many individuals, each contributing uniquely to its success,
and I sincerely appreciate their support. I am grateful to William P. Dempsey and Zhuowei Du for their
pivotal roles in conducting the behavioral and imaging experiments, where Colton D. Smith also played
a crucial role. Special thanks are due to Don B. Arnold, Thai V. Truong, and Scott E. Fraser for their
insightful guidance, which has profoundly influenced the direction and refinement of the research. The
efforts in data management by the Informatics Systems Research Division at the University of Southern
California Information Sciences Institute, especially those of Carl Kesselman and Karl Czajkowski, have
been critical for our project’s success. I thank Andrey Andreev, Thai V. Truong, and Scott E. Fraser for
their notable efforts in designing and building the SPIM and the apparatus for tail-flick conditioning. This
contribution has significantly enhanced our experimental capabilities. Acknowledgment is also given to
the team involved in the synapse identification protocol, including S. Applebaum, K. Watanabe, H. Jin, and
others, whose meticulous work has contributed significantly to our findings. I am particularly grateful
to my dissertation committee members Scott Fraser, Remo Rohs, Donald Arnold, Fengzhu Sun, and Thai
Truong for valuable suggestions for future research. Finally, I thank the National Institute of Mental Health
for their support under Grant MH107238.
2.8 Code Availability
The analysis and figure code is implemented in Matlab and provided at:
https://github.com/LemonJust/synapse-redistribution.
50

Chapter 3
Flexible Computational Pipeline for Cell-Type-Aware Analysis of
Synaptic Changes
3.1 Synopsis
In this chapter, we shift our focus to inhibitory synapses, recognizing their crucial role in modulating
neural circuits, as discussed in multiple reviews (Ehrlich et al., 2009; Giorgi and Marinelli, 2021; Hájos,
2021; Krabbe et al., 2018; Pare and Duvarci, 2012; Wolff et al., 2014). This project builds on our previous
research on changes in excitatory synapses during fear memory formation (Chapter 2) and continues our
collaboration with Zhuowei Du and Don Arnold. While maintaining the core experimental paradigm of
fear conditioning in zebrafish from our earlier work, we have introduced distinct molecular tools and
additional experimental procedures tailored to the unique goals of this project.
A comprehensive investigation of inhibitory synapses on a large scale presents unique challenges,
necessitating the development of specialized analytical tools and methodologies. Our exploration of inhibitory synapses revolves around a specific objective: the identification of perisomatic synaptic connections (the synaptic connections that are located close to neuron cell bodies) while considering the specific
cell types involved. This focus on perisomatic connections arises from two fundamental distinctions between inhibitory and excitatory synapses.
51

Firstly, unlike excitatory synapses, even minor changes in inhibitory synapses can substantially affect
neural circuit behavior, given the modulatory role of inhibition. In contrast to our previous study of excitatory synapses that focused on synaptic changes on a broad scale, emphasizing distinct behaviors within
large pallium subregions, it is crucial to employ methods sensitive to smaller changes when considering
inhibitory synapses.
Secondly, fear conditioning can introduce opposing alterations within the same confined region due
to the diversity of inhibitory neuron types (Bloodgood et al., 2013). For example, increasing neuronal
excitability requires reducing the inhibition targeting excitatory neurons while simultaneously increasing
the inhibition applied to inhibitory neurons. This intricacy underscores the necessity of examining changes
in inhibitory neurons while considering their specific targets and functions - differing markedly from the
broad patterns we revealed in our earlier study of excitatory synapses.
This chapter introduces a data processing pipeline tailored to detect localized changes in inhibitory
synapses across expansive brain regions. These adaptable tools can be applied more broadly and accommodate a wide variety of studies of synaptic changes. Created in close collaboration with our experimental neuroscience team, the pipeline is designed to be flexible, easy to maintain, and adaptable to evolving
experimental conditions. Our aim is to achieve traceability, flexibility, and accessibility. Through this
pipeline, we offer a specific toolkit for investigating the role of inhibitory synapses in fear memory formation, as detailed in the following subsections:
• Design and implementation: In this section, we outline the design and implementation of the
pipeline, focusing on traceability, flexibility, and accessibility as central objectives. Our approach
includes precise data management, flexible and maintainable code architecture, and user-friendly
configuration options, making it suitable for computational scientists and biologists.
52

• Usage scenarios: Here, we give an overview of potential practical applications of our pipeline,
showcasing its role in advancing our understanding of brain function and structure. We introduce
three usage scenarios that illustrate the utility of our pipeline in real-world research contexts.
• Application: In the final section, we offer a practical demonstration of the implementation of our
pipeline in the context of an ongoing research project. We focus on detecting changes in inhibitory
synapses following fear conditioning, taking into account the cell type of synaptic targets. This
section provides an overview of data collection, workflow, and preliminary findings, illustrating
the application of our pipeline to an individual sample and highlighting its potential for studying
inhibitory synapse dynamics.
3.2 Design and Implementation
The central goal of this project is to integrate well-established analytical components- such as registration,
segmentation, and classification, for which multiple tools already exist- into a unified, flexible, yet precisely
controlled framework that ensures quality data management. Three primary objectives guide the design
and implementation:
1. Traceability: Given the complexity of the pipeline, which consists of numerous steps, various parameters, and interdependencies, traceability is essential. The pipeline must ensure a clear record of
the steps executed and their associated parameter. Additionally, the diverse data produced at each
pipeline step requires meticulous organization.
2. Flexibility: The pipeline must adapt to the ongoing development of our project, where data sets
undergo continuous changes due to evolving biological experiments that introduce novel labels and
methods. Because multiple approaches exist for various pipeline steps (for example, for segmentation or classification), some of them could perform better than others as the dataset evolves. This
53

requires that the pipeline’s structure and underlying codebase remain flexible and open to experimentation, allowing it to easily incorporate new methods and experiment with them.
3. Accessibility: Our objective was to ensure accessibility for biologists, regardless of their programming skills, enabling them to combine existing methods into a customized pipeline, search for parameters that best suit the data, and independently execute it without relying on the assistance of a
programmer.
To meet the objectives of traceability, flexibility, and accessibility, we implemented specific solutions
in the construction of the pipeline. We designed a database for precise tracking of each pipeline execution
run, including tasks and parameters, and comprehensive data logging. The code architecture has been
meticulously designed to facilitate the integration of new methods. To simplify pipeline execution, users
can specify the entire process through a human-readable configuration file. Further enhancing ease of use,
the pipeline is distributed as a Docker container, eliminating the complexities of library installations and
ensuring cross-platform compatibility.
3.2.1 Modular Tasks and Configurable Pipeline
The architecture of the pipeline is organized into distinct modular tasks, each with well-defined inputs
and outputs. These tasks include registration, segmentation, classification, transformation, assignment,
and synapse pairing (Figure 3.1). This modular approach allows for the seamless integration of multiple
methods and implementations, provided they conform to the same input and output specifications.
The pipeline incorporates a dedicated database to ensure rigorous data logging and traceability. This
database captures all tabular data generated by the pipeline, such as the centroids of segmented synapses
and nuclei, the file locations of both generated and input image files, and comprehensive metadata for
each pipeline execution, which includes executed tasks and parameters. The architecture aligns with the
54

pipeline’s defined input-output system, thereby facilitating the inclusion of new methods while maintaining the tracking capabilities. As a result, every pipeline run is exhaustively tracked, maintaining a
transparent and searchable record.
To enhance user accessibility, the pipeline allows tasks to be combined and customized through a
straightforward configuration file, as further detailed in Section 3.2.2. This feature allows users, irrespective of their programming expertise, to tailor the pipeline to the specific needs of their research. It also
ensures that the pipeline remains adaptable to the evolving requirements of various projects. While the
pipeline comes with some methods already implemented, it is designed to accommodate the implementation of additional methods. Specific tasks and methods currently implemented for each task within the
pipeline are detailed below:
• Initialization: Initialization task serves as the foundation for subsequent analyses and ensures that
the pipeline is fully equipped with the required data, models, and parameters before proceeding
with further steps. At this stage, the pipeline is provided with the location of raw data files, crucial
metadata, and pre-trained classification and segmentation models. The existence of all the files and
models is verified, and in cases where data is online, the data is downloaded.
• Registration: Registration task is responsible for image alignment. It calculates the transformations
that can be later used by the Transformation task. For this purpose, the ANTsPy library, a Python
wrapper for the Advanced Normalization Tools (ANTs) biomedical image processing library (Avants
et al., 2011a), is utilized.
• Transformation: Transformation task executes the calculated transformations on images and point
clouds, yielding transformed image files and point-cloud centroids. The ANTsPy library is used for
this task as well.
55

Figure 3.1: Schematic representation illustrating the data flow through individual tasks. Blue blocks depict
tasks such as Initialization, Registration, Transformation, Segmentation, Classification, Assignment, and
Pairing. Green blocks signify data inputs and outputs, while yellow blocks represent pre-trained models.
Arrows indicate the flow of data, with dotted lines marking optional inputs or outputs. A database icon on
a block specifies that the corresponding data is stored entirely within the database for efficient retrieval and
manipulation, and to facilitate the establishment of the relationships by the Pairing and Assignment tasks.
Blocks without the database mark indicate that the data is stored on disk while its location is cataloged in
the database.
• Segmentation: Segmentation isolates objects of interest, such as synaptic candidates and cell nuclei,
from the image data. The resulting segmentation mask and an associated table containing object
metadata (mask ID, centroids, probability) are stored. The StarDist library, which specializes in starconvex object detection for 2D and 3D images (Schmidt et al., 2018), is used for this task.
• Classification: Classification task augments the analysis by categorizing the segmented objects into
distinct classes, such as excitatory neurons, inhibitory neurons, or glial cells. The output is a table
with class assignments for each segmented object. Custom methods are used for this task, tailored
to the specific classification goals.
56

• Assignment: Assignment task facilitates mapping between centroids, potentially of different object
types. It creates a one-to-many mapping and produces a table that establishes these connections
based on specified criteria. In the context of this project, it can be used to establish neuron-synapse
connections under the assumption that synapses are connected to neuron cell bodies if they are
within a certain distance.
• Pairing: Pairing task is designed to create unique one-to-one mappings between specified centroids.
This task generates a table that delineates these unique pairings based on specified criteria. In the
context of this project, it is used to define the lost, gained and persistent synapses, as described in
2.4.1.d
3.2.2 Pipeline Configuration
The pipeline’s modular structure enables users to define and execute tasks based on their analysis requirements. The user defines the pipeline execution workflow in a YAML configuration file that describes the
pipeline in terms of tasks. The configuration file serves as the blueprint for the entire analysis, detailing the
steps to be executed, the input data, the chosen methods, models, and parameters. For example, in Section
3.3.2, we demonstrate how the entire data processing workflow for our first project- focused on synaptic
alterations in excitatory synapses following fear conditioning- can be configured via a single YAML file in
only 65 lines.
The flexible nature of the pipeline allows for iterative analysis and adjustments. As research needs
evolve or new experimental conditions are introduced, users have the flexibility to revisit and modify
earlier steps. This iterative functionality allows the user to experiment with a diverse set of parameters
and methods for each task to evaluate their impact on the entire analysis process. To help keep things
organised and avoid the computational overhead, upon each execution, the pipeline identifies tasks within
the configuration file that have previously been executed with the specified parameters and inputs. Users
57

have the option to run these tasks again or bypass them, thus facilitating ongoing data exploration and
methodological refinement within a single configuration file.
3.3 Usage Scenarios
In this section, we illustrate the practical applicability of the pipeline through three distinct usage scenarios, each designed to showcase specific capabilities in a real-world research context. These include
cell-type-specific synaptic connectivity, time-dependent synaptic connectivity patterns, and a composite
scenario that combines elements of both, termed time-dependent and cell-type-specific synaptic connectivity patterns. These examples serve to underscore the flexibility of the pipeline and its user-friendly
design.
3.3.1 Cell-Type Specific Synaptic Connectivity
In this scenario, we underscore the pipeline’s core functionality to assign synapses to objects of interest, demonstrated here through the analysis of perisomatic synaptic connections targeting excitatory and
inhibitory neuron types. This provides a versatile framework for researchers studying neural circuitry
and potentially other fields. Although the current implementation focuses on the segmentation and classification of cell bodies, it is possible to include other anatomical features such as dendrites, relying on
the addition of appropriate methods to the pipeline. As illustrated in Figure 3.2, researchers initiate the
analysis with multilabel images that include labeled synapses and markers for different cell subtypes. If
available, an additional channel for cell nuclei can further refine cell detection.
The core pipeline for synaptic connectivity analysis consists of five fundamental steps. These steps
include image registration and transformation, synapse and cell segmentation, cell classification, and the
assignment of synapses to specific cell types. Image registration and transformation are essential to align
synapses and cells in the same space, ensuring precise spatial mapping across different modalities. This
58

Figure 3.2: Workflow for Cell-Type Specific Synaptic Connectivity Analysis. Blue blocks represent individual tasks within the pipeline, such as image registration, segmentation, and assignment. Green blocks
denote the data inputs and outputs at each stage, including multi-label images and spatial coordinates of
synaptic and cellular elements. Yellow blocks illustrate the pre-trained models employed in tasks like segmentation and classification. Arrows indicate the flow of data between tasks and data elements, outlining
the progression of the analysis.
step provides the foundation upon which subsequent tasks are executed. Instance segmentation models
are used for two critical tasks: isolating synapses and identifying cells. These models extract the spatial
coordinates of individual synapses and cells from images. When available, nuclei labels can also be incorporated into the segmentation process, often allowing one to use pre-trained models to enhance cell detection. If desired, a classification method can categorize these segmented nuclei into specific cell types. An
optional synapse classification step can be introduced post-segmentation to improve synapse specificity.
Since segmentation models are generally more complex and data-intensive, a useful strategy is to identify
synapse candidates through segmentation first and then refine these candidates using a fine-tuned classification model. Finally, the Assignment task is executed to establish the connectivity between synapses
59

and identified cell types. It establishes one-to-many mappings between the cells and synapses, broadening our understanding of synaptic patterns within neural circuits. This workflow enables researchers to
delve deeper into synaptic connectivity, exploring interactions among various neuron subtypes, and thus
improves our grasp of neural connectivity patterns.
3.3.2 Time-Dependent Synaptic Connectivity Patterns
Figure 3.3: Workflow for Time-Dependent Synaptic Connectivity Analysis. Blue blocks represent individual tasks within the pipeline, such as image registration, segmentation, and pairing. Optional classification
steps are not shown. Green blocks denote the data inputs and outputs at each stage, including the spatial coordinates of the synaptic elements at two-time points. The yellow block illustrates the pre-trained
model employed in synapse segmentation. Arrows indicate the flow of data between tasks and data elements, outlining the progression of the analysis.
This scenario emphasizes the pipeline’s capability for longitudinal analysis of synaptic connectivity
patterns, facilitating researchers in studying synaptic changes across multiple time points. Adapted from
our initial project on excitatory synapses, as depicted in Figures 3.3 and Figure 2.6, the pipeline is built to
accommodate time-dependent analyses of synaptic alterations.
In this workflow, researchers initiate the analysis by acquiring labeled synapse images at multiple
time points. Subsequent synapse segmentation isolates individual synaptic elements within these images
60

at each time point. To improve identification accuracy, optional classification steps may be introduced
following segmentation. The pairing task is then performed to identify the same synapses across different
time points, identifying those that are lost, gained, or remain unchanged. This approach enables precise
tracking of synaptic alterations over time.
In highlighting the pipeline’s user-friendly configuration, the code snippet below exemplifies the setup
for this analysis scenario using a YAML file. Note how the use of YAML features simplifies the configuration, specifically using & and * symbols to label and reference individual blocks. For instance, an image
defined on line 5 is later referenced on line 26 to specify a registration task. This streamlined configuration
process simplifies task customization and parameter adjustments for researchers.
1 db_file: /location/of/database_file.sqlite
2
3 initialisation:
4 raw_images:
5 - image: &live_tp1
6 file_name: /data/images/synapses_TP1.tif
7 channel: 0
8 label: gfp
9 image_space: live1 # name this image space, can be anything
10 resolution_xyz: [0.23, 0.23, 0.67]
11 - image: &live_tp2
12 file_name: /data/images/synapses_TP2.tif
13 channel: 0
14 label: gfp
15 image_space: live2
16 resolution_xyz: [0.23, 0.23, 0.67]
17
18 segmentation_models:
19 - model: &stardist_synapse
20 model_folder: /data/models/segmentation/stardist_synapse_001
21 model_type: stardist3D
22 segmentation_type: synapse
23
24 registrations:
25 - registration: &tp1_tp2_affine
26 fixed_image: *live_tp2
27 moving_image: *live_tp1
28 registration_type: Affine
29 transformation_folder: /data/transformations
30
31 segmentations:
32 - segmentation: &synapse_segmentation_tp1
33 image: *live_tp1
34 model: *stardist_synapse
35 parameters:
36 n_tiles: !!python/tuple [1,1,4]
37 prob_thr: 0.1
38 scale_probability: 10000
39
40 - segmentation: &synapse_segmentation_tp2
61

41 image: *live_tp2
42 model: *stardist_synapse
43 parameters:
44 n_tiles: !!python/tuple [1,1,4]
45 prob_thr: 0.1
46 scale_probability: 10000
47
48 transformations:
49 - transformation: &tp1_tp2_transformation
50 points:
51 from_segmentation: *synapse_segmentation_tp1 # segmentation that produced the points
52 transforms:
53 registration: *tp1_tp2_affine
54 direction: forward
55
56 pairs:
57 - pairing: &tp1_tp2_pairing
58 points_1:
59 from_segmentation: *synapse_segmentation_tp2 # segmentation that produced the points
60 points_2:
61 from_transformation: *tp1_tp2_transformation
62 method: unique_nearest_neighbour
63 parameters:
64 max_distance: 4 # in pixels
65 distance_unit: physical # physical or voxel
3.3.3 Time-Dependent & Cell-Type Specific Synaptic Connectivity Patterns
In this scenario, the pipeline integrates features from both core scenarios to provide a comprehensive
analysis of synaptic changes over time in relation to specific cell types. Such an approach offers valuable
information on the temporal dynamics of neural connectivity within specific cellular contexts. This integrated scenario is applied to the study of changes in inhibitory synapses following fear conditioning, as
detailed in Section 4.4 and illustrated in Figure 3.4. By combining these pipelines, researchers can identify
which cell subtypes are most affected by synaptic gain or loss, and assess the influence of external factors
or stimuli on neural circuit connectivity. This multidimensional analysis enriches our understanding of
the mechanisms underlying neural plasticity, learning, and memory.
62

3.4 Synaptic Changes in Inhibitory Synapses
Following Fear Conditioning
In this section, we demonstrate the application of our pipeline to an ongoing research project, focusing on
the local dynamics of inhibitory synapses following fear conditioning. This implementation corresponds
to our third usage scenario, which combines the examination of time-dependent synaptic connectivity
patterns with cell-type specificity. Unlike our previous investigations of excitatory synapses, which looked
at broader subregional synaptic changes, the current study aims to detect small local alterations due to
distinctive characteristics of inhibitory synapses and their role in neural circuit modulation. To achieve
this, our analysis targets perisomatic synapses and investigates their changes after fear conditioning in
relation to specific neuron types, inhibitory or excitatory.
In this section, we detail both the methodological approach and preliminary results of our pipeline as
applied to a single zebrafish sample. We offer an overview of the experimental setup and data collection
carried out by our collaborators and present a detailed account of the data flow, with concrete examples
covering image registration, synapse segmentation, cell classification, and synapse-to-cell assignment for
the selected sample. Our preliminary results are promising; our pipeline has demonstrated the ability to
identify single synaptic changes in the perisomatic region of both inhibitory and excitatory neurons. While
we share our initial findings here, we want to emphasize that our project is continuously evolving, and the
full potential of our pipeline will become evident as we apply it to larger datasets in future studies.
3.4.1 Experimental Design and Data Collection
The experimental design is based on our previous work using the Tail Flick Conditioning paradigm (TFC),
as described in Section 2.4. While we continue to utilize two-timepoint imaging in live fish, this project
63

introduces new molecular tools and experimental techniques, specifically tailored for visualizing and detecting inhibitory synapses and their target neurons. A notable addition is the incorporation of imaging
in fixed tissue, which provides a comprehensive approach to neuron segmentation and classification, and
following synapse assignment. The design of the experiment is shown schematically in Figure 3.4.
Figure 3.4: Schematic Overview of the Fear Conditioning Experiment and Analysis Focusing on Inhibitory
Synapses. (A) Depicts the imaging process, illustrating the collection of 3D images at two time points using Gad1b::GFP label in live fish both before and after Fear Conditioning (FC). Following the time point
2 imaging, fish are sacrificed, stained with antibodies, and a 3D image in fixed tissue is acquired using
Gad1b::GFP, DAPI, and HuC/D labels. (B) Describes the image processing pipeline, starting with synapse
detection in live images, followed by nuclei segmentation in the fixed image, and followed by the classification of segmented nuclei into inhibitory or excitatory neurons based on the colocolisation of labels
from HuC/D and Gad1b::GFP channels. (C) Presents the organization of different inhibitory neuron types:
PV, VIP, and SOM and their synaptic connections to different cellular compartments of other inhibitory
and excitatory neurons. It also highlights how the co-localization of the three labels identifies a neuron
as excitatory (DAPI and HuC/D positive, Gad1b::GFP negative) or inhibitory (positive for all three labels).
(D) Illustrates the analysis concept, emphasizing the assignment of synapses to excitatory and inhibitory
neurons to detect synaptic changes relative to these specific neuron types.
64

3.4.1.a Live Imaging of Synapses Structures Using Gad1b::GFP
In this study, we employ a transgenic zebrafish model (Gad1b::GFP) specifically designed to label inhibitory
neurons. The label is in the cytoplasm of these neurons, highlighting their cell bodies and projections, such
as axons and dendrites. Although Gad1b::GFP does not specifically target presynaptic sites, presynaptic
boutons can still be identified in the labeled images as distinct brighter dots along the axons, Figure 3.5.
Labeling is sparse, given that inhibitory neurons comprise only about 10% of the total neurons in the
pallium, allowing for clear resolution of individual synapses. To capture synaptic changes, we acquire 3D
images of the larval pallium in zebrafish at two different time points, before and after TFC.
Figure 3.5: Imaging inhibitory synapses in larval zebrafish. (A) An individual slice from a stack of images
of Gad1b::GFP in the pallium (red box highlighted region, Inset) showing synaptic puncta, projections, and
cell bodies of inhibitory neurons. (B) A magnified view of the region outlined in A. Orange triangles mark
potential locations of the cell bodies of excitatory neurons or glial cells. (Scale bar, A 10 um, B 5 um)
3.4.1.b Mapping the Locations of Excitatory and Inhibitory Neuron Cell Bodies in Live Images
Our imaging strategy combines live imaging with subsequent immunofluorescence staining, which allows
us to accurately map the locations of both excitatory and inhibitory neuron cell bodies in live imaging
data. First, live images are captured before and after Tail Flick Conditioning (TFC); these images are used
65

Figure 3.6: Immunofluorescence staining allows to distinguish between glia cells and inhibitory and excitatory neurons. (A) An individual slice from a stack of images of Gad1b::GFP after fixation, showing the
cell bodies, projections, and synapses of inhibitory neurons. (B) Same slice as in A with all neuronal cell
bodies labeled with HuC/D (neuronal protein) antibody. (C) Same slice as in A with all cell nuclei labeled
with DAPI antibody (binds to DNA). (D, E, F) Enlarged regions outlined in A, B, C. Glia cells only appear in
the DAPI channel, excitatory neurons in DAPI and HuC/D, and inhibitory neurons are visible in all three
channels. (Scale bars A,B,C 50 um., D, E, F 10 um. )
to identify the locations of the synapses. Afterward, the fish are fixed and stained with antibodies to label
all neurons and nuclei. This is schematically illustrated in Figures 3.4.A and the resulting images are shown
in Figure 3.6. By imaging the GFP channel in both live and fixed fish, we are able to align the live and fixed
images, creating a precise map of the locations of excitatory and inhibitory neuron cell bodies relative
to the synapses. This enables a targeted exploration of the synaptic changes in the vicinity of these cell
bodies. Immunofluorescence staining distinguishes between glial cells, inhibitory neurons, and excitatory
neurons. Specifically, inhibitory neurons are visiblle in all three channels, Gad1b::GFP, HuC/D, and DAPI,
excitatory neurons appear only in HuC/D and DAPI channels, and glia cells only in DAPI.
66

3.4.2 Configuring the Pipeline for Cell-Type-Specific Synaptic Change Detection
Our pipeline is engineered to detect cell-type-specific synaptic changes across time, and it consists of several interconnected processing stages as depicted in Figure 3.7. The pipeline’s steps are categorized into
Image Registration, Synapse-Related Processing, Neuron-Related Processing, and Synapse-to-Neuron Assignment. The workflow begins with the registration of all acquired images to a post-FC reference image,
the live image obtained after fear conditioning, utilizing the Gad1b::GFP channel. This registration produces transformation files that are used in subsequent steps. Synapses are then segmented and classified
in both pre- and post-FC live images. The segmented synapses from the pre-FC image are mapped into the
coordinate space of the post-FC image using the calculated transformation files. Next, these synapses are
paired to detect changes, such as gain and loss, or persistence of the synapse. In parallel, nuclei segmentation occurs on the DAPI channel of the fixed image, followed by classification into excitatory neurons,
inhibitory neurons, or glia cells, informed by additional Gad1b::GFP and HUC/D channels. In the final
stage, these segmented and classified neurons are aligned with the post-FC image using the initial transformation files, allowing us to assign synapses to specific neuron types. An added layer of complexity is
incorporated to resolve these assignments in the context of synapses that remained unchanged to ensure
that synapses at both time points are assigned to the target neuron. This workflow provides spatial data
on the distribution of synaptic changes relative to both inhibitory and excitatory neurons.
3.4.3 Results
Our results provide empirical evidence for the pipeline’s capability to detect and categorize synaptic
changes in the context of fear conditioning. Specifically, we focus on two representative examples to
highlight its sensitivity, as illustrated in Figure 3.8:
67

Figure 3.7
68

Figure 3.7: Workflow for Cell-Type Specific Synaptic Change Detection. Blue blocks represent individual
tasks within the pipeline. Green blocks denote the data inputs and outputs at each stage. Yellow blocks
illustrate the pre-trained models employed in tasks like segmentation and classification. Arrows indicate
the flow of data between tasks and data elements, illustrating the progression of the analysis. Processing
steps are categorized into registration (in yellow), synapse-related processes (in blue), neuron-related processes (in green), and assignment (in red). Arrows pointing downward from the blocks indicate that the
data will be utilized in subsequent steps.
Figure 3.8: Cell-Type-Specific Synaptic Changes Following Fear Conditioning Identified by the Pipeline.
(A) Gained synapse adjacent to an excitatory neuron and (B) lost synapse near an inhibitory neuron. Both
(A) and (B) panels include the DAPI channel in blue and HUC/D in red, imaged in fixed fish after the live
imaging sessions. Live imaging sessions consist of two time points: TP1 and TP2, with TP2 taken 5 hours
after TP1, visualizing Gad1b::GFP in green. In the fixed fish images, an additional GFP channel (not shown)
was taken to align the DAPI and HUC/D channels with the live GFP images from TP1 and TP2. The DAPI,
HUC/D, and TP1 live images shown are registered to the TP2 live image space using affine transformation.
The white arrows indicate the specific locations of the synaptic changes, the gained synapse in panel (A)
and the lost synapse in panel (B).
1. Synapse Loss Assigned to Inhibitory Neuron: A synapse that was present prior to fear conditioning
was no longer detectable afterward. It was specifically attributed to an inhibitory neuron, as verified
by the congruent labeling with DAPI, HUC/D, and Gad1b::GFP.
69

2. Synapse Gain Assigned to Excitatory Neuron: Conversely, a new synapse appeared following fear
conditioning and was assigned to an excitatory neuron. The absence of Gad1b::GFP labeling, in
conjunction with positive DAPI and HUC/D labeling, verified this assignment.
The successful identification and classification of these distinct synaptic changes serve as a validation of
the pipeline’s effectiveness in capturing subtle shifts in synaptic connectivity.
70

3.5 Limitations and Future Directions
3.5.1 Training Data Limitations
It is essential to recognize a fundamental constraint in our pipeline: its performance is intrinsically linked
to the quality of its training data. Our current training dataset for synapse detection, primarily based on
human annotation, poses significant challenges. In fluorescence imaging of synaptic structures, there is
often a notable discrepancy in how different annotators perceive and label synapses. This ambiguity and
variability in human-annotated datasets are not unique to synaptic data but are prevalent in deep learning
applications (Rizos and Schuller, 2020; Tayyab et al., 2023). Different experts may interpret the same data in
different ways, leading to a high degree of uncertainty in the annotations. Such variability directly affects
our detection algorithm, transferring the inherent ambiguity of the data into the pipeline’s outcomes.
To mitigate this issue, a more reliable method for identifying synapses is required, one that minimizes
the reliance on human judgment. A feasible approach is the introduction of two distinct synaptic labels:
pre-synaptic and post-synaptic. The colocalization of both pre- and post-synaptic labels could then be
used as a more definitive indicator of a synapse’s presence. While relying on colocalization may not be
completely error-free, it offers a significant improvement over the current standard by providing a clearer,
more objective criterion for identifying synapses. Implementing this approach could greatly enhance the
accuracy of our training data, and by extension, the effectiveness of the detection algorithms.
3.5.2 Introducing Human-in-the-Loop for Quality Control
Given the noted challenges with our training data, a promising direction for future work involves the introduction of a human-in-the-loop system for quality control (Maadi et al., 2021). This approach would
allow a human expert to periodically review, validate, and correct the pipeline’s output. Implementing
71

such a system would enhance our pipeline’s accuracy and reliability and provide continuous learning opportunities for the machine learning models. As these models are iteratively refined with expert feedback,
they can better handle the complexities and nuances of synaptic data. This direction aligns with the evolving nature of our field, where collaborative efforts between computational tools and human expertise can
significantly advance our understanding of neural circuits and their dynamics.
3.5.3 Integrating Model Training Control and Continuous Updates
Integrating a system for logging model training directly within our pipeline promises to significantly enhance its capabilities. Currently, models are trained outside the pipeline and then imported for use. Although the pipeline monitors each model application, it does not provide insight into the parameters and
data that were used during the training process. We aim to embed a feature that records all training parameters and datasets directly within the pipeline tracking database. MLFlow (Zaharia et al., 2018) and
DVC (Barrak et al., 2021) could be integrated into the pipeline to track changes to training data and model
parameters. These open-source tools are instrumental for the efficient versioning of large datasets and
machine learning models, providing a streamlined approach to tracking and managing changes. This integration would enable a more detailed traceability of the pipeline’s performance, allowing users not only to
recognize the use of a new model version, but also to understand the specific training modifications that
led to performance changes.
Logging model training combined with the continuous model updates, would fit perfectly with the
evolving nature of our pipeline. Consider a scenario where the model conducts synapse detection across
an entire sample, followed by an expert’s targeted corrections in specific areas. These expert-validated
areas then can be used as new training data in the fine-tuning of the model, resulting in a new model
version capable of more refined predictions in future analyses. This iterative process ensures that the
model progressively refines its accuracy and reliability.
72

By documenting the changes made to the training data at each model version, researchers can conduct comparative analyses of different annotation strategies on the pipeline performance. For example,
they could initiate the process with the same model baseline, but apply varying degrees of stringency in
synapse classification during expert corrections. When carefully documented, this approach allows for a
direct assessment of different labeling strategies on model performance, enhancing transparency in model
development and providing valuable insights into the influence of annotation techniques.
Enhanced tracking capability, coupled with expert insights, promises to transform our pipeline into a
more dynamic and robust system, making it an even more powerful resource for researchers.
73

3.6 Conclusion
This study introduced a robust, scalable, and adaptable computational pipeline tailored for the detection
and classification of cell-type-specific synaptic changes. Designed with traceability, flexibility, and accessibility as core objectives, the pipeline accommodates researchers from diverse backgrounds and expertise
levels. While the current focus has been on mapping inhibitory synapses in the context of fear conditioning, the architecture allows for a broad range of applications. The modular structure of the pipeline further
enables code-free customization through a straightforward configuration file and ensures that researchers
with diverse backgrounds, including those with limited programming proficiency, can engage with the
pipeline effectively and tailor it to their needs.
In a broader biological context, this work augments our existing understanding of fear memory formation by providing the tools to shed light on the role of inhibitory synapses in neural circuit modulation. We
have shown that the pipeline is effective in identifying both the loss and gain of synapses and in mapping
these changes to specific types of neurons. Importantly, while the pipeline is tailored to study inhibitory
synapses, its flexible architecture enables adaptation for a variety of synaptic types and research contexts.
In line with best practices in software development, the pipeline is engineered for extensibility, facilitating
future adaptations to meet evolving research demands. Therefore, it serves as a valuable tool for both
the current study and forthcoming investigations into synaptic changes across various forms of learning,
memory, or pathological conditions.
By introducing this computational tool, we not only address current challenges related to inhibitory
synapses but also lay a foundational framework for future large-scale studies of synaptic change across
diverse neurobiological settings. The flexibility and extensibility of the pipeline are expected to enrich
future neuroscience research, enabling a deeper understanding of the intricacies of synaptic dynamics.
74

3.7 Acknowledgments
I am deeply thankful for the exceptional collaboration with Don Arnold and Zhuowei Duo in this research.
Ideas have evolved and materialized through our collective input, showcasing a synergy where each of us
contributes and values each other’s expertise. Don’s project oversight and significant biological insights
have been instrumental in guiding our progress. Zhuowei’s active role in both conducting experiments and
participating in study design has been invaluable. This dynamic interaction has been crucial in shaping
the study’s design and its successful execution. I am also grateful for funding from the National Institute
of Neurological Disorders and Stroke under Grant [1UF1NS122082-01]. This support has been essential
not only to facilitate our research but also to foster an environment conducive to such collaboration.
3.8 Code Availability
The code is provided as an open-source Python package at https://github.com/lemonjust/sprinkle.
75

Chapter 4
VoDEx: a Python Library for Time Annotation and Management of
Volumetric Functional Imaging Data
4.1 Synopsis
Functional imaging is widely used in neuroscience studies for recording brain activity in parallel with behavioral and physiological data of organisms (Dombeck et al., 2010; T. H. Kim and Schnitzer, 2022; Macé
et al., 2011; O’Craven and Kanwisher, 2000). Such studies often have complex experimental designs, with
the aim of characterizing neural responses to experimental tasks and/or detecting differences in brain activity patterns among various stimuli, behaviors, and physiological states. Accurate analysis of functional
imaging data requires accurate annotations of the time course of the experiment and synchronization of
the time annotations with the imaging data.
Linking time annotations to volumetric imaging data presents an additional challenge, as volumes are
acquired as a series of optical sections: a series of 2D images are taken sequentially at different depths
inside the sample, which are then assembled into a 3D dataset. To correctly interpret the volumetric data,
it is crucial not only to track the correspondence between image frames and experimental conditions, but
also the exact location of these frames within a volume. Such a complexity of time annotation, combined
76

with the unprecedented amount of data produced by functional imaging experiments, makes manual data
handling tedious and unreliable.
In recent studies involving large-scale volumetric functional imaging (Haesemeyer et al., 2018,
Marquez-Legorreta et al., 2022; Mu et al., 2019), the data management and time annotation functionalities were integrated into the entire study-specific analysis pipeline, hindering modular validation, and
precluding extraction and application of these functionalities to other studies in the wider community.
This chapter describes an open-source Python library that I developed to address the challenge of
synchronizing time annotations and volumetric information with 3D imaging data, VoDEx (Volumetric
Data and Experiment Manager), and which has been recently published ( Nadtochiy, A., Luu, P., Fraser,
S. E., & Truong, T. V. (2023). VoDEx: A python library for time annotation and management of volumetric
functional imaging data (H. Peng, Ed.). Bioinformatics. https://doi.org/10.1093/bioinformatics/btad568
) VoDEx integrates information about individual image frames, volumes, and experimental conditions
and allows the retrieval of sub-portions of the 3D-time series datasets based on any of these identifiers
without the need to load the entire dataset into memory. It is designed as a low-entry-barrier tool to store
information in a simple, accessible way, enabling later data verification and sharing in accordance with
the FAIR (Findable, Accessible, Interoperable, and Reusable) principles (Dempsey et al., 2022; Wilkinson
et al., 2016).
To demonstrate the capabilities of VoDEx, we present its application to the study of numerosity in
zebrafish larvae, where it plays a key role in the processing of whole-brain functional imaging data acquired
using light-sheet fluorescence microscopy.
4.2 Design and Implementation
VoDEx is implemented both as an open source Python package and a napari (Sofroniew et al., 2022) plugin
for interactive use with a GUI. Python’s rich ecosystem of libraries, such as NumPy (Harris et al., 2020),
77

SciPy (Virtanen et al., 2020), and scikit-image (Van der Walt et al., 2014), makes VoDEx a useful tool for
image analysis and allows for integration into a wide range of analysis pipelines.
VoDEx uses an SQLite database to store data, taking advantage of its lightweight and portable nature,
zero configuration need, and efficient data storage and query capabilities (Hipp, 2020).
4.2.1 Modules and Functionality
VoDEx contains classes that assist in the creation, organization, and storage of information related to image
acquisition and time annotation, allowing for the search and retrieval of image data based on specific
conditions. This functionality is split into five modules: core, annotation, dbmethods, experiment, and
loaders.
• The core module provides the basic functionality for retrieving image data information.
• The annotation module handles the construction, validation, and storage of time annotation. VoDEx
keeps track of cycle iterations for cyclic events, which is important in behavioral experiments where
the subject might become habituated to the repeated stimulus or learn over the course of the experiment.
• The dbmethods module abstracts the SQL calls, providing an easy-to-use interface to populate the
SQLite database in which VoDEx stores information (Figure 4.1), and to query the database.
• The loaders module contains classes designed to load image data from specific file types, with current
support for TIFF, and allows for easy addition of support for other file formats.
• The experiment module contains the Experiment class, connecting all the functionalities of the
VoDEx package and serving as the main point of entry for interacting with the package.
78

Figure 4.1: Vodex Experiment Database Schema. The diagram illustrates the structure of the Vodex SQLite
database used to store information related to experiments. It details various tables and their respective
fields: Files: Contains ’FileName’ (filenames relative to the main directory) and ’NumFrames’ (number of
frames in each file). AnnotationTypes: Includes ’Name’ (types of annotations) and ’Description’ (optional
descriptions of annotation types). Frames: Links each frame to a file (’FileId’) and specifies a frame within
that file (’FrameInFile’). Cycles: For annotations derived from cycles, stores the cycle as a JSON string
(’Structure’) with a corresponding ’AnnotationTypeId’. AnnotationTypeLabels: For each annotation
type (’AnnotationTypeId’), lists label names (’Name’) and optional descriptions (’Description’). Volumes:
Connects each frame (’FrameId’) to a volume (’VolumeId’) and a slice within that volume (’SliceInVolume’).
CycleIterations: Associates each frame (’FrameId’) with a cycle (’CycleId’) and its iteration (’CycleIteration’). Annotations: Links each frame (’FrameId’) to a label (’AnnotationTypeLabelId’). Options: Keyvalue pairs for additional information, including ’data_dir’ (image files directory), ’frames_per_volume’
(frames per volume parameter), and parameters for the head and tail frames as well as the count of full
volumes.
79

4.2.2 Pipeline: Data Mapping and Querying
The VoDEx pipeline consists of two steps: data mapping and data querying. An illustration of the VoDEx
pipeline applied to a Toy Dataset is shown in Fig.4.2 (Toy Dataset is available via the project website at
https://lemonjust.github.io/vodex/data/). During the mapping step, VoDEx creates a mapping between
image frames, their location within associated files, the image volumes they correspond to, experimental
conditions, and cycle iterations. This information is saved to a database, allowing users to save the experiment description for sharing or to return to it later. In the second step, users can investigate the data by
querying the database for image frames or volumes that were recorded during specific combinations of
the mapped conditions. The querying process does not require prior knowledge of database query syntax
and is conducted via methods provided by VoDEx. When requesting frames, VoDEx returns the indices
of all the image frames matching the request. When requesting volumes, VoDEx selects those frames that
constitute full volumes and returns the corresponding volume indices. The indexing of frames and volumes start from the beginning of the recording. The image data can thus be loaded based on these indices
as a 3D or a 4D numpy array for frames or volumes, respectively. The package offers a graphical user
interface (GUI) through its integration with napari via the napari-vodex plugin, enabling users to easily
navigate the full VoDEx pipeline and to load selected volumes directly into the napari viewer. The two
steps of the pipeline can be performed independently using either the script or the GUI.This permits users
to annotate the data using the GUI, and to then switch to a script to query the data; alternately, users can
perform data annotation in a script, load this annotation into a GUI, and then interactively explore the
data in napari. Comprehensive documentation of VoDEx, including examples, tutorials, and toy datasets,
are provided at the project website https://lemonjust.github.io/vodex/. The documentation is continuously
updated through the use of mkdocs and mkdocstrings packages, as well as GitHub actions, ensuring that
any changes to the API are promptly reflected in the documentation. The package has undergone rigorous
testing, achieving 100% test coverage, through the use of pytest.
80

Figure 4.2: Illustration of the VoDEx pipeline applied to a Toy Dataset.
The Toy Dataset is available via the project website at lemonjust.github.io/vodex/data/. The dataset comprises 42 image frames, divided into three TIFF files. Each volume consists of 10 frames, resulting in 4 full
volumes, with 2 additional frames at the end of the recording. In the dataset, three conditions are tracked:
light, label, and shape. The background of the frame indicates the light (on/off), the middle of the screen
shows the label (c1, c2, c3), and the screen displays either a circle or a square.
In Step 1, the user inputs information about the imaging data and the experimental conditions into VoDEx,
either through a Python script or a GUI in the napari plugin. Specifically, the user provides the information
for the whole recording to encode light, but only one cycle iteration of the label and shape. Note that the
shape conditions switch in the middle of a volume. VoDEx then integrates this information, automatically
determining the number of frames in each image file, estimating which frames correspond to which volumes, and repeating the provided cycles to cover the duration of the entire recording while keeping track
of both conditions and cycle iterations. The information is stored in a database created by VoDEx. For
instance, frame number 30 is stored as a 4th frame inside the 3rd TIFF file. It represents the last slice (10th
slice) in the 3rd volume. This frame features a square shape, with the label "c3" and the light "on". It was
recorded during the second iteration of the shape cycle and the first iteration of the label cycle.
In Step 2, the user can search and access imaging data based on mapped experimental conditions using
either a Python script or the GUI in the napari plugin. Conditions can be combined using "and" or "or"
logic. For instance, to retrieve the volumes where the label is c3 and the light is on, VoDEx points to volume
index 3. When searching for the label c3 or the light is on, it returns indices for both volumes 2 and 3.
If conditions do not span an entire volume, one can pinpoint individual image frames that align with the
condition (not depicted). Once the frame or volume indices are determined, VoDEx can load the volumes
as numpy arrays for in-depth Python processing or show them directly in the napari viewer. Alternatively,
the volume or frame indices can serve as discrete time markers for temporal analysis. This empowers researchers to isolate pertinent intervals from associated time series, such as extracted neural activity.
Figure adapted from Nadtochiy et al., 2023
81

4.3 Usage Scenarios
This section explores the practical applications of VoDEx, a versatile tool for functional volumetric data
analysis. We examine how VoDEx supports two common approaches in processing calcium imaging data,
including high-throughput assays and analysis of single-cell neuronal activity.
4.3.1 High-Throughput assays on Calcium Data Processing
The first approach, common in high-throughput assays, uses raw imaging data directly and aims to identify
large-scale differences in brain activity under various conditions. This type of analysis can be used in drug
screening experiments (Lin et al., 2018). VoDEx is an ideal tool for data handling in such experiments, as
it allows easy loading and comparisons of image volumes or slices from different conditions.
4.3.2 Individual Neuron Activity on Calcium Data Processing
The second approach starts by extracting activity traces of individual neurons from imaging data, usually
with tools such as CaImAn (Giovannucci et al., 2019a) or Suite2p (Pachitariu et al., 2016). The focus then
shifts to these activity traces rather than the raw data. VoDEx simplifies this process by streamlining data
preprocessing, helping to adjust 3D data for 2D tools, and pinpointing parts of the extracted time series
linked to different experimental conditions.
Although highly effective for 2D data, CaImAn and Suite2p are not directly suitable for large 3D
datasets. A common workaround is to use tools such as SimpleITK (Beare et al., 2018) or ANTs (Avants
et al., 2011b) for 3D motion correction and then process individual image slices with these 2D tools. The
same neuron might appear on adjacent slices; thus detected cells from different slices are merged based
on activity and location, and, finally, the cell signals are extracted. This method can be complex and timeconsuming.
82

VoDEx eases this by offering batch processing (Section 4.3.4.b), simplifying the 3D motion correction
setup. Moreover, its ability to access individual slices from a full recording helps bridge 2D and 3D analysis.
After extracting the neural signals, VoDEx can isolate signal parts tied to certain experimental conditions
for further analysis or building the regressor for activity prediction, when using regression analyses on the
signals. Lastly, annotations made with VoDEx can be directly shared as a compact database file for VoDEx
users or exported as a CSV file for wider accessibility.
4.3.3 Transitions between behaviors
VoDEx is particularly useful for experiments in which time annotation is derived from experimentally
measured events, such as the behavior of an organism or physiological data. Unlike well-controlled stimuli
sequences that align with the rate of volumetric image collection, transitions between behaviors in such
experiments can occur at various times during the recording of individual volumes. VoDEx provides an
easy way to detect and manage these types of events.
4.3.4 Code examples
In this section, we provide two practical examples to illustrate the user-friendly nature of VoDEx for functional imaging data processing: VoDEx Initialization and Batch Processing. For detailed instructions on a
wide range of other tasks, please refer to the Quick Start section of the documentation
(lemonjust.github.io/vodex/qstart/new_experiment/). This resource covers tasks such as adding time annotations, isolating time points for specific experimental conditions, loading volumes, loading slices, and
data sharing.
In addition, some tasks can be accomplished using a graphical user interface. A comprehensive tutorial
on the GUI usage is available in the napari-vodex plugin description
83

(lemonjust.github.io/vodex/napari/how-to/). This tutorial offers a step-by-step guide for users who prefer
a graphical approach to using VoDEx’s features.
4.3.4.a Initialization
Interaction with the data occurs through an object of the experiment class of the VoDEx package that
contains all the information about the data set and the time annotations.
To begin processing the 3D movie, we need to initialize an object of the experiment class by providing
the file locations and the number of frames per volume. Time annotation information can be included now
or added later. The following code snippet demonstrates one way to initialize the object:
1 import vodex as vx
2 # Specify the directory path to the raw data files
3 data_dir = "path/to/raw_data/"
4 # Specify the number of frames per volume and the starting slice
5 frames_per_volume = 67
6 starting_slice = 0
7 # Create the experiment object from the directory using the specified parameters
8 experiment = vx.Experiment.from_dir(data_dir,
9 frames_per_volume,
10 starting_slice,
11 verbose=True)
12 # Save the experiment for future use
13 experiment.save("experiment.db")
For more detailed information and alternative methods to initialize VoDEx, please refer to the documentation or the Quick Start Guide available at https://lemonjust.github.io/vodex/qstart/new_experiment/.
Alternatively, the napari-vodex plugin can be used to initialize the experiment using a GUI; a step-bystep guide is available at https://lemonjust.github.io/vodex/napari/how-to/.
Once created, the object can be saved and loaded later. Save the experiment to a local SQLite database:
1 # Save the experiment for future use
2 experiment.save("experiment.db")
Load the saved experiment:
1 # Load the experiment from the saved database
2 experiment = vx.Experiment.load("experiment.db")
84

4.3.4.b Batch Processing of 3D Movies
Once VoDEx is initialized, the experiment object can be used for easy batch processing of the data set. All
the necessary methods, including splitting the full brain volumes into batches and loading the volumes,
are provided by VoDEx. A template for a batch-processing code is shown below:
1 # Split the volumes into chunks of the defined size that will be loaded into RAM at once
2 chunks = experiment.batch_volumes(
3 batch_size, # the number of volumes in each batch
4 full_only=True, # if True, only full volumes are returned
5 overlap=overlap ) # the number of volumes that overlap between batches
6 )
7 for chunk in chunks:
8 # Load the full volumes in the chunk
9 data = experiment.load_volumes(chunk, verbose=False)
10 # Perform processing
11 # ...
85

4.4 Application: The Study of Numerosity in Zebrafish Larvae
To demonstrate the capabilities of VoDEx, we present its application to the study of numerosity in zebrafish
larvae, where it played a key role in the processing of whole-brain functional imaging data acquired using
light-sheet fluorescence microscopy.
Numerosity, the ability to perceive and evaluate discrete quantities in a set, is a fundamental aspect of
cognition that underpins complex behaviors and decision-making processes in humans and many other
species (Nieder, 2020). The zebrafish larva, with its small size, genetic accessibility, and optical transparency, provides an ideal model to study the neural basis for numerosity at the cellular level (Messina
et al., 2022).
Previous research identified a region of the zebrafish pallium that selectively responds to changes in
the numerosity of visual stimuli (Messina et al., 2022), suggesting an evolutionarily conserved mechanism
for approximate numerical magnitude estimation. However, that study focused on a subregion of the
pallium, leaving the role of other brain regions in numerosity processing largely unexplored. The study
presented in this supplementary note aims to address this knowledge gap by investigating the neural
activity throughout the brain of larval zebrafish while they are presented with visual numerosity stimuli
consisting of different numbers of dots in various geometric patterns. We describe below how VoDex is
used in this numerosity study’s data management and analysis.
4.4.1 Overview of the Experiment and Data Acquisition
Zebrafish larvae were presented with a series of visual stimuli in a pseudo-random order with variable
timing. The numerosity stimuli included a blank screen and screen with one to five dots in varying geometric patterns, commonly used to control for potential confounding effects (Gebuis et al., 2016; Pennock
et al., 2021; Piazza et al., 2004). VoDEx facilitated the interpretation of the responses to the stimuli, which
required accurate annotation and tracking of various stimulus patterns. VoDEx efficiently managed the
86

sets of visual stimulus patterns and enabled the processing of large volumetric imaging datasets on a standard computer. The implementation was carried out in Jupyter notebooks and in a custom Python package
specifically designed for this study, showcasing the versatility of integrating VoDEx into a comprehensive
analysis pipeline.
4.4.2 Dataset Specification
To illustrate the application of VoDEx in analyzing the neural activity recorded during the presentation
of the visual numerosity stimuli, we will use an individual dataset as an example. The neural activity,
reported by the genetically encoded calcium indicator H2B::jGCaMP7f1 (Dana et al., 2019), was captured
using light-sheet microscopy (Keomanee-Dizon et al., 2022) at single-cell resolution. Each brain volume
was captured as a sequence of frames and contained 67 frames per volume. This dataset was collected
during a numerosity stimuli sequence in which the fish was presented with numerosities 1, 2, 3, 4, and 5.
The recording comprises a total of 1900 brain volumes acquired over a period of approximately 2 hours,
organized into 17 TIFF files by the acquisition software (Edelstein et al., 2014), with each file having a size
of about 4Gb.
4.4.3 Visual Numerosity Estimation
During the numerosity stimuli presentation, zebrafish larvae were exposed to visual stimuli that aim to isolate neural responses specific to number processing by controlling for both numerical and non-numerical
variables. The stimuli consisted of black dots presented on a red background, with the quantity of dots
varying from one to five. To account for non-numerical variables that may co-vary with the number of
dots, two specific geometric parameters were controlled: the relative position of the dots and the size of
individual dots. The relative position was regulated by maintaining either an equivalent convex hull or an
equivalent inter-distance, ensuring consistent spatial relationships between the dots. The size of the dots
87

Figure 4.3: Visual stimuli used in the numerosity stimuli presentation. The figure illustrates the different
geometric parameters used to generate the stimuli across different numerosities. The stimuli maintain
either a constant convex hull (smallest convex polygon that encloses all of the elements) or inter-distance
(average distance between the dots), while the size and circumference of the dots are controlled by maintaining a constant radius, total area, or total perimeter. Convex hull or inter-distance have no impact on
numerosity "1", while convex hull is not taken into account for numerosity "2". A “blank” screen, consisting of the same red background but without any dot pattern, is used to reset between different numerosity
stimuli. The patterns were generated using GeNEsIS (Zanon et al., 2021).
Figure adapted from Nadtochiy et al., 2023
was maintained by controlling the dot radius, total dot area, or total dot perimeter for each numerosity,
Figure 4.3. The stimuli were presented in a pseudo-random order, Figure 4.4, which included all possible
combinations of relative dot positions and individual dot sizes while varying numerosities. The sequence
consisted of 30 different visual stimuli based on combinations of parameters: 2 possible relative positions,
3 possible dot sizes, and 5 numerosities. Such design of the visual stimuli allowed us to isolate the neural
responses specific to numerosity and ensure that these responses were independent of variations in other
visual cues.
88

Figure 4.4: Pseudo-random stimulus cycle for numerosity response isolation in zebrafish larvae. Each
stimulus consists of a 3-second presentation of the dot pattern, followed by a blank screen lasting between
15 and 27 seconds. One cycle has a total duration of 372 seconds. The cycles alternate between two
conditions: one with an equivalent convex hull, and the other with an equivalent inter-distance. Each
condition includes 15 stimuli, covering all numerosities while maintaining a constant radius, total area, and
total perimeter. Different cycles feature distinct dot patterns while adhering to the respective geometric
constraints. The cycle is repeated 18 times per sample.
Figure adapted from Nadtochiy et al., 2023
4.4.4 Data Processing and Analysis of a Single Fish Sample
The data processing pipeline for a single fish sample involved four key steps:
• The raw calcium movie was processed to remove slow changes in the calcium signal using a sliding
window approach to calculating a relative change in fluorescence (dF/F) per voxel.
• A Statistical Parametric Map (SPM) was created by calculating the difference between brain activity
during stimulus presentation and the blank screen (no stimuli) period for each voxel. This SPM
highlighted the voxels in the brain images that exhibited significant responses to the stimuli.
• Adjacent voxels with similar scores in the SPM were grouped together to identify individual cells,
each group roughly set to the size of a single cell. This grouping step enabled the extraction of cell
signals from the raw movie. To ensure consistency and comparability across cells, the extracted cell
89

signals were further normalized and slow changes in the calcium signal were removed, similar to
the processing of raw calcium movies.
• The cell responses to different numerosity stimuli and different non-numerical variables were compared to assess the significance of differences in cell responses to different visual stimuli. This allowed us to identify the cells that were responsive to a particular numerosity regardless of the dot
positions and individual dot sizes.
VoDEx proved to be a valuable tool throughout this data processing and analysis pipeline.
4.4.5 VoDEx utility for Numerosity Study Data Processing
In the context of processing numerosity data, VoDEx serves as the foundational framework for NumAn,
a specialized package developed for the analysis of numerosity datasets (Section 4.4.5.a). VoDEx offers
a user-friendly interface to create batch processing scripts that directly handle brain volumes (Section
4.3.4.b). This feature greatly simplifies the management and processing of extensive datasets, such as our
numerosity dataset, which comprises roughly 60 GB of data. We utilize batch processing to normalize raw
calcium movies and extract cell signals from them. VoDEx also streamlines the creation of the Statistical
Parametric Map (SPM) by selectively loading relevant imaging data.
Our precise analysis of the numerosity dataset, which contains 1900 brain volumes across 30 distinct
conditions, is based on accurate annotation and tracking of stimulus patterns. VoDEx’s time annotation and
query capabilities play a crucial role in ensuring the faithful interpretation of neural responses, addressing
the complexities inherent in our stimuli.
VoDEx simplifies the addition of time annotations and helps to isolate neural responses specific to
numerosity while distinguishing them from responses related to other geometric parameters. With VoDEx,
we can monitor both numerical and non-numerical variables in visual stimuli, enabling precise analysis of
signals associated with specific combinations of these variables.
90

4.4.5.a NumAn: a Python Tollbox for Numerosity Analysis
NumAn, which stands for Numerosity Analysis, is a set of methods designed for neuron segmentation and
calcium signal analysis. NumAn relies on VoDEx as its underlying framework. VoDEx serves as the cornerstone of this package, providing essential capabilities for managing complex numerosity stimuli data.
It enables the selective loading of data corresponding to individual stimuli presentations and facilitates the
isolation of specific time points of interest within the extracted neural signal traces, Figure 4.5.
Figure 4.5: Segmentation of numerosity-tuned cells. Schematic of the processing pipeline. The pipeline
allowed us to detect neurons in 3D functional imaging data tuned to different numbers of stimuli. The
pipeline is based on the NumAn python package. (A) The raw calcium movie is normalized to remove
slow changes in the calcium signal by calculating a relative change in fluorescence (dF/F) per voxel with a
sliding window. (B) The t-statistic for the difference between two visual stimuli is calculated per voxel. (C)
Adjacent voxels with similar t-score are grouped into individual cells. (D) The cell signal is extracted from
the raw movie and normalized, after that (E) the cell response to different number stimuli is compared
using bootstrap.
Existing tools for neural segmentation (Pachitariu et al., 2017, Giovannucci et al., 2019b) are designed
primarily for 2D data and aim to identify all active neurons within a recorded volume. However, this task
is computationally challenging and requires data with a high signal-to-noise ratio to be effective.
91

In NumAn, we have taken a different approach inspired by fMRI analysis. We create a 3D statistical
map to highlight differences in neural activity during different visual stimuli (Friston et al., 1994). Although
our method may be less complex than some other segmentation techniques, it excels when dealing with
data that has a low signal-to-noise ratio, a situation where other tools often struggle. A single neuron can
occupy multiple voxels in an image and these voxels typically respond similarly to stimuli. To address this,
we group neighboring voxels with similar statistical properties, allowing us to identify individual neurons
and extract their signals. You can see a visual representation of this process in Figure 4.5.
4.4.5.b Time Annotation of Numerosity Stimuli
Before the analysis of neural responses to visual stimuli with NumAn, we incorporate time annotation
into VoDEx. At the first stage of the analysis, we only need to distinguish between the presentation of
different numerosity stimuli; thus, we do not need detailed information about the stimuli geometry and
will only add the annotation for the number of dots presented on the screen. We provide a code snippet
that demonstrates how to add a time annotation through code in Section 4.3.4. Below, we will show how to
load an annotation from Table 4.1 from a CSV file. Please note that the duration of stimuli can be provided
in various units such as frames, seconds, volumes, etc. However, if units other than frames are used, it
is required to provide a method for converting them to frames. In our example, we specify the stimulus
duration in volumes and use the timing_conversion argument to indicate how volumes can be converted
to frames.
1 # Load the experiment from the saved database
2 experiment = vx.Experiment.load("experiment.db")
3 # Load the CSV file with the numerosity annotation
4 numerosity_df = pd.read_csv("numerosity_annotation.csv", index_col = False)
5 # Add the time annotation to the experiment
6 # The timing_conversion specifies that
7 # 1 volume corresponds to frames_per_volume number of frames (67)
8 # The frames_per_volume is obtained from the experiment metadata
9 experiment.add_annotations_from_df(numerosity_df,
10 cycles = True,
11 timing_conversion= {"volumes":1,
12 "frames":experiment.frames_per_volume})
13 # Save the changes to the experiment
14 experiment.save("experiment.db")
92

duration_volumes name group description
3 b number blank, no dots
1 d4 number 4 dots on the screen
9 b number blank, no dots
1 d3 number 3 dots on the screen
7 b number blank, no dots
1 d5 number 5 dots on the screen
5 b number blank, no dots
1 d2 number 2 dots on the screen
7 b number blank, no dots
1 d1 number 1 dot on the screen
9 b number blank, no dots
1 d4 number 4 dots on the screen
5 b number blank, no dots
1 d2 number 2 dots on the screen
7 b number blank, no dots
1 d5 number 5 dots on the screen
7 b number blank, no dots
1 d3 number 3 dots on the screen
5 b number blank, no dots
1 d1 number 1 dot on the screen
9 b number blank, no dots
1 d4 number 4 dots on the screen
9 b number blank, no dots
1 d1 number 1 dot on the screen
5 b number blank, no dots
1 d3 number 3 dots on the screen
9 b number blank, no dots
1 d2 number 2 dots on the screen
7 b number blank, no dots
1 d5 number 5 dots on the screen
6 b number blank, no dots
Table 4.1: Annotation for the Number of Dots Presented on the Screen
93

After individual neuron signals have been extracted with NumAn, we extend our annotations to account for other nonnumerical variables, allowing us to explore their influence on cell responses. To illustrate the implementation of this expanded time annotation, we provide a code snippet below. In this
snippet, we add two separate annotations: a dot position and an individual dot size. As before, we will
load the annotation from a CSV file. Please note that we can combine multiple annotations in one file, as
shown below (Table 4.2).
duration_volumes name group description
124 ch position equivalent convex hull
124 id position equivalent inter-distance
4 cr size constant radius
10 cr size constant radius
8 cr size constant radius
6 cr size constant radius
8 cr size constant radius
10 ta size equivalent total area
6 ta size equivalent total area
8 ta size equivalent total area
8 ta size equivalent total area
6 ta size equivalent total area
10 tp size equivalent total perimeter
10 tp size equivalent total perimeter
6 tp size equivalent total perimeter
10 tp size equivalent total perimeter
14 tp size equivalent total perimeter
Table 4.2: Annotation for Geometric Parameters of Visual Stimuli
1 # Load the experiment from the saved database
2 experiment = vx.Experiment.load("experiment.db")
3 # Load the CSV file containing the spread and size annotations
4 covariate_df = pd.read_csv("covariate_annotation.csv", index_col = False)
5 # Add the time annotation to the experiment
6 experiment.add_annotations_from_df(covariate_df,
7 cycles = True,
8 timing_conversion= {"volumes":1,
9 "frames": experiment.frames_per_volume})
10 # Save the changes to the experiment database
11 experiment.save("experiment.db")
94

4.4.5.c Exploring Neural Signal Traces
Analyzing cell responses in the context of complex visual stimuli involves comparing activity patterns
across different sets of stimuli parameters. Since each timestep of the extracted cell signals corresponds to
one volume for each cell, we can leverage VoDEx’s querying capabilities to retrieve cell signals associated
with specific combinations of numerical and non-numerical variables, as demonstrated below.
1 # Retrieve cell signal corresponding to the 5 dots with a constant radius
2 # (16 such timepoints)
3 d5_cr_idx = experiment.choose_volumes([("number", "d5"),("size", "cr")])
4 # cell_signal: numpy array of shape (time_steps, )
5 # representing the cell signal previously extracted from the volumetric movie
6 d5_cr_signal = cell_signal[d5_cr_idx ]
7
8 # Retrieve cell signal corresponding to the 5 dots, a constant radius, and equivalent convex hull
9 # (8 such timepoints)
10 d5_ch_idx = experiment.choose_volumes([("number", "d5"),("size", "cr"),("position", "ch")])
11 d5_cr_ch_signal = cell_signal[d5_ch_idx]
With the ability to load the necessary signals, researchers can perform various data analysis tasks or
organize the cell signals into a table for subsequent analysis in external software, as demonstrated below:
1 # Retrieve all time points (volumes) during stimulus presentation for numerosities 1 to 5
2 stim_volumes = experiment.choose_volumes([("number", "d1"), ("number", "d2"),
3 ("number", "d3"), ("number", "d4"),
4 ("number", "d5")], logic="or")
5 # Get cell signal for these time points
6 stim_signal = cell_signal[stim_volumes]
7 # Get annotation information for these time points
8 annotation_dict = experiment.get_volume_annotations(stim_volumes)
9 # Create a summary dataframe combining cell signals and annotations
10 annotation_dict['cell_signal'] = stim_signal
11 annotation_df = pd.DataFrame(annotation_dict)
12 # Save the annotation dataframe to a CSV file
13 annotation_df.to_csv("stim_full_annotation.csv", index=False)
In this code snippet, we first retrieve the time points during stimulus presentation for numerosities 1
to 5. We then extract the corresponding cell signals and obtain the annotation information for these time
points using VoDEx. Finally, we combine the cell signals and annotations into a summary dataframe and
save it as a CSV file (Table 4.3). With the created annotation dataframe, one can perform various statistical
analyses, including multi-factor ANOVA, to examine the effects of different factors on cell responses.
95

number size position volumes cell_signal
d4 cr ch 3 0.222
d3 cr ch 13 0.286
d5 cr ch 21 0.068
... ... ... ... ...
... ... ... ... ...
d3 tp id 1959 0.130
d2 tp id 1969 0.265
d5 tp id 1977 0.077
Table 4.3: Summary of Cell Signals and Annotations for Stimulus Presentation
4.4.6 Results
We recorded the whole-brain neural activity of zebrafish larvae at 8 days post fertilization (dpf) while
subjecting them to repeated visual stimuli representing different numerosities. Specifically, the stimuli
consisted of a blank screen (without the numerosity stimulus), 2 dots, and 5 dots.
To process and analyze the recorded neural data, we used the VoDEx and NumAn packages. The
pipeline is available as a set of Jupyter Notebooks. This pipeline allowed us to identify distinct groups of
neurons exhibiting varying levels of activity in response to the presentation of different stimuli.
Our findings revealed that neurons that responded differently to 2 versus 5 dots were mainly located in
the forebrain and displayed a phenomenon known as number tuning, indicating their specific sensitivity to
numerical distinctions, while the neurons that differentiated between 2 dots and a blank screen or between
5 dots and a blank screen were localized to the optic tectum, Figure 4.6. This result is consistent with
existing literature in the field. These preliminary results can be found in Messina et al. (2022).
96

Figure 4.6: Using Whole-brain functional imaging to find neural substrate of zebrafish numerosity capability. (A) Schematic drawing of the brain. The forebrain includes the pallium (Pa) and the habenula (Hb).
The midbrain contains the optic tectum (OT). (B) Representative results from the forebrain of a 8-dpf larvae
(region outlined with the red box in A). The background standard deviation image (magenta) highlights all
time-varying activity. Identified neurons (green) show different levels of activity during different stimuli:
two dots versus five, two and five dots versus a blank screen. (C) Normalized activity traces of the two
neurons selected by arrowheads in panel B averaged over n = 50 trials. Neuron 1 is tuned to 5, neuron 2 is
tuned to 2, P < 0.05, bootstrapping with resampling. Error bars represent SEM, n = 50. (Scale bar, 100 um).
Figure adapted from Messina et al., 2022
97

4.5 Limitations and Future Directions
4.5.1 Expansion for Image Format Support
Currently, VoDEx’s capabilities are constrained by its limited compatibility with only TIFF images. To
address this limitation, a key aspect of VoDEx’s future development is encouraging community-driven expansion, particularly regarding support for new image formats. VoDEx has been designed with a modular
architecture, enabling users to easily integrate additional image formats. This approach acknowledges that
the diversity of imaging formats in neuroscience and biological research is vast and continually evolving,
making it impractical for a single developer or team to anticipate and cater to all possible formats.
The core of this expansion lies in providing comprehensive guidelines on how users can extend the capabilities of VoDEx. Rather than implementing specific format support directly, VoDEx will offer detailed
descriptions and templates to guide users in adding new formats. This approach entails creating and documenting a clear, user-friendly process for developing new loader classes. By following these guidelines,
users can adapt VoDEx to their specific imaging formats, making the tool more versatile and applicable
across various research contexts. This direction not only enhances the flexibility of VoDEx but also fosters a collaborative development environment. By enabling researchers and developers to contribute their
extensions, VoDEx becomes a more comprehensive tool, enriched by the diverse experiences and needs of
its user community.
4.5.2 Enabling User-Created Database Queries
Another critical area for future development is the enhancement of database query flexibility in VoDEx.
At present, while VoDEx allows users to request specific volumes or frames based on annotations, this
represents only a fraction of the potential querying capabilities. Full access to the database could enable
98

queries based on more intricate conditions, such as specific slices or cycle iterations, which are currently
inaccessible to users without SQL expertise.
Recognizing that experimental designs in neuroscience can be incredibly diverse, with specific and
sometimes unique data interrogation needs, VoDEx aims to provide a framework where users can add
their custom queries. The focus will be on developing and sharing a set of guidelines and templates that
allow users to understand and leverage the underlying database structure of VoDEx. These resources will
guide users through the process of designing and testing new query methods, ensuring that these additions
are compatible with the existing VoDEx framework.
This approach underscores VoDEx’s commitment to being a user-centric tool, adaptable to the specific requirements of individual research projects. By enabling researchers to tailor the database querying
process to their needs, VoDEx becomes not just a tool for the community but a platform shaped by the
community’s expertise and ingenuity. This strategy aligns with the broader goals of open-source software,
promoting innovation, inclusivity, and collaboration in scientific research.
4.5.3 Enhancing Experiment Sharing Capabilities
A vital enhancement for VoDEx involves augmenting its experiment sharing capabilities. This improvement will facilitate better collaboration among researchers and promote open science practices. To maintain the integrity of data when shared, VoDEx plans to implement tools that verify data consistency, such
as MD5 checksums. This feature will allow users to confirm that the data has not been altered or corrupted
during transfer. The process involves generating an MD5 checksum for the dataset at the source location,
which can then be stored in the database and used to verify the data at the destination. This verification
ensures that collaborators receive the data exactly as intended, preserving the reliability of the shared
experimental results.
99

4.6 Conclusion
The last decade has witnessed significant advancements in fluorescent imaging tools and labeling techniques, profoundly impacting the studies of neuronal activity. The shift to imaging neuronal activity at
single-cell resolution across the entire brain and complex behavioral assays has generated increasingly
complex 3D data sets.
In this evolving landscape, VoDEx serves as an essential tool specifically designed to manage the challenges associated with volumetric functional imaging data and complex time annotation. It fills a critical
gap in existing analytical toolsets by providing a user-friendly and reliable solution for handling these
novel datasets, ensuring that researchers can efficiently interpret their data.
VoDEx’s dual availability as both a standalone Python package and a Napari plugin makes it accessible
to a broad spectrum of researchers, from those with minimal programming experience to those engaged in
complex data analyses. Its design is highly adaptable and can be seamlessly integrated with other Python
packages, making it a flexible tool suitable for various neuroscientific applications.
The utility of VoDEx has been demonstrated in its application to the numerosity study in zebrafish
larvae, where it demonstrated its ability to streamline data processing and handle complex visual stimuli.
Given the industry’s growing focus on robust data management and the FAIR principles, VoDEx is
well-positioned to become a part of the data management and analysis ecosystem in neuroscience. It aligns
with ongoing community efforts to develop open-source, modular software tools. We envision that VoDEx
will join the growing list of tools that promote efficient data management and analysis in neuroscientific
studies (Meunier et al., 2020; Viejo et al., 2023).
In summary, VoDEx serves as a robust, flexible, and user-friendly tool that tackles the challenges posed
by modern volumetric functional imaging techniques. Its simple design, adaptability, and ease of use make
it a valuable tool for the next generation of neuroscientific research.
Our work on VoDEx was published in Bioinformatics (Nadtochiy et al., 2023)
100

4.7 Acknowledgments
I extend my gratitude to my advisors, Scott Fraser and Thai Truong, for their support throughout this
project. Special thanks are due to Peter Luu for his dedicated efforts in executing the numerosity imaging
experiments and for trying out numerous versions of the VoDEx code as it went through refinements.
His contributions have been essential to the success of this work. I also wish to express my appreciation to Caroline Brennan, Giorgio Vallortigara, and their respective lab members. Although not directly
involved in the conceptualization and development of VoDEx, their enthusiasm about the numerosity
research has inspired this project. This research was generously funded by the Human Frontier Science Program [RPG0008/2017] and the National Institutes of Health [1U01NS122082-01, 1UF1NS126562-01,
1R34NS126800-01].
4.8 Code Availability
The code is provided as an open-source Python package at https://github.com/lemonjust/vodex.
101

Chapter 5
Conclusion
This thesis presents three contributions to computational neuroscience, each addressing distinct challenges
in analyzing brain function and structure on a large scale.
1. Subregional Synaptic Changes in Zebrafish (Chapter 2): Our first contribution, as detailed in Chapter
2, is a robust analytical framework employing Support Vector Machines (SVMs). This approach
has successfully identified subregional synaptic changes crucial for fear memory formation in the
ventrolateral pallium of zebrafish. This methodology, published in PNAS (Dempsey et al., 2022), sets
the stage for a nuanced understanding of synaptic modifications on a large scale.
2. Scalable Detection of Synaptic Changes (Chapter 3): The second contribution introduces a flexible computational pipeline for detecting synaptic changes, adaptable to the evolving computational methods and experimental paradigms in neuroscience. While this work is still in progress,
it promises to enhance our understanding of synaptic dynamics, contributing to the field’s expansion. This work is still in progress. We plan to refine and publish it in the next few years.
3. VoDEx: A Tool for 3D Volumetric Functional Imaging (Chapter 4): The final contribution is VoDEx,
a computational tool designed for 3D volumetric functional imaging challenges, which has been
published in Bioinformatics (Nadtochiy et al., 2023). Its adaptability and user-friendliness make it
invaluable for future studies in neuronal activity in 3D spaces.
102

Using the synergy between these tools, furture research could investigate the connections between
synaptic changes, neuronal activity, and behavior, enriching our understanding of brain function and
structure. For example, VoDEx could enable analysis of brain activity during fear conditioning phases,
while the tools developed in Chapters 2 and 3 would be used to detect and analyze synaptic changes.
This integration would allow for a simultaneous study of synaptic changes and neuronal activity patterns,
linking them directly to behavioral data, and offering a foundational framework for integrated large-scale
studies.
The future of computational neuroscience looks promising, with technologies like ours paving the way
for more detailed and comprehensive brain studies. Our work positions itself at the forefront of this exciting journey, contributing to the unraveling of complex neural mechanisms and fostering advancements in
the field.
103

Bibliography
Ahrens, M. B., & Engert, F. (2015). Large-scale imaging in small brains. Current Opinion in Neurobiology,
32, 78–86. https://doi.org/10.1016/j.conb.2015.01.007
Ahrens, M. B., Li, J. M., Orger, M. B., Robson, D. N., Schier, A. F., Engert, F., & Portugues, R. (2012).
Brain-wide neuronal dynamics during motor adaptation in zebrafish. Nature, 485(7399), 471–477.
https://doi.org/10.1038/nature11057
Ahrens, M. B., Orger, M. B., Robson, D. N., Li, J. M., & Keller, P. J. (2013). Whole-brain functional imaging
at cellular resolution using light-sheet microscopy. Nature Methods, 10(5), 413–420.
https://doi.org/10.1038/nmeth.2434
Appelbaum, L., Wang, G., Yokogawa, T., Skariah, G. M., Smith, S. J., Mourrain, P., & Mignot, E. (2010).
Circadian and homeostatic regulation of structural synaptic plasticity in hypocretin neurons.
Neuron, 68(1), 87–98. https://doi.org/10.1016/j.neuron.2010.09.006
Ascoli, G. A., Alonso-Nanclares, L., Anderson, S. A., Barrionuevo, G., Benavides-Piccione, R.,
Burkhalter, A., Buzsáki, G., Cauli, B., DeFelipe, J., Fairén, A., Feldmeyer, D., Fishell, G., Fregnac, Y.,
Freund, T. F., Gardner, D., Gardner, E. P., Goldberg, J. H., Helmstaedter, M., Hestrin, S., . . .
Yuste, R. (2008). Petilla terminology: Nomenclature of features of GABAergic interneurons of the
cerebral cortex. Nature Reviews Neuroscience, 9(7), 557–568. https://doi.org/10.1038/nrn2402
Avants, B. B., Tustison, N. J., Song, G., Cook, P. A., Klein, A., & Gee, J. C. (2011a). A reproducible
evaluation of ANTs similarity metric performance in brain image registration. NeuroImage, 54(3),
2033–2044. https://doi.org/10.1016/j.neuroimage.2010.09.025
Avants, B. B., Tustison, N. J., Song, G., Cook, P. A., Klein, A., & Gee, J. C. (2011b). A reproducible
evaluation of ANTs similarity metric performance in brain image registration. NeuroImage, 54(3),
2033–2044. https://doi.org/10.1016/j.neuroimage.2010.09.025
Baird, G. S., Zacharias, D. A., & Tsien, R. Y. (1999). Circular permutation and receptor insertion within
green fluorescent proteins. Proceedings of the National Academy of Sciences, 96(20), 11241–11246.
https://doi.org/10.1073/pnas.96.20.11241
Barrak, A., Eghan, E. E., & Adams, B. (2021). On the co-evolution of ml pipelines and source code -
empirical study of dvc projects. 2021 IEEE International Conference on Software Analysis, Evolution
and Reengineering (SANER). https://doi.org/10.1109/saner50967.2021.00046
104

Barron, H. C. (2021). Neural inhibition for continual learning and memory. Current Opinion in
Neurobiology, 67, 85–94. https://doi.org/10.1016/j.conb.2020.09.007
Beare, R., Lowekamp, B., & Yaniv, Z. (2018). Image segmentation, registration and characterization in r
with simpleitk. Journal of Statistical Software, 86(8). https://doi.org/10.18637/jss.v086.i08
Beyeler, A., Chang, C.-J., Silvestre, M., Lévêque, C., Namburi, P., Wildes, C. P., & Tye, K. M. (2018).
Organization of valence-encoding and projection-defined neurons in the basolateral amygdala.
Cell Reports, 22(4), 905–918. https://doi.org/10.1016/j.celrep.2017.12.097
Bliss, T. V. P., & Lømo, T. (1973). Long-lasting potentiation of synaptic transmission in the dentate area of
the anaesthetized rabbit following stimulation of the perforant path. The Journal of Physiology,
232(2), 331–356. https://doi.org/10.1113/jphysiol.1973.sp010273
Bloodgood, B. L., Sharma, N., Browne, H. A., Trepman, A. Z., & Greenberg, M. E. (2013). The
activity-dependent transcription factor NPAS4 regulates domain-specific inhibition. Nature,
503(7474), 121–125. https://doi.org/10.1038/nature12743
Bosch, M., Castro, J., Saneyoshi, T., Matsuno, H., Sur, M., & Hayashi, Y. (2014). Structural and molecular
remodeling of dendritic spine substructures during long-term potentiation. Neuron, 82(2),
444–459. https://doi.org/10.1016/j.neuron.2014.03.021
Brereton, R. G., & Lloyd, G. R. (2010). Support vector machines for classification and regression. The
Analyst, 135(2), 230–267. https://doi.org/10.1039/b918972f
Buzsáki, G., & Wang, X.-J. (2012). Mechanisms of gamma oscillations. Annual Review of Neuroscience,
35(1), 203–225. https://doi.org/10.1146/annurev-neuro-062111-150444
Callahan, R. A., Roberts, R., Sengupta, M., Kimura, Y., Higashijima, S.-i., & Bagnall, M. W. (2019). Spinal
v2b neurons reveal a role for ipsilateral inhibition in speed control. eLife, 8.
https://doi.org/10.7554/elife.47837
Caroni, P., Donato, F., & Muller, D. (2012). Structural plasticity upon learning: Regulation and functions.
Nature Reviews Neuroscience, 13(7), 478–490. https://doi.org/10.1038/nrn3258
Chen, J. L., Villa, K. L., Cha, J. W., So, P. T., Kubota, Y., & Nedivi, E. (2012). Clustered dynamics of
inhibitory synapses and dendritic spines in the adult neocortex. Neuron, 74(2), 361–373.
https://doi.org/10.1016/j.neuron.2012.02.030
Chen, T.-W., Wardill, T. J., Sun, Y., Pulver, S. R., Renninger, S. L., Baohan, A., Schreiter, E. R., Kerr, R. A.,
Orger, M. B., Jayaraman, V., Looger, L. L., Svoboda, K., & Kim, D. S. (2013). Ultrasensitive
fluorescent proteins for imaging neuronal activity. Nature, 499(7458), 295–300.
https://doi.org/10.1038/nature12354
Choi, J.-E., Kim, J., & Kim, J. (2020). Capturing activated neurons and synapses. Neuroscience Research,
152, 25–34. https://doi.org/10.1016/j.neures.2019.12.020
105

Choi, J.-H., Sim, S.-E., Kim, J.-i., Choi, D. I., Oh, J., Ye, S., Lee, J., Kim, T., Ko, H.-G., Lim, C.-S., &
Kaang, B.-K. (2018). Interregional synaptic maps among engram cells underlie memory
formation. Science, 360(6387), 430–435. https://doi.org/10.1126/science.aas9204
Cizeron, M., Qiu, Z., Koniaris, B., Gokhale, R., Komiyama, N. H., Fransén, E., & Grant, S. G. N. (2020). A
brainwide atlas of synapses across the mouse life span. Science, 369(6501), 270–275.
https://doi.org/10.1126/science.aba3163
Clem, R. L., & Huganir, R. L. (2010). Calcium-permeable ampa receptor dynamics mediate fear memory
erasure. Science, 330(6007), 1108–1112. https://doi.org/10.1126/science.1195298
Cong, L., Wang, Z., Chai, Y., Hang, W., Shang, C., Yang, W., Bai, L., Du, J., Wang, K., & Wen, Q. (2017).
Rapid whole brain imaging of neural activity in freely behaving larval zebrafish (danio rerio).
eLife, 6. https://doi.org/10.7554/elife.28158
Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.
https://doi.org/10.1007/bf00994018
Dana, H., Sun, Y., Mohar, B., Hulse, B. K., Kerlin, A. M., Hasseman, J. P., Tsegaye, G., Tsang, A., Wong, A.,
Patel, R., Macklin, J. J., Chen, Y., Konnerth, A., Jayaraman, V., Looger, L. L., Schreiter, E. R.,
Svoboda, K., & Kim, D. S. (2019). High-performance calcium sensors for imaging activity in
neuronal populations and microcompartments. Nature Methods, 16(7), 649–657.
https://doi.org/10.1038/s41592-019-0435-6
Davis, M. (1994). The role of the amygdala in emotional learning. In International review of neurobiology
(pp. 225–266). Elsevier. https://doi.org/10.1016/s0074-7742(08)60305-0
Dehaene, S., & Changeux, J.-P. (1993). Development of elementary numerical abilities: A neuronal model.
Journal of Cognitive Neuroscience, 5(4), 390–407. https://doi.org/10.1162/jocn.1993.5.4.390
Dempsey, W. P., Du, Z., Nadtochiy, A., Smith, C. D., Czajkowski, K., Andreev, A., Robson, D. N., Li, J. M.,
Applebaum, S., Truong, T. V., Kesselman, C., Fraser, S. E., & Arnold, D. B. (2022). Regional
synapse gain and loss accompany memory formation in larval zebrafish. Proceedings of the
National Academy of Sciences, 119(3). https://doi.org/10.1073/pnas.2107661119
Denk, W., Strickler, J. H., & Webb, W. W. (1990). Two-photon laser scanning fluorescence microscopy.
Science, 248(4951), 73–76. https://doi.org/10.1126/science.2321027
Dombeck, D. A., Harvey, C. D., Tian, L., Looger, L. L., & Tank, D. W. (2010). Functional imaging of
hippocampal place cells at cellular resolution during virtual navigation. Nature Neuroscience,
13(11), 1433–1440. https://doi.org/10.1038/nn.2648
Dunn, T. W., Mu, Y., Narayan, S., Randlett, O., Naumann, E. A., Yang, C.-T., Schier, A. F., Freeman, J.,
Engert, F., & Ahrens, M. B. (2016). Brain-wide mapping of neural activity controlling zebrafish
exploratory locomotion. eLife, 5. https://doi.org/10.7554/elife.12741
Edelstein, A. D., Tsuchida, M. A., Amodaj, N., Pinkard, H., Vale, R. D., & Stuurman, N. (2014). Advanced
methods of microscope control using umanager software. Journal of Biological Methods, 1(2), e10.
https://doi.org/10.14440/jbm.2014.36
106

Ehrlich, I., Humeau, Y., Grenier, F., Ciocchi, S., Herry, C., & Lüthi, A. (2009). Amygdala inhibitory circuits
and the control of fear memory. Neuron, 62(6), 757–771.
https://doi.org/10.1016/j.neuron.2009.05.026
El-Husseini, A. E.-D., Schnell, E., Chetkovich, D. M., Nicoll, R. A., & Bredt, D. S. (2000). PSD-95
involvement in maturation of excitatory synapses. Science, 290(5495), 1364–1368.
https://doi.org/10.1126/science.290.5495.1364
El-Husseini, A. E.-D., Schnell, E., Dakoji, S., Sweeney, N., Zhou, Q., Prange, O., Gauthier-Campbell, C.,
Aguilera-Moreno, A., Nicoll, R. A., & Bredt, D. S. (2002). Synaptic strength regulated by palmitate
cycling on psd-95. Cell, 108(6), 849–863. https://doi.org/10.1016/s0092-8674(02)00683-9
Fauth, M., & Tetzlaff, C. (2016). Opposing effects of neuronal activity on structural plasticity. Frontiers in
Neuroanatomy, 10. https://doi.org/10.3389/fnana.2016.00075
Feinberg, E. H., VanHoven, M. K., Bendesky, A., Wang, G., Fetter, R. D., Shen, K., & Bargmann, C. I.
(2008). GFP reconstitution across synaptic partners (GRASP) defines cell contacts and synapses in
living nervous systems. Neuron, 57(3), 353–363. https://doi.org/10.1016/j.neuron.2007.11.030
Flores, C. E., & Mandez, P. (2014). Shaping inhibition: Activity dependent structural plasticity of
GABAergic synapses. Frontiers in Cellular Neuroscience, 8.
https://doi.org/10.3389/fncel.2014.00327
Freund, T. F., & Katona, I. (2007). Perisomatic inhibition. Neuron, 56(1), 33–42.
https://doi.org/10.1016/j.neuron.2007.09.012
Friedrich, R. W., Genoud, C., & Wanner, A. A. (2013). Analyzing the structure and function of neuronal
circuits in zebrafish. Frontiers in Neural Circuits, 7. https://doi.org/10.3389/fncir.2013.00071
Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J.-P., Frith, C. D., & Frackowiak, R. S. J. (1994).
Statistical parametric maps in functional imaging: A general linear approach. Human Brain
Mapping, 2(4), 189–210. https://doi.org/10.1002/hbm.460020402
Fu, Y., Tucciarone, J. M., Espinosa, J. S., Sheng, N., Darcy, D. P., Nicoll, R. A., Huang, Z. J., & Stryker, M. P.
(2014). A cortical circuit for gain control by behavioral state. Cell, 156(6), 1139–1152.
https://doi.org/10.1016/j.cell.2014.01.050
Gebuis, T., Kadosh, R. C., & Gevers, W. (2016). Sensory-integration system rather than approximate
number system underlies numerosity processing: A critical review. Acta Psychologica, 171, 17–35.
https://doi.org/10.1016/j.actpsy.2016.09.003
Gentet, L. J., Kremer, Y., Taniguchi, H., Huang, Z. J., Staiger, J. F., & Petersen, C. C. H. (2012). Unique
functional properties of somatostatin-expressing GABAergic neurons in mouse barrel cortex.
Nature Neuroscience, 15(4), 607–612. https://doi.org/10.1038/nn.3051
Giorgi, C., & Marinelli, S. (2021). Roles and transcriptional responses of inhibitory neurons in learning
and memory. Frontiers in Molecular Neuroscience, 14. https://doi.org/10.3389/fnmol.2021.689952
107

Giovannucci, A., Friedrich, J., Gunn, P., Kalfon, J., Brown, B. L., Koay, S. A., Taxidis, J., Najafi, F.,
Gauthier, J. L., Zhou, P., Khakh, B. S., Tank, D. W., Chklovskii, D. B., & Pnevmatikakis, E. A.
(2019a). CaImAn an open source tool for scalable calcium imaging data analysis. eLife, 8.
https://doi.org/10.7554/elife.38173
Giovannucci, A., Friedrich, J., Gunn, P., Kalfon, J., Brown, B. L., Koay, S. A., Taxidis, J., Najafi, F.,
Gauthier, J. L., Zhou, P., Khakh, B. S., Tank, D. W., Chklovskii, D. B., & Pnevmatikakis, E. A.
(2019b). CaImAn an open source tool for scalable calcium imaging data analysis. eLife, 8.
https://doi.org/10.7554/elife.38173
Gorgolewski, K. J., Auer, T., Calhoun, V. D., et al. (2016). The brain imaging data structure, a format for
organizing and describing outputs of neuroimaging experiments. Scientific Data, 3(1).
https://doi.org/10.1038/sdata.2016.44
Grewe, B. F., Gründemann, J., Kitch, L. J., Lecoq, J. A., Parker, J. G., Marshall, J. D., Larkin, M. C.,
Jercog, P. E., Grenier, F., Li, J. Z., Lüthi, A., & Schnitzer, M. J. (2017). Neural ensemble dynamics
underlying a long-term associative memory. Nature, 543(7647), 670–675.
https://doi.org/10.1038/nature21682
Gross, G. G., Junge, J. A., Mora, R. J., Kwon, H.-B., Olson, C. A., Takahashi, T. T., Liman, E. R.,
Ellis-Davies, G. C., McGee, A. W., Sabatini, B. L., Roberts, R. W., & Arnold, D. B. (2013).
Recombinant probes for visualizing endogenous synaptic proteins in living neurons. Neuron,
78(6), 971–985. https://doi.org/10.1016/j.neuron.2013.04.017
Haesemeyer, M., Robson, D. N., Li, J. M., Schier, A. F., & Engert, F. (2018). A brain-wide circuit model of
heat-evoked swimming behavior in larval zebrafish. Neuron, 98(4), 817–831.e6.
https://doi.org/10.1016/j.neuron.2018.04.013
Hájos, N. (2021). Interneuron types and their circuits in the basolateral amygdala. Frontiers in Neural
Circuits, 15. https://doi.org/10.3389/fncir.2021.687257
Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E.,
Taylor, J., Berg, S., Smith, N. J., Kern, R., Picus, M., Hoyer, S., van Kerkwijk, M. H., Brett, M.,
Haldane, A., del Río, J. F., Wiebe, M., Peterson, P., . . . Oliphant, T. E. (2020). Array programming
with NumPy. Nature, 585(7825), 357–362. https://doi.org/10.1038/s41586-020-2649-2
Hebb, D. O. (1949). Organization of behavior: A neurophysiological theory. (No Title).
Hipp, R. D. (2020). SQLite. https://www.sqlite.org/index.html
Huisken, J., Swoger, J., Del Bene, F., Wittbrodt, J., & Stelzer, E. H. K. (2004). Optical sectioning deep inside
live embryos by selective plane illumination microscopy. Science, 305(5686), 1007–1009.
https://doi.org/10.1126/science.1100035
Hyde, D. C., & Spelke, E. S. (2011). Neural signatures of number processing in human infants: Evidence
for two core systems underlying numerical cognition. Developmental Science, 14(2), 360–371.
https://doi.org/10.1111/j.1467-7687.2010.00987.x
108

Iascone, D. M., Li, Y., Sümbül, U., Doron, M., Chen, H., Andreu, V., Goudy, F., Blockus, H., Abbott, L. F.,
Segev, I., Peng, H., & Polleux, F. (2020). Whole-neuron synaptic mapping reveals spatially precise
excitatory/inhibitory balance limiting dendritic and somatic spiking. Neuron, 106(4), 566–578.e8.
https://doi.org/10.1016/j.neuron.2020.02.015
Ito, M. (1989). Long-term depression. Annual Review of Neuroscience, 12(1), 85–102.
https://doi.org/10.1146/annurev.ne.12.030189.000505
Josselyn, S. A., Köhler, S., & Frankland, P. W. (2015). Finding the engram. Nature Reviews Neuroscience,
16(9), 521–534. https://doi.org/10.1038/nrn4000
Kandel, E., Koester, J., Mack, S., & Siegelbaum, S. (2021). Principles of neural science, sixth edition. McGraw
Hill LLC. https://books.google.com/books?id=IYoEEAAAQBAJ
Kannan, M., Gross, G. G., Arnold, D. B., & Higley, M. J. (2016). Visual deprivation during the critical
period enhances layer 2/3 GABAergic inhibition in mouse v1. The Journal of Neuroscience, 36(22),
5914–5919. https://doi.org/10.1523/jneurosci.0051-16.2016
Katz, G. B., Benbassat, A., & Sipper, M. (2016). Chapter 6 - development of counting ability: An
evolutionary computation point of view. In A. Henik (Ed.), Continuous issues in numerical
cognition (pp. 123–145). Academic Press.
https://doi.org/https://doi.org/10.1016/B978-0-12-801637-4.00006-8
Keller, P. J., Schmidt, A. D., Wittbrodt, J., & Stelzer, E. H. (2008). Reconstruction of zebrafish early
embryonic development by scanned light sheet microscopy. Science, 322(5904), 1065–1069.
https://doi.org/10.1126/science.1162493
Keomanee-Dizon, K., Jones, M., Luu, P., Fraser, S. E., & Truong, T. V. (2022). Extended depth-of-field
light-sheet microscopy improves imaging of large volumes at high numerical aperture. Applied
Physics Letters, 121(16). https://doi.org/10.1063/5.0101426
Kim, J., Zhao, T., Petralia, R. S., Yu, Y., Peng, H., Myers, E., & Magee, J. C. (2011). mGRASP enables
mapping mammalian synaptic connectivity with light microscopy. Nature Methods, 9(1), 96–102.
https://doi.org/10.1038/nmeth.1784
Kim, T. H., & Schnitzer, M. J. (2022). Fluorescence imaging of large-scale neural ensemble dynamics. Cell,
185(1), 9–41. https://doi.org/10.1016/j.cell.2021.12.007
Kim, W. B., & Cho, J.-H. (2017). Encoding of discriminative fear memory by input-specific ltp in the
amygdala. Neuron, 95(5), 1129–1146.e5. https://doi.org/10.1016/j.neuron.2017.08.004
Klausberger, T., & Somogyi, P. (2008). Neuronal diversity and temporal dynamics: The unity of
hippocampal circuit operations. Science, 321(5885), 53–57.
https://doi.org/10.1126/science.1149381
Kobylkov, D., Mayer, U., Zanon, M., & Vallortigara, G. (2022). Number neurons in the nidopallium of
young domestic chicks. Proceedings of the National Academy of Sciences, 119(32).
https://doi.org/10.1073/pnas.2201039119
109

Krabbe, S., Gründemann, J., & Lüthi, A. (2018). Amygdala inhibitory circuits regulate associative fear
conditioning. Biological Psychiatry, 83(10), 800–809.
https://doi.org/10.1016/j.biopsych.2017.10.006
Kubota, Y. (2014). Untangling GABAergic wiring in the cortical microcircuit. Current Opinion in
Neurobiology, 26, 7–14. https://doi.org/10.1016/j.conb.2013.10.003
Kulikov, V., Guo, S.-M., Stone, M., Goodman, A., Carpenter, A., Bathe, M., & Lempitsky, V. (2019).
DoGNet: A deep architecture for synapse detection in multiplexed fluorescence images
(J. Vogelstein, Ed.). PLOS Computational Biology, 15(5), e1007012.
https://doi.org/10.1371/journal.pcbi.1007012
Kwon, T., Merchán-Pérez, A., Verde, E. M. R., Rodríguez, J.-R., DeFelipe, J., & Yuste, R. (2018).
Ultrastructural, molecular and functional mapping of GABAergic synapses on dendritic spines
and shafts of neocortical pyramidal neurons. Cerebral Cortex, 29(7), 2771–2781.
https://doi.org/10.1093/cercor/bhy143
Lai, C. S. W., Adler, A., & Gan, W.-B. (2018). Fear extinction reverses dendritic spine formation induced by
fear conditioning in the mouse auditory cortex. Proceedings of the National Academy of Sciences,
115(37), 9306–9311. https://doi.org/10.1073/pnas.1801504115
Lai, C. S. W., Franke, T. F., & Gan, W.-B. (2012). Opposite effects of fear conditioning and extinction on
dendritic spine remodelling. Nature, 483(7387), 87–91. https://doi.org/10.1038/nature10792
Lee, C., Lee, B. H., Jung, H., Lee, C., Sung, Y., Kim, H., Kim, J., Shim, J. Y., Kim, J.-i., Choi, D. I., Park, H. Y.,
& Kaang, B.-K. (2023). Hippocampal engram networks for fear memory recruit new synapses and
modify pre-existing synapses in vivo. Current Biology, 33(3), 507–516.e3.
https://doi.org/10.1016/j.cub.2022.12.038
Lee, H.-K., & Kirkwood, A. (2019). Mechanisms of homeostatic synaptic plasticity in vivo. Frontiers in
Cellular Neuroscience, 13. https://doi.org/10.3389/fncel.2019.00520
Lee, S.-H., & Dan, Y. (2012). Neuromodulation of brain states. Neuron, 76(1), 209–222.
https://doi.org/10.1016/j.neuron.2012.09.012
Lee, S., Kim, S.-J., Kwon, O.-B., Lee, J. H., & Kim, J.-H. (2013). Inhibitory networks of the amygdala for
emotional memory. Frontiers in Neural Circuits, 7. https://doi.org/10.3389/fncir.2013.00129
Levoy, M., Ng, R., Adams, A., Footer, M., & Horowitz, M. (2006). Light field microscopy. ACM
Transactions on Graphics, 25(3), 924–934. https://doi.org/10.1145/1141911.1141976
Li, H., Penzo, M. A., Taniguchi, H., Kopec, C. D., Huang, Z. J., & Li, B. (2013). Experience-dependent
modification of a central amygdala fear circuit. Nature Neuroscience, 16(3), 332–339.
https://doi.org/10.1038/nn.3322
Lin, X., Duan, X., Jacobs, C., et al. (2018). High-throughput brain activity mapping and machine learning
as a foundation for systems neuropharmacology. Nature Communications, 9(1).
https://doi.org/10.1038/s41467-018-07289-5
110

Lømo, T. (1966). Frequency potentiation of excitatory synaptic activity in dentate area of hippocampal
formation. Acta physiologica Scandinavica.
Lovett-Barron, M., Turi, G. F., Kaifosh, P., Lee, P. H., Bolze, F., Sun, X.-H., Nicoud, J.-F., Zemelman, B. V.,
Sternson, S. M., & Losonczy, A. (2012). Regulation of neuronal input transformations by tunable
dendritic inhibition. Nature Neuroscience, 15(3), 423–430. https://doi.org/10.1038/nn.3024
Ma, W.-p., Liu, B.-h., Li, Y.-t., Huang, Z. J., Zhang, L. I., & Tao, H. W. (2010). Visual representations by
cortical somatostatin inhibitory neurons—selective but with weak and delayed responses. The
Journal of Neuroscience, 30(43), 14371–14379. https://doi.org/10.1523/jneurosci.3248-10.2010
Maadi, M., Akbarzadeh Khorshidi, H., & Aickelin, U. (2021). A review on human–ai interaction in
machine learning and insights for medical applications. International Journal of Environmental
Research and Public Health, 18(4), 2121. https://doi.org/10.3390/ijerph18042121
Maccaferri, G. (2004). Stratum oriens horizontal interneurone diversity and hippocampal network
dynamics. The Journal of Physiology, 562(1), 73–80.
https://doi.org/10.1113/jphysiol.2004.077081
Macé, E., Montaldo, G., Cohen, I., Baulac, M., Fink, M., & Tanter, M. (2011). Functional ultrasound
imaging of the brain. Nature Methods, 8(8), 662–664. https://doi.org/10.1038/nmeth.1641
Madaan, S., Keomanee-Dizon, K., Jones, M., Zhong, C., Nadtochiy, A., Luu, P., Fraser, S. E., &
Truong, T. V. (2021). Single-objective selective-volume illumination microscopy enables
high-contrast light-field imaging. Optics Letters, 46(12), 2860. https://doi.org/10.1364/ol.413849
Malenka, R. C., & Bear, M. F. (2004). Ltp and ltd. Neuron, 44(1), 5–21.
https://doi.org/10.1016/j.neuron.2004.09.012
Maren, S. (2015). Out with the old and in with the new: Synaptic mechanisms of extinction in the
amygdala. Brain Research, 1621, 231–238. https://doi.org/10.1016/j.brainres.2014.10.010
Margolin, J. F., Friedman, J. R., Meyer, W. K., Vissing, H., Thiesen, H. J., & Rauscher, F. J. (1994).
Krüppel-associated boxes are potent transcriptional repression domains. Proceedings of the
National Academy of Sciences, 91(10), 4509–4513. https://doi.org/10.1073/pnas.91.10.4509
Markram, H., Toledo-Rodriguez, M., Wang, Y., Gupta, A., Silberberg, G., & Wu, C. (2004). Interneurons of
the neocortical inhibitory system. Nature Reviews Neuroscience, 5(10), 793–807.
https://doi.org/10.1038/nrn1519
Marques, J. C., Li, M., Schaak, D., Robson, D. N., & Li, J. M. (2019). Internal state dynamics shape
brainwide activity and foraging behaviour. Nature, 577(7789), 239–243.
https://doi.org/10.1038/s41586-019-1858-z
Marquez-Legorreta, E., Constantin, L., Piber, M., Favre-Bulle, I. A., Taylor, M. A., Blevins, A. S.,
Giacomotto, J., Bassett, D. S., Vanwalleghem, G. C., & Scott, E. K. (2022). Brain-wide visual
habituation networks in wild type and fmr1 zebrafish. Nature Communications, 13(1).
https://doi.org/10.1038/s41467-022-28299-4
111

Messina, A., Potrich, D., Perrino, M., Sheardown, E., Petrazzini, M. E. M., Luu, P., Nadtochiy, A.,
Truong, T. V., Sovrano, V. A., Fraser, S. E., Brennan, C. H., & Vallortigara, G. (2022). Quantity as a
fish views it: Behavior and neurobiology. Frontiers in Neuroanatomy, 16.
https://doi.org/10.3389/fnana.2022.943504
Meunier, D., Pascarella, A., Altukhov, D., Jas, M., Combrisson, E., Lajnef, T., Bertrand-Dubois, D.,
Hadid, V., Alamian, G., Alves, J., Barlaam, F., Saive, A.-L., Dehgan, A., & Jerbi, K. (2020).
NeuroPycon: An open-source python toolbox for fast multi-modal and reproducible brain
connectivity pipelines. NeuroImage, 219, 117020.
https://doi.org/10.1016/j.neuroimage.2020.117020
Meyer, D., Bonhoeffer, T., & Scheuss, V. (2014). Balance and stability of synaptic structures during
synaptic plasticity. Neuron, 82(2), 430–443. https://doi.org/10.1016/j.neuron.2014.02.031
Miyawaki, A., Llopis, J., Heim, R., McCaffery, J. M., Adams, J. A., Ikura, M., & Tsien, R. Y. (1997).
Fluorescent indicators for ca2+based on green fluorescent proteins and calmodulin. Nature,
388(6645), 882–887. https://doi.org/10.1038/42264
Mu, Y., Bennett, D. V., Rubinov, M., Narayan, S., Yang, C.-T., Tanimoto, M., Mensh, B. D., Looger, L. L., &
Ahrens, M. B. (2019). Glia accumulate evidence that actions are futile and suppress unsuccessful
behavior. Cell, 178(1), 27–43.e19. https://doi.org/10.1016/j.cell.2019.05.050
Nabavi, S., Fox, R., Proulx, C. D., Lin, J. Y., Tsien, R. Y., & Malinow, R. (2014). Engineering a memory with
ltd and ltp. Nature, 511(7509), 348–352. https://doi.org/10.1038/nature13294
Nadtochiy, A., Luu, P., Fraser, S. E., & Truong, T. V. (2023). VoDEx: A python library for time annotation
and management of volumetric functional imaging data (H. Peng, Ed.). Bioinformatics.
https://doi.org/10.1093/bioinformatics/btad568
Nakai, J., Ohkura, M., & Imoto, K. (2001). A high signal-to-noise ca2+ probe composed of a single green
fluorescent protein. Nature Biotechnology, 19(2), 137–141. https://doi.org/10.1038/84397
Namburi, P., Beyeler, A., Yorozu, S., Calhoon, G. G., Halbert, S. A., Wichmann, R., Holden, S. S.,
Mertens, K. L., Anahtar, M., Felix-Ortiz, A. C., Wickersham, I. R., Gray, J. M., & Tye, K. M. (2015).
A circuit mechanism for differentiating positive and negative associations. Nature, 520(7549),
675–678. https://doi.org/10.1038/nature14366
Nieder, A. (2018). Evolution of cognitive and neural solutions enabling numerosity judgements: Lessons
from primates and corvids. Philosophical Transactions of the Royal Society B: Biological Sciences,
373(1740), 20160514. https://doi.org/10.1098/rstb.2016.0514
Nieder, A. (2020). The adaptive value of numerical competence. Trends in Ecology and Evolution, 35(7),
605–617. https://doi.org/10.1016/j.tree.2020.02.009
O’Craven, K. M., & Kanwisher, N. (2000). Mental imagery of faces and places activates corresponding
stimulus-specific brain regions. Journal of Cognitive Neuroscience, 12(6), 1013–1023.
https://doi.org/10.1162/08989290051137549
112

Ostroff, L. E., Cain, C. K., Bedont, J., Monfils, M. H., & LeDoux, J. E. (2010). Fear and safety learning
differentially affect synapse size and dendritic translation in the lateral amygdala. Proceedings of
the National Academy of Sciences, 107(20), 9418–9423. https://doi.org/10.1073/pnas.0913384107
Pachitariu, M., Stringer, C., Dipoppa, M., Schröder, S., Rossi, L. F., Dalgleish, H., Carandini, M., &
Harris, K. D. (2016). Suite2p: Beyond 10, 000 neurons with standard two-photon microscopy.
https://doi.org/10.1101/061507
Pachitariu, M., Stringer, C., & Harris, K. D. (2017). Robustness of spike deconvolution for calcium imaging
of neural spiking. https://doi.org/10.1101/156786
Pan, S., Mayoral, S. R., Choi, H. S., Chan, J. R., & Kheirbek, M. A. (2020). Preservation of a remote fear
memory requires new myelin formation. Nature Neuroscience, 23(4), 487–499.
https://doi.org/10.1038/s41593-019-0582-1
Pare, D., & Duvarci, S. (2012). Amygdala microcircuits mediating fear expression and extinction. Current
Opinion in Neurobiology, 22(4), 717–723. https://doi.org/10.1016/j.conb.2012.02.014
Pennock, I. M. L., Schmidt, T. T., Zorbek, D., & Blankenburg, F. (2021). Representation of visual
numerosity information during working memory in humans: An fmri decoding study. Human
Brain Mapping, 42(9), 2778–2789. https://doi.org/10.1002/hbm.25402
Perez-Alvarez, A., Navarrete, M., Covelo, A., Martin, E. D., & Araque, A. (2014). Structural and functional
plasticity of astrocyte processes and dendritic spine interactions. Journal of Neuroscience, 34(38),
12738–12744. https://doi.org/10.1523/jneurosci.2401-14.2014
Pi, H.-J., Hangya, B., Kvitsiani, D., Sanders, J. I., Huang, Z. J., & Kepecs, A. (2013). Cortical interneurons
that specialize in disinhibitory control. Nature, 503(7477), 521–524.
https://doi.org/10.1038/nature12676
Piazza, M., Izard, V., Pinel, P., Bihan, D. L., & Dehaene, S. (2004). Tuning curves for approximate
numerosity in the human intraparietal sulcus. Neuron, 44(3), 547–555.
https://doi.org/10.1016/j.neuron.2004.10.014
Portugues, R., Feierstein, C. E., Engert, F., & Orger, M. B. (2014). Whole-brain activity maps reveal
stereotyped, distributed networks for visuomotor behavior. Neuron, 81(6), 1328–1343.
https://doi.org/10.1016/j.neuron.2014.01.019
Prevedel, R., Yoon, Y.-G., Hoffmann, M., Pak, N., Wetzstein, G., Kato, S., Schrödel, T., Raskar, R.,
Zimmer, M., Boyden, E. S., & Vaziri, A. (2014). Simultaneous whole-animal 3d imaging of
neuronal activity using light-field microscopy. Nature Methods, 11(7), 727–730.
https://doi.org/10.1038/nmeth.2964
Rizos, G., & Schuller, B. W. (2020). Average jane, where art thou? – recent avenues in efficient machine
learning under subjectivity uncertainty. In Communications in computer and information science
(pp. 42–55). Springer International Publishing. https://doi.org/10.1007/978-3-030-50146-4_4
113

Roy, D. S., Muralidhar, S., Smith, L. M., & Tonegawa, S. (2017). Silent memory engrams as the basis for
retrograde amnesia. Proceedings of the National Academy of Sciences, 114(46).
https://doi.org/10.1073/pnas.1714248114
Rübel, O., Tritt, A., Dichter, B., Braun, T., Cain, N., Clack, N., Davidson, T. J., Dougherty, M.,
Fillion-Robin, J.-C., Graddis, N., Grauer, M., Kiggins, J. T., Niu, L., Ozturk, D., Schroeder, W.,
Soltesz, I., Sommer, F. T., Svoboda, K., Lydia, N., . . . Bouchard, K. (2019). NWB:n 2.0: An accessible
data standard for neurophysiology. https://doi.org/10.1101/523035
Rudy, B., Fishell, G., Lee, S., & Hjerling-Leffler, J. (2010). Three groups of interneurons account for nearly
100% of neocortical GABAergic neurons. Developmental Neurobiology, 71(1), 45–61.
https://doi.org/10.1002/dneu.20853
Rudy, B., & McBain, C. J. (2001). Kv3 channels: Voltage-gated k+ channels designed for high-frequency
repetitive firing. Trends in Neurosciences, 24(9), 517–526.
https://doi.org/10.1016/s0166-2236(00)01892-0
Ryan, T. J., Roy, D. S., Pignatelli, M., Arons, A., & Tonegawa, S. (2015). Engram cells retain memory under
retrograde amnesia. Science, 348(6238), 1007–1013. https://doi.org/10.1126/science.aaa5542
Salas, C., Broglio, C., Durán, E., Gómez, A., Ocaña, F. M., Jiménez-Moya, F., & Rodríguez, F. (2006).
Neuropsychology of learning and memory in teleost fish. Zebrafish, 3(2), 157–171.
https://doi.org/10.1089/zeb.2006.3.157
Schmidt, U., Weigert, M., Broaddus, C., & Myers, G. (2018). Cell detection with star-convex polygons. In
Medical image computing and computer assisted intervention – MICCAI 2018 (pp. 265–273).
Springer International Publishing. https://doi.org/10.1007/978-3-030-00934-2_30
Scholl, B., Thomas, C. I., Ryan, M. A., Kamasawa, N., & Fitzpatrick, D. (2020). Cortical response selectivity
derives from strength in numbers of synapses. Nature, 590(7844), 111–114.
https://doi.org/10.1038/s41586-020-03044-3
Siegelbaum, S. A., & Kandel, E. R. (1991). Learning-related synaptic plasticity: Ltp and ltd. Current
Opinion in Neurobiology, 1(1), 113–120. https://doi.org/10.1016/0959-4388(91)90018-3
Singh, S., & Topolnik, L. (2023). Inhibitory circuits in fear memory and fear-related disorders. Frontiers in
Neural Circuits, 17. https://doi.org/10.3389/fncir.2023.1122314
Sinnen, B. L., Bowen, A. B., Forte, J. S., Hiester, B. G., Crosby, K. C., Gibson, E. S., Dell’Acqua, M. L., &
Kennedy, M. J. (2017). Optogenetic control of synaptic composition and function. Neuron, 93(3),
646–660.e5. https://doi.org/10.1016/j.neuron.2016.12.037
Sofroniew, N., Lambert, T., Evans, K., Nunez-Iglesias, J., Bokota, G., Winston, P., Peña-Castellanos, G.,
Yamauchi, K., Bussonnier, M., Doncila Pop, D., Can Solak, A., Liu, Z., Wadhwa, P., Burt, A.,
Buckley, G., Sweet, A., Migas, L., Hilsenstein, V., Gaifas, L., . . . McGovern, A. (2022). Napari: A
multi-dimensional image viewer for python. https://doi.org/10.5281/ZENODO.3555620
114

Tasic, B., Menon, V., Nguyen, T. N., Kim, T. K., Jarsky, T., Yao, Z., Levi, B., Gray, L. T., Sorensen, S. A.,
Dolbeare, T., Bertagnolli, D., Goldy, J., Shapovalova, N., Parry, S., Lee, C., Smith, K., Bernard, A.,
Madisen, L., Sunkin, S. M., . . . Zeng, H. (2016). Adult mouse cortical cell taxonomy revealed by
single cell transcriptomics. Nature Neuroscience, 19(2), 335–346. https://doi.org/10.1038/nn.4216
Tayyab, M., Metz, L. M., Li, D. K., Kolind, S., Carruthers, R., Traboulsee, A., & Tam, R. C. (2023).
Accounting for uncertainty in training data to improve machine learning performance in
predicting new disease activity in early multiple sclerosis. Frontiers in Neurology, 14.
https://doi.org/10.3389/fneur.2023.1165267
Tonegawa, S., Pignatelli, M., Roy, D. S., & Ryan, T. J. (2015). Memory engram storage and retrieval.
Current Opinion in Neurobiology, 35, 101–109. https://doi.org/10.1016/j.conb.2015.07.009
Trinh, L. A., Hochgreb, T., Graham, M., Wu, D., Ruf-Zamojski, F., Jayasena, C. S., Saxena, A., Hawk, R.,
Gonzalez-Serricchio, A., Dixson, A., Chow, E., Gonzales, C., Leung, H.-Y., Solomon, I.,
Bronner-Fraser, M., Megason, S. G., & Fraser, S. E. (2011). A versatile gene trap to visualize and
interrogate the function of the vertebrate proteome. Genes amp; Development, 25(21), 2306–2320.
https://doi.org/10.1101/gad.174037.111
Truong, T. V., Holland, D. B., Madaan, S., Andreev, A., Keomanee-Dizon, K., Troll, J. V., Koo, D. E. S.,
McFall-Ngai, M. J., & Fraser, S. E. (2020). High-contrast, synchronous volumetric imaging with
selective volume illumination microscopy. Communications Biology, 3(1).
https://doi.org/10.1038/s42003-020-0787-6
Truong, T. V., Supatto, W., Koos, D. S., Choi, J. M., & Fraser, S. E. (2011). Deep and fast live imaging with
two-photon scanned light-sheet microscopy. Nature Methods, 8(9), 757–760.
https://doi.org/10.1038/nmeth.1652
Van der Walt, S., Schönberger, J. L., Nunez-Iglesias, J., Boulogne, F., Warner, J. D., Yager, N., Gouillart, E.,
& Yu, T. (2014). Scikit-image: Image processing in python. PeerJ, 2, e453.
Viejo, G., Levenstein, D., Carrasco, S. S., Mehrotra, D., Mahallati, S., Vite, G. R., Denny, H., Sjulson, L.,
Battaglia, F. P., & Peyrache, A. (2023). Pynapple: A toolbox for data analysis in neuroscience.
https://doi.org/10.7554/elife.85786.1
Villa, K. L., Berry, K. P., Subramanian, J., Cha, J. W., Oh, W. C., Kwon, H.-B., Kubota, Y., So, P. T., &
Nedivi, E. (2016). Inhibitory synapses are repeatedly assembled and removed at persistent sites
in vivo. Neuron, 89(4), 756–769. https://doi.org/10.1016/j.neuron.2016.01.010
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E.,
Peterson, P., Weckesser, W., Bright, J., van der Walt, S. J., Brett, M., Wilson, J., Millman, K. J.,
Mayorov, N., Nelson, A. R. J., Jones, E., Kern, R., Larson, E., . . . SciPy 1.0 Contributors. (2020).
SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17,
261–272. https://doi.org/10.1038/s41592-019-0686-2
Walker, A. S., Neves, G., Grillo, F., Jackson, R. E., Rigby, M., O’Donnell, C., Lowe, A. S.,
Vizcay-Barrena, G., Fleck, R. A., & Burrone, J. (2017). Distance-dependent gradient in
NMDAR-driven spine calcium signals along tapering dendrites. Proceedings of the National
Academy of Sciences, 114(10). https://doi.org/10.1073/pnas.1607462114
115

Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N.,
Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T.,
Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R., . . . Mons, B. (2016). The
FAIR guiding principles for scientific data management and stewardship. Scientific Data, 3(1).
https://doi.org/10.1038/sdata.2016.18
Wilson, N. R., Runyan, C. A., Wang, F. L., & Sur, M. (2012). Division and subtraction by distinct cortical
inhibitory networks in vivo. Nature, 488(7411), 343–348. https://doi.org/10.1038/nature11347
Witzgall, R., O’Leary, E., Leaf, A., Onaldi, D., & Bonventre, J. V. (1994). The krüppel-associated box-a
(krab-a) domain of zinc finger proteins mediates transcriptional repression. Proceedings of the
National Academy of Sciences, 91(10), 4514–4518. https://doi.org/10.1073/pnas.91.10.4514
Wolff, S. B. E., Gründemann, J., Tovote, P., Krabbe, S., Jacobson, G. A., Müller, C., Herry, C., Ehrlich, I.,
Friedrich, R. W., Letzkus, J. J., & Lüthi, A. (2014). Amygdala interneuron subtypes control fear
learning through disinhibition. Nature, 509(7501), 453–458. https://doi.org/10.1038/nature13258
Xu, Z., Adler, A., Li, H., Pérez-Cuesta, L. M., Lai, B., Li, W., & Gan, W.-B. (2019). Fear conditioning and
extinction induce opposing changes in dendritic spine remodeling and somatic activity of layer 5
pyramidal neurons in the mouse motor cortex. Scientific Reports, 9(1).
https://doi.org/10.1038/s41598-019-40549-y
Yang, E., Zwart, M. F., James, B., Rubinov, M., Wei, Z., Narayan, S., Vladimirov, N., Mensh, B. D.,
Fitzgerald, J. E., & Ahrens, M. B. (2022). A brainstem integrator for self-location memory and
positional homeostasis in zebrafish. Cell, 185(26), 5011–5027.e20.
https://doi.org/10.1016/j.cell.2022.11.022
Yiu, A. P., Mercaldo, V., Yan, C., Richards, B., Rashid, A. J., Hsiang, H.-L. L., Pressey, J., Mahadevan, V.,
Tran, M. M., Kushner, S. A., Woodin, M. A., Frankland, P. W., & Josselyn, S. A. (2014). Neurons
are recruited to a memory trace based on relative neuronal excitability immediately before
training. Neuron, 83(3), 722–735. https://doi.org/10.1016/j.neuron.2014.07.017
Zaharia, M. A., Chen, A., Davidson, A., Ghodsi, A., Hong, S. A., Konwinski, A., Murching, S., Nykodym, T.,
Ogilvie, P., Parkhe, M., Xie, F., & Zumar, C. (2018). Accelerating the machine learning lifecycle
with mlflow. IEEE Data Eng. Bull., 41, 39–45. https://api.semanticscholar.org/CorpusID:83459546
Zanon, M., Potrich, D., Bortot, M., & Vallortigara, G. (2021). Towards a standardization of non-symbolic
numerical experiments: GeNEsIS, a flexible and user-friendly tool to generate controlled stimuli.
Behavior Research Methods, 54(1), 146–157. https://doi.org/10.3758/s13428-021-01580-y
Zeisel, A., Muñoz-Manchado, A. B., Codeluppi, S., Lönnerberg, P., Manno, G. L., Juréus, A., Marques, S.,
Munguba, H., He, L., Betsholtz, C., Rolny, C., Castelo-Branco, G., Hjerling-Leffler, J., &
Linnarsson, S. (2015). Cell types in the mouse cortex and hippocampus revealed by single-cell
RNA-seq. Science, 347(6226), 1138–1142. https://doi.org/10.1126/science.aaa1934
Zhang, S., Xu, M., Kamigaki, T., Do, J. P. H., Chang, W.-C., Jenvay, S., Miyamichi, K., Luo, L., & Dan, Y.
(2014). Long-range and local circuits for top-down modulation of visual cortex processing.
Science, 345(6197), 660–665. https://doi.org/10.1126/science.1254126
116

Zhu, F., Cizeron, M., Qiu, Z., Benavides-Piccione, R., Kopanitsa, M. V., Skene, N. G., Koniaris, B.,
DeFelipe, J., Fransén, E., Komiyama, N. H., & Grant, S. G. (2018). Architecture of the mouse brain
synaptome. Neuron, 99(4), 781–799.e10. https://doi.org/10.1016/j.neuron.2018.07.007
117

Abstract (if available)

Abstract The last decade has witnessed a transformative shift in the field of neuroscience, largely fueled by advancements in fluorescence imaging tools and labeling techniques. This shift toward studying synaptic changes at the level of individual synapses and neural activity at single-cell resolution across the entire brain has generated increasingly complex data sets. This thesis introduces three contributions that address the methodological challenges accompanying these advancements.
The first contribution is an analytical framework employing Support Vector Machines (SVMs), offering a paradigmatic departure from traditional aggregate synaptic studies. This approach allows for the detection of subregional synaptic changes that would otherwise remain obscure.
The second contribution is a computational pipeline explicitly designed for the scalable analysis of synaptic changes. It is engineered to accommodate evolving computational methods and unique experimental paradigms emerging in this growing field.
Lastly, the thesis describes VoDEx, a specialized tool for managing complex 3D data sets generated by functional volumetric imaging. VoDEx streamlines the integration of experimental timelines and behavioral events with acquired imaging data, thereby improving the accuracy and reproducibility of functional analyses.
Together, these computational tools represent significant advancements in the large-scale analysis of brain function and structure, offering the adaptability and precision required for future neuroscientific research.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Deciphering protein-nucleic acid interactions with artificial intelligence

PDF

Characterizing brain aging with neuroimaging, health, and genetic data

PDF

Mapping 3D genome structures: a data driven modeling method for integrated structural analysis

PDF

Exploration of human microbiome through metagenomic analysis and computational algorithms

PDF

Fast search and clustering for large-scale next generation sequencing data

PDF

Identification and analysis of shared epigenetic changes in extraembryonic development and tumorigenesis

PDF

Computational algorithms for studying human genetic variations -- structural variations and variable number tandem repeats

PDF

Sharpening the edge of tools for microbial diversity analysis

PDF

Computer vision approaches to the analysis of animal behavior

PDF

Early detection of lung cancer by characterizing circulating rare cells using peripheral blood liquid biopsy

PDF

Geometric interpretation of biological data: algorithmic solutions for next generation sequencing analysis at massive scale

PDF

Learning controllable data generation for scalable model training

PDF

Detection, classification and functional annotation of mouse L1 retrotransposon promoters

PDF

Developing statistical and algorithmic methods for shotgun metagenomics and time series analysis

PDF

Enhancing phenotype prediction through integrative analysis of heterogeneous microbiome studies

PDF

Validating structural variations: from traditional algorithms to deep learning approaches

PDF

3D modeling of eukaryotic genomes

PDF

A parallel computation framework for EONS synaptic modeling platform for parameter optimization and drug discovery

PDF

Analyzing disease progression using cross-sectional data

PDF

Structure – dynamics – function analysis of class A GPCRs and exploration of chemical space using integrative computational approaches

Asset Metadata

Creator Nadtochiy, Anna (author)

Core Title Computational tools for large-scale analysis of brain function and structure

School College of Letters, Arts and Sciences

Degree Doctor of Philosophy

Degree Program Computational Biology and Bioinformatics

Degree Conferral Date 2023-12

Publication Date 01/05/2024

Defense Date 09/29/2023

Publisher Los Angeles, California (original), University of Southern California (original), University of Southern California. Libraries (digital)

Tag 3D data,4D data,computational tools,fluorescence imaging,functional volumetric imaging,napari,neural activity,Neuroscience,OAI-PMH Harvest,support vector machines,SVM,synaptic changes

Format theses (aat)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Fraser, Scott (committee chair), Arnold, Donald (committee member), Rohs, Remo (committee member), Sun, Fengzhu (committee member), Truong, Thai (committee member)

Creator Email nadtochi@usc.edu,nadtochyaj@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC113800520

Unique identifier UC113800520

Identifier etd-NadtochiyA-12591.pdf (filename)

Legacy Identifier etd-NadtochiyA-12591

Document Type Dissertation

Format theses (aat)

Rights Nadtochiy, Anna

Internet Media Type application/pdf

Type texts

Source 20240105-usctheses-batch-1118 (batch), University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu