University of Southern California Dissertations and Theses

Machine learning and image processing of fluorescence lifetime imaging microscopy enables tracking and analysis of subcellular metabolism

(USC Thesis Other)

Machine learning and image processing of fluorescence lifetime imaging microscopy enables tracking and analysis of subcellular metabolism

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Copyright 2022 Peiyu Wang

Machine Learning and Image Processing
of Fluorescence Lifetime Imaging Microscopy
Enables Tracking and Analysis of
Subcellular Metabolism

by

Peiyu Wang

A Thesis Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
Doctor of Philosophy
(BIOMEDICAL ENGINEERING)

Dec 2022

ii

Acknowledgments
First and foremost, I thank my advisor, Prof. Scott. E. Fraser for the great help he provided
during my Ph.D. He helped me progress in many aspects, and I am immensely grateful. I
probably have more stories than a whole thesis in which he helped me, not only in research but
also in cultivating a logical mind, developing an eye for details, and communicating thoughts
clearly to the audience. He is companionate and cares about his students and the people
around him, which makes him a fantastic instructor. He is a model in science and life and the
man I think everyone should strive to be.
I would also like to thank every member of Fraser’s lab. They are a family to me, and the best
company one can wish for in graduate school. I love how passionate they are about their work
and helpful they are when others need help. I want to thank Prof. Francesco Cutrale especially.
He gave me a lot of great suggestions, and without his motivation, I wouldn’t have landed my
first job at Medtronic.
I would also like to thank my parents. My father has always been a great role model for me. His
personal experiences encouraged me to switch majors and work in biomedical engineering,
where I can make a difference in people’s lives. I also like to thank my mother. She is an
extraordinary woman who made many sacrifices for my father and my career path. I speak for
everyone in my family that we are grateful for everything she did.
Finally, I would like to thank my girlfriend, Qiong Jia. You made me a better man and gave me a
good reason to continue in difficult times. Thank you for being by my side.
iii

Table of Contents
Acknowledgments............................................................................................................................ ii
Table of Contents ............................................................................................................................ iii
List of Figures ................................................................................................................................... v
List of Tables ................................................................................................................................. viii
Abstract ....................................................................................................................................... ix
Chapter 1. Fluorescent Lifetime Imaging Microscopy Provides Important Biological Insights ...... 1
1.1 Fluorescence microscopy: a powerful tool to understand cells and cellular activities 1
1.2 The process for generating fluorescence ........................................................................... 6
1.3 Limitations of fluorescence imaging ................................................................................... 8
1.4 Fluorescence lifetime used as contrast ............................................................................ 14
1.4.1 Measuring fluorescence lifetime in the time-domain is straightforward in the
collection but complicated for analysis ......................................................................... 15
1.4.2 Fluorescence lifetime in the frequency domain: straightforward representation
and analysis .................................................................................................................... 19
1.5 Scatter error in phasor analysis brings uncertainties in lifetime measurements ............ 27
Chapter 2. NADH autofluorescence FLIM Enables Metabolic Imaging with High Spatial and
Temporal Resolution ................................................................................................... 31
2.1 Cellular metabolism is key to understanding cellular behavior ....................................... 31
2.2 NADH lifetime changes with cellular metabolism ............................................................ 36
2.3 Frequency domain FLIM provides clear contrast for NADH lifetime ................................ 39
2.4 Challenges of metabolic NADH FLIM imaging and analysis .............................................. 43
Chapter 3. Extracting Metabolic Information from FLIM Phasor Plots ........................................ 48
3.1 Metabolic NADH lifetime readings are hidden in noisy phasor clusters .......................... 48
3.2 Weighted mode ensures accurate features extraction from phasor clusters .................. 51
3.3 Converting phasor features to metabolic readings .......................................................... 54
Chapter 4. Correlated Non-Local Means for FLIM Phasor Analysis .............................................. 62
4.1 Median filtering for FLIM phasor analysis degrades lifetime details in subcellular
structures ....................................................................................................................... 62
4.2 Correlated non-local means supplements structural information from the photon
intensity map to lifetime phasor maps .......................................................................... 64
iv

4.3 Calculations in correlated non-local means ...................................................................... 68
4.4 Correlated non-local means preserves lifetime information in structural details ........... 69
4.5 Correlated non-local means accurately reconstructs cellular metabolic signatures
for NADH FLIM imaging ................................................................................................. 74
4.6 Advantages and potential pitfalls of correlated non-local means.................................... 77
Chapter 5. Complex Wavelet Filtering for FLIM Phasor Analysis ................................................. 80
5.1 Localized spatial frequency filtering with complex wavelet filtering ............................... 80
5.2 Calculations for the complex wavelet filtering in phasor analysis ................................... 85
5.3 Complex wavelet filtering accurately reconstructs structural details .............................. 86
5.4 Complex wavelet filtering shows accurate structural reconstruction of lifetime
information with metabolic FLIM imaging .................................................................... 91
5.5 CWF accurately reconstructs structural details for τ-STED imaging ................................. 93
5.6 Discussion on complex wavelet filtering for phasor analysis on FLIM ............................. 94
5.7 Comparing correlated non-local means with complex wavelet filter .............................. 96
Chapter 6. Subcellular Metabolic Imaging and Analysis of Glucose Stimulated Insulin
Secretion in INS-1E Cells ........................................................................................... 101
6.1 Establishing a workflow for metabolic imaging with subcellular accuracy .................... 101
6.2 Pancreatic beta cells metabolism is key to maintaining normal blood glucose level .... 103
6.3 Segmentation of cellular compartment for metabolic analysis with machine
learning ........................................................................................................................ 105
6.4 INS-1E GSIS subcellular metabolic analysis ..................................................................... 115
6.5 Subcellular metabolic redox ratio analysis of INS-1E cells under GSIS ........................... 119
Conclusion ................................................................................................................................... 124
References .................................................................................................................................. 127

v

List of Figures
Chapter 1
1.1 Fluorescence microscopy enables imaging of specific cellular structures through
fluorescent molecules with high specificity and subcellular resolution ........ 5
1.2 Fluorescence is the process of emitting energy as a photon by a fluorophore,
explained in a Jablonski diagram. .................................................................................. 7
1.3 Detailed Jablonski diagram with sublevels and triplet state explains
photobleaching and why fluorophores' excitation and emission spectrum is broad.
....................................................................................................................................... 11
1.4 The broad excitation and emission spectrum for fluorophores can cause
fluorescence bleed-through in fluorescence microscopy. .......................................... 13
1.5 Time Domain FLIM measures the lifetime by building a decay curve for the
fluorescence and inferring the lifetime by fitting the curve to exponential decay. ... 17
1.6 In Time Domain FLIM (TD-FLIM) images, lifetime is a contrast to distinguish
different fluorophores. .................................................................................................. 19
1.7 Lifetime in FD-FLIM is measured by detecting the delay in time (phase) and
modulation decrease of the fluorescence with respect to the excitation and
displayed with the phasor plot. ..................................................................................... 21
1.8 The phasor plot can be constructed by decomposing the exponential decay
measured in TD-FLIM into sine and cosine wave components with Fourier
transform. ...................................................................................................................... 22
1.9 Phasor analysis allows for grouping pixels on the photon intensity map based on
lifetime. ......................................................................................................................... 24
1.10 Grouping and color-coding pixels of a similar lifetime for autofluorescence of a
Convallaria sample based on their location inside the phasor plots. .......................... 25
1.11 Phasor analysis allows straightforward unmixing of mixed lifetimes. In TD-FLIM,
unmixing of lifetime relies on complicated statistical tests. ......................................... 27
1.12 Scatter error arises in the phasor plot when the photon counts are low and blurs
the actual location of the phasor signature. ................................................................. 29
Chapter 2
2.1 Cellular metabolism, also known as metabolic pathways, is the set of chemical
reactions that consume or generate energy in the cell. ............................................... 33
2.2 NADH has substantial changes in lifetime depending on its binding states with
enzymes, which can indicate cellular metabolic states. ............................................... 37
vi

2.3 NADH lifetime changes with respect to the enzyme it is bound to, which can be
displayed in the phasor plot. ....................................................................................... 39
2.4 The metabolic states of the sample can be indicated by the location of its NADH
phasor signature. ......................................................................................................... 40
2.5 The lifetime phasor analysis of the NADH in a mouse islet allows for characterizing
regions of different metabolic states. ......................................................................... 42
Chapter 3
3.1 Pixels with similar lifetimes form phasor clusters in the phasor plot. .......................... 48
3.2 Representing phasor information with the centroid of a phasor cluster can be
inaccurate, especially in the presence of fluorescence bleed-through. ...................... 52
3.3 Representing phasor information with the mode of a phasor cluster is inaccurate
when the photon counts are low. ................................................................................ 53
3.4 The redox ratio for cellular metabolism can be approximated by calculating the
percentage of free and bound NADH concentration. .................................................. 57
3.5 The metabolic trendline for NADH FLIM imaging can be customized for every data
point. .............................................................................................................................. 59
3.6 The metabolic map for autofluorescence NADH FD-FLIM indicates the redox ratio at
every image location. ................................................................................................... 60
Chapter 4
4.1 Median filters commonly used to reduce scatter error for FLIM phasor analysis
degrade lifetime information in subcellular structures. .............................................. 63
4.2 Comparing photon intensity map and their respective lifetime phasor plots under
different photon counts show that photon intensity map is less subjective to low
photon counts. ............................................................................................................... 65
4.3 Correlated Non-local Means (CNLM) for supplements structural information to FLIM
phasor maps. ............................................................................................................... 67
4.4 CNLM filtering for fluorescent antibody staining on cell lines shows better
convergence than median filters. ................................................................................ 71
4.5 Comparisons of lifetime fluorescence images with different filters show that CNLM
has better noise reduction and detail preservation than median filters. ................... 73
4.6 CNLM filtering shows better scatter error reduction and more accurate preservation
of structural details than median filters for FLIM metabolic imaging. ........................ 76
Chapter 5
5.1 Wavelet transform decomposes images into smaller images with different frequency
components and directionalities. ................................................................................ 82
vii

5.2 Dual-tree wavelet transform has mother wavelets with more directionalities than
discrete wavelet transforms. ....................................................................................... 83
5.3 Workflow for Complex Wavelet Filtering for FLIM phasor plots. ................................ 84
5.4 Complex Wavelet Filter (CWF) enables low photon count FLIM datasets to
generate similar phasor plots and information extraction as high photon count
ideal datasets. ................................................................................................................ 88
5.5 G-coordinate error maps show that CWF achieves better structural reconstruction
of FLIM phasor data than other filtering methods. ..................................................... 89
5.6 CWF provides superior filtering for autofluorescence NADH metabolic live-
imaging of HEK-293 Cells. ............................................................................................ 92
5.7 In t-STED, CWF provides noise filtering in the phasor analysis, leading to enhanced
resolution and accurate structure reconstruction. ..................................................... 94
5.8 Comparing the performance of the CWF and CNLM shows that CWF works better
for regions with lower photon counts while CNLM for areas with higher photon
counts. .......................................................................................................................... 98
5.9 Mean Squared Error (MSE) with respect to photon counts shows that CWF works
better for regions with lower photon counts while CNLM for areas with higher
photon counts. .............................................................................................................. 99
Chapter 6
6.1 Basic Structure of the U-Net, a convolutional neural network model that yields good
segmentation for biomedical applications with low sample sizes. ........................... 107
6.2 Four input channels are used in each U-net model to segment the individual cell
instances, the mitochondria, and the nucleus compartments in FLIM datasets. ..... 109
6.3 Segmentation ground truth generated with staining dyes. ....................................... 111
6.4 Distance transformation is applied in the ground truth generation for individual
cellular instances and nucleus segmentation to ensure high quality. ........................ 114
6.5 Imaging pipeline for subcellular metabolic analysis of INS-1E cells under GSIS. ...... 116
6.6 Image analysis pipeline for subcellular metabolic analysis of INS-1E cells under GSIS.118
6.7 The calibrated redox ratio trendlines show different metabolic changes in the
mitochondria and cytosol. ......................................................................................... 120
6.8 Calibrated redox ratio histogram for the INS-1E cell show population metabolic
profiles of the cytosol and the mitochondria at different time points of GSIS. ......... 121
6.9 Scatter plot of redox ratios compared to their previous time points shows the
stability of the redox ratio of the different cellular compartments under GSIS. ...... 122

viii

List of Tables
Chapter 2
2.1 NADH state and lifetime with respect to different metabolic states of the sample. .. 38
Chapter 4
4.1 The standard deviation of a single fluorescence dye with different filtering show
CNLM has improved performance than median filters. ............................................... 70
Chapter 6
6.1 Staining dyes and their imaging conditions for creating the segmentation ground
truth. ........................................................................................................................... 113
6.2 The number of training sets and quantitative performance for segmentation
models for different cellular compartments. ............................................................. 113
6.3 The number of data sets and individual cells of INS-1E cells collected in the GSIS
experiments. ............................................................................................................... 117

ix

Abstract
Fluorescence Lifetime Imaging Microscopy (FLIM) is a fluorescence microscopy modality that
measures the time for fluorophores to emit fluorescence once excited, defined as the
fluorescence lifetime. FLIM is a powerful tool for metabolic imaging. It measures the lifetime of
Nicotinamide Adenine Dinucleotide plus Hydrogen (NADH), a coenzyme central to metabolism
whose lifetime changes with cellular metabolic states. FLIM provides spatial resolution sufficient
to resolve subcellular components. Yet, due to three significant hurdles, most researchers do not
extract metabolic information at a subcellular level with FLIM: First, FLIM signals in previous
research are not accurately converted to meaningful metabolic readings that account for the
various subcellular components. Second, noise carried by FLIM signal buries the metabolic
information, and conventional post-imaging filters for removing the noise do not preserve
lifetime readings in small cellular structures. Third, different subcellular components in FLIM
datasets are not effectively segmented for subcellular analysis. This thesis tackles these
problems. We first introduce an analysis workflow for FLIM metabolic readings that considers
the different binding enzymes in various subcellular components. Two filtering methods are
presented that preserve accurate lifetime readings in delicate cellular structures. Further,
machine learning is used to segment individual cells and subcellular components to extract their
metabolic trends independently. These tools provide a workflow that reveals the metabolic
dynamics in different subcellular components. Our workflow is applied to INS-1E cells to analyze
subcellular metabolic responses of INS-1E cells during glucose-induced insulin release. This
workflow can be easily modified to serve as a template for other similar research projects to
accurately analyze cellular metabolism.
1

Chapter 1. Fluorescent Lifetime Imaging Microscopy Provides Important
Biological Insights
1.1 Fluorescence microscopy: a powerful tool to understand cells and cellular activities
The cell is the fundamental building block of most living organisms, making the studies of
cellular structure, function, and behavior central topics in biological research (Welch, 1978;
Marchetti, et al., 2007; Khetani & Bhatia, 2008). Many approaches have been developed to
study cell physiology, including chemical assays or mass spectrometry to probe cellular
compositions and computational tools that model cellular behaviors and responses to the
environment (Mookerjee, Gerencser, Nicholls, & Brand, 2017; TeSlaa & Teitell, 2014). Yet the
most direct, effective, and versatile way to capture any cell’s behavior is cellular imaging
(Lakowicz J. R., 1999; Fine, Amos, Durbin, & McNaughton, 1988; Nwaneshiudu, et al., 2012).
Imaging cells allows for direct observation of their activities, including their growth and
reproduction, interactions with other cells, responses to the environment, and their processes
of apoptosis and death (Roberts-Dalton, et al., 2017; Wang, et al., 2021). Many imaging
modalities provide resolutions that can distinguish subcellular components and organelles. This
provides important insight into how these organelles coordinate to achieve different cellular
functions (Wang, Feri, Salim, & Jahnsson, 2018; Swinkels, et al., 2019; Pan, Yan, Li, & Xu, 2018;
Shotton., 1989).
Two essential aspects of cellular imaging have been improved over the years. The first is the
spatial resolution of imaging modalities. High spatial resolution is necessary for imaging cells as
2

they are typically several micrometers, with subcellular compartments as small as tens of
nanometers. Optical microscopes that reach resolutions of around 100 nm are now
commercially available from all major microscope manufacturers. The new generation of
super-resolution microscopes provide even better resolution, as high as 10 nm, and accurately
locate and track small cellular components such as secretion granules and ribosomes
(Schermelleh, 2019; Gustafsson, 1999; Hell & Wichmann, 1994).
The second significant improvement of imaging modalities has been to enhance the contrast to
distinguish different cellular compartments better. Cellular components are always located in a
complicated environment, surrounded by dozens of organelles and hundreds of other
compounds inside the cells. To distinguish one cellular part from another, one has to rely on the
properties of that cellular component that is different from its surroundings to generate
contrast. Some contrasts are intrinsic, including physical properties such as differences in light
absorption coefficients and refractive index (Allen & David, 1969; Maunsbach, 1967; Thorn,
2016). Some imaging modalities, such as Differential Interference Contrast (DIC) microscopy,
utilizes these physical properties to generate contrast for cellular components with good optical
resolution (Pluta, 1994; Wang, Shen, Setlow, & Li, 2015). However, in many cases, the physical
properties of small cellular components such as organelles, receptor proteins, and secretory
vesicles are hard to distinguish from their surrounding milieu and fail to generate stable
contrast.
To enhance contrast and localize specific targets, scientists use extrinsic fluorescent markers to
label structures and proteins of interest. Fluorescence molecules, known as fluorophores, can
3

be engineered to selectively bind to specific organelles, functional groups, or proteins. Many
synthetic dyes were designed to tag particular structures; for example, Tetramethyl Rhodamine
Methyl ester (TMRM), a positively charged red fluorescence dye, accumulates in healthy
mitochondria and detects mitochondrial membrane depolarization (Farkas, Wei, Febbroriello,
Carson, & Loew, 1989; Raymo, 2012). Immunolabeling, another commonly used fluorescence
labeling technique, uses antibody molecules to bind to the target of interest and then fuses a
fluorescence molecule to the antibody to help researchers to visualize their distribution
(Morton, 1970; Franke, et al., 1979). Genetic labeling uses fluorescent proteins such as Green
Fluorescence Protein (GFP) to label structures of interest by inserting the DNA sequence of the
fluorescent protein at desired locations of the gene, so they are expressed at those structures
(Lang, et al., 2012; Barrangou & Doudna, 2016; Pédelacq, Cabantous, Tran, Terwilliger, &
Waldo, 2006; Shimomura, Johnson, & Saiga, 1962). Calcium indicators and dyes are molecular
probes that exhibit a significant increase in fluorescence upon calcium binding and are often
used to measure neural activities (Paredes, Etzler, Watts, Zheng, & Lechleiter, 2008; Dana, et
al., 2016; Tsien, 1980). Many fluorescent labels provide bright contrast even at low
concentrations, are stable over a long period, and do not interfere with the normal function of
the sample if used carefully. Fluorescence microscopes were subsequently designed to image
fluorescent labels. These modalities include epifluorescence microscopes, confocal
microscopes, and two-photon microscopes, which allow the sample to be imaged with high
resolution (Thorn, 2016; Nwaneshiudu, et al., 2012; Shotton., 1989; Denk, Strickler, & Webb,
1990).
4

One significant advantage of using fluorescent labels is that multiple labels can be used
simultaneously, each tagging a different structure so that the interactions of cellular
components can be studied. An example is shown in Figure 1.1, where the mitochondria, F-
actin, and the nucleus are each tagged with different fluorescence stains, and the fluorescence
is collected with a fluorescence microscope. Tagging multiple structures simultaneously allows
researchers to study many interactions that happen between cellular components, such as
colocalization of proteins and organelles, molecular conformation, and association of the
proteins (Tang, Iwahara, & Clore, 2006; Cutrale, et al., 2017; Gross, et al., 2013) In the rest of
this chapter, we will explain in detail how fluorescence is used for generating contrasts in
cellular imaging, as well as the potential pitfalls for fluorescence imaging.

5

Figure 1.1 Fluorescent labels provide extrinsic contrast that enables imaging of specific
cellular structures. Imaging is one of the most direct and effective ways to understand
cellular activities. Many microscopes designed for fluorescent imaging have high spatial
resolution, which allows visualization of cellular compartments in detail. Fluorescence
labels enhance the contrast of specific imaging targets over other cellular structures and
the environment. Multiple fluorescence labels can be used simultaneously to study
interactions between cellular components. Fluorescence properties such as the
fluorescence color can be used as a contrast to distinguish different fluorescence
molecules. In the figure, the mitochondria, nuclei, and actin of H9C2 cardiomyocytes
were labeled with different fluorescent molecules and imaged with a confocal microscope
with high resolution. The H9C2 nuclei were stained with Hoechst 33342, actin stained
with Phalloidin-iFluor 488 Reagent, and mitochondria with Cox4-Alexa Fluor 594
antibodies. Each fluorescence label had a different color. Spectral filters were used to
collect the image of each fluorophore separately. The images were stacked together to
form the image shown above.
6

1.2 The process for generating fluorescence
Fluorescence is the emission of photons by fluorescent molecules that have absorbed light or
other electromagnetic radiation (Brand & Gohlke, 1972; Lakowicz J. R., Introduction to
fluorescence. In Principles of fluorescence spectroscopy, 1999). The fluorescence process can
be illustrated in a Jablonski diagram shown in Figure 1.2. The Jablonski diagram plots the
electronic levels electrons can occupy and energy transitions between these electronic levels.
Horizontal lines in Figure 1.2 represent the electronic energy levels of the fluorophore; the
straight vertical line in the figure shows the energy transitions between electronic levels. To
emit fluorescence, the fluorophore first absorbs energy through the absorption of photons of
light or other electromagnetic radiation. The energy the fluorophore absorbs is transferred to
an electron that gets excited from a lower energy level to a higher energy level. This is
indicated by the blue error pointing from the lowest energy state (𝑆 0
) to a higher energy state
(𝑆 𝑛 with 𝑛 > 0). After the excitation, the electron first relaxes to the lowest vibrational level of
the first excited state (S1) by dissipating energy through non-radiative processes like emitting
heat. After that, it settles back to the ground state by emitting the fluorescence photon with
energy 𝐸 𝑒𝑚
. In the process of fluorescence, the fluorophore stays in the excited state from a
fraction of a nanosecond to hundreds of nanoseconds before emitting the fluorescence photon.
The time for a fluorophore to emit fluorescence after the excitation is defined as the
fluorescence lifetime (Wolf, 2007; Lakowicz, Szmacinski, Nowaczyk, & Johnson, Fluorescence
lifetime imaging of free and protein-bound NADH., 1992). Different fluorophores have unique
electronic level compositions, and these electronic levels determine many fundamental
properties of fluorescence, including the absorption/emission spectra, lifetime, the probability
7

of photobleaching, and the likelihood of inducing phototoxicity, which will be explained in
detail in the rest of this chapter (Lichtman & Conchello, 2005).
The fluorophore only absorbs and emits photons with energy levels compatible with the energy
difference between the energy states. Because fluorophores have different energy structures,
each fluorophore will have distinguished excitation and emission spectra. The excitation and
emission spectra are very common contrasts to distinguish fluorophores. Many light sources
are designed to emit specific wavelengths to excite particular fluorophores. Meanwhile,
Figure 1.2 The Jablonski diagram illustrates the electronic states of molecules and the
transitions between these electronic states to generate fluorescence. A fluorophore
is excited to a higher energy level by an excitation photon with energy E Ex. After part
of that energy is dissipated through internal conversion (black curved arrow) and the
fluorophores reach the first excited state S 1, the rest of the energy is emitted as a
fluorescence photon with energy E Em. The emission happens not instantly after the
excitation but usually takes several nanoseconds. The time between the excitation
and the emission is called the fluorescence lifetime. Different fluorophores have
different energy structures for the electronic states, which results in different
excitation and emission spectra. The different fluorophores generally exhibit
different lifetimes. Both properties could be used to generate fluorescence contrast.
8

detectors often use optical wavelength filters to collect photons with a particular wavelength in
hopes of detecting specific fluorophores (Wolf, 2007).
Brightness is another essential property of fluorescence. It indicates the fluorescence output
per fluorophore. It is positively related to how strongly a fluorescent molecule absorbs light at
a particular wavelength, measured by the extinction coefficient, and how efficiently the
absorbed light is converted into emitted light, measured by the quantum yield. Many
commonly used fluorophores have both high quantum yields and extinction coefficients. For
example, two bright fluorophores, Enhanced Green Fluorescence Protein (EGFP) and
fluorescein, have quantum yields of 0.60 and 0.95 and extinction coefficients of 55,000 M
−1
cm
−1

and 70,000 M
−1
cm
−1
, respectively. In comparison, tyrosine, a fluorescent amino acid that is
much dimmer, has a much smaller quantum yield, only 0.13, and an extinction coefficient of
only 1405 M
−1
cm
−1
. (Lakowicz J. R., 1999; Patterson, Day, & Piston, 2001)
1.3 Limitations of fluorescence imaging
A few properties of fluorescence negatively affect the quality of fluorescence imaging. These
properties include photobleaching, phototoxicity, and fluorescence bleed-through. We will
explain these aspects in a detailed Jablonski diagram shown in Figure 1.3. Figure 1.3 shows the
triplet energy states and the sublevels of every energy state. In Figure 1.2, all electronic states
are labeled with S, representing the singlet electronic states. In these electronic states, the
excited electron is paired with the ground-state electron, and the pair have opposite spins,
resulting in a total net spin of zero. Triplet states in Figure 1.3 are shown in the column to the
right of the singlet states. In the triplet state, the excited electron is no longer paired with the
9

ground-state electron. This means they have the same spin (instead of the opposite spins as in
singlet states), and the total net spin is not zero. The transition between singlet states and
triplet states is defined as intersystem crossing. The spin transition involved in excitation to a
triplet state from a singlet state is usually forbidden. Therefore, intersystem crossing is less
likely to happen. However, when excited repetitively, the dye more likely goes through
intersystem crossing and gets excited to the triplet excited state which can cause
photobleaching.
Photobleaching is a phenomenon that fluorescent dyes become dimmer after repeated
excitation cycles in a short period of time. The fluorescence cycle can be repeated for most
fluorophores, and the fluorophore can keep generating fluorescence. These fluorescence
cycles only occur for energy transitions between singlet states. However, suppose under
repetitive excitation the dye goes through intersystem crossing and get excited to the triplet
excited state to return to the singlet ground state. In that case, the excited electron is quenched
by oxygen molecules in the environment (Wilkinson, McGarvey, & Olea, 1994). This process
generates singlet oxygen. These molecules react with the dyes and cause a photochemical
modification of the dye resulting in the irreversible loss of its ability to fluoresce. (Stracke,
Heupel, & Thiel, 1999; Demchenko, 2020; Lippincott-Schwartz, Altan-Bonnet, & Patterson,
2003).
Phototoxicity is another problem associated with fluorescence and is related to photobleaching.
The singlet oxygen mentioned in the mechanism for photobleaching can generate secondary
reactive oxygen species (ROS). Both singlet oxygen and ROS can damage proteins, nucleic acids,
10

and other cellular components, causing damage to cellular structures, inhibiting normal cellular
functions, and even inducing apoptosis. (Tirlapur, Konig, Peuckert, Krieg, & Halbhuber, 2001;
Laissue, 2017; Kalies, Kuetemeyer, & Heisterkamp, 2011).
The amount of photobleaching and phototoxicity increases with the amount of light exposure.
Hence when imaging fluorescent molecules, it is essential to carefully limit the excitation
energy. To avoid photobleaching and phototoxicity, researchers minimize both the power for
excitation and the repetition cycles exerted on fluorescent dyes. Therefore, fluorescence
images are often constructed with few photon counts, which causes another problem: Poisson
noise. The photon count at every pixel has fluctuations, just like the fluctuations seen when
counting the heads and tails tossing a fair coin with only a handful of trials. These fluctuations
are also known as counting errors and substantially affect the image quality when photon
counts are low. Because this counting error can be modeled with the Poisson distribution, it is
also called Poisson noise. Structures such as the mitochondria and cellular membranes can be
buried within the counting error, making it difficult to see sharp edges and details within the
image. (Rodrigues, Sanches, & Bioucas-Dias, 2008; Schaefer, Schuster, & Herz, 2001). Poisson
noise is the dominating noise source for images with low photon counts. Extracting valid
information in images with heavy Poisson noise is a significant topic for this thesis, which will be
discussed in detail in Chapters 3 and 4.

11

Figure 1.3. Detailed Jablonski diagram with sublevels and triplet state can be used to
explain photobleaching and why excitation and emission spectra are broad for
fluorophores. Triplet states are electronic states in which the excited electron is no
longer paired with the ground state electron. The transition between singlet states and
triplet states is defined as intersystem crossing and is usually forbidden. However, with
repetitive excitation, the fluorophore is more likely be excited to a triplet state. To
return to the singlet ground state, the excited electron is quenched by oxygen molecules
in the environment. This process generates singlet oxygen, which reacts with the dyes,
and causes a photochemical modification of the dye resulting in the irreversible loss of
its ability to fluoresce. This phenomenon is referred to as photobleaching. The singlet
oxygen, and reactive oxygen species generated by them, can damage the sample, and
cause phototoxicity. Each energy state for the electron is composed of several
vibrational states. Energy transitions for fluorescence can happen between this family of
vibrational energy states, as shown in the arrows with different colors. Therefore,
excitation and emission of fluorophores are not limited to single wavelengths,
broadening the emission and excitation spectra.

12

Another common problem fluorescence microscopy users often face is fluorescence bleed-
through. It refers to the phenomenon that when collecting the fluorescence of interest,
fluorescence from other fluorophores can be unintentionally collected by the detector. As
shown in Figure 1.3, electronic states are coupled with multiple vibronic energy states. These
energy states are shown as the thinner lines in each electronic state. The energy transition in
fluorescence can occur in these sublevels, which broaden the absorption and emission spectra.
In Figure 1.5, we show the emission spectra of green fluorescence protein (GFP), yellow
fluorescence protein (YFP), and red fluorescence protein (RFP). Although their naming, based
on their emission color, might suggest that they each emit single-color fluorescence, their
emission spectra are broad and significantly overlap. As shown in Figure 1.4, there is at least an
overlap of 2 fluorescence proteins at most wavelengths. Although not shown, the same case is
with their excitation spectra. When multiple fluorophores exist inside the sample, it is likely
that they are excited together because of their overlapping excitation spectra. As for the
detectors, even with optimal spectral filters applied, they cannot separate the fluorophore
species' fluorescence because of the emission spectra overlap. Fluorescence bleed-through
adds to the uncertainty of what the collected signal truly is. One might believe bleed-through
can be reduced by narrowing the collection wavelength range with optical bandpass filters.
However, as shown in Figure 1.4, such attempts fail, reducing the number of photons collected,
which adds to the previous explained Poisson noise issues. Fortunately, one way to
compensate for fluorescence bleed-through is to add additional contrast by using fluorescence
lifetime as a contrast.

13

Figure 1.4. The wide emission spectrum for fluorophores can cause fluorescence bleed-
through in fluorescence microscopy. This figure shows the emission spectrum of Green
Fluorescence Protein (GFP), Yellow Fluorescence Protein (YFP), and Red Fluorescence
Protein (RFP). Although their naming suggests that they only emit one color, their
emission spectra are wide and have noticeable overlaps, as shown in the figure on the
right. When collecting fluorescence from one fluorophore, bandpass filters are often
used to attempt to collect the fluorescence from a single fluorophore. However, there is
at least an overlap of 2 fluorescence proteins at most wavelengths, and no matter how
narrow the bandpass filter is, it is not enough to collect fluorescence from one single
fluorophore. This phenomenon is referred to as fluorescence bleed-through. It is a
common problem in many experiments when multiple fluorophores are used to label
different cellular structures. Figure generated with data provided by Thermo Fisher.
14

1.4 Fluorescence lifetime used as contrast
The fluorescence lifetime is different for fluorophores. Therefore, it can serve as a contrast to
distinguish them. Fluorescence lifetime provides several unique advantages compared to the
excitation and emission spectra. First, fluorescence lifetime is not as heavily affected by
fluorescence bleed-through as color. Therefore, it is beneficial to have fluorescence lifetime as
an additional contrast to distinguish different fluorophores. Second, fluorescence lifetime is
less affected by scattering. Photons with smaller wavelengths are scattered more inside the
sample than photons with higher wavelengths. Therefore, the color of the emitted light
changes with the imaging depth inside the sample. Lifetime, on the other hand, is consistent
regardless of the imaging depth. Third, fluorescence lifetime is sensitive in detecting some
phenomena. For example, fluorescence resonance energy transfer (FRET) causes changes in
lifetime, which make them easily detectable and quantifiable with lifetime measurements.
Microscopes detecting these phenomena improve their accuracy with lifetime calculations
(Biskup, et al., 2007).
For this thesis, the most crucial property of lifetime as contrast is that lifetime is sensitive to the
molecular conformation of many fluorophores. Particularly, reduced Nicotinamide Adenine
Dinucleotide (NADH), the primary imaging target for this thesis, has sensitive lifetimes to
changing surroundings. When NADH is free and not bound to an enzyme, NADH will have
direct contact with water molecules. Water molecules quench excited NADH, leading them to
release their energy faster. At this state, NADH’s lifetime is around 0.4 ns. When NADH is
bound to an enzyme, the enzyme shields NADH from direct contact with water. Therefore,
15

NADH is less likely to be quenched by water, and its lifetime increases to around 2 to 3 ns
( Lakowicz, Szmacinski, Nowaczyk, & Johnson, Fluorescence lifetime imaging of free and
protein-bound NADH., 1992). In comparison, their excitation and emission wavelength changes
are not as substantial. This property of NADH will further be explained in detail in the next
chapter. We will discuss how measuring the lifetime of NADH could give us a unique window to
measure metabolism, which is the principle of NADH autofluorescence metabolic imaging, the
primary application for this thesis.
The lifetime in fluorescence imaging can be measured with Fluorescence Lifetime Imaging
Microscopy (FLIM). Depending on the measuring mechanism, FLIM can be divided into Time-
Domain FLIM(TD-FLIM) and Frequency Domain FLIM(FD-FLIM). We will describe the details of
the two mechanisms and further explain their respective advantages and disadvantages.
1.4.1 Measuring fluorescence lifetime in the time-domain is straightforward in the collection but
complicated for analysis
Time Domain-FLIM measures the time it takes for fluorophore to emit fluorescence once
excited (Wang, Periasamy, Herman, & Coleman, 1992; Gadella Jr, Jovin, & Clegg, 1993; Datta,
Heaster, Sharick, Gillette, & Skala, 2020). After a laser with a short duration (a laser pulse)
excites the fluorophore at one location, a fast-gated image intensifier or Time-Correlated Single
Photon Counting (TCSPC) module serves as the detector to record the time between the
emission of the laser pulse and the arrival of the fluorescence photon. It is not sufficient to
determine the lifetime with readings from only one laser pulse. First, readings from only one or
a few photons suffer heavily from counting errors. Second, when a pixel has more than one
16

fluorophore, the different fluorescence species and their relative concentrations must be
calculated. Enough photons from all fluorescence species must be collected to perform the
calculation. To collect sufficient photons, the above-described procedure is repeated through
many cycles with a laser with a high repeat rate, so a histogram of the fluorescence lifetime can
be built. This histogram will have an envelope that follows an exponential decay, as shown in
Figure 1.5. Mathematical models can be used to fit the fluorescence intensity decay for
calculating the time constant of the exponential decay. If multiple fluorescence species are
present in the imaging sample, the histogram of the fluorescence lifetime will follow a
combination of multiple exponential decays, and the mathematical fitting will use statistical
tests to determine how many different exponential components are involved in the decay curve
and their relative contributions (Gerritsen, 2009). After fitting every pixel in the image with
decay constants, a fluorescence image based on the fitted lifetime for each pixel is built, as
shown in Figure. 1.6, which is the lifetime image for a convallaria sample. In the left panel of
Figure 1.6, the contrast is based on photon counts detected on each pixel. On the right panel,
the contrast is based on the lifetime readout, independent of the fluorescence emission
spectrum and the photon counts at each pixel. Notice that on the right panels, many pixels with
similar photon counts in the left panel have different colors. This indicates that they are
different fluorescence species. Note that this shows lifetime as contrast distinguishes
fluorophores that are not separable by fluorescence intensity.
17

Figure 1.5. Time Domain FLIM measures the lifetime by measuring the time it takes for
fluorophores to emit fluorescence once excited. A pulsed laser acts as the excitation
source, and the arrival time of the fluorescence is detected with either fast-gated image
intensifiers or TCSPC. The pulsed laser repetitively excites the fluorophores at a single
location, and the detector collects the arrival time of the photons. Sufficient photons have
to be collected to build an accurate envelope for the histogram of the fluorescence
intensity over time, as shown in the lower left image. This envelope for the histogram
follows an exponential decay curve, which is shown as the red line on the lower right
image. It is further fitted to an exponential decay model to derive the decay constant. The
number of laser repetitions is shown above each image. The τ in the equation is the decay
constant used to describe the lifetime. Calculating τ usually requires setting assumptions
of the sample and performing complicated statistical tests.
18

TD-FLIM has several disadvantages despite being straight forward to understand. First, the
collection of the arrival time of the photons is inefficient in TD-FLIM. During the process of
collecting fluorescence between laser pulses, most instruments are too slow to deal with more
than the first photons detected, and the other fluorescence photons will not contribute to the
generation of the image and the calculation of lifetime. To minimize the counting loss, the
laser intensity can be turned down. However, this causes imaging to take longer, which can be
problematic when imaging a moving sample or through large volumes or when the reactions to
be captured is fast. Second, fitting exponential decay and calculating the decay constant is
Figure 1.6. In Time Domain FLIM (TD-FLIM) images, lifetime as contrast distinguishes
fluorophores that are not separable by fluorescence intensity. The auto-fluorescence of
a convallaria sample is imaged with a Leica SP8 Falcon system with 488nm excitation.
Left: Convallaria image with photon counts as a contrast. Right: the same convallaria
image, using lifetime as a contrast. With TD-FLIM, the lifetime of each pixel is estimated
by fitting the fluorescence lifetime curve of every pixel with exponential decays. After
extracting the decay constants from every pixel, each pixel is color-coded by its lifetime,
which is the contrast for the image on the right. Lifetime as contrast is independent of
the photon counts of each pixel. Color coding of the image on the right is done by the
lifetime fitting performed by the fast FLIM module of Leica SP8 Falcon, which uses a
single-exponential fitting model.
19

complicated. This becomes more challenging when multiple fluorophores are present and
multiple exponential components are intermixed in the decay curve. Determining the lifetimes
of a mix of fluorophores and their relative contribution requires running statistical tests that are
not easy for researchers without extensive knowledge of the sample and plenty of imaging
experience and mathematical background.
1.4.2 Fluorescence lifetime in the frequency domain: straightforward representation and analysis
Frequency domain-FLIM is more elegant than time domain-FLIM in representing and analyzing
the lifetime data. It reduces the complexity of the analysis of lifetime by performing the
measurements and calculations in the frequency domain. (Behne, Sanchez, Moll, & Gratton,
2006; Digman, Caiolfa, Zamai, & Gratton, 2008; Colyer, Lee, & Gratton, 2008; Datta, Heaster,
Sharick, Gillette, & Skala, 2020). In FD-FLIM, fluorophores are excited with an intensity-
modulated light source, and fluorescence emission mirrors the excitation light, with a delay in
time (phase shift) and a decrease in modulation depth, as shown in Fig 1.7. The FLIM
microscope directly measures the phase shift and the modulation depth. Fluorophores with
shorter lifetimes will have a minor phase difference and a smaller modulation decrease;
fluorophores with a larger lifetime will have a more significant phase difference and a more
significant modulation decrease. The modulation decreases, and the phase difference can be
represented in a shorthand way as a complex number in a polar form with magnitude and a
phase, as shown in Figure 1.7(b). The magnitude of this complex number shows the
modulation decrease. For lifetime representation, it is the ratio of fluorescence modulation and
excitation modulation. The phase of this complex number equals the phase difference between
20

the fluorescence and the excitation. This representation is called the lifetime phasor. The
phasor can also be analyzed in the form of real and imaginary parts. In this representation, the
real and imaginary parts are equal to the phasor’s magnitude times the cosine and the sine of
the phase, respectively. For FLIM phasor representation, the real part is labeled as G, and the
imaginary part is labeled as S. Plotting S over G generates a graphical visualization of the
lifetime called the phasor plot. This representation is shown in Figure 1.7 (b). Fluorophores
with different lifetimes show different modulation decreases and phase differences, resulting in
different phasor representations, as shown in Figures 1.7(c) and 1.7 (d).
21

Figure 1.7. Lifetime in FD-FLIM is measured by detecting the delay in time (phase) and
modulation decrease of the fluorescence with respect to the excitation and displayed with
the phasor plot. (a) In FD-FLIM, the excitation is modulated to a frequency that is
approximately reciprocal to the lifetime of the fluorophore detected (black line). The
fluorescence follows the same modulation frequency, however with a phase difference and
a modulation difference. With the phase difference and their ratio of modulation, the
lifetime of the fluorophore can be estimated. (b) The modulation and the phase difference
can be represented by a phasor, a complex constant that is a shorthand for a sinusoidal
wave. The complex constant has a magnitude that is ratio between the modulation of the
fluorescence and the excitation, and a phase that is the phase difference between the
fluorescence and the excitation. This constant can be represented in the form of a real and
imaginary part. The real part is labeled as G, and the imaginary part is labeled as S.
(c)Fluorophores with shorter lifetimes will have a smaller phase difference and a smaller
modulation decrease, while fluorophores with a larger lifetime will have a larger phase
difference and a bigger modulation decrease. (d) Single exponential decay curves will
always have a phasor signature that is on the universal circle, with shorter lifetime placed
on the right further apart from the origin.
22

Lifetime data in TD-FLIM can also be transformed into the frequency domain with Fourier
transform. Fourier transform decomposes time domain data into cosine and sine waves with
their relative contributions at specific frequencies. When applying the Fourier transform to the
collected exponential decay shown in Figure 1.5, the Fourier transform calculates the amplitude
of the relative cosine and sine wave contributions. These magnitudes are further normalized
with the intensity of the decay curve. The cosine and sine components are equivalent to the G
and S shown in Figure 1.7 (b), with which the phasor plot is generated.

Figure 1.8. Lifetime data in TD-FLIM can be transformed into the frequency domain with
Fourier transform. With Fourier transform, the phasor plot can be constructed by
decomposing the exponential decay measured in TD-FLIM into sine and cosine wave
components. The exponential decay curve in (a) is collected with TCSPC. Figure (b)
shows the cosine and sine wave components. The magnitudes of the waves are the
Fourier coefficients, which are further normalized based on the intensity of the signal.
The amplitude of the normalized sine component is labeled as S, and the cosine is
named G. When projecting the Fourier coefficients into the Cartesian coordinates, with
the cosine coefficient as the horizontal axis and the sine on the vertical, the phasor plot
is generated. This phasor plot is the same as Figures 1.7 (b) and (d).
23

The phasor plot allows for graphical presentation and lifetime analysis, which is much easier
than mathematical fitting to calculate lifetimes. The lifetime of fluorophores can be easily
measured from the phasor plot. If an exponential decay has only one term, which is the case
for most fluorophores, its phasor representation will be on the universal circle shown in Figure
1.7(d). A shorter lifetime will have a phasor signature to the right of the origin, while a longer
lifetime will have phasor signatures closer to the origin. With FD-FLIM, every pixel of the
photon intensity map will have a corresponding phasor location inside the phasor plot. Pixels
with similar lifetime information will have similar locations on the phasor plots, forming a
phasor cluster. This graphical representation of lifetime in the phasor map allows easy
grouping of pixels with similar lifetimes, and the different lifetimes are directly observed in the
phasor plot. When doing phasor analysis, the phasor plot and the photon intensity image are
put side by side, and pixels in the intensity image can be color-coded by their location inside the
phasor plot, as shown in Figure 1.9. In Figure 1.10, this principle is applied to the convallaria
sample shown in Figure 1.6. Every pixel in Figure 1.10 (c) is color-coded with lifetime
information based on its phasor signature and its associated color in Figure 1.10 (d). It allows
direct visualization of the lifetime distributions inside the intensity image to correlate lifetime
with structures inside the sample.

24

Figure 1.9. Phasor analysis allows grouping pixels on the photon intensity map based on
lifetime. The phasor plot and the original image can be put side by side when
performing phasor analysis. On the left above is a simulated image where the different
sectors on each corner have different lifetimes. The lifetime information is displayed by
the phasor plot on the right. Every pixel in the intensity map has a corresponding
phasor location inside the phasor plot. The phasor plot for an image is a 2D histogram
for the G and S coordinates for all the pixels in the image. Pixels with similar lifetime
information will have similar locations on the phasor plots, forming a phasor cluster. By
assigning different colors to the phasor plot locations, the structural information can be
color-coded based on their lifetime. This allows researchers to visually associate
lifetime with structures on the photon intensity map.

25

Figure 1.10. FD-FLIM and phasor plots allow color-coding pixels of similar lifetimes based on
their location inside the phasor plots. The auto-fluorescence of the same convallaria sample
as in Figure 1.6 is imaged with a Leica SP8 Falcon system with 488nm excitation and
processed with phasor analysis. Every pixel in FLIM has its phasor coordinate in the phasor
plot. The location of the phasor coordinate indicates its lifetime composition. Pixels can be
grouped inside the intensity image based on their corresponding location inside the phasor
plots and assigned a specific color to indicate that they have similar lifetimes. (a) Photon
intensity image of the convallaria sample. The contrast shows how many photons are
collected from every pixel. (b) Phasor plot as a histogram of G and S components from the
field of view of (a). (c) FD FLIM image of the convallaria sample, color-coded based on
lifetime. Pixels with similar colors indicate that their lifetime information is similar. (d) Phasor
plot color-coding map that (c) is based on. Image (c) was obtained by color coding each pixel
with the color assigned to its corresponding phasor signature in (d). The phasor
representation allows direct visualization of the lifetime distributions inside the intensity
image to correlate lifetime with structural information.

26

An essential benefit of phasor analysis is that unmixing of mixed lifetimes becomes
straightforward. In a real-world scenario, every pixel of an image is very likely to contain
photons from more than one fluorophore. An important task for researchers is to calculate the
relative contributions of these fluorophores. When a pixel contains a mixture of two
fluorophores with different lifetimes, the pixel’s phasor signature will be on the line connecting
the individual fluorophores’ phasor signatures. It can be mathematically derived that the
pixel’s distance to each phasor signature is linearly related to the relative contribution of these
fluorophores. The shorter the distance, the higher the relative contribution, as shown in Fig. 1.
11. With this property, the relative concentrations of the different fluorophores can be easily
calculated. If more components exist, then the phasor signature of the mixture must be
located inside the convex set connecting all the phasor signatures of the individual species.
27

Fig 1.11. Phasor analysis allows straightforward unmixing of mixed lifetimes. In TD-
FLIM, unmixing of lifetime relies on complicated statistical tests. In comparison, the
unmixing of fluorophores in the FD-FLIM phasor plots can be directly visualized. When
analyzing the phasor signature of a pixel that is a blend of fluorophores two single
exponentials, its phasor signature will lie on the line connecting the individual
fluorophores’ phasor signatures on the semi-universal circle. The distance to each
phasor signature is linearly related to the relative concentration of the fluorophores,
the shorter the distance, the higher the relative concentration. (a) The blue and red
curves show two single exponential decay curves, and the magenta is a mixture of
these two, with a relative ratio of 3:1. (b) The respective phasor signatures of the three
curves in (a) are shown in a phasor plot. The magenta signature is on the line
connecting the separate two dots, and its distance to the red phasor signature and the
blue phasor signature is 3:1. As it has a higher concentration of the longer exponential
decay, its phasor signature is significantly closer. The unmixing of the fluorophores is
much easier inside the phasor plot.
28

1.5 Scatter error in phasor analysis brings uncertainties in lifetime measurements
As mentioned above, to avoid phototoxicity and photobleaching, the exposure of excitation
light to the fluorophores is limited, and fluorescence micrographs are often constructed with
very few photons per pixel. Therefore, most fluorescence images carry a significant amount of
Poisson noise. Although phasor analysis reduces the complexity of processing and analyzing
FLIM data, it is still affected by Poisson noise. This effect is demonstrated by imaging a field of
view with one single fluorophore species (ER-Tracker™ Red from ThermoFisher) in Figure 1.12.
In theory, such a field of view has one single lifetime, therefore, it will have a tight spot as its
phasor signature inside the phasor plot. This can be achieved with an extremely high photon
count, as shown in Fig 1. 12(a). However, as the number of photons collected for each pixel
decreases, the variance of the calculated G and S increases, and the tight spot in the ideal
scenario transforms into a cloud. This effect is defined as scatter error (Cutrale, et al., 2017).
The G and S values for every pixel in the field of view will be centered around the ideal phasor
signature. However, the fewer the photon counts, the bigger the variance of the G and S
coordinate values, and the more scatter error is induced. Unfortunately, the number of
photons acquired for most experiments is similar to that depicted in Figure 1.12 (d). When
using exogenous labels, it is possible to increase the fluorescence by increasing the
concentration of the fluorophores. Even so, the photon counts are often only in the range of
several hundred. When imaging NADH, which is an endogenous molecule, the natural
abundance of NADH inside the sample provides very limited fluorescence, and most FLIM
images are only constructed with tens of photons per pixel. In this case, the phasor plot is
heavily affected by scatter error, making it extremely difficult to extract accurate lifetime
29

readings. Filtering is needed to accurately analyze these phasor plots, which is one major topic
discussed in future chapters.
Figure 1.12. Scatter error arises in phasor analysis when the photon counts are low
and blurs the actual location of the phasor signature. This is demonstrated by a
phasor plot for a field of view of pure ER-Tracker™ Red dye with varying photon
counts. In an ideal scenario where the image is constructed with vast numbers of
photons, the phasor signature that only contains one single fluorophore is a tight spot,
as shown in (a). However, when decreasing the number of photons in each pixel,
uncertainties arise inside the lifetime calculations and transform the tight spot in the
phasor plot into a cloud. This expansion in the phasor plot is defined as scatter error.
The fewer photons count each pixel has, the more substantial will the scatter error be.
Unfortunately, most FLIM experiments are constructed with photon counts that are at
best similar to (c), and when imaging endogenous labels like NADH, their phasor plots
are most likely to be similar to (d). Under these circumstances, it is difficult to extract
accurate lifetime readings. Filtering is needed to analyze these phasor plots accurately,
which will be a major topic in future chapters.
30

This thesis will show how to deal with the scatter error inside the phasor plots with two
separate filtering techniques. But before introducing these techniques, the next chapter will
first focus on a major application of FLIM with phasor analysis: autofluorescence metabolic
imaging. Metabolic imaging has been widely used in many different types of research over the
past decades; however, in many, not all the information inside the FLIM datasets is extracted.
The next chapter will introduce the basics of metabolic imaging and will further present a
workflow to improve the analysis process for metabolic imaging.

31

Chapter 2. NADH autofluorescence FLIM Enables Metabolic Imaging with
High Spatial and Temporal Resolution
2.1 Cellular metabolism is key to understanding cellular behavior
Cellular behaviors are supported by complex sequences of biochemical reactions known as
cellular metabolism, which are also known as metabolic pathways (Nielsen & Keasling, 2016;
DeBerardinis, 2012). Almost all cellular functions are supported by these biochemical reactions,
including the generation and consumption of energy, responses to environmental stimuli,
communication between cells and neurons, cell growth, and apoptosis. Cellular metabolism is
central to many diseases that are related to cellular dysfunction, including diabetes, Alzheimer's
disease, and cancer. Because of its major role in cellular activities, cellular metabolism is a
central topic in the science of biology (Zhao, et al., 2010; Zhu, Zhao, Yang, Ding, & Zhao, 2015).
The activities related to cellular metabolism are complicated and highly coordinated. The
reactants, products, and intermediates involved in metabolic pathways are called metabolites.
A complete set of metabolic pathways for any cellular function can involve hundreds of
different metabolites. Every organelle in the cell has a designated group of metabolic
pathways, and they have elaborate mechanisms to control their metabolites and enzymes. The
concentration of the metabolites and many different enzymes tightly regulate what
biochemical reactions occur and the reaction speed (Zinser, Paltauf, & Daum, 1993; Rovira &
Smith, 2019; DeBerardinis, 2012). Figure 2.1 displays a subset of the metabolic pathways
responsible for glutamine metabolism to show the complexity of the metabolic pathways. This
32

pathway helps rapidly proliferating cells meet the increased demand for Adenosine 5'-
triphosphate (ATP, the primary energy source for cells), biosynthetic precursors, and reducing
agents. Although only a subset of the metabolic pathway is shown, it already involves dozens
of different metabolites. Each subcellular compartment, for example, the cellular membrane or
the mitochondria, is responsible for a specific set of metabolic reactions. The reactions are
cascaded from one organelle to another, and the functions of these organelles are closely
related. Understanding the role of each component in the cell and the coordination of the
subcellular compartments are essential keys to deciphering the secrets behind cellular
metabolism.
33

Figure 2.1 Cellular metabolism, also known as metabolic pathways, is the set of chemical
reactions that support the functions of the cell. Cellular metabolism is complicated and
highly coordinated. This figure depicts a part of the metabolic pathways involved in
glutamine metabolism. These pathways help rapidly proliferating cells meet the
increased demand for ATP, biosynthetic precursors, and reducing agents. Although only
a subset of the metabolic pathway is shown, it already involves dozens of different
metabolites. Each subcellular compartment, for example, the cellular membrane or the
mitochondria, is responsible for a specific set of metabolic reactions. The reactions are
cascaded from one organelle to another, and the functions of these organelles are closely
related. Note that the metabolic pathways shown in the figure are all oxidation and
reduction reactions (redox reactions), which are chemical reactions that involve the
transfer of electrons. The transfer of electrons in these reactions is carried out by NADH
and FADH 2, two important enzymes in all living organisms. Image Origin:
https://www.cellsignal.com/path-ways/glutamine-metabolism
34

Many techniques have been developed to study cellular metabolism. One standard metabolic
test is called Seahorse (Mookerjee, Gerencser, Nicholls, & Brand, 2017; TeSlaa & Teitell, 2014).
Seahorse measures the oxygen consumption and the proton released in the immediate
surroundings of the cell, with which the cell’s metabolic state is inferred. One of the most
significant advantages of seahorse is that the analysis is non-invasive and does not require any
addition of dyes, labels, or reporters. Therefore, the sample can be used for additional
measurements after the seahorse analysis. However, there are several cons to this technique.
First, seahorse does not provide any spatial resolutions within a cell. Seahorse measurements
are taken from the extracellular matrix, meaning each cell acts as a black box, and the actions
within the cells are obscured from the measurements. Besides, seahorse studies use
disposable assay kits, which are relatively expensive, and when a metabolic study requires large
numbers of samples, the cost could build up quickly.
Another common way to study cellular metabolism is using mass spectrometry for
metabolomics (Junot, Fenaille, Colsch, & Bécher, 2014; Domenick, Gill, Vedam-Mai, & Yost,
2020; Ren, Zhang, Kong, & Wang, 2018). It is a powerful tool for measuring the relative
concentration of hundreds of different metabolites inside a sample by measuring the ion’s
mass-to-charge ratio. Mass spectrometry allows researchers to probe what reactions are taking
place and build mathematical models to explain and predict cell reactions. However, one major
disadvantage of mass spectrometry is that there is no spatial resolution for the measurements.
With mass spectrometry experiments, the sample is vaporized and ionized, and it is hard to
pinpoint the subcellular location of where the data is collected. Besides, with the ionization
and vaporization of the samples, there is no possibility of reusing the sample for time-lapse
35

experiments. If a time-lapse study is desired, the only way is to examine the different time
points on different batches of cells. This induces systematic error for metabolic measurements
and loses sight of individual cellular responses.
Metabolic changes are cascades of reactions coordinated over multiple organelles; therefore,
studying metabolism requires profiling metabolic changes over time and in different locations.
Most questions regarding cellular metabolism are time sensitive. For example, how rapidly do
metabolic changes occur in response to external stimuli? What are the orders of metabolic
reactions of the cell before it recovers to its normal state? To study these questions, it is
essential to provide metabolic measurements with temporal resolution. Besides temporal
resolution, spatial resolution is equally crucial because cellular metabolism is location
dependent. Cells are filled with organelles; each organelle has its specific function and a set of
metabolic pathways. When an environmental change occurs, it is crucial to understand which
part of the cell reacted to the stimulus. Did all organelles react simultaneously, or did a portion
of the cell respond consecutively behind others? Answering these questions requires the
measurements to have the fine spatial resolution to resolve subcellular components.
Fluorescence microscopy provides both the spatial and temporal resolution needed to study
metabolism. To study metabolism by fluorescence imaging, a fluorophore is required that
generates contrast for changing metabolism. In other words, this fluorophore needs to alter its
fluorescence properties when cellular metabolism is changing. This chapter will explain how
this is achieved by imaging NADH with fluorescence lifetime imaging microscopy.
36

2.2 NADH lifetime changes with cellular metabolism
Most cellular pathways are oxidation and reduction reactions (redox reactions), which are
chemical reactions that involve the transfer of electrons. Inside cells, electrons are carried
through by coenzymes. One essential coenzyme that plays a significant role in metabolism is
Nicotinamide adenine dinucleotide (NAD). NAD has two forms, the reduced form:
Nicotinamide adenine dinucleotide plus hydrogen (NADH), and the oxidized form: NAD
+
. NAD
+

is not fluorescent and cannot be directly imaged. On the other hand, NADH’s reduced form is
auto-fluorescent, meaning itself is a fluorophore, and no additional tagging or staining steps are
required for fluorescence imaging. The excitation wavelength of NADH is around 370 nm.
However, 370 nm is in the ultraviolet wavelength region. Photons at this wavelength carry
higher energy than regular light, and imaging cells with them could easily lead to phototoxicity,
intoxicating the imaging sample and affecting their metabolic state. To avoid this, NADH can be
imaged with 2-photon excitation, meaning two photons with lower energy are used to excite
NADH instead of a high-energy photon. To achieve that, NADH is excited with photons of 740
nm wavelength, roughly two times the wavelength and half the energy of photons of 370 nm
wavelength. As the scattering of light is weaker with a higher wavelength, imaging at 740 allows
imaging deeper inside the sample. Experiments described in this thesis used 740 nm as the
NADH excitation wavelength and collected NADH fluorescence from 420 nm – 530 nm.
37

NADH can bind various enzymes in metabolic pathways, depending on the reaction it is
involved in. As the carrier for transferring electrons from one molecule to another, NAD(H) is
heavily involved in glycolysis, oxidative phosphorylation, pyruvate oxidation, and the citric acid
cycle. The fluorescence properties of NADH change with their environment and are used to
infer the metabolic state of the imaging sample. Researchers have used NADH to indicate
redox states for many years ( Lakowicz, Szmacinski, Nowaczyk, & Johnson, 1992; Wang, et al.,
2021). As a rule of thumb, higher bound NADH concentrations usually indicate more oxidative
phosphorylation in the sample; on the other hand, higher concentrations of free NADH indicate
more glycolysis. As described in chapter 1, NADH’s lifetime depends on its binding state with
the enzyme. When NADH is free and not bound to an enzyme, NADH will have direct contact
Figure2.2 NADH has substantial changes in lifetime depending on its binding states with
enzymes. The lifetime changes can be used to infer NADH’s cellular metabolic states.
NADH is the reduced form of NAD and is heavily involved in metabolic reactions. NADH is
autofluorescent and can be imaged without additional tagging. (a): Difference of emission
spectrum for bound and free NADH. (b): Difference of lifetime for bound and free NADH.
Binding with an enzyme causes changes in both the emission spectrum and the lifetime of
NADH. However, the change is more substantial in lifetime. Since changes in the lifetime
are much more significant compared to the wavelength, measurement in lifetime can more
accurately indicate changes in metabolism. This is the basis of auto-fluorescence metabolic
imaging of FLIM. Image origin: Long, 2013
38

with water molecules. Water molecules help NADH release their energy faster, causing them to
have a smaller lifetime value (around 0.4 ns). When NADH is bound to an enzyme, the enzyme
shields NADH from direct contact with water. Therefore, NADH has a lower probability of being
quenched by water. This does not substantially affect the emission wavelength, as shown in Fig
2.2(a); however, its lifetime becomes considerably longer, varying from 1.5ns to 3.2 ns,
depending on the enzyme it is binding to. The different NADH lifetime values depending on the
bounding enzyme are shown in Figure 2.3. This property gives us a unique window for probing
cell metabolism by measuring the lifetime of NADH. A summary of the changes is summarized
in table 2.1.

Table 2.1. NADH lifetime changes with respect to different metabolic states of the sample, and
therefore can be used as a fluorescence probe to image cellular metabolism.

39

2.3 Frequency domain FLIM provides clear contrast for NADH lifetime
The NADH lifetime difference, which reflects the sample’s metabolic state, is easily visualized
and analyzed with Frequency Domain FLIM (FD-FLIM) and the phasor plot. When NADH is
bound to an enzyme, or when NADH is free from enzymes, its fluorescence decay curve will
follow a single exponential decay. Each single-term exponential decay curve has its unique
position inside the phasor plots on the universal circle, as shown in Fig 2.3. Free NADH with
0.4ns lifetime has a lifetime signature at the far side to the right of the origin in the phasor plot.
When NADH is bound to enzymes, the lifetime of NADH will increase, and the signature of
NADH will move further to the left of the universal circle. By visualizing the phasor signature of
Figure 2.3. NADH has different lifetimes depending on the enzyme it is bound to. NADH has a
single exponential lifetime decay when NADH is free or bound to a single enzyme. The NADH
lifetime with different enzymes is shown in the phasor plot. The phasor signature of a single
exponential decay is always on the circle shown in the figure. On the right is a list of enzymes
NADH binds to and the respective lifetime. When NADH is free, it has a very short
lifetime(0.4ns), and its lifetime signature is to the right of the phasor plot. With increasing
lifetimes by binding to different enzymes, the phasor signature of NADH moves further to the
left on the universal circle. The phasor plot was analyzed with the first harmonic with an
80MHz laser pulse. Image origin: Leben, 2019

40

NADH on the phasor plot, it is easy to examine whether NADH is free or bound to a certain
enzyme.
Another important task for metabolic imaging users is calculating the sample's free and bound
NADH ratio. In any given pixel of an NADH fluorescence image, NADH is never 100% free or
100% bound; instead, it is a mixture of both. The ratio between these two compartments must
be calculated to understand whether glycolysis or oxidative phosphorylation dominates.
Phasor analysis easily provides this information with the linear properties of the phasor analysis
Figure 2.4. The metabolic states of the sample can be indicated by the location of its NADH
phasor signature. The red and blue points on the universal circles indicate 100% free and
100% bound NADH lifetime, respectively. When NADH is a combination of free and bound
NADH, its phasor location is on the line connecting the two phasor signatures, and the
ratio of free versus bound NADH, which is the ratio between glycolysis and oxidative
phosphorylation, could be calculated by examining its distance two to these phasor points
based on the linearity of phasor analysis. The regions to the right have higher free-NADH
concentrations than bound-NADH, indicating that more glycolysis is taking place. On the
other hand, regions to the left have more bound-NADH than free-NADH, and oxidative
phosphorylation dominates. The different colors indicate the different relative proportions
of oxidative phosphorylation versus glycolysis.

Metabolic Phasor Plot
41

inherent in Fourier analysis explained in Figure 1.11. The NADH phasor location lies in the line
connecting the free and bound NADH phasor signatures. The free versus bound NADH ratio can
be calculated by measuring its distance to these phasor signatures. The closer the pixel’s
phasor signature is to one of them, the higher its relative concentration of that particular
component. A visual demonstration is displayed in Figure 2.4. Different ratios of free and
bound NADH concentrations are shown with different colors. In this figure, the phasor
positions tilting to the red have more free-NADH than bound-NADH, indicating that more
glycolysis is taking place. On the other hand, for bluer phasor positions, there is more bound-
NADH than free-NADH, and oxidative phosphorylation dominates. The different colors indicate
the different relative proportions of oxidative phosphorylation versus glycolysis.
This color-coding scheme can be combined with the principles explained in Figure 1.9, where
the pixels in the photon intensity map are color-coded with lifetime information, so that
structures in the photon intensity map are color-coded with their metabolic states. Figure 2.5
demonstrates how it is applied to a real dataset of a mouse islet. The autofluorescence NADH
FLIM dataset was collected from a wild-type mouse with a Leica SP8 Falcon system. The
intensity image is shown in Figure 2.5 (a), with the phasor plot shown in Figure 2.5(b). A color
code based on the metabolic states, shown in Figure 2.5(d), is assigned to the phasor plot to
correlate metabolic information with structural information. Figure 2.5(c) is the color-coded
intensity image based on the phasor color code shown in Figure 2.5(d). Colors indicate the
metabolic states: blue means there is more glycolysis, red more oxidative phosphorylation. It
can be directly visualized that regions in the mouse islets have different metabolic conditions,
probably indicating areas occupied by different cell types or performing different functions.
42

This shows a significant advantage of metabolic FLIM imaging. Metabolic information can be
displayed with the same spatial resolution of the FLIM microscope, which can be as high as
several hundred nanometers.

Figure 2.5. The NADH lifetime phasor analysis of a mouse islet allows correlating metabolic
information with structures in the photon intensity map. Pixels are color-coded in a
photon intensity image based on their corresponding by their metabolic states indicated by
the NADH lifetime. (a) The photon intensity image of the mouse islet. This image only
informs how many photons are collected from every pixel. (b) The phasor plot as a
histogram of G and S components from the dataset (a). (c) The FD FLIM image of a mouse
islet color-coded based on the metabolic states. (d) The phasor plot color-coding map that
indicates the metabolic states. Image (c) was obtained by color coding each pixel with the
color that is assigned to its corresponding phasor signature in (d). Notice that regions with
similar colors indicate similar metabolic states. This allows direct visualization of the
distribution of regions of different metabolic states and correlate them with structures in
(a). Data collected with Leica SP8 Falcon system. NADH excitation: 740 nm; NADH detection
range: 420nm – 500nm.
43

2.4 Challenges of metabolic NADH FLIM imaging and analysis
Three major hurdles have not been properly addressed by the research community, especially
when it comes to extracting metabolic information at a subcellular level. First, although
metabolic imaging with phasor analysis has already been used for over two decades,
researchers have not yet found a way to successfully transfer the information from the phasor
plot into accurate biological meanings (Datta, Heaster, Sharick, Gillette, & Skala, 2020; Stringari,
et al., 2012). Many research publications only display metabolic signatures of the experiment
group and the control group in the phasor plots to demonstrate the difference between the
groups; however, the biological meaning of these phasor signatures is rarely summarized in a
quantitative manner. No statistical tests can be run if the biological implications are not
extracted with numbers, and the results cannot be confidently concluded. The quantification of
the biological meanings behind the NADH phasor signatures is challenging because when
deciphering NADH’s lifetime, the process must be calibrated to the enzymes involved, as the
enzyme species will affect the lifetimes of NADH. This is essential for subcellular metabolic
analysis, as the different cellular compartments have different enzyme species binding to
NADH. Many analyses workflows for metabolic FLIM are only calibrated for one single enzyme
species and therefore are not suitable for subcellular metabolism analysis.

44

The second hurdle for proper analysis of NADH FLIM signal is that NADH images usually have
low photon counts and require proper filtering for the extraction of valid information (Ma,
Digman, Malacrida, & Gratton, 2016). Unlike fluorescence labels and fluorescence dyes, NADH
is not a fluorophore that can be added to the sample. Most imaging relies only on the natural
abundance of NADH to supply fluorescence. The quantum yield, defined as how many emitted
photons could be generated per absorbed photon, is only 0.02 for Free NADH and 0.1 for bound
NADH. This is relatively low compared to other fluorophores often used in fluorescence
microscopy. For example, GFP has a quantum yield of 0.6; Fluorescein has a quantum yield of
0.93 (Lakowicz J. R., Introduction to fluorescence. In Principles of fluorescence spectroscopy,
1999). Because of the low abundance and the low quantum yield of NADH, only a limited
amount of NADH fluorescence is collected to build the NADH lifetime profile. Many metabolic
studies involve time lapses to track metabolic changes over time. Suppose the time intervals do
not allow complete replenishment of NADH molecules. In that case, the limited photons
collected from NADH are further shared between multiple time points, causing the images to
be even more photon starved.
Another problem that causes low photons in NADH imaging is that most imaging samples are
subject to phototoxicity, which causes unexpected metabolic changes inside the sample. If one
wishes to increase the laser power or the number of times the sample is scanned, one could
risk inducing phototoxicity inside the sample. Once phototoxicity is induced, the metabolic
signatures collected are more likely to represent procedures such as self-reparation or
apoptosis (the self-initiating procedure for cells to die) instead of the question of interest.
45

Therefore, metabolic imaging is performed with low laser power, limiting the number of
photons collected.
The lack of photons induces scatter error inside the phasor plots, as mentioned in the above
chapter. This scatter error clouds the actual location of the metabolic phasor signature one
wishes to study. Making things even worse, the metabolic changes of some stimulus or
pathological changes are very subtle, and the changes in the phasor coordinates could be hard
to extract within the noise. Post-processing and filtering have been involved in the phasor
analysis to counter the effect of scatter error. The most common way is to apply median filters
onto the phasor maps during the phasor analysis process (Cutrale, et al., 2017). However, this
reduction of scatter error comes with a price. The following chapters will explain how lifetime
information of structural details, especially those with high spatial frequency, suffer from
median filtering in the phasor analysis. These high spatial frequency details inside a biological
sample usually represent membranes, vesicles, mitochondria, and components with distinct
structural details. These components are often the interest of the studies. Better filtering
techniques must be introduced to resolve these problems.
The third challenge for metabolic imaging arises from the heterogeneity in metabolism. Every
cell is composed of many different components that host different metabolic pathways. To
accurately quantify the various metabolic processes inside the cell, it is crucial that the
metabolic imaging will have enough resolution to resolve these structures so that their
metabolic signatures can be analyzed separately. Meanwhile, in many cases, different cell
types are imaged inside the field of view, especially for complicated biological samples that are
46

not single cell lines. The different cell types inside the same field of view will have different
metabolic responses to the environment and stimulus. Even when imaging cells that belong to
the same cell type, not every cell will have the same response. Sometimes the cells will
respond in waves under stimulus, depending on their surroundings to ambient nutrition (Van
Schravendijk, Kiekens, & Pipeleers, 1992). Researchers are interested in studying this
heterogeneity amongst cells and revealing the cellular population dynamics behind these
phenomena. FLIM is able to resolve metabolic changes of different cell types and cellular
components, however, current studies of phasor analysis do not segment the field of view into
individual cells and cellular compartments. Instead, most of the feature extraction of the phasor
plots is done on the entire field of view. Although studies might still find differences in the
phasor plots, this difference could be an artifact due to the composition of the field of view, not
due to actual metabolic changes inside each cell or cell component.
This Ph.D. thesis presents a workflow to solve the previously mentioned three problems. First,
chapter 3 presents the procedures to quantify the metabolic changes shown in phasor plots
and extract a parameter calibrated to the involved enzyme. It allows extracting biological
meaning from different subcellular components. In chapters 4 and 5, two different filtering
techniques are presented to extract valid phasor information from the scatter error, especially
when the photon counts are low. These two filtering techniques not only work for NADH
metabolic imaging but other phasor applications as well. Chapter 6 will talk about how to
tackle the segmentation of different cellular compartments with machine learning with the
example of INS-1E cell lines. After segmenting subcellular compartments, their metabolic
47

changes under the glucose stimulus can be tracked separately, leading to metabolic analysis
that has great precision and reveals the metabolic relations between the organelles.

48

Chapter 3. Extracting Metabolic Information from FLIM Phasor Plots
3.1 Metabolic NADH lifetime readings are hidden in noisy phasor clusters
In previous chapters, we explained how fluorescence lifetime information is displayed in phasor
plots (Datta, Heaster, Sharick, Gillette, & Skala, 2020). The phasor plot displays the 2D
histogram for the G and S components calculated from every pixel when representing the
lifetime information for multiple pixels. Pixels with similar lifetimes form phasor clusters in the
phasor plot, as shown in Figure 3.1. Much information regarding the fluorescence lifetime of
the imaged sample can be extracted from the phasor plot. The most direct information is how
Figure 3.1. Pixels with similar lifetimes form phasor clusters in the phasor plot. In FD-
FLIM, the phasor plot is a 2D histogram of the G and S phasor components. The color
code in the figure indicates the histogram counts. The lifetime information displayed
is a summary of the statistical distribution of the lifetime of the collected photons. As
shown in the figure, pixels with similar lifetimes form phasor clusters that likely follow
a 2D gaussian distribution. Multiple phasor clusters emerge when fluorescence
species with different lifetimes exist in the image. By observing the phasor plot, one
can estimate the number of fluorescence species in the image. Statistical tools can be
used to summarize the lifetime information inside the phasor cluster.

49

many different fluorescent species are contributing to the image, and how many pixels each
fluorophore occupies. In Figure 3.1, two phasor clusters are shown in the phasor plots,
indicating that this image most likely contains fluorescence from two separate fluorescence
species. Meanwhile, the phasor cluster on the right has a much higher histogram count,
indicating that this fluorescence species occupies more pixels in the field of view.
In many cases, especially for NADH metabolic imaging, we want to analyze the lifetime
composition from the phasor clusters. The location of the NADH phasor cluster provides an
intuitive analysis on the metabolic state of the sample, which is explained in Chapter 2.
However, to accurately decipher the information inside the phasor clusters, it is not sufficient
to only visualize the lifetime in the phasor plot. The information from the phasor cluster has to
be quantified. Quantification of the features of phasor clusters makes it easier to keep track of
and examine the lifetime properties inside different datasets. It also enables performing
statistical tests on the different phasor distributions, which are necessary to conclude metabolic
differences from different regions or different experimental conditions with confidence.
Quantifying the phasor information is challenging in the presence of noise. In Figure 1.12, we
have shown how low photon counts introduce scatter error. The G and S values of the phasor
cluster is positioned close to the ideal phasor signature, however, the less the photon counts,
the bigger the variance of the G and S coordinate values and the more scatter error is induced.
This is also shown in Figure 3.1, where both clusters have significant spreads around their
centers. Note that counterintuitively, the wide spread of the cluster does not necessarily
indicate that the field of view contains more of the corresponding fluorescence species. It likely
50

indicates that the species did not contribute enough fluorescence photons and therefore
introduced more scatter error. To extract what fluorescence species the phasor cluster
represents, the scatter error has to be reduced reveal to the actual phasor coordinates of the
fluorescence species. This requires knowledge of the fluorophores’ lifetime, understanding of
the biological sample, and background in statistics. One way to approach this task is with
filtering techniques, which will be discussed in the latter two chapters. Most filtering can
eliminate scatter error to a certain extent, and the phasor cluster will tighten closer to the ideal
phasor coordinates, it will still not be a single point. After filtering, it is still essential to
parameterize the features of the phasor clusters in a way that is less subjective to the
remaining scatter error.
Another challenge to quantify the information in the phasor cluster is that most samples
contain more than one single fluorescence species, and multiple phasor clusters will be
presented inside the phasor plot. These phasor clusters can intertwine with each other, as
shown in Figure 1.10(b). When there is a fluorescence species that is our main interest to be
analyzed, as in the case of metabolic NADH lifetime imaging, it is important to separate the
NADH lifetime information from the noise that other species inject into the phasor plots. The
rest of this chapter first presents a workflow to extract NADH information that is commonly
used in previous research. This workflow is simple, unfortunately, it is also inaccurate. Then,
we will introduce a workflow that achieves better accuracy in summarizing the phasor
information in an objective way.
51

3.2 Weighted mode ensures accurate features extraction from phasor clusters
The most typical attribute researchers extract from phasor clusters is the centroid (Wang, et al.,
2021; Stringari, et al., 2012). To calculate the centroid, the mean value of all the G values and S
values from the Region of Interest (ROI) are respectively calculated. It is one of the most
essential properties of the phasor distribution, as it is supposed to represent the location of the
phasor signature when no scatter error is introduced. This is the case if the phasor cluster
follows a perfect 2D Gaussian Distribution. However, the phasor cluster suffers from bias when
other unwanted fluorescence species bleed through the detection channel. This is a common
issue for many NADH metabolic imaging experiments, as many metabolites and cellular
components, such as lipid droplets, collagen, and fluorite, are also autofluorescent, and many
have broad excitation and emission spectra. When measuring the lifetime of NADH for
metabolic imaging, even if the excitation and detection settings are focused on NADH,
fluorescence from the other molecules is likely to be detected. The phasor cluster often looks
like what is depicted in Fig 3.2. The cluster to the right comes from NADH, and the other signal
on the left originates from other fluorescence species. The separation of the cluster is not as
obvious in most cases; instead, it is more likely to be like Figure 1.10(b), where the phasor
clusters are intertwined. Phasor clusters from noise intermingle with the NADH phasor cluster,
and they are not as easily separated as in Fig 3.2. One might attempt to filter the fluorophore
species by thresholding the photon counts in the image. However, for NADH imaging, it is likely
that the unwanted fluorophores have similar photon counts as NADH. Under these
circumstances, the calculated NADH centroid will be skewed toward the other fluorescence
52

species and therefore not represent the metabolic properties, at least not in the way to
quantify with attributes of free and bound NADH.
An alternate attribute to represent the phasor cluster is the cluster’s mode, which is the highest
peak of the phasor cluster. It is represented by the G and S coordinate with the highest
histogram count. One advantage of the mode is that its calculation will not be disrupted by
unwanted fluorescence species as long as they only take up a low percentage of the area of the
field of view from which the phasor plot is reconstructed. While using the mode avoids the
Figure 3.2. Representing phasor information with the centroid of a phasor cluster can be
inaccurate, especially in the presence of fluorescence bleed-through. The cluster centroid
is often used to describe the NADH phasor cluster in FLIM metabolic imaging studies.
However, the NADH imaging settings excite and collect fluorescence from other auto-
fluorescent species like collagen and lipid. When these species contaminate the collected
fluorescence, the calculated centroid is pulled towards these noise components. The
white cross marks the ideal phasor representative for the NADH phasor cluster in this
figure. Because of the presence of the noise components represented in the blue circle,
calculations for the centroid will lead to a phasor coordinate labeled in magenta.
Therefore, using the centroid as a representation for phasor information is not
recommended.
53

contamination of unwanted fluorophores other than NADH, it can be inaccurate in the
presence of scatter error. When zooming in to the top of the phasor cluster with heavy scatter
error, as shown in Figure 3.3, the cluster is spread out and could contain artificial peaks. The
highest peak might be an artifact formed due to statistical chance, and the mode fails to be the
accurate ideal phasor representation of the fluorophore. This is a common problem for NADH
metabolic imaging, as the photon counts for these images are generally very low, and therefore
the phasor cluster contains heavy scatter error.
To accurately describe the ideal phasor coordinates of the phasor cluster, we define a metric
called the weighted mode. The calculation is similar to calculating the mode; however, it is not
defined by one single peak but by a group of coordinates with relatively high histogram counts
Figure 3.3. Representing phasor information with the mode of a phasor cluster is
inaccurate in the presence of scatter error. When photon counts are low, the
calculations of G and S contain a lot of uncertainties, which are shown as scatter errors in
the phasor plot. Scatter error can generate artificial peaks in the phasor cluster, and the
highest peak(mode) might be an artifact. The two figures above show 2 different views
of the same phasor cluster. Because of scatter error, the cluster is not smooth and
contains artificial spikes. This is the case for most NADH datasets, as they are usually
constructed with very low photon counts. In this case, we recommend using the
weighted mode, which is a weighted mean of phasor locations that have the highest
count. The effect of scatter error will be averaged out, and the calculation is not
affected by unwanted fluorescence species as in the centroid calculation.
54

inside the phasor cluster. The weighted mode of the phasor cluster is calculated as the
weighted mean of these coordinates, and the weight of each coordinate is its histogram counts.
To calculate the weighted mode, the G and S values of the phasor cluster are sorted based on
their histogram counts in descending order. Then, The G and S coordinates for the weighted
mode, denoted as G wm and S wm, are calculated with first sorted n phasor coordinates based on
the following equations:
in which G i and S i are the G and S values for the phasor coordinate that had the i
th
highest
count, and N i is the histogram count at that location. As the mean calculation is weighted by
the histogram count, the phasor coordinates with higher histogram values have more
substantial contributions to the calculated weighted mode. The weighted mode can be
considered as combining the benefits of the centroid and the mode of the phasor cluster and
overcoming their disadvantages. By averaging through the highest cluster peaks, the effect of
scatter error is mitigated. Meanwhile, as only the highest peaks in the phasor clusters
participate in the calculation, signals from fluorophores that are unwanted will not be included
in the calculation.
3.3 Converting phasor features to metabolic readings
The binding state of NADH can be interpreted with the weighted modes of the NADH phasor
clusters. Two attributes need to be deciphered for the NADH binding states: the enzyme
𝐺 𝑤𝑚
=
∑ 𝐺 𝑖 ×𝑁 𝑖 𝑛 1
∑ 𝑁 𝑖 𝑛 1
(3.1)

𝑆 𝑤𝑚
=
∑ 𝑆 𝑖 ×𝑁 𝑖 𝑛 1
∑ 𝑁 𝑖 𝑛 1
(3.2)
55

species binding the NADH, and the ratio between bound and free NADH. NADH has distinct
lifetime signatures on the universal circle when bound to different enzymes, as shown in Figure
2.3. If the enzyme is determined, the linear property for phasor analysis explained in Figure 1.
10 can be applied to quantify the bound and free NADH ratio for any given location on the line
between free and bound NADH phasor signatures. The free and bound NADH ratio can be
associated with the sample's glycolysis and oxidative phosphorylation levels.
The line connecting the free and bound NADH phasor signatures is essential for quantifying the
free and bound NADH ratio. The rest of this thesis will refer to this line as the metabolic trend
line. The free-to-bound NADH redox ratio can be calculated by projecting the phasor signature
of the weighted mode to the trendline and then calculating its distance to the free and bound
NADH phasor signature. This process is shown in Fig. 3.4, where the distances from the phasor
cluster parameter to the free and bound NADH phasor cluster are represented as free NADH
contribution and bound NADH contribution.
To better describe subcellular metabolism, we introduce the NADH redox ratio. The redox ratio
is defined as the ratio of reduced NAD and oxidized NAD (Stringari, et al., 2012). The redox
ratio is more suitable to describe subcellular activities than glycolysis and oxidative
phosphorylation. This is because glycolysis mainly occurs inside the cytosol, and oxidative
phosphorylation mostly happens inside the mitochondria. Therefore, using glycolysis and
oxidative phosphorylation terms to describe subcellular metabolism in different organelles can
be confusing. It has been shown by Bird et al. 2005 and by Stringari et al. 2012 that the
56

NADH/NAD+ redox ratio is associated with the ratio of free to bound NADH. This relation is
summarized in equation 3.3.
Redox Ratio =
𝑁𝐴𝐷𝐻 𝑁𝐴𝐷 +
≈
𝐹𝑟𝑒𝑒 𝑁𝐴𝐷𝐻 𝐵𝑜𝑢𝑛𝑑 𝑁𝐴𝐷𝐻 (3.3)
Each NADH-related term represents the concentration of the substance. Higher free NADH
percentages will indicate a higher redox ratio inside the sample, while higher bound NADH
percentages indicate a lower redox ratio.
The calculation of the redox ratio requires the lifetime phasor signatures of purely free NADH
and purely bound NADH to be determined on the phasor plot. Free NADH has a lifetime of
0.4ns, and the phasor location of free NADH is permanently fixed at the lower right corner of
the universal circle. To determine the phasor location for bound NADH, a common practice is
to assume a particular enzyme is binding to NADH. The most frequently used one is pyruvate
dehydrogenase (PDH, bound NADH lifetime: 2.65ns). Unfortunately, assuming all data points,
no matter which pathway, organelle, or cellular compartment the data is collected from, have
the same NADH binding endpoint is inaccurate. It is detrimental for subcellular metabolic
analysis, as most organelles will have a permanent bias for the calculated redox ratio.

57

Another common practice to find the bound NADH phasor signature is to use the free NADH
phasor signature and all data points to generate a linear regression line and use the intersection
of the regression line and the universal circle as the bound NADH phasor signature point (Wang,
et al., 2021). This trendline is objective when only one NADH binding enzyme is involved, yet it
does not separate the different metabolic pathways represented by the different subcellular
compartments. Meanwhile, the balance of the data will heavily influence the slope of the line.
The data balance refers to the ratio between the number of samples between the groups. This
balance can be easily broken when dividing groups by organelles. Unbalanced ratios, in this
case, will favor the one that has more samples and tilt the trendline closer to their distribution.
Figure 3.4. NADH phasor signatures are projected on a metabolic trend line that
connects the free and bound NADH phasor signatures, and the distances from the
projected point to these phasor signatures are used to quantify cellular metabolism. The
redox ratio is approximated by calculating the ratio of free and bound NADH
concentration. To determine the metabolic trend line, the free NADH has a fixed phasor
signature. The bound NADH phasor signature changes depending on the binding
enzyme. In many studies, the bound NADH phasor signature is determined by manually
choosing an NADH binding enzyme or performing linear regression with all data points.
Unfortunately, predefining the bound NADH contribution point is inaccurate because it
fails to accommodate more than one enzyme for metabolic calculations.
58

Moreover, metabolic phasor signatures far from the average have high leverages, meaning
outlier samples and samples with errors could potentially cause significant changes to the
regression line.
The calculation must meet the following requirements to extract valid subcellular metabolic
information. First, the NADH binding enzyme must be considered when the redox ratio is
calculated. Second, the quantification process of each data point is independent of the dataset
compositions. To achieve these two requirements, we propose a workflow that customizes the
metabolic trendline and redox ratio for each phasor cluster separately. The procedures are
described as follows: First, for every phasor cluster to be analyzed, the weighted mode of the
phasor cluster is calculated. Then, a line is drawn from the free NADH phasor signature to cross
each weighted mode and extend until it intersects with the universal circle. Each intersection
will be used as the point for bound NADH for its associated phasor cluster. Further, the
distance of the weighted mode to the free and bound NADH phasor signature is measured, with
which the redox ratio can be calculated. This process is demonstrated in Fig 3. 5. It ensures that
the metabolic calculation is adjusted for the NADH binding enzyme for every data point and is
independent of the number of datasets in the analysis and the sample composition.

59

This
procedure can be extended to plot a metabolic map for all pixels in a region of interest. The
described procedures in Figure 3.5 are applied to every pixel to achieve this. Each position
inside the phasor map will have one calculated redox ratio, as shown in Figure 3.6. A metabolic
map is obtained by pseudo-coloring the original intensity map based on this phasor redox ratio
(Figure 3.6 (d)). This thesis will refer to this color-coded map as the metabolic redox map.
Notice that the color-coded map is generalized for all binding enzymes for NADH. The color
only indicates the redox ratio but does not label the different enzymes that could be involved in
NADH binding.
Figure 3.5. The metabolic trendline for NADH FLIM imaging must be customized for
every data point. After fixing the free NADH phasor signature, the bound NADH
phasor signature for each data point is determined by extending lines from the free
NADH phasor signature to each data point until they intersect with the universal
circle. Each intersection is the bound NADH phasor signature for that data point.
Two different metabolic trendlines were determined for the two data points labeled
in yellow and green, and the redox ratio for each data point was calculated on their
respective trendline. This process ensures that the metabolic calculation is adjusted
for the NADH binding enzyme for every data point.
60

Now that an objective strategy to extract metabolic information from the phasor clusters is
established, the next step is to make sure that the phasor cluster does indeed contain the
information of interest. The major obstacle to accurately translating the lifetime information is
scatter error. NADH Metabolic images are often constructed with low photon counts and
contain substantial scatter error. The scatter error prevents us from identifying accurate
metabolic information. The following two chapters will discuss two filtering techniques that
Figure 3.6. The metabolic map for autofluorescence NADH FD-FLIM indicates the redox
ratio at every image location. (a) displays the original phasor plot, (b) shows the photon
intensity map. (c) is generated by performing the process illustrated in Figure 3.5 to every
location in the phasor plot and assigning the color based on the calculated redox ratio.
For each pixel in (b), its phasor signature is calculated and placed in figure (c) to obtain
the color it will be assigned to for generating (d). (d) is the color-coded metabolic map. It
shows a direct visualization of the distribution of the redox ratio. Notice that the color
only indicates the redox ratio but does not resolve the different enzymes that could be
involved in NADH binding. This process uses the pixel-wise correlation from the original
intensity image to the phasor plot and the correlation of each location of the phasor plot
to a certain redox ratio.
61

alleviate the noise issue and help us extract valid metabolic information from phasor plots,
even when the photon counts are low.

62

Chapter 4. Correlated Non-Local Means for FLIM Phasor Analysis
4.1 Median filtering for FLIM phasor analysis degrades lifetime details in subcellular
structures
In the previous chapters, we have shown how FLIM and phasor analysis can be powerful tools
for NADH metabolic imaging. However, NADH images are often constructed with very low
photon counts. When the photon counts are low, FLIM suffers from counting error, which is
also known as Poisson noise. (Cutrale, et al., 2017; Raghav & Raheja, 2014) Phasor analysis is
an optimized tool for frequency domain FLIM; however, it is still subject to Poisson noise when
limited photons are collected. The Poisson noise appears in phasor plots as scatter error, which
blurs the actual position of a signal by forming a cloud, as shown in Fig 1.11. (Cutrale, et al.,
2017; Wang, et al., 2021)
Filtering techniques designed for phasor analysis can decrease the effect of the scatter error.
Enrico Gratton’s group at the University of California, Irvine, introduced a procedure to apply
median filters in FLIM phasor analysis, as shown in Fig 4. 1. The median filter is one of the most
commonly used filters in image processing and is very effective in reducing salt-and-pepper
noise. The adopted filter for phasor analysis is currently the most frequently used filtering
procedure for eliminating scatter error (Stringari, et al., 2012; Datta, Alfonso-García, Cino, &
Gratton, 2015). Within median filters, every pixel’s value is ranked with the pixel values from its
local neighborhood and is replaced by the median value for these pixels. The filter is applied
separately to the G and S phasor maps when using median filtering in phasor analysis. To get a
better reduction of scatter error, median filtering is often applied repeatedly several times.
63

After filtering, the phasor plot is generated by summarizing the two filtered phasor maps with a
2D histogram for G and S. Median filtering is very effective for phasor analysis, and the
tightening of the phasor cluster is substantial. Besides its effectiveness, the median filter is fast
and easy to implement and has been used by many FD-FLIM users. Unfortunately, the
reduction of scatter error that median filters achieved comes with a price of degrading details.
Figure 4.1. Median filters frequently used to reduce scatter error for FLIM phasor
analysis degrade lifetime information in subcellular structures. Scatter error is a
significant obstacle to extracting accurate lifetime readings in the phasor plots,
especially when photon counts are low. Median filters are adapted into the phasor
analysis process to help reduce scatter error, which is illustrated in this figure. In phasor
analysis, the unfiltered G and S maps are calculated from the raw FLIM data set and are
used to generate the phasor plot. For median filters in phasor analysis, median filters
are applied on the G and S maps separately. G and S readings that were outliers will be
replaced by the median pixel value of their surroundings, and the reconstructed phasor
cluster in the phasor plot will be less spread out. Although this strategy reduces the
scatter noise to a certain extent, which is shown as the tightening of the cloud on the
phasor plot, median filters degrade the high spatial frequency components inside the
phasor maps. Structures like edges and puncta will be assigned G and S values similar to
their surroundings. Therefore, median filters are not ideal when lifetime information for
detailed structures is to be analyzed.
64

Median filters, not only for phasor analysis but in other applications as well, are known to have
a smoothing effect on edges. In the process of replacing each pixel with the median of its
surrounding, edges and puncta are likely to be replaced by their surrounding pixels. High spatial
frequency details inside the images will likely be degraded (Shrivastava & Gupta, 2011).
Therefore, in FLIM phasor analysis, median filtering is helpful to get a general estimate of the
phasor distributions of the entire field of view. However, it is inadequate to extract lifetime
information of delicate structures inside the image. When summarizing metabolic information
at a subcellular level, median filters are not suitable anymore. It is essential to develop filters
for phasor analysis to retain the lifetime information in structural details while effectively
reducing scatter error. The following two chapters will demonstrate two filtering methods to
overcome this hurdle and extract valid lifetime information inside high spatial frequency
components for FLIM datasets.
4.2 Correlated non-local means supplements structural information from the photon
intensity map to lifetime phasor maps
The photon intensity map is less subjective to the low photon counts than the phasor maps, as
demonstrated in Figure 4.2. Figure 4.2(a) and (c) are photon intensity maps, while Figure 4.2
(b) and (d) are the corresponding phasor plots. Note that figures (a) and (b) are constructed
with photons just one-tenth compared to figures (c) and (d). The phasor plot shown in (b) has a
much larger spread than figure (a). However, while the phasor plots are very different, the
65

normalized photon intensity maps are highly similar. The intensity maps are much more robust
than the phasor maps when the photon counts are low. This is expected. In both TD-FLIM and
FD-FLIM acquisition, the collected photons from one single pixel are divided into multiple
lifetime bins. This reduces the number of photons per bin and increases the counting error. The
counting error is inherited by further lifetime calculations, including phasor analysis. However,
for the photon intensity map, photons are not split into bins; therefore, the intensity maps are
(d)
(h)
(g)
(c)

(a) 1 Frame Accumulation
(c) 10 Frame Accumulation
(b) Phasor Plot for (a)
(d) Phasor Plot for (c)

Figure 4.2. Comparing photon intensity map and their respective lifetime phasor plots
under different photon counts show that the photon intensity map is more robust to
low photon counts than the phasor plot. FLIM images were taken on the
autofluorescence of convallaria slides. (a, c): Normalized photon intensity images of
the same field of view. (c) had ten times photon count accumulation compared to
image (a). (b, d): The respective phasor plots of (a) and (c). The high degradation to
photon counts for the phasor plot is due to dividing the limited photons into lifetime
bins, which increases the level of Poisson noise in each bin. Since a more stable
structure is observed in the photon intensity map, the photon intensity map can be
used to supplement structural information in the phasor calculations. Scale Bar for (a)
and (c): 10um
66

more likely to hold more structural details than the phasor maps when the photon counts are
low.
The filtering strategy presented in this chapter reduces Poisson noise inside the phasor maps by
using the photon intensity map to supplement structural information. To extract structural
details from the photon intensity map, a well-established filtering strategy called non-local
means is modified to apply to phasor analysis (Buades, Coll, & Morel, 2011; Boulanger, et al.,
2009). In non-local means, the filtering window extends beyond the neighboring pixels,
potentially to the entire field of view (FOV). The structural similarity of one pixel to other pixels
in the filtering window is calculated. Pixel pairs with similar structures are assigned a higher
weight for a weighted mean calculation. The net effect of this weight assigning scheme is that
after the weighted mean calculation, pixels with similar structures in the filtering window are
averaged together and share similar values. For our filtering strategy for phasor analysis, the
weights between pixel pairs for non-local means are calculated based on structural similarity on
the photon intensity map. They are then transferred to the corresponding pixel in the G and S
phasor maps for phasor filtering. Since the structural details between the photon intensity map
and the phasor maps are correlated with non-local means, we name this filtering method
Correlated Non-Local Means (CNLM). The process for CNLM is illustrated in Figure 4.3. The
next session of this chapter demonstrates the filtering results of CNLM on multi-label
fluorescence and autofluorescence images of FLIM. The results show that CNLM leads to better
noise reduction and increased Signal to Noise Ratio (SNR) with minimal impact on spatial
resolution compared to median filters, especially when the available photon counts are low.
67

Figure 4.3. Correlated Non-local Means (CNLM) for supplements structural
information to FLIM phasor maps. To extract structural details from the photon
intensity map, a well-established filtering strategy called non-local means is
modified to apply to phasor analysis. In non-local means, the filtering window
extends beyond the neighboring pixels, potentially to the entire field of view. The
structural similarity of one pixel to other pixels in the filtering window is
calculated. Pixel pairs with similar structures are assigned a higher weight for a
weighted mean calculation. The net effect of this weight assigning scheme is that
after the weighted mean calculation, pixels with similar structures in the filtering
window are averaged together and share the same value. Weights for every pixel
with respect to other pixels in the image are calculated based on the pixels’
neighborhood similarity on the intensity map. The higher the similarity between
pixels’ neighborhoods, the larger the weight assigned to the pairs. The
magnitudes of the weights are indicated as the thickness of the arrows in the
bottom left image. Weights are further transferred to each phasor map
separately to calculate the weighted means on each phasor component which
finally generates the filtered phasor plot. With CNLM, the filtered phasor plot
preserves structural details more accurately with much less scatter error.
68

4.3 Calculations in correlated non-local means
In CNLM, the G and S phasor maps are calculated first with equations (4.1) and (4.2).
𝐾 𝑖 represents the filtering window for pixel 𝑖 in which pixel-wise weight calculation for non-
local means is performed, and it can extend to the full image if needed. In CNLM, the filtered G
and S values of pixel 𝑖 , labeled as Ĝ(𝑖 ) and Ŝ(𝑖 ) , is calculated with the weighted average of
every pixel in the search window 𝐾 𝑖 based on the following equations:
Ĝ(i) = ∑
1
Z(i)
e
−
‖v(N
i
)−v(N
j
)‖
2
h
2
G(j)
jЄK
i
(4.1)
Ŝ(i) = ∑
1
Z(i)
e
−
‖v(N
i
)−v(N
j
)‖
2
h
2
S(j)
jЄK
i
(4.2)
for each pixel j, 𝑁 𝑗 denotes a square neighborhood centered on j. 𝑣 (𝑁 𝑗 ) to represent the
image neighborhood vector in the photon intensity image. 𝑍 (𝑖 ) = ∑ 𝑒 −
‖𝑣 (𝑁 𝑖 )−𝑣 (𝑁 𝑗 )‖
2
ℎ
2
𝑗 Є𝐾 𝑖 is a
normalizing term and parameter h controls the extent of averaging. Notice that Ĝ and Ŝ are
weighted averages of the original G and S values, while the weights are calculated based on the
photon intensity map.

69

4.4 Correlated non-local means preserves lifetime information in structural details
Fluorescence lifetime can be used as a contrast to discriminate fluorophores based on their
respective positions on the FLIM phasor plots (Maddipatla & Tankam, 2020). However, in the
presence of scatter error, the phasor clusters of the fluorophores expand and intermix, and it
becomes difficult to separate the phasor clusters apart. As mentioned above, although median
filters reduce scatter error, this reduction comes with the price of degrading structural details.
If one attempts to separate fluorophores based on median filtered phasor plots, the separation
results are likely to be unnatural because of the loss of high spatial frequency details. On the
other hand, CNLM reduces the scatter error while preserving fine details in the phasor maps,
leading to a more realistic reconstruction for phasor-based separation.
When examining the phasor cluster after filtering, CNLM provides more reduction of scattering
error compared to median filtering in the phasor plots. To prove this, FLIM data were collected
on fixed H9C2 cardiomyocytes stained for mitochondria (Cox4-Alexa 594) and F-actin
(Phalloidin-iFluor 488 Reagent). The amount of scatter error reduced can be determined by
measuring the full-width half max of the phasor clusters on the phasor plots and the magnitude
of the standard deviation of the calculated G and S values for single fluorescence species. The
70

results are shown in Fig 4.4 and Table 4.1. In Fig 4.4, Cox4-Alexa 594 and iFluor 488 were each
collected with separate detectors, and the unfiltered, median filtered and CNLM filtered phasor
plots for each stain are displayed. A direct comparison of the size of the phasor cluster shows
that CNLM has a better reduction of scatter error compared to median filters. This is
quantitatively shown with the standard deviation for G and S values for pixels that contain
structural information in Table 4.1. In median filters, the G and S values on the structural
edges are blended with their neighbors during the filtering process. Neighboring pixels that
contain significant noise levels also contribute to the filtered phasor clusters, and the reduction
of scatter error is not complete. On the other hand, in CNLM, non-structural pixels will
contribute very little to pixels of structures because weights are based on structural similarity.
Therefore, the noise reduction of scatter error is very effective.

Table 4.1. The standard deviation of a single fluorescence dye with different
filtering show CNLM has improved performance than median filters. The calculation
was done on stained mitochondria and F-actin in H9C2 cardiomyocytes
independently.
Fluorescent labels: Mitochondria: Alexa 594. F-actin: Alexa 488
71

To measure how well CNLM and median filters discriminate different fluorophores, phasor
analysis, the two fluorophores were collected together in one wide band detector and
separated based on their phasor information. The results are shown in Figure 4.5. Comparing
the median filtered and the CNLM filtered phasor plots in Figures 4.5 (b) and (c), it is easy to see
that the scatter error is reduced substantially more with CNLM filtering, and the clusters of the
fluorophores are more separated. Further, the noise reduction can be quantified by drawing a
line profile of the phasor plot across the center of the Alexa 488 phasor signature (Fig. 4.5i).
After fitting the line profiles to a Gaussian curve, it can be seen that CNLM results in a higher
peak value of about 300% and reduces the full-width half max (FWHM) almost to half (55%).
Fig 4.4. CNLM filtering for fluorescent antibody staining on cell lines shows better
convergence of phasor clusters compared to median filters. FLIM phasor images of
stained mitochondria and F-actin in H9C2 cardiomyocytes are processed with different
filters. Each stain was detected using a separate filter. The phasor signature of each
fluorophore ideally has only one G value and one S value, and the phasor signature will
be one spot in the phasor plot. However, the unfiltered phasor cluster is spread out
because of scatter error. Median filters and CNLM reduce the scatter error level in the
phasor plot. A visual inspection of the size of the phasor cluster shows that CNLM has a
better reduction of scatter error than median filters.
72

While reducing the scatter error inside the phasor plots, CNLM preserves much better details
compared to the standard filtering technique. To analyze the structural details of filtering,
each pixel of the photon intensity map is color-coded based on its respective location in the
phasor plot to correlate the structures with their lifetime represented in the phasor plot. The
results are shown in Figure 4.5 (d-f). The color coding is based on the phasor color shown in
each inset. The results are compared to the separation ground truth in Figure. 4.5(h), which is
generated by collecting the two fluorophores with separate detectors. In absence of denoising,
the detailed structures or the organelles are heavily distorted (Figure. 4.5d). A median filter
slightly improves the image features; however, edge-degradation is evident, leading to
unnatural organelle patterns in Figure 4.5(e). CNLM filtering results shown in Figure 4.5 (f)
extracted much finer details and more realistic spatial structures for the organelles, which is
further shown by the structural similarity with respect to the ground truth.

73

Fig 4.5. Comparisons of lifetime (FLIM) fluorescence images with different filters show
that CNLM has better noise reduction and detail preservation than median filters. FLIM
phasor images were taken of labeled mitochondria with Alexa-594 and labeled F-actin
with Alexa-488 in H9C2 cardiomyocytes. (a-c) FLIM phasor plots filtered with CNLM show
better scatter noise reduction than median filters. Phasor plots shown are: (a) unfiltered,
(b) Median filtered, and (c) CNLM filtered phasor plots. (d-f) Color coding based on the
phasor plot shows that CNLM preserves fine structural details with the phasor
information when median filters corrode them. Color coding was based on: (d)
unfiltered, (e) median filtered, and (f) CNLM filtered phasor plots. (g)Photon intensity
image independent of lifetime. (h) Ground truth images were taken by separated
detectors recognized by wavelength for F- actin and mitochondria. F-Actin: Magenta,
Alexa 488. Mitochondria: Cyan, Alexa 594. (i) Line profiles over the stained F-actin phasor
cluster with different filtering methods show greater scatter noise reduction with CNLM
compared to median filters. Inset marks the location of the line profiling location. The
location is identical for all phasor plots. Scale Bar: 10um
74

4.5 Correlated non-local means accurately reconstructs cellular metabolic signatures for
NADH FLIM imaging
One of the most widely used applications for FLIM is NADH Metabolic Imaging (Ma, Digman,
Malacrida, & Gratton, 2016; Stringari, et al., 2012; Wang, et al., 2021). Filtering for NADH
metabolic imaging is essential, as NADH has low photon yields that result in scatter error in
their phasor analysis. Meanwhile, extra care has to be taken in metabolic imaging since
overexposure to the excitation laser could induce phototoxicity, which changes the metabolism
and leads to inaccurate metabolic analysis. Therefore, it is essential to extract valid metabolic
information with low excitation power. In other words, ideally, metabolic information is
extracted with as few photons as possible. Unfortunately, median filters cannot accurately
reconstruct the signal with high spatial details, especially when the photon count is low. It will
be demonstrated that CNLM preserves the details in phasor analysis with much better
structural accuracy than median filters.
To show how CNLM accurately reconstructs the desired FLIM phasor distributions when median
filters cannot, the NADH signal was imaged in HEK 293 cells, and the unfiltered, median filtered,
and CNLM filtered phasor plots are compared in Figure 4.6. The NADH signatures inside and
outside the nucleus differ because of their binding ratios to enzymes. Inside the nucleus, there
is little enzyme NADH can bind to, and their NADH concentration is mostly unbound. On the
other hand, NADH in other regions will have a higher binding ratio with enzymes. The free
NADH inside the nucleus and the bound NADH outside the nucleus shows two different FLIM
signatures in the phasor plot. CNLM accurately depicts the two different phasor signatures
75

when unfiltered and median filtered phasor plots cannot. When no filtering is performed, the
phasor plot is dominated by noise, and the actual signals are buried in it, as shown in Figure 4.6
(a). Figure 4.6 (b) shows that median filters reduce the scatter error. However, the two
separate clusters for the nucleus and the cytoplasm are not evident. CNLM filtering restored
the two distinct phasor clusters, as shown in Figure 4.6 (c). The effects of the filters can be
examined by adding a line profiling on the phasor plot, shown in Figure 4.6 (h). Comparing the
line profiles of the three filtering strategies, only the CNLM filtering depicts the two separate
clusters of the nucleus and the cytoplasm.
We further verify how the structural details inside the phasor maps are preserved with the
different filters. The photon intensity map in Figure 4.6(d-f) is color-coded based on the phasor
plot depicted in each inset. The unfiltered raw phasor plot (Figure 4.6(d)) shows substantial
scattered noise. With median filters, artificial clusters were formed, and the nucleus
membrane was not successfully reconstructed in Figure. 4.6 (e). In contrast, CNLM was able to
show structural details such as the smooth edges of the nucleus membrane, as shown in Figure
4.6(f).

76

Figure 4.6. CNLM filtering shows better scatter error reduction and more accurate
preservation of structural details than median filters for FLIM metabolic imaging. NADH
autofluorescence imaging is almost always associated with low photon counts. Filtering
techniques are essential to extract valid metabolic information from FLIM datasets. FLIM
data of auto-fluorescence NADH was taken with HEK cells. (a-c) Phasor plot of NADH
autofluorescence with different filtering show only CNLM successfully separates NADH
states inside the cell. The phasor clusters in CNLM are natural results of the two NADH
populations within and outside the nucleus. (d-f) Color coding of NADH autofluorescence
based on phasor plot show only CNLM preserves the structural details with phasor
information and reduces artifacts, while median filters result in clustered artifacts.
(g)Photon intensity image independent of lifetime. (h) Line profiles over the phasor
clusters. In (h), the line profiles over the phasor clusters with different filtering methods
show that only CNLM generated 2 NADH phasor clusters originating from the cytoplasm and
nucleus. The inset marks the line profiling location; the location is identical for all phasor
plots. Scale Bar: 10um
77

4.6 Advantages and potential pitfalls of correlated non-local means
The proposed CNLM filtering has substantial improvements in reducing the scatter error while
preserving fine structural details that median filters cannot. There are two main reasons for the
outstanding performance of CNLM compared to median filters. First, non-local means is a
filtering strategy that enhances structural details, while median filtering is designed to remove
sudden structural changes. Median filters blur the lifetime information in high spatial
frequency structures, such as puncta and edges, with the neighboring pixels. In non-local
means, pixel values with similar neighborhood textures and patterns inside the filtering window
support each other. This preserves detailed structures that repetitively appear inside the
image. Another reason for the superior performance is that CNLM uses the photon intensity
map to supplement information for filtering in the phasor analysis. The photon intensity maps
are much less noisy than the phasor maps (G and S maps), especially when the photon counts
are low. Since the photon intensity map and the phasor maps are correlated, the structural
details between the two maps are transferable. It is possible to further improve the
performance of CNLM by selecting the temporal bins in FLIM specific for the filtering target
when integrating them into the photon intensity map.
CNLM prefers scenarios where the structures that are associated with fluorescence have
distinct shapes from each other. One assumption for CNLM is that similar structures or
surroundings in the sample have similar phasor signatures. Most organelles have different
shapes observable inside the intensity maps. For organelles that have similar structures, their
surroundings generally vary (Ker, Wang, Rao, & Lim, 2017). The non-local means strategy
78

examines the differences among these neighborhoods. It even compares structures and
surroundings among the same organelles and accounts for the heterogeneity that is not
observable by the human eye. However, for structures that are similar, the filtering window in
CNLM need to be increased for a higher probability of finding subtle differences between these
structures.
Compared to median filtering, CNLM is more computationally intensive, especially when the
search window is large. On one 512 × 512 FLIM image, CNLM took almost 13 times longer than
the median filter to complete on the same computer (44.1s compared to 2.99s), with a 9x9
filtering window for the CNLM and a 5x5 filtering window for median filters. However, the
heavy computation time could be remedied by clever programming. This computational time
could be reduced by implementing parallel processing or using PCA to reduce the
dimensionality when the similarity between pixels is calculated.
CNLM opens the possibility for many applications when photobleaching or phototoxicity is an
issue, as it provides better filtering performance than traditional filters with fewer photons
required (Rodrigues, Sanches, & Bioucas-Dias, 2008). As CNLM reduces the photons needed to
acquire meaningful data, imaging with lower laser power or faster imaging speed is allowed.
The advantages of using CNLM will be critical for experiments for live-cell imaging like NADH
metabolic imaging with FLIM or long-term imaging for studying organ development and
regenerations. Meanwhile, these same principles of noise filtering in CNLM might be adapted
to help noise reduction in correlative microscopy, where potentially CNLM can filter correlated
images by extracting similarity matrix with non-local means on the higher Signal-To-Noise(SNR)
79

images and strengthen the structures in the low SNR images (Caplan, Niethammer, Taylor, &
Czymmek, 2011).

80

Chapter 5. Complex Wavelet Filtering for FLIM Phasor Analysis
5.1 Localized spatial frequency filtering with complex wavelet filtering
The previous chapter introduced Correlated Non-Local Means (CNLM) to filter the scatter noise
associated with phasor analysis. CNLM supplements structural details from the photon
intensity map to the lifetime phasor maps with a non-local mean strategy. Despite the
outstanding performance to preserve lifetime information in structural details, there are
several downsides to CNLM. First, CNLM is slow. This can be troublesome when computation
speed is important or when many datasets await to be processed. Second, CNLM could
average the phasor values of similar structures leading to inaccurate phasor readings if the
filtering parameter is not carefully selected.
In this chapter, we introduce a different filtering strategy for FLIM phasor analysis based on
wavelet analysis, which is fast in calculation and robust against structural similarities of filtering
targets. Phasor information is decomposed into different frequency components with wavelet
filters, and the noise is eliminated within each frequency band. The principles of wavelet
analysis are similar to the Fourier transform. Fourier transform decomposes a one-dimensional
signal into oscillating waves of different frequency components with varying phase shifts. The
same process is transferable to images, where a 2D Fourier transform decomposes the image
into oscillating waves with different frequencies, directionality, and phases. However, the 2D
Fourier transform only reveals the distribution of the spatial frequencies but not the spatial
location of the different spatial frequencies. The spatial frequency components with different
locations in the image are not shown. This is not advantageous for biological datasets, as
81

different regions are supposed to contain different frequency components, and the filtering of
the frequency components has to be region dependent. Therefore, the 2D Fourier transform
will not be helpful for our cause.
Wavelet analysis enables pinpointing the frequency composition with a spatial resolution
(Luisier, Vonesch, Blu, & Unser, 2009; Chen, Xie, & Zhao, 2013). The Discrete Wavelet
Transform (DWT) decomposes the image into locally oscillating waves called wavelets.
Wavelets are associated with different frequencies, and when convolving the locally oscillating
wavelets to the signal to be analyzed, wavelet coefficients at different locations of the signal
are extracted. The wavelet coefficient indicates how much of the corresponding wavelet is
located at that particular location. Figure 5.1 demonstrates how wavelet transforms allow
analyzing image details with spatial frequency information and locational information
simultaneously. The wavelet transform is applied in different depth levels, the number of
frequencies to be analyzed, and each level extracts features of a particular frequency
component. The lowest level extracts the highest frequency component and leaves an
approximation image that contains the lower frequency components of the original image, half
the size in length compared to the approximation of the previous image. The process is further
repeated for the next level. For a 2D image, the frequency components are divided into
different directions; the most frequently seen ones are the vertical, horizontal, and diagonal
compartments. The magnitude of the spatial frequency at every pixel is represented as the
wavelet coefficient, which is displayed in the figures to the right. After the wavelet transform
extracts image features to wavelet coefficients, the wavelet coefficients that are small in value
are typically noise and can be removed to enhance image quality. In this way, wavelet
82

denoising suppresses noise that does not belong to that frequency band. Advanced wavelet
transforms exist that decompose images with even finer details, like the dual-tree wavelet
transform. The dual-tree complex wavelet transform is an extension of the Discrete Wavelet
Transform. With the dual-tree complex wavelet transform, two separate DWT are calculated,
one for the real part and one for the imaginary parts. The complex value inside its calculation
allows for additional directionality for the wavelet filters which are shown in Figure 5.2.
Figure 5.1. Wavelet transform decomposes images into smaller images with different
frequency components and different directionalities. Wavelet transform uses
localized oscillating waves called wavelets as the base function for decomposition and
yields wavelet coefficients that represent the amount of decomposition at particular
spatial frequencies at specific locations. The original image in the figure above is
decomposed into two levels. The level 1 images contain high-frequency components,
and the level 2 images contain lower-frequency components. The 2D wavelet
transform has directionalities (the horizontal, vertical, and diagonal in the figure
above) based on the directionalities of the wavelets. After the decomposition of each
level, an approximation image remains with all the lower frequency components and
approximates the original image with half the size in length. The wavelet coefficients
that are small in value are typically noise and are removed to enhance the image
quality. This is the basis for wavelet denoising.
83

The filtering strategy that is presented in this chapter combines a dual-tree complex wavelet
filter with an Anscombe transform, which is referred to as Complex Wavelet Filter(CWF) in this
thesis. (Kingsbury, A dual-tree complex wavelet transform with improved orthogonality and
symmetry properties., 2000). The dual-tree complex wavelet filtering works ideal for data sets
that have Gaussian distributed noise values. However, the scatter noise results from Poisson
noise, which has a Poisson distribution. To ensure the best performance for the dual-tree
complex wavelet filter, the Anscombe transform is introduced into the analysis. The Anscombe
transform is a variance-stabilizing transformation to transform a random variable with a
Poisson distribution into one with an approximately standard Gaussian distribution. In the
filtering process, it is applied to the phasor maps before the dual-tree wavelet transforms. The
process for CWF is depicted in Fig 5.3. CWF minimizes the effects of Poisson noise and adopts
the strength of complex wavelet filtering in high dynamic range images. We will show that CWF
Figure 5.2. Dual-tree wavelet transform has mother wavelets with more directionalities
than discrete wavelet transforms. With the dual-tree complex wavelet transform, two
separate DWT are calculated, one for the real part and one for the imaginary parts.
This allows for better preservation of structural details. Dual-tree wavelet transform is
the basis of the Complex Wavelet Transform (CWF) for phasor analysis and ensures the
preservation of fine details in subcellular structures. Image Origin: Kingsbury, 2000.
84

drastically outperforms median filters in low photon images. CWF will be especially useful in
situations where available photons are limited, for example in metabolic imaging.
Figure 5.3. Workflow for Complex Wavelet Filtering for FLIM phasor analysis. Raw data
for FLIM is transformed into real (G) and imaginary (S) phasor maps with Fourier
transform. G and S are further normalized with the photon intensity map and plotted
against each other, to generate the phasor plot. In Complex Wavelet Filter (CWF), the
Dual-Tree Complex Wavelet Transform (DTCWT) optimally applies to data whose noise
distribution follows a Gaussian distribution. Since the distribution of the phasor map is
Poisson, the Anscombe transform is applied to the intensity map and the unnormalized
G and S phasor maps respectively to convert their noise distribution profiles to
Gaussian. After the Anscombe transform, the dual-tree complex wavelet transform is
applied on the G and the S map. Afterwards, the Anscombe transformed intensity map
is used to normalize the G and S maps. Further, we use the inverse Anscombe transform
to reconstruct the G and S maps, which are further plotted against each other to form
the filter phasor plot.
85

5.2 Calculations for the complex wavelet filtering in phasor analysis
The process for CWF is described in Figure 5.3. The phasor maps G and S are first calculated
with Equations (5.1) and (5.2), and the photon intensity map with 5.3:
𝐺 𝑖 _𝑜𝑟𝑔 (𝜔 ) = ∑ 𝐼 (𝑡 )cos(𝑛𝜔𝑡 )𝛥𝑡
𝑡 𝑠 𝑡 𝑓 (5.1)
𝑆 𝑖 _𝑜𝑟𝑔 (𝜔 ) = ∑ 𝐼 (𝑡 )sin(𝑛𝜔𝑡 )𝛥𝑡
𝑡 𝑠 𝑡 𝑓 (5.2)
𝐼 𝑖 = ∑ 𝐼 (𝑡 )𝛥𝑡
𝑡 𝑠 𝑡 𝑓 (5.3)
Notice that the G and S maps are not normalized. Once the three maps are calculated, the
Anscombe transform is applied on each image to convert the Poisson noise to Gaussian with
Equation 5.4:
𝐴 ′ = 2√𝐴 +
3
8
(5.4)
Equation 5.4 is calculated for both the two phasor maps G, S, and the photon intensity map.
Afterward, a dual-tree complex wavelet transform is applied to the Anscombe transformed G
and S maps. The dual-tree complex wavelet transform is described by Kingsbury et al 2001.
After the images are filtered by the wavelet transform, the images are normalized with:
𝐺 𝑖 (𝜔 ) =
𝐺 𝑓𝑖𝑙𝑡𝑒𝑟𝑒𝑑 𝐼 𝑓𝑖𝑙𝑡𝑒𝑟𝑒𝑑 (5.5)
𝑆 𝑖 (𝜔 ) =
𝑆 𝑓𝑖𝑙𝑡𝑒𝑟𝑒𝑑 𝐼 𝑓𝑖𝑙𝑡𝑒𝑟𝑒𝑑 (5.5)
86

Finally, the filtered G and S maps are transformed back with an inverse Anscombe transform
and can then be plotted against each other for a 2D histogram to generate the phasor plot.
5.3 Complex wavelet filtering accurately reconstructs structural details
The performance of CWF is analyzed with stained cell lines, the simplest scenario for FLIM
imaging. Fixed Cos-7 cells are imaged with antibody-labeled mitochondria and created in two
separate datasets, one ideal dataset with a vast number of photons as the ground truth and
one simulating the real-life scenario with a limited photon budget. The ground truth dataset
was created by 50 integrated frames over the same field of view, and a single frame image was
segmented out of the 50-frame dataset as our real-life sample. This 1-frame dataset is more
similar to what is acquired in a real dataset scenario. The 1-frame dataset is processed with the
complex wavelet filter and other filtering methods in hopes of extracting similar information
from the ideal dataset.
The phasor plot for the 50-frame dataset, the unfiltered 1-frame dataset, and the complex
wavelet filtered dataset are shown in Figures 5.4(c), (f), and (i), respectively. Comparing Figures
5.4 (c) and (f), it can be seen that under low photon counts, the phasor plot contains heavy
scatter error. Figures 5.2 (c) and (i) show that CWF was able to correct the noise and produce a
phasor distribution similar in size, shape, and most importantly, in G and S position to the
ground truth.
Pixel grouping of the structural image is performed based on their phasor plot coordinates to
demonstrate the validity of the CWF. Pixels above 1/3 of the maximum histogram count in
each phasor plot are highlighted as an unbiased method of phasor selection. The resulting
87

selections are shown in the insets of Figure 5.4(c), (f), and (i) in red, and the selected pixels are
shown in Figures 5.4 (a), (d), and (g) in red as well. The ideal scenario for filtering is to select
the same pixels with a filtered 1-frame dataset as the ground truth 50-frame dataset. For the
unfiltered 1-frame dataset, although a large area of the phasor plot was selected, the selection
region in Figure 5.4 (d) is scattered and very limited, with much structural information missing.
However, when the complex wavelet filter was applied, the selected region became much more
similar to the 50-frame dataset, with more structure covered.
To visualize the effect of CWF along with other filtering strategies on structural details, the G-
coordinate maps of the different filtering methods are shown in Figure 5.5. The unfiltered one-
frame data set contained a substantial amount of noise, and the G values in some regions
deviated significantly compared with the 50-frame ground-truth dataset. Median and real
wavelet filters (applying only the DWT on the G phasor map) were able to reconstruct the
structures to a limited extent and generated artifacts. In contrast, CWF recovered many details
with the limited amount of information given by the limited photon image. The CWF-filtered
image has the highest similarity to the ground truth dataset, both within and outside the
mitochondria.
88

Figure. 5.4 Complex Wavelet Filter (CWF) enables low photon count FLIM datasets to
generate similar phasor plots and information extraction similar to the high photon
count datasets. The grey-scale, intensity images of Tom20 stained mitochondria (a, b)
are overlaid with FLIM phasor-selected pixels (red overlay). The unfiltered phasor was
calculated from the lifetimes integrated from 50-frames (c) with the selected lifetimes
in the phasor (c inset) that map back to the images in a and b and serves as the ground
truth. A single frame was extracted from our ground truth dataset for (d-i). The single
frame with the unfiltered phasor (d-f) demonstrates how noisy the phasor is in the
photon starved condition and the phasor selection (f inset) misses most of the Tom20
pixels (highlighted red in d and e). However, CWF application to the single-photon
starved image (g-i) shows successful noise reduction and the distribution of the ground
truth can be recapitulated (i). Phasor-based pixel selection (I inset) shows CWF
appropriately maps Tom20 pixels to the phasor from the previously diffuse
distribution.

89

Figure 5.5. G-coordinate error maps show that CWF achieves better structural
reconstruction of FLIM phasor data than other filtering methods. FLIM G-coordinate
maps (a-e) are arranged in left-to-right order of increasing error. (f-j) highlight the
detail of those errors. The zoomed-in views (k-o) are generated from f through j,
respectively. Error maps (f-o) are generated by plotting the absolute difference of
the filtered G coordinate maps with respect to the accumulated 50-frame ground
truth. The color-scale bars show the colorimetric values of the G coordinate and
Mean Square Error (MSE). The ground truth dataset (a) has zero error (f and k) by
definition as this G-coordinate map is compared to itself. Mitochondrial networks
were drawn over the completely black error maps (f and k) in cyan. Single-frame CWF
was able to recover the most information, without introducing artifacts (b) and has
the least error (g and l) of the other filtering methods (b-d). The real wavelet filter
seems to have diminished detail in the G-coordinate map (c) but surprisingly has a
similar error calculation (h and m) to the median filter (d, i, and n). The unfiltered
single-frame G-coordinate map is extremely noisy, with heavy deviation from the
actual G values in some regions (e) and, not surprisingly, has the most error (j and o).
90

The error maps of each filtering method are displayed by calculating the absolute difference of
each pixel between the filtered image and the ground truth in Figure. 5.5. Brighter colors in the
error maps indicate higher error values, and an image with no error would be completely black,
as depicted in the error maps for the 50-frame dataset. Median filtering and the real wavelet
filter have much higher errors compared to CWF, indicated by the overall brighter color in the
error maps. Mitochondria in the median filtered image are surrounded by a yellow shell in the
error maps. This shell indicates the erosion of the high spatial frequency edges. The real
wavelet filter, on the other hand, creates artificial structural patterns, as indicated in the G
coordinate maps inside regions of mitochondria. These effects have not been seen under CWF.
CWF did not generate any structural artifacts within the image. Moreover, the error compared
to the ground truth is much lower than other filtering methods, both inside and the outer
mitochondrial regions.
To quantitatively analyze the performance of complex wavelet filtering over median filtering
and real wavelet filtering, the mean square error (MSE) of the real component (G) is calculated
using the 50-frame accumulated image as the ground truth. The MSE value of each filtering is
displayed inside the error map. Noise reduction is calculated as the MSE reduction compared to
the ground truth dataset. CWF has a nearly four-fold decrease in MSE and has at least about
20% better noise reduction than other filtering methods.

91

5.4 Complex wavelet filtering shows accurate structural reconstruction of lifetime
information with metabolic FLIM imaging
As mentioned in the previous chapters, FLIM phasor analysis is heavily used for NADH
metabolic imaging (Lakowicz, Szmacinski, Nowaczyk, & Johnson, Fluorescence lifetime imaging
of free and protein-bound NADH, 1992; Stringari, et al., 2012). To analyze the filtering
performance of CWF on auto-fluorescent NADH FLIM, HEK-293 cells are imaged and filtered
with CWF and median filters. The results are shown in Figure. 5.6. NADH molecules outside the
nucleus are relatively more concentrated and have a longer lifetime as they have a higher
binding ratio to enzymes; in the nucleus, NADH is present with a lower concentration and with
a higher percentage of free NADH. Therefore, two clusters would arise from the two
populations of NADH that come from the inside and outside the nucleus. Figures 5.6(a) and (b)
show that the separation of the phasor plot is hardly seen when no filtering or when median
filters are applied on the original phasor map. On the other hand, with CWF, shown in Figure
5.6 (c), the two populations with their respective phasor clusters are clearly separated.
We further color-coded the structural image based on phasor plots under different filtering.
When no filters are applied (Figure 5.6(a)), speckled color variance in the uniform structure like
in the nucleus is observed. This is caused by the scatter error in the phasor plot. The median
filter (Figure 5.6(b)) partially reduced the scatter error, but unnatural clusters are formed,
especially at the edges of the nucleus membrane. This is the effect of degradation of high
spatial frequency components induced by median filters. With CWF (Figure 5.69c)), the nucleus
92

membrane and the mitochondrial puncta are well reconstructed smoothly and realistically,
proving that the extracted FLIM information aligns well with the structure of the cells.

Figure 5.6. CWF provides superior filtering for autofluorescence NADH metabolic live-
imaging of HEK-293 Cells. Since most NADH images collected with FLIM have very low
photon counts, it is important to reduce the amount of scatter error to extract valid
metabolic information. The unfiltered raw data(a), median filtered (b), & CWF filtered (c)
are shown and within each condition, from left-to-right the images are phasor plots,
phasor selections, the corresponding images, and zoomed regions of interest (from dashed
boxes). The two-dimensional gradient plot shows the degree of bound NADH (Oxidative
Phosphorylation) and free NADH (Glycolysis) in the image overlaid on the intensity values
(grayscale). In the phasor plot, we have shown that CWF can extract valid and meaningful
information from the noisy data and reconstruct the two different NADH populations
within and outside the nucleus, while the non-filtered image and the median filter images
cannot. Meanwhile, when carefully examining the structures in the color-coded intensity
maps, we can see that median filters generate artifacts along with the high spatial
frequency components such as along the nuclear membrane and creating what looks like
structure in the nucleus (b). On the other hand, CWF has smooth reconstruction with valid
alignment with the cell structures (c).
93

5.5 CWF accurately reconstructs structural details for τ-STED imaging
Stimulated emission depletion (STED) microscopy is a super-resolution microscopy modality
that selectively deactivates fluorophores to minimize the area of illumination at the focal point,
thereby enhancing its spatial resolution (Schermelleh, 2019; Gustafsson, 1999; Hell &
Wichmann, 1994). Although STED provides sub-diffraction limit resolution for fluorescence
imaging, it suffers from low photon budgets in most scenarios (Vicidomini, Bianchini, & Diaspro,
2018). Moreover, selective deactivation of fluorophores, the basis of STED, requires high
depletion power, which can lead to many problems like phototoxicity. To tackle these
problems, researchers have combined phasor FLIM with STED, realizing that valid information
for STED is concentrated in certain parts of the phasor cluster. This has led to technologies like
τ-STED, which further improved the imaging resolution while lowering the laser power
required. However, the problem of low photon counts still exists, and the degrading effect of
high spatial frequency components traditionally used in median filters is fatal.
In Figure 5.7, τ-STED images of mitochondria in live Hela cells stained with Nile red are shown
and processed with different filtering. Figure 5.7a shows the unfiltered image with substantial
noise due to the low photon counts, and the respective phasor plot contains much scatter
error. Median filters slightly reduce the scatter error of the phasor plot; however, median
filters generate erosions on the high-frequency components that lead to unnatural edges along
the mitochondria membrane (Figure 5.7 (b)). CWF shows a better reduction of scatter error in
the phasor plot than median filters, and the processed membrane is more natural and
smoother. This can be further confirmed by a line profiling along the arrows indicated in Figure
94

5.7 (a-c). Structures were buried under noise for the raw image. The median filter
reconstruction was noisier and contained more bumps at the edges, while the CWF
reconstruction was more uniform and realistic.

Figure. 5.7 CWF provides noise filtering in τ-STED phasor analysis, leading to enhanced
resolution and accurate structure reconstruction. When imaging with STED, the
fluorophores are forced to release their energy by the depletion laser, causing a shift in
lifetime; therefore, we can incorporate lifetime measurements to reduce noise and
enhance resolution in STED. This is the basis of τ-STED. However, STED images are
usually constructed with very few photons and require filtering techniques to reduce
the scatter error. Mitochondria of live Hela Cells stained with Nile red are imaged with
t-STED, and the unfiltered, median filtered, and CWF filtered phasor plots and the
reconstructed images are shown in figures (a-c). Arrows in the structural images
indicate the position and direction of the line profiling. Median filters lead to unnatural
clustering of the mitochondrial membrane of the reconstructed image. With CWF, the
membrane reconstruction is smooth and uniform.
95

5.6 Discussion on complex wavelet filtering for phasor analysis on FLIM
The proposed CWF for FLIM phasor analysis greatly outperforms the traditional median filters
for FLIM phasor analysis. Median filtering of images is a relatively simple way to clean up a
noisy image and it delivers useful results for a range of applications. However, as the matrices
of pixels to perform the median filter grow larger, the sharpness of the image is reduced, and
the high spatial frequency components are washed out. This problem is worse in the case of
photon-starved specimens. Compared to median filters, CWF reduces the scatter error in the
phasor plot by at least 20%. Moreover, CWF well preserved high spatial frequency components
of the images, including mitochondria and membrane, while the median filters generated
filtering artifacts. CWF also has more outstanding performances when the photon budget is
low, therefore enabling imaging with lower laser power and fasting imaging speed.
There are two major changes in our filtering strategy compared to using traditional filters. First,
the Anscombe transform was added to address the non-gaussian distribution of scatter error.
The Anscombe transform mediates the high variance of the photon-limited images, making the
standard deviation approximately constant before proceeding with the wavelet transform.
Second, with the dual-tree complex wavelet transform, oriented wavelets are produced to
better address different directionalities, which is useful for preserving edges and fine
structures. Combining these two changes enables the CWF to preserve fine structures when
only limited photons can be collected for every pixel.
Compared to median filters, CWF is easier to use as it does not require manual adjustment of
hyperparameters such as filter size. For a median filter, the optimal filter size will depend on
96

the pixel size of the image and the size of the target imaging structure, which in many cases
takes multiple attempts to adjust. This process is eliminated with CWF and therefore is not only
more user-friendly but also more objective. This filtering strategy needs no subjective
adjustments by the user to give an optimal result. Since phasors have been used in other
imaging modalities like hyperspectral imaging, this approach could easily be transferable to
those analyses.
5.7 Comparing correlated non-local means with complex wavelet filter
We introduced two filtering methods for FLIM phasor analysis in chapters 4 and 5: CNLM and
CWF. Both filtering methods reduce scatter error in phasor plots and preserve structural
lifetime information much better than the median filter, the currently most frequently used
filter for phasor analysis. CNLM and CWF have very similar results in most cases. However,
each can provide different advantages. We will dive into the differences between these two
filters to provide a guide for choosing between them.
The most important aspect influencing the decision on which filter to use is the dataset's
photon count. In Figures 5.8 and 5.9, we will quantify and compare the performances of CNLM
and CWF with respect to the number of photons collected in a data set from fixed Cos-7 cells
imaged with antibody-labeled mitochondria. Figure 5.8 shows the error map of the CNLM and
CWF filtered dataset concerning a ground truth dataset on a zoomed-in portion with high
resolution and various photon count levels. The ground truth dataset was collected with 50
laser repetitions and is shown on the lower left. The maximum photon count for the ground
truth is over 3000; the scatter error in this data set is minimal. Filtering was performed with a
97

single frame of the dataset. The squared error maps with respect to the ground truth for the G
components of unfiltered, median filtered, CNLM filtered, and CWF filtered datasets are shown
to the right. In these error maps, the darker the location is, the lower the error level. The
figure shows that the median filtered image has a significantly higher error level than the CNLM
and CWF filtered images. Details inside the error maps show that the CNLM filtered image has
lower errors where the photon counts are higher. The rectangular structure in the CNLM error
maps corresponds to an area with minimal photon counts, where CNLM made more significant
errors in filtering lifetime information. On the other hand, CWF shows a more uniform error
over the field of view. Notice that the overall error level for the CWF filtered image is low, even
for regions on the left where almost no photons were collected.
We further quantify the error level in Figure 5.9. The mean square error magnitude with
respect to photon counts is plotted for all the filtered datasets and the unfiltered one-frame
98

dataset. The error bar represents the standard deviation. Both CNLM and CWF have
substantially less error than the median filter. With extremely low photon counts (below 15),
the CWF has a lower mean square error value. For pixels with larger numbers of photon counts,
the CNLM slightly outperforms CWF.
Figure 5.8. Comparing the performance of the CWF and CNLM shows their respective
ideal working scenario. CWF, CNLM, and the median filter were applied to a stained
mitochondria dataset. The ground truth image (b) is constructed with 50 image frame
repetitions. Filtering was performed with a single frame of the dataset. The squared
error map is displayed on the right. Darker regions inside the error map represent lower
errors. Generally, median filters(e) perform the worst, as the overall color is brighter
(higher errors), and a clustering effect can be seen in the structure. The complex
wavelet filtered image(d) has lower error values than the CNLM filtered(f) one where
the photon counts are low. CNLM, on the other hand, has lower errors in the middle
part of the image where the photon counts are higher. This indicates that when the
structure of interest is constructed with high photon counts, CNLM might lead to a
better result than CWF. For structures with lower photon counts, CWF generates a more
uniform output, which means the mean error value might be smaller than CNLM.
(a)
(b)
(c)
(d) (f)
(e)
99

The foundations behind the two filtering methods are different and lead to different optimal
scenarios for usage. CNLM uses structural information obtained from the photon intensity map
to support the phasor maps. Therefore, it works best when components with different
lifetimes have different shapes. This includes scenarios when only one fluorescence species is
involved in constructing the image or when staining dyes are used to label structures that are
not alike. CNLM works well for NADH metabolic imaging. This is because mitochondria,
cytosol, and nucleus have very distinct NADH readings, and their NADH lifetime readings are

Photon Counts
Squared Error in G
Figure 5.9. Mean Squared Error (MSE) with respect to photon counts shows CNLM works
better for regions with higher photon counts, while CWF slightly outperforms CNLM when
photon counts are low. The squared error was calculated with the filtered 1 image
repetition image and taking the 50-image repetition as the ground truth. The MSE values
with respect to the photon counts in the 1 rep image were plotted in the figure above.
Error bars indicate the standard deviation of the error values. Generally median filters
perform the worst. CNLM has lower errors where the photon counts are large, while the
CWF filter slightly outperforms CNLM where the photon counts are less than 15.
100

different from each other as well. CWF filters, on the other hand decompose the lifetime
information into different wavelet components and perform wavelet thresholding for each
wavelet component. It works better than CNLM when structural information on the photon
intensity map is not a reliable contrast to distinguish lifetime components. This refers to
scenarios when the structures are similar, like when differentiating some secretory vesicles
against ribosomes. CWF works better than CNLM when the photon intensity image itself is too
noisy or when too little detail can be obtained within it because there are no structural details
for CNLM to extract. However, in this case, the lifetime information of interest will be buried
within scatter error, and even CWF might not salvage any of it.

101

Chapter 6. Subcellular Metabolic Imaging and Analysis of Glucose
Stimulated Insulin Secretion in INS-1E Cells
6.1 Establishing a workflow for metabolic imaging with subcellular accuracy
In the previous chapters, we introduced tools and workflows to lay the foundations for
representing NADH lifetime accurately with phasor analysis in subcellular structures. Chapter 1
introduced the lifetime representation with the frequency domain phasor plot. Chapter 2
explained how NADH lifetime is used to infer the redox ratio. Chapter 3 presented the workflow
to accurately extract metabolic information from NADH phasor clusters and reduce the
contamination from other fluorescence species in the sample. Chapters 4 and 5 introduced two
different filtering techniques: correlated non-local means and complex wavelet filter. Both
reduce the scatter noise in phasor clusters to reveal accurate lifetime details in delicate
subcellular structures.
The subcellular spatial resolution of FLIM enables the measurement of the metabolic
heterogeneity in the cells (Van Schravendijk, Kiekens, & Pipeleers, 1992; Gutierrez, Gromada, &
Sussel, 2017). NADH lifetime information can be pinpointed to locations and structures with
resolution as high as couple hundred nanometers, which means that metabolic differences
between cells and the metabolic differences between organelle can be resolved. There are two
types of cellular heterogeneity that need to be considered. The first one is the cellular
heterogeneity within the cells. This type of heterogeneity arises from the different internal
structures and organelles. Each subcellular structure fulfills unique functions and hosts
102

distinctive pathways, and the metabolic states and redox states of different structures are
different. When measuring cellular metabolism, the individual response of different cellular
components and their distinctive redox changes must not be averaged in the calculation
process.
The second heterogeneity in cellular metabolism is the heterogeneity among cells. Each tissue
and organism contain cells from different cell types. Each cell type has unique functions and
metabolic pathways, and they coordinate to fulfill the functions of the tissue and organism.
The heterogeneity among cells also exists for cells of the same cell type. Changes in metabolism
in a group of cells are not always uniform, as these cells could have different metabolic states
and stimulus responses depending on their surroundings and previous metabolic states. To
account for the heterogeneity among cells, the basic unit for studying metabolism must be the
individual cell. Each cell needs to be treated as a single sample, and the measured metabolic
information needs to be viewed as a populational feature, such as the percentage of the
individual cells that changed their metabolism or the maximum extent cells change their
metabolism.
In many previous metabolic analyses, researchers used the field of view as the basic unit for
summarizing metabolic signatures and lost track of both types of cellular heterogeneity. In this
chapter, we will introduce a workflow to measure the heterogeneity in cellular metabolism and
perform statistical tests to quantify metabolic changes. To demonstrate this, we use INS-1E
cells as our sample and study their metabolic responses under the glucose stimuli. Although
103

INS-1E cells are the samples for this chapter, the presented workflow can also be modified for
other research subjects.
6.2 Pancreatic beta cells metabolism is key to maintaining normal blood glucose level
Pancreatic beta cells (PBC) are the primary modulators for blood glucose levels inside the
human body (Aspinwall, Lakey, & Kennedy, 1999; Cerasi, Luft, & Efendic, 1972; Wang, et al.,
2021). PBCs are located in the pancreatic islet. They make up 50-70% of the cell populations in
the human islet. One of PBCs’ unique functions is to secrete insulin. This hormone reduces
blood glucose levels by promoting glucose absorption from the blood into the liver, fat, and
skeletal muscle cells. Rising glucose levels in the bloodstream trigger insulin release by beta
cells; this process is called Glucose Stimulated Insulin Secretion (GSIS) (Hedeskov, 1980; Van
Schravendijk, Kiekens, & Pipeleers, 1992). GSIS has two separate insulin-releasing phases. The
first phase is acute, where beta cells secrete a large amount of insulin within less than 10
minutes. After the first phase, insulin secretion will have a substantial drop but slowly picks up
again in the second phase. In the second phase, insulin secretion is slower than in the first, but
the duration lasts several hours. Since beta cells are the only cells that synthesize and secrete
insulin in mammals, their dysfunction, reduction in population, and failure to secrete insulin are
long suspected to be the primary cause of diabetes (Pipeleers & Ling, 1992; Wang, et al., 2021).
The study of pancreatic beta cells has been one significant effort to understand diabetes
pathology and treatment in the past several decades. A critical topic for PBC studies is whether
there are metabolic differences in healthy versus diabetic pancreatic beta cells in their resting
state and during GSIS (Maechler, Carobbio, & Rubi, 2006). Another aspect of PBCs to be
104

understood is whether all cells exhibit metabolic changes simultaneously under GSIS or do
some cells respond first to outside stimuli (Van Schravendijk, Kiekens, & Pipeleers, 1992). Some
studies have attempted to study PBC metabolism using techniques such as seahorse and mass
spectrometry. However, these techniques are very limited in temporal and spatial resolution
(Pirman, et al., 2013). NADH metabolic FLIM imaging, on the other hand, is an ideal technique
for studying metabolism under GSIS, as it can collect metabolic information with subcellular
resolution and track the same group of cells over time to study metabolic changes under the
stimulus.
To understand the functions of PBC, especially their insulin-releasing pathway, cell lines were
developed to be easily acquired, handled, and studied in mass. INS-1E cells are one of the most
widely used cell lines and are involved in numerous studies investigating the mechanisms of
GSIS. INS-1E cells are rat insulinoma cells and are glucose-responsive with substantial insulin
secretion levels. INS-1E cells are usually cultivated in RPMI-1640 buffer with 11.1mM glucose.
To stimulate GSIS, researchers first place them in a starvation buffer with low glucose levels
(1.1mM or 2.8mM) to halt insulin synthesis. Then, a high glucose level media (16.7mM or
25mM) stimulates insulin release (Vetterli, Brun, Giovannoni, Bosco, & Maechler, 2011; Wang,
et al., 2021).
In our study, we trigger GSIS with the same process described above and analyze the metabolic
signature by imaging with NADH autofluorescence FLIM during the GSIS process. We leverage
the spatial resolution provided by FLIM to study the heterogeneities inside individual INS-1E
cells and among the cell population. This is accomplished by using machine learning to
105

segment individual cell instances and different cellular compartments, after which statistical
tests can summarize the metabolic changes of different cellular compartments to understand
populational metabolic features.
6.3 Segmentation of cellular compartment for metabolic analysis with machine learning
In image processing, segmentation is the process of partitioning a field of view into multiple
subsets. Segmentation allows locating objects and boundaries, simplifying the contents of the
field of view, and assigning meanings for regions inside the image. Over the past decades,
researchers have developed multiple techniques and algorithms for segmenting images
(Haralick & Shapiro, 1985; Minaee, et al., 2021). Tremendous improvements have taken place in
recent years with machine learning, especially with convolutional neural networks (Hesamian,
Jia, He, & Kennedy, 2019; Du & Gao, 2017). Convolutional neural networks are built by
connecting layers of parameters called neurons. Each layer of neurons adds one procedure to
process and combine information obtained from neurons in the previous layers. With
convolution, features are extracted from each layer with predesigned filters. Like all machine
learning algorithms, convolutional neural networks rely on tuning the matrix of the network
with training data to learn features of the desired output. Training data consist of input and
output pairs. The inputs are images on which segmentation will be performed on; the outputs
are the already segmented results of its corresponding inputs. By training with these training
pairs, the algorithm slowly learns features of the desired output and eventually learns how to
segment images it has not seen before.
106

In our workflow, a convolutional neural network called U-Net is used as the base model to
develop our segmentation algorithms (Ronneberger, Fischer, & Brox, 2015). U-Net was
introduced in 2015 for biomedical image segmentation by a group at the University of Freiburg,
Germany. U-Net has excellent precision for segmentation, even when the structures are
blurred together. Another great property of U-Net is that it achieves high accuracy even with
limited training datasets. This is important for many applications in biomedical engineering,
where the training sets are hard to acquire. The basic structure of U-Net is shown in Fig 6.1.
This network is called U-Net for apparent reasons: the two symmetric branches in the network
that form the U shape architecture. The left arm of the U-Net is a “contracting path” where
successive convolutions are used, each followed by a rectified linear unit and a max-pooling
operation to acquire more complicated features while reducing spatial information progressing
along the branch. On the right arm, which is called the expansive pathway, the exact opposite
takes place. It uses up convolutions to expand spatial information but reduces the feature
complexity. The middle arrows are concatenating pathways where the data from the
contracting pathway is directly concatenated to the expansion pathway. The original U-net
takes a single image as input. However, FLIM phasor datasets have four separate channels
containing meaningful information: the photon intensity map, the two phasor maps (G and S),
and an additional bright-field image generated with the transmitted light collected with the T-
107

PMT detector. These images are represented in the upper row of Figure. 6.2. We modified the
standard U-Net model to take all four channels as input.
The cellular compartments that contain the most metabolic actions are segmented with the
machine learning model. The biggest organelle inside a cell is the nucleus, which contains most
of the cellular DNA and takes up around 30% to 50% of the cell size. However, NADH metabolic
wise, the nucleus is not the most interesting compartment. Most NADH-related metabolic
pathways do not take place in the nucleus. When imaging the NADH inside the nucleus, one
Figure 6.1 U-Net is a convolutional neural network model that yields good
segmentation for biomedical applications with low sample sizes. U-net is the basic
model for our segmentation algorithms, with which different cellular compartments
are segmented inside the cell. U-Net has two symmetrical branches; the left one in
the diagram converts the original image into features with convolution. With each
layer of convolution, the features derived will become more complicated. The right
one operates in reverse and restores the features to the image space. The two
branches are connected by concatenating operations (gray arrows), which directly
compare the two branches. U-net is widely used in many segmentation tasks for its
outstanding segmentation accuracy, especially for biomedical images where training
data is limited. Image: Ranneberger, 2015.
108

will find the NADH concentration is relatively low, and most of its NADH has a very short
lifetime, indicating that it’s not bound to enzymes. Therefore, our metabolic calculations will
exclude the regions where the nucleus resides. One most crucial organelle for metabolism is
the mitochondrion. Mitochondria are the primary energy production plant inside the cell,
which hosts many pathways involved in the production of ATP. For INS-1E cells, the
mitochondria are also involved in the pathway to detect the excess glucose in the environment
(Anello, et al., 2005). NADH is actively engaged in the metabolic pathways inside the
mitochondria, which makes the mitochondria an interesting subject for our NADH metabolic
FLIM imaging. Besides the mitochondria, the cytosol, which is the aqueous component of the
cytoplasm in a cell, is an essential cellular compartment. Many glycolysis pathways take place
here; these pathways are heavily dependent on the redox ratio inside the cytoplasm. The
cytoplasm is hard to segment out by itself, but we can use the whole cell and subtract the
nucleus and the mitochondria as an appropriate approximation. Finally, since the basic unit for
summarizing metabolism is single-cell, individual cell instances need to be segmented. Unique
cell segmentation allows the statistical analysis of populational metabolic features, which will
be presented later in this chapter.
109

Separate machine learning models are constructed for each cellular compartment mentioned
above (the nucleus, the mitochondria, and individual cell instances). Each segmentation model
requires specific segmentation training data sets that consist of input-output pairs. To generate
the ground-truth outputs, staining dyes are used on live INS-1E cells to label the desired
compartments. These dyes are applied separately onto different plates of INS-1E cells: each
training set only labels one cell compartment and only trains one single segmentation model.
The most essential requirement of the dyes is that they must not contaminate NADH with
fluorescence bleed-through, and their emission spectrum must be far from that of NADH.
Therefore, most of the dyes that were chosen to have red or infrared emission spectra. The
Figure 6.2 The input channels and segmentation results for the customized machine
learning model to perform subcellular metabolism analysis. The photon intensity map,
the two phasor maps (G and S), and an additional bright-field image generated with
the transmitted light collected with the T-PMT are used as the input; the individual cell
instances, the mitochondria, and the nucleus are the segmentation targets. The four
input channels are shown in the upper row, and the three segmentation targets are
shown below. Separate U-net models for each cellular compartment. Each input
channel contains unique information, and all four input channels are used in the U-net
model to segment the cellular compartment. All images share the same scale bar.
110

dyes used in their staining and imaging condition are summarized in Table 6.1. The staining dye
for the mitochondrion and the nuclei directly labeled their target organelles. For individual cell
segmentation, the cellular membranes are stained and imaged, which are further used to mark
out the regions for each cell.
Once the datasets are collected, the images of the stain dyes are post-processed to generate
the segmented ground truth data. The first step of post-processing is to create maps by
thresholding the photon counts in the images of the staining dyes. This was sufficient to
generate the ground truth data for the mitochondria. Additional procedures were required to
create the ground truth for the individual cell instances and the nucleus. For cell instances, a
standard cluster counting algorithm is applied to separate the cells. For the DNA label, the
mitochondria were also stained because mitochondria carry DNA molecules. Fortunately, the
lifetime of the dye inside the mitochondria is different from that of the nucleus. With this
property, an additional lifetime filter is applied, easily separating the nucleus from the
mitochondria. All training models used data augmentation to increase the number of training
data the network observes. Demonstrations of the images collected with the staining dyes and
the derived segmented ground truths are shown in Figure 6.3.
111

Segmentation models for different compartments have specific adjustments according to their
tasks. For mitochondria segmentation, the NADH photon intensity channel’s stability plays an
important role in the segmentation. Before this channel is fed into the first layer of the
network, a Contrast Limited Adaptive Histogram Equalization (CLAHE) is applied to the image.
This is a local contrast enhancement operation that calibrates the pixels intensity based on
Figure 6.3. Segmentation ground truth is generated with staining dyes. Each
segmentation U-Net model has a separate training set. To create the ground truth for
the training sets, fluorescence dyes were used to specifically label the component of
interest. The upper rows are the stain images collected with the staining dye. The
lower row images are corresponding segmentation ground truth. Fluorescence dyes
were used separately on different dishes of INS-1E cells. Thresholding was the first
procedure to generate the ground truth images from the staining images. This was
sufficient for the ground truth generation for mitochondria. For cell instance
segmentation, we reversed the thresholding result to get the actual area of the cells.
For the nucleus-stained images, the dye also stained the DNA in the mitochondria.
Fortunately, the staining dye exhibited a different lifetime inside the mitochondria
than in the nucleus, a property we used to apply an additional lifetime filter that
segmented the nucleus.
112

histograms computed over different tile regions of the image. With CLAHE, local details are
enhanced in regions that are darker or brighter than other regions of the image. For individual
cell instance segmentation and nucleus segmentation, the segmented results need circular
boundaries like the cell membrane and nucleus. However, U-Nets sometimes output
fragmented regions with clusters of pixels. To ensure that the network generates continuous
output instead of fragmented ones, we used distance transformation on the original
segmentation ground truth data set, where all non-feature pixels will be assigned a value
corresponding to the distance to the nearest feature. This process for distance transformation
is shown in Figure 6.3. After the output is generated by the nucleus and individual cell instance
segmentation model, all non-zero-pixel vales are labeled as the corresponding compartment.
The segmentation models have reached outstanding results for every cellular compartment,
even with the limited number of training data. The number of training data and the quantified
performance with the accuracy and Dice coefficient are summarized in Table 6.2. The
individual cell instance segmentation required a larger training set to generate precise results.
This is because INS-1E cells like to cluster together and form continuous islands. With close
contact of the cells, the outer boundary is difficult to identify and requires more training sets.
The results of the three segmentation compartments of one sample are shown in Figure 6.2.
The displayed sample was not a training set for any segmentation models. The quality of the
segmentation can be estimated by searching for desired features of the segmentation in the
NADH photon intensity map. The nucleus membrane is visible where the round boundaries
mark the position of the nucleus membrane. Mitochondria have a slightly higher concentration
of NADH, and the brighter spots in the NADH photon intensity map can be used as an estimate
113

of regions where mitochondria reside. The boundary of cellular membranes is hard to identify
by the human eye, but the cellular segmentation is corresponding well to the cell boundary
where it can be identified in the bright field image and the photon intensity image.
Table 6.2. The number of training sets and quantitative performance for segmentation
models for different cellular compartments. The results show good segmentation
precision.
Table 6.1 Staining dyes and their imaging conditions for creating the segmentation
ground truth.
114

Figure 6.4 Distance transformation is applied to in ground truth generation for
individual cellular instances and nucleus segmentation to ensure high quality. U-
Net models have the tendency to assign fragmented clusters for segmentation
results. This is not ideal for individual cellular instances and nucleus segmentation,
as the segmented results need circular boundaries like the cell membrane and
nucleus. To ensure that the network generates continuous output instead of
fragmented ones, the distance transformation is used on the ground truth output
to assign each pixel with its distance to the feature’s boundary. This is a standard
procedure for segmentation for similar structures and ensures that the segmented
results from the neural networks are continuous regions instead of fragmented
clusters. After the output is generated by the nucleus and individual cell instance
segmentation model, all non-zero-pixel vales are labeled as the corresponding
compartment.
115

6.4 INS-1E GSIS subcellular metabolic analysis
Now that the cellular compartments are segmented to track their metabolic changes
individually, we can set up the experiment to trigger the metabolic changes. Our experiments
and analysis aim to understand subcellular metabolic changes during GSIS in INS-1E cells. INS-
1E cells were seeded in 8 well imaging chambers (Thermo Scientific™ 154941) with 40k/L
density and cultivated in 11mM Glucose RPMI media and 5% CO 2 for six days before the
experiment. To trigger GSIS, insulin synthesis inside INS-1E cells is first halted by putting the
samples into low glucose starvation media (100ul KREBS buffer with 2.8mM glucose) for 40
minutes inside the imaging chamber. Afterward, one dose (300ul) of high glucose KREB media is
injected into the imaging well so that the media reaches a final concentration of 16.7mM
glucose. The volume ratio of the starvation media to the injection media is set to 1:3 to
guarantee that glucose is evenly mixed after the injection. Besides imaging the metabolic
changes of INS-1E cells under high glucose stimulation, Exendin-4 is another interesting
substance to be studied for INS-1E cells. Exendin-4 is a peptide agonist of the glucagon-like
peptide (GLP) receptor that promotes insulin secretion and is often used as a treatment for
diabetes. (Xu, Stoffers, Habener, & Bonner-Weir, 1999). An experiment set introducing the INS
cells to high glucose(16.7mM) plus exendin-4(10nM) after starvation is added to track the
induced metabolic changes. In the control set, the buffer used in starvation is injected into the
cell chambers after the starvation period, and the cells remain inactive for insulin secretion.
Time-lapse images were taken with FLIM to track metabolic changes over time. The data were
collected on a Leica SP8 Falcon system in the Translational Imaging Center at the University of
116

Southern California. The same field of view was imaged four times: 10 minutes before the
injection and 1 minute, 5 minutes, and 30 minutes after the injection. We imaged 10 minutes
before the stimulation instead of right before the stimulation to avoid phototoxicity build-up.
This imaging and treatment timeline is depicted in Fig 6.5. In the imaging experiment, the
sample was excited with a Spectra-Physics infra-red tunable laser at 740 nm, and the FLIM
detection channel was set from 410 nm to 530 nm.
After data collection, the workflows described in the previous chapters were applied to extract
the subcellular metabolic changes. All data sets were filtered with the complex wavelet filter
introduced in Chapter 5 to reduce scatter error. The metabolic redox map, introduced in
Chapter 4, is then applied to the FLIM data set to extract metabolic information. In parallel, the
segmentation models for the mitochondria, the nucleus, and the individual cell instances
created masks for each cellular component. To guarantee the quality of the analysis for
individual cell instances, two prerequisites were set for a cell region to be analyzed. First, the
Figure 6.5 Imaging pipeline for subcellular metabolic analysis of INS-1E cells under GSIS.
Two treatment conditions were designed to induce metabolic changes in the cells: high
glucose (16.7mM) and high glucose (16.7mM) with exendin -4 (10nM). The control set
does not trigger any insulin secretion. When the stimulus was injected into the imaging
wells, the volume ratio of the injected media and the original media was 3:1 to ensure
that the stimuli were uniformly distributed in the imaging well. The same field of view
was imaged four times: 10 minutes before and 1 minute, 5 minutes, and 30 minutes
after the injection. We imaged 10 minutes before instead of right before the
stimulation to avoid phototoxicity build-up.
117

region marked as an individual cell must have a nucleus segmented by the nucleus
segmentation model. This guarantees that no gaps between cells are analyzed. Second, since
INS-1E cells have minimal movement during the 40-minute imaging window, each analyzed cell
must have good overlaps across the four time points. This requirement guarantees the overall
stability of the imaging procedure and simplifies the analysis process for tracking cells. Most
cells in the experiment meet the previously stated two requirements. The number of datasets
generated, and the total number of cells analyzed in each condition are summarized in Table
6.3. The individual cell segmentation, the mitochondria segmentation map, and the cytosol
segmentation map (full map minus the nucleus map and the mitochondria map) are applied as
masks on the metabolic redox map, and the metabolic treads for each cellular compartment
are extracted. The overall analysis pipeline is demonstrated in Fig 6.6. This workflow allows us
to analyze how each compartment’s redox ratio changed individually during the stimulus.
Moreover, because of the individual cell instances segmentation, it is possible to explore the
populational metabolic features that occur under different stimuli, answering the question of
how the populational distribution of the redox ratio changed over time in mitochondria and the
cytosol.

Table 6.3 The number of data sets and individual cells of INS-1E cells that were
collected in the GSIS experiments.
118

Figure 6.6. Image analysis pipeline for subcellular metabolic analysis of INS-1E cells under
GSIS. The metabolic redox ratio map for each data set is generated based on the phasor
plot and the autofluorescence intensity map. The individual cell instance segmentation
map is applied to the metabolic redox ratio map to segment individual cells. Each cell
must meet two requirements to be considered a candidate to be analyzed. First, a nucleus
segmented out by the nucleus segmentation model must exist in the cell instance
segmentation. Second, the individual cell instance has stable regions within the four time
points collected. The cytosol and the mitochondria are labeled as cyan and magenta in the
figures on the lower row. This workflow allows researchers to analyze how each
compartment’s redox ratio changed during the stimulus. Moreover, because of the
individual cell instances segmentation, it is possible to explore the populational metabolic
features.
119

6.5 Subcellular metabolic redox ratio analysis of INS-1E cells under GSIS
With the metabolic analysis workflow introduced so far, the redox ratio change inside the
mitochondria and cytosol in individual INS-1E cells during GSIS can be analyzed. The most
direct analysis is to plot the redox ratio changes over time under the experiment and control
sets, as shown in Fig 6.7. As promised, each individual cell was used as a sample to generate
the plot. Since changes in the redox ratio are more important than the redox ratio itself, all
redox ratio readings were calibrated with the redox ratio of the first imaging time point. For
the control, the redox ratio should have little to no change since no additional stimulus is
provided. The control line (blue line) is stable over time, indicating that no metabolic changes
occurred as expected. This also provides evidence that FLIM imaging caused little changes in
metabolism. With high glucose stimulation, the redox ratio decreased after injection of the
stimulus in both the mitochondria and the cytoplasm. A decrease in the redox ratio means that
the NADH binding ratio with enzymes increases, indicating more activity inside the region. The
redox ratio decrease is more substantial inside the cytosol than in the mitochondria at all time
points. One-way ANOVA tests and pairwise t-tests with Bonferroni correction were used to test
for statistical differences in the redox ratio between the experimental sets and the control at
each time point. The high glucose experiment set showed a statistical difference with respect to
the control at all time points. However, in the experiment set of high glucose plus exendin-4,
the only case where the metabolic trend differs from the control is at the 5-minute time point
inside the mitochondria. Checking the amount of insulin released with the insulin ELISA test
120

(data not shown) shows that high glucose plus exendin-4 still triggers insulin release. This is an
interesting phenomenon, indicating that exendin-4 activated a different metabolic pathway
than the high glucose stimulation.
The trend line analysis in Figure 6.7 provides a general idea of the redox ratio changes, yet it
does not show the metabolic heterogeneity among the cells. Histograms can be used to
visualize the redox ratio populational distributions to examine the populational metabolic
heterogeneity. These histograms are shown in Figure. 6.8. The INS-1E cell baseline metabolic
fluctuations are represented by the control sets where no additional stimulus is added. Adding
high glucose to the INS-1E cell media causes the histogram to tilt to the left, which shows a
decrease in the redox ratio inside the cell population. This tilt is not observable with high
Figure 6.7 The calibrated redox ratio trendlines shows different metabolic changes in the
mitochondria and cytosol. Every cell individual cell is used as one sample count to plot the
trendlines. Every cell individual cell is used as one sample count to plot the previous plot. The redox
ratio in each compartment is calibrated to their respective redox ratio of 10 minutes before the
injection. Error bars indicate the standard deviation of the redox ratio calculated from all the
samples in their respective condition. With high glucose stimulation, the redox ratio decreased after
injection of the stimulus in both the mitochondria and the cytoplasm. This is not seen with high
glucose stimulation with exendin-4. This could be an indication that extendin-4 triggers a different
metabolic pathway than glucose, one that involves fewer changes in the redox ratio.
121

glucose plus exendin-4 treatment. The normality of the histogram represents whether the cells
are divided into groups of different metabolic states. A normally distributed histogram
indicates that the group of cells does not have separate metabolic conditions. The chi-square
goodness of fit test is used for each histogram to see whether the population is a normal
distribution. The P values are labeled above each histogram. All histograms have P values
greater than 0.05. Therefore, all of the histograms are normally distributed. In this case, we
cannot conclude that cells reacted to stimulus in separate waves. But an interesting effect is
shown in the high glucose treatment, especially inside the cytosol. Inside the histograms
Figure 6.8 Calibrated redox ratio histogram for the INS-1E cell show population metabolic
profiles of the cytosol and the mitochondria at different time points of GSIS. These
histograms are calibrated because every cell compartment’s redox ratio is calibrated to
their respective redox ratio of 10 minutes before the injection. The P values in the title
indicate the P values of the chi-square goodness of fit test to determine whether they are
normally distributed. This histogram shows a populational metabolic profile of the cells
under different stimuli and time points. All the P values shown here are less than 0.05,
therefore we cannot conclude that the cells are responding to any stimulus in separate
waves. However, we do see rather obvious tails in the histogram plots, for high glucose
stimulation, which could indicate that there is a small population that was slow to
respond or did not respond to the stimulus at all.
122

labeled in orange, while the histograms are skewed to the left, a tail remains on the right. This
tail on the right indicates that a small portion of the cells was very slow to respond or did not
respond to the high glucose treatment.
The two previously shown analyses reveal populational metabolic states and how the overall
redox ratios change. However, cells can change dynamically while maintaining an overall stable
metabolic envelope. How fast these dynamic these changes are and how unified the redox
ratio changes are among the cells still need to be explored. To tackle this, the redox ratio
change of each cell compartment of every cell is calibrated with respect to the previous time
Figure 6.9 Scatter plot of redox ratios compared to their previous time points shows
the stability of the redox ratio of the different cellular compartments under GSIS.
The x-axis shows the redox ratio of the previous time point in reference to the title in
each plot, the y-axis is the redox ratio of the current time point. The R
2
values are
calculated for each plot to indicate the stability of the redox ratio, the higher the
value, the more stable the redox ratio. The calculated R
2
values are rather low for
most scatter plots, however, the R
2
in the cytosol with high glucose plus exendin4
stimulation is substantially higher at the 1-minute and 5-minute time points. This
could indicate that exendin-4 could be stabilizing the redox ratio inside the cytosol in
the first 5 minutes, which indicates a different metabolic pathway triggered by
exendin-4 with high glucose than high glucose alone.

123

point. This analysis provides a real-time redox ratio change which shows more transient
dynamics changes. A scatter plot of the redox ratios of every time point versus their previous
time point is used to analyze the transient fluctuations. These scatter plots are shown in Figure
6.9. To quantitatively classify the stability of the redox ratios, the R
2
values of the data in each
plot are calculated. R
2
is a statistical measure of fit for the regression model. It indicates how
much variation of a dependent variable is explained by the independent variable. In this case,
the association of the redox ratio of one time point versus that of the previous time point is
shown. Higher R
2
values indicate strong associations between time points, meaning that the
redox ratios are stable over time. The R
2
values are relatively low for the scatter plots in Figure
6.9. However, the R
2
values in the cytosol with high glucose plus exendin4 stimulation at the 1-
minute and 5-minute time points are substantially higher than their peers. This indicates that
exendin-4 stabilizes the redox ratio inside the cytosol in the first 5 minutes. This effect
becomes even more informative when associated with the trend figures in Fig 6.6. In Fig 6.6,
both the control and the high glucose with exendin-4 trend lines were very flat, indicating an
overall low redox fluctuation. However, the information presented in Fig 6.9 shows that
beneath the trend line, the redox ratio in the cytosol is much more stable for cells treated
simultaneously with high glucose and exendin-4. In contrast, the redox ratio is much more
dynamic for cells in other groups.

124

Conclusion
Imaging NADH with FLIM is a powerful tool that can reveal metabolic information with
extraordinary spatial and temporal resolution. However, most current research does not take
full advantage of the spatial resolution FLIM provides to extract subcellular metabolic
information. The most important three reasons are: First, the lifetime information in FLIM
datasets is not accurately converted into quantifiable metabolic readings. Many attempts to
characterize the metabolic changes fail to calibrate the readings to the enzyme involved in the
metabolic reactions. Therefore, they cannot be used for subcellular calculations where the
enzymes differ in cellular compartments. Second, metabolic FLIM datasets contain heavy
scatter error that originates from the low photon counts inherited in NADH imaging. Scatter
error obscures researchers from summarizing the lifetime information of structures that have a
size close to the resolution limit of the microscope. Third, no automatic segmentation methods
for segmenting the different compartments inside the cells exist for FLIM datasets. Without
these automated segmentation methods, researchers cannot distinguish the metabolic trends
of different organelles and thus are not able to account for cellular heterogeneity.
In this thesis, we presented a workflow for subcellular metabolic imaging and analysis on INS-1E
cells to derive findings of metabolic changes. The metabolic findings presented have not been
shown by any other research group to the best of our knowledge. The workflow includes three
parts. The first part is the metabolic analysis of FLIM datasets with phasors, which involves
acquiring the experimental datasets, filtering the raw data (with either correlated non-local
means or complex wavelet filters), and then deriving the redox ratio from FLIM phasor plots.
125

The second part is to segment individual cell instances and their mitochondria and cytosol to
track their metabolism separately. This is achieved by building machine learning models for
each cellular compartment mentioned. Finally, analyses and statistical tests are used to derive
biological findings of the metabolism changes inside different compartments under different
stimulations. The workflow presented allows for performing subcellular metabolic analysis with
statistical support that previous research has never achieved.
Although this workflow is applied to one specific cell line and one specific question, it can be
easily modified for other research questions. The process of deriving the metabolic readings
from the phasor plots is standard and can be transferred to any project. The biggest hurdle is
to transfer the machine learning segmentation networks to other cell lines and imaging
conditions. Training new networks will require the collection of training datasets and providing
ground truth segmentation, which is the most time-consuming process of this project.
Changing cell lines or imaging settings, for example, the image magnification or pixel size, will
require obtaining new datasets for training and retraining a new machine learning model. We
tested our already trained machine learning models on other cell lines and imaging conditions
and found them robust to a certain extent. The individual cell segmentation model can identify
cells similar in size to INS-1E cells, like HEK-293 cells. We also successfully segmented cells
imaged with around 10% larger or smaller pixel sizes. However, it awaits further testing to be
sure whether this process applies to more changes in the imaging settings. Nevertheless, this
project required fewer training sets than other machine learning projects. Most machine
learning projects require thousands, if not more data. On the other hand, the training sets that
were used for our projects were minimal. We collected the training set for each segmentation
126

within four 4-hour imaging sessions. Moreover, using the network we already trained, one can
easily apply transfer learning for their own needs. This will help the researchers to reduce the
number of training sets needed to prepare their own network.
There are potentially more metabolic analyses that can be done with the high spatial and
temporal resolution of FLIM. After collecting and segmenting the different cell compartments,
it is possible to associate the metabolic changes with more information about the cells, like the
size of the cell, the mitochondria content, or its surrounding environments. Many other pieces
of information can also be used inside the analysis, for example, whether it has direct contact
with the media or is surrounded by other cells. Performing these tasks will require gathering
more accurate information about the cells, which could be achieved by collecting data in larger
fields of view and 3D, and with shorter temporal intervals. This will probably require more
sensitive detectors to collect more photons from NADH and more computational powers and
algorithms to analyze the datasets.
All the code that is used for analysis is shared in the GitHub Repository:
https://github.com/peiyuwan/TIC_FLIM_Collaborators/
I sincerely believe that the workflow we created is a much more accurate way to do metabolic
analysis with NADH FLIM than other work we have seen. I hope that the methods presented in
this thesis could prove to be useful for other researchers as well.

127

References
Lakowicz, J. R., Szmacinski, H., Nowaczyk, K., & Johnson, M. L. (1992). Fluorescence lifetime
imaging of free and protein-bound NADH. Proceedings of the National Academy of
Sciences, 89(4), 1271-1275.
Allen, R. D., & David, G. B. (1969). The Zeiss-Nomarski differential interference equipment for
transmitted-light microscopy. Zeitschrift fur wissenschaftliche Mikroskopie und
mikroskopische Technik, 69(4), 193-221.
Anello, M., Lupi, R., Spampinato, D., Piro, M., Masini, , M., Boggi, U., . . . Marchetti, P. (2005).
Functional and morphological alterations of mitochondria in pancreatic beta cells from
type 2 diabetic patients. Diabetologia, 48(2), 282-289.
Aspinwall, C. A., Lakey, J. R., & Kennedy, R. T. (1999). Insulin-stimulated insulin secretion in
single pancreatic beta cells. Journal of Biological Chemistry, 274(10), 6360-6365.
Barrangou, R., & Doudna, J. A. (2016). Applications of CRISPR technologies in research and
beyond. Nature biotechnology, 34(9), 933-941.
Behne, M. J., Sanchez, S., Moll, I., & Gratton, E. (2006). Calcium‐fluorescence lifetime imaging in
ex‐vivo skin. Experimental Biology 2006 (Part I), A117-A117.
Biskup, C., Zimmer, T., Kelbauskas, L., Hoffmann, B., Klöcker, N., Wolfgang, B., . . . Benndorf, K.
(2007). Multi‐dimensional fluorescence lifetime and FRET measurements. Microscopy
research and technique, 70(5), 442-451.
Boulanger, J., Kervrann, C., Bouthemy, P., Elbau, P., Sibarita, J.-B., & Salamero, J. (2009). Patch-
based nonlocal functional for denoising fluorescence microscopy image sequences. IEEE
transactions on medical imaging, 29(2), 442-454.
Brand, U. L., & Gohlke, J. R. (1972). Fluorescence probes for structure. Annual Review of
Biochemistry, 41(1), 843-868.
Buades, A., Coll, B., & Morel, J.-M. (2011). Non-local means denoising. Image Processing On
Line, 1, 208-212.
Caplan, J., Niethammer, M., Taylor, R. M., & Czymmek, K. J. (2011). The power of correlative
microscopy: multi-modal, multi-scale, multi-dimensional. Current opinion in structural
biology, 21(5), 686-693.
Cerasi, E., Luft, R., & Efendic, S. (1972). Decreased sensitivity of the pancreatic beta cells to
glucose in prediabetic and diabetic subjects: a glucose dose-response study. Diabetes,
21(4), 224-234.
Chen, G., Xie, W., & Zhao, Y. (2013). Wavelet-based denoising: A brief review. 2013 fourth
international conference on intelligent control and information processing, pp. 570-574.
128

Colyer, R. A., Lee, C., & Gratton, E. (2008). A novel fluorescence lifetime imaging system that
optimizes photon efficiency. Microscopy research and technique, 71(3), 201-213.
Costes, S. V., Daelemans, D., Cho, E. H., Dobbin, Z., Pavlakis, G., & Lockett, S. (2004). Automatic
and quantitative measurement of protein-protein colocalization in live cells. Biophysical
journal, 86(6), 3993-4003.
Cutrale, F., Trivedi, V., Trinh, L. A., Chiu, C.-L., Choi, J. M., Artiga, M. S., & Fraser, S. E. (2017).
Hyperspectral phasor analysis enables multiplexed 5D in vivo imaging. Nature methods,
14(2), 149-152.
Dana, H., Mohar, B., Sun, Y., Narayan, S., Gordus, A., Hasseman, J. P., . . . Patel, R. (2016).
Sensitive red protein calcium indicators for imaging neural activity. elife, 5, p.e12727.
Datta, R., Alfonso-García, A., Cino, R., & Gratton, E. (2015). Fluorescence lifetime imaging of
endogenous biomarker of oxidative stress. Scientific reports, 5(1), 1-10.
Datta, R., Heaster, T. M., Sharick, J. T., Gillette, A. A., & Skala, M. C. (2020). Fluorescence
lifetime imaging microscopy: fundamentals and advances in instrumentation, analysis,
and applications. Journal of biomedical optics,, 25(7), 071203.
DeBerardinis, R. J. (2012). Cellular metabolism and disease: what do metabolic outliers teach
us? Cell, 148(6), 1132-1144.
Demchenko, A. P. (2020). Photobleaching of organic fluorophores: quantitative
characterization, mechanisms, protection. Methods and Applications in Fluorescence,
8(2), 022001.
Denk, W., Strickler, J. H., & Webb, W. W. (1990). Two-photon laser scanning fluorescence
microscopy. Science, 248(4951), 73-76.
Digman, M. A., Caiolfa, V. R., Zamai, M., & Gratton, E. (2008). The phasor approach to
fluorescence lifetime imaging analysis. Biophysical journal, 94(2), L14-L16.
Domenick, T. M., Gill, E. L., Vedam-Mai, V., & Yost, R. A. (2020). Mass spectrometry-based
cellular metabolomics: current approaches, applications, and future directions.
Analytical Chemistry, No 1: 546-566.
Du, C., & Gao, S. (2017). Image segmentation-based multi-focus image fusion through multi-
scale convolutional neural network. IEEE access, 5, 15750-15761.
Farkas, D. L., Wei, M. D., Febbroriello, P., Carson, J. H., & Loew, L. M. (1989). Simultaneous
imaging of cell and mitochondrial membrane potentials. Biophys J., 56(6), 1053-1069.
Fine, A., Amos, W. B., Durbin, R. M., & McNaughton, P. A. (1988). Confocal microscopy:
applications in neurobiology. Trends in neurosciences, 11(8), 346-351.
129

Franke, W. W., Applehans, B., Schmid, E., Freudenstein, C., Osborn, M., & Weber, K. (1979).
Identification and characterization of epithelial cells in mammalian tissues by
immunofluorescence microscopy using antibodies to prekeratin. Differentiation, 15(1‐3),
7-25.
Gadella Jr, T. W., Jovin, T. M., & Clegg, R. M. (1993). Fluorescence lifetime imaging microscopy
(FLIM): spatial resolution of microstructures on the nanosecond time scale. Biophysical
chemistry,, 48(2), 221-239.
Gómez-Varela, A. I., Stamov, D. R., Miranda, A., Alves, R., Barata-Antunes, C., Dambournet,
D., . . . De Beule, P. A. (2020). Simultaneous co-localized super-resolution fluorescence
microscopy and atomic force microscopy: combined SIM and AFM platform for.
Scientific reports, 10(1), 1-10.
Gross, G. G., Junge, A. J., Mora, R. J., Kwon, H.-B., Olson, C., Takahashi, T. T., . . . Arnold, D. B.
(2013). Recombinant probes for visualizing endogenous synaptic proteins in living
neurons. Neuron, 78(6), 971-985.
Gustafsson, M. G. (1999). Extended resolution fluorescence microscopy. Current opinion in
structural biology, 9(5), 627-628.
Gutierrez, G., Gromada, J., & Sussel, L. (2017). Heterogeneity of the pancreatic beta cell.
Frontiers in genetics, 8, 22.
Haralick, R. M., & Shapiro, L. G. (1985). Image segmentation techniques. Computer vision,
graphics, and image processing, 29(1), 100-132.
Hedeskov, C. J. (1980). Mechanism of glucose-induced insulin secretion. Physiological reviews,
60(2), 442-509.
Hell, S. W., & Wichmann, J. (1994). Breaking the diffraction resolution limit by stimulated
emission: stimulated-emission-depletion fluorescence microscopy. Optics letters, 780-
782.
Hesamian, M. H., Jia, W., He, X., & Kennedy, P. (2019). Deep learning techniques for medical
image segmentation: achievements and challenges. Journal of digital imaging, 32(4),
582-596.
Junot, C., Fenaille, F., Colsch, B., & Bécher, F. (2014). High resolution mass spectrometry based
techniques at the crossroads of metabolic pathways. Mass spectrometry reviews, 33(6),
471-500.
Kalies, S., Kuetemeyer, K., & Heisterkamp, A. (2011). Mechanisms of high-order photobleaching
and its relationship to intracellular ablation. Biomedical optics express, 2(4), 805–816.
130

Ker, J., Wang, L., Rao, J., & Lim, T. (2017). Deep learning applications in medical image analysis.
IEEE Access, 6, 9375-9389.
Khetani, S. R., & Bhatia, S. N. (2008). Microscale culture of human liver cells for drug
development. Nature biotechnology, 26(1), 120-126.
Kingsbury, N. (1998). The dual-tree complex wavelet transform: a new technique for shift
invariance and directional filters. IEEE digital signal processing workshop, Vol. 86, pp.
120-131.
Kingsbury, N. (2000). A dual-tree complex wavelet transform with improved orthogonality and
symmetry properties. Proceedings 2000 international conference on image processing,
Vol. 2, pp. 375-378.
Klonis, N., Rug, M., Harper, I., Wickham, M., Cowman, A., & Tilley, L. (2002). Fluorescence
photobleaching analysis for the study of cellular dynamics. European Biophysics Journal,
31(1), 36-51.
Laissue, P. P. (2017). Assessing phototoxicity in live fluorescence imaging. Nature methods,
14(7), 657-661.
Lakowicz, J. R. (1999). Introduction to fluorescence. In Principles of fluorescence spectroscopy.
Boston, MA.: Springer, .
Lakowicz, J. R. (1999). Principles of Fluorescence Spectroscopy, 2nd Ed. New York, London,
Moscow, Dordrecht: Kluwer Academic/Plenum Publishers,.
Lakowicz, J. R., Szmacinski, H., Nowaczyk, K., & Johnson, M. L. (1992). Fluorescence lifetime
imaging of free and protein-bound NADH. Proceedings of the National Academy of
Sciences, 89(4), 1271-1275.
Lang, K., Davis, L., Wallace, S., Mahesh, M., Cox, D. J., Blackman, M. L., . . . Chin, J. W. (2012).
Genetic encoding of bicyclononynes and trans-cyclooctenes for site-specific protein
labeling in vitro and in live mammalian cells via rapid fluorogenic Diels–Alder reactions.
Journal of the American Chemical Society, 134(25), 10317-10320.
Lichtman, J. W., & Conchello, J. A. (2005). Fluorescence microscopy. Nature methods,, 2(12),
910-919.
Lippincott-Schwartz, J., Altan-Bonnet, N., & Patterson, G. H. (2003). Photobleaching and
photoactivation: following protein dynamics in living cells. Nature cell biology, S7-14.
Luisier, F., Vonesch, C., Blu, T., & Unser, M. (2009). Fast Haar-wavelet denoising of
multidimensional fluorescence microscopy data. EEE International Symposium on
Biomedical Imaging: From Nano to Macro, pp. 310-313.
131

Ma, N., Digman, M. A., Malacrida, L., & Gratton, E. (2016). Measurements of absolute
concentrations of NADH in cells using the phasor FLIM method. Biomedical optics
express, 7(7), 2441-2452.
Maddipatla, R., & Tankam, P. (2020). Bleed-through elimination method in a dual-channel
fluorescence microscopy system. Multiphoton Microscopy in the Biomedical Sciences XX,
Vol. 11244, pp. 96-100.
Maechler, P., Carobbio, S., & Rubi, B. (2006). In beta-cells, mitochondria integrate and generate
metabolic signals controlling insulin secretion. The international journal of biochemistry
& cell biology, 38(5-6), 696-709.
Marchetti, P., Bugliani, M., Lupi, R., Marselli, L., Masini, M., Boggi, U., . . . Cnop, M. (2007). The
endoplasmic reticulum in pancreatic beta cells of type 2 diabetes patients. Diabetologia,
50(12), 2486-2494.
Maunsbach, A. B. (1967). Phase contrast and interference microscopy for cell biologists.
Minaee, S., Boykov, Y. Y., Porikli, F., Plaza, A. J., Kehtarnavaz, N., & Terzopoulos, D. (2021).
Image segmentation using deep learning: A survey. IEEE transactions on pattern analysis
and machine intelligence.
Mookerjee, S. A., Gerencser, A. A., Nicholls, D. G., & Brand, M. D. (2017). Quantifying
intracellular rates of glycolytic and oxidative ATP production and consumption using
extracellular flux measurements. Journal of Biological Chemistry, 292(17), 7189-7207.
Morton, D. F. (1970). Immunological factors which influence response to immunotherapy in
malignant melanoma. Surgery , Surgery, 68(1), 158-63.
Nielsen, J., & Keasling, J. D. (2016). Engineering cellular metabolism. Cell, 164(6), 1185-1197.
Nwaneshiudu, A., Kuschal, C., Sakamoto, F. H., Anderson, R. R., Schwarzenberger, K., & Young,
R. C. (2012). Introduction to confocal microscopy. Journal of Investigative Dermatology,
132(12), 1-5.
Pan, L., Yan, R., Li, W., & Xu, K. (2018). Super-resolution microscopy reveals the native
ultrastructure of the erythrocyte cytoskeleton. Cell reports, 22(5), 1151-1158.
Paredes, R. M., Etzler, J. C., Watts, L. T., Zheng, W., & Lechleiter, J. D. (2008). Methods. Chemical
calcium indicators., 46(3), 143-151.
Patterson, G., Day, R. N., & Piston, D. (2001). Fluorescent protein spectra. Journal of cell science,
114(5), 837-838.
Pédelacq, J. D., Cabantous, S., Tran, T., Terwilliger, T. C., & Waldo, G. S. (2006). Engineering and
characterization of a superfolder green fluorescent protein. Nature biotechnology,
24(1), 79-88.
132

Pipeleers, D., & Ling, Z. (1992). Pancreatic beta cells in insulin-dependent diabetes. Diabetes
Metabolism Reviews, 8, 209-209.
Pirman, D. A., Efuet, E., Ding, X., Pan, Y., Yan, L., Fischer, S. M., . . . Yang, P. (2013). Changes in
cancer cell metabolism revealed by direct sample analysis with MALDI mass
spectrometry. PloS one 8, 8(4), e61379.
Pluta, M. (1994). Nomarski's DIC microscopy: a review. Phase contrast and differential
interference contrast imaging techniques and applications, 1846, 10-25.
Raghav, M., & Raheja, S. (2014). Image denoising techniques: literature review. nternational
Journal of engineering and computer science, 3(5).
Raymo, F. M. (2012). Photoactivatable synthetic dyes for fluorescence imaging at the
nanoscale. The journal of physical chemistry letters, 3(17), 2379-2385.
Ren, J., Zhang, A., Kong, L., & Wang, X. (2018). Advances in mass spectrometry-based
metabolomics for investigation of metabolites. RSC advances, 8(40), 22335-22350.
Roberts-Dalton, H. D., Cocks, A., Falcon-Perez, J. M., Sayers, E. J., Webber, J. P., Watson, P., . . .
Jones, A. T. (2017). Fluorescence labelling of extracellular vesicles using a novel thiol-
based strategy for quantitative analysis of cellular delivery and intracellular traffic.
Nanoscale, 9(36), 13693-13706.
Rodrigues, I., Sanches, J., & Bioucas-Dias, J. (2008). Denoising of medical images corrupted by
Poisson noise. 2008 15th IEEE International Conference on Image Processing, pp. 1756-
1759.
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical
image segmentation. International Conference on Medical image computing and
computer-assisted intervention, pp. 234-241.
Rovira, A. G., & Smith, A. G. (2019). PPR proteins–orchestrators of organelle RNA metabolism.
P. hysiologia plantarum,, 166(1), 451-459.
Schaefer, L. H., Schuster, D., & Herz, H. (2001). Generalized approach for accelerated maximum
likelihood based image restoration applied to three‐dimensional fluorescence
microscopy. Journal of microscopy, 204(2), 99-107.
Schermelleh, L. F. (2019). Super-resolution microscopy demystified. Nature cell biology, 21(1),
72-84.
Shimomura, O., Johnson, F. H., & Saiga, Y. (1962). Extraction, purification and properties of
aequorin, a bioluminescent protein from the luminous hydromedusan, Aequorea.
Journal of cellular and comparative physiology, 59(3), 223-239.
133

Shotton., M. (1989). Confocal scanning optical microscopy and its applications for biological
specimens. Journal of Cell Science, 94(2), 175-206.
Shrivastava, A., & Gupta, V. B. (2011). Methods for the determination of limit of detection and
limit of quantitation of the analytical methods. Chron. Young Sci, 2(1), 21-25.
Stracke, F., Heupel, M., & Thiel, E. (1999). Singlet molecular oxygen photosensitized by
rhodamine dyes: Correlation with photophysical properties. J. Photochem., Photobiol. A
126:51–58.
Stringari, C., Edwards, R. A., Pate, K. T., Waterman, L. M., Donovan, P. J., & Gratton, E. (2012).
Metabolic trajectory of cellular differentiation in small intestine by Phasor Fluorescence
Lifetime Microscopy of NADH. Scientific reports, 2(1), 1-9.
Swinkels, M., Slotman, J. A., Leebeek, F. W., Voorberg, J., Bierings, R., & Jansen, G. (2019).
Super-Resolution Immunofluorescence Imaging of Platelet Granules. Blood,, 134, 3613.
Tang, C., Iwahara, J., & Clore, M. G. (2006). Visualization of transient encounter complexes in
protein–protein association. Nature, 444(7117), 383-386.
TeSlaa, T., & Teitell, M. A. (2014). Techniques to monitor glycolysis. Methods in enzymology,
542: 91-114.
Thorn, K. (2016). A quick guide to light microscopy in cell biology. Molecular biology of the cell,
27(2), 219-222.
Tirlapur, U. K., Konig, K., Peuckert, C., Krieg, R., & Halbhuber, K. J. (2001). Femtosecond near-
infrared laser pulses elicit generation of reactive. Exp. Cell Res., 263:88-97.
Tsien, R. Y. (1980). New calcium indicators and buffers with high selectivity against magnesium
and protons: design, synthesis, and properties of prototype structures. Biochemistry,
19(11), 2396-2404.
Van Schravendijk, C. F., Kiekens, R., & Pipeleers, D. G. (1992). Pancreatic beta cell heterogeneity
in glucose-induced insulin secretion. Journal of Biological Chemistry, 267(30), 21344-
21348.
Vetterli, L., Brun, T., Giovannoni, L., Bosco, D., & Maechler, P. (2011). Resveratrol potentiates
glucose-stimulated insulin secretion in INS-1E β-cells and human islets through a SIRT1-
dependent mechanism. Journal of Biological Chemistry, 286(8), 6049-6060.
Vicidomini, G., Bianchini, P., & Diaspro, A. (2018). STED super-resolved microscopy. Nature
methods, 15(3), 173-182.
Wang, L., Feri, M., Salim, A., & Jahnsson, K. (2018). Small-molecule fluorescent probes for live-
cell super-resolution microscopy. Journal of the American Chemical Society, 141(7),
2770-2781.
134

Wang, P., Hecht, F., Ossato, G., Tille, S., Fraser, S. E., & Junge, J. A. (2021). Complex wavelet
filter improves FLIM phasors for photon starved imaging experiments. Biomedical Optics
Express, 12(6), 3463-3473.
Wang, S., Shen, A., Setlow, P., & Li, Y. Q. (2015). Characterization of the dynamic germination of
individual Clostridium difficile spores using Raman spectroscopy and differential
interference contrast microscopy. Journal of bacteriology, 197(14), 2361-2373.
Wang, X. F., Periasamy, A., Herman, B., & Coleman, D. M. (1992). Fluorescence lifetime imaging
microscopy (FLIM): instrumentation and applications. Critical Reviews in Analytical
Chemistry,, 23(5), 369-395.
Wang, Z., Gurlo, T., Matveyenko, A. V., Elashoff, D., Wang, P., Rosenberger, M., . . . Butler, P. C.
(2021). Live-cell imaging of glucose-induced metabolic coupling of β and α cell
metabolism in health and type 2 diabetes. Communications biology, 4(1), pp.1-11.
Welch, G. R. (1978). On the role of organized multienzyme systems in cellular metabolism: a
general synthesis. Progress in biophysics and molecular biology, 103-191.
Wilkinson, F., McGarvey, D. J., & Olea, A. F. (1994). Excited Triplet State Interactions with
Molecular Oxygen: Influence of Charge Transfer on the Bimolecular Quenching Rate
Constants and the Yields of Singlet Oxygen [O*2(1.DELTA.g)] for Substituted
Naphthalenes in Various Solvents. The Journal of Physical Chemistry,, 98 (14). 3762-
3769.
Wolf, D. E. (2007). Fundamentals of fluorescence and fluorescence microscopy. Methods in cell
biology, 81, 63-91.
Xu, G., Stoffers, D. A., Habener, J. F., & Bonner-Weir, S. (1999). Exendin-4 stimulates both beta-
cell replication and neogenesis, resulting in increased beta-cell mass and improved
glucose tolerance in diabetic rats. Diabetes, 48(12), 2270-2276.
Zhao, S., Xu, W., Jiang, W., Yu, W., Lin, Y., Zhang, T., & Yao, J. (2010). Regulation of cellular
metabolism by protein lysine acetylation. Science, 327(5968), 1000-1004.
Zhu, L., Zhao, Q., Yang, T., Ding, W., & Zhao, Y. (2015). Cellular metabolism and macrophage
functional polarization. International reviews of immunology, 34(1), 82-100.
Zinser, E. R., Paltauf, F. R., & Daum, G. (1993). Sterol composition of yeast organelle membranes
and subcellular distribution of enzymes involved in sterol metabolism. Journal of
bacteriology, 175(10), 2853-2858.

Abstract (if available)

Abstract Fluorescence Lifetime Imaging Microscopy (FLIM) is a fluorescence microscopy modality that measures the time for fluorophores to emit fluorescence once excited, defined as the fluorescence lifetime. FLIM is a powerful tool for metabolic imaging. It measures the lifetime of Nicotinamide Adenine Dinucleotide plus Hydrogen (NADH), a coenzyme central to metabolism whose lifetime changes with cellular metabolic states. FLIM provides spatial resolution sufficient to resolve subcellular components. Yet, due to three significant hurdles, most researchers do not extract metabolic information at a subcellular level with FLIM: First, FLIM signals in previous research are not accurately converted to meaningful metabolic readings that account for the various subcellular components. Second, noise carried by FLIM signal buries the metabolic information, and conventional post-imaging filters for removing the noise do not preserve lifetime readings in small cellular structures. Third, different subcellular components in FLIM datasets are not effectively segmented for subcellular analysis. This thesis tackles these problems. We first introduce an analysis workflow for FLIM metabolic readings that considers the different binding enzymes in various subcellular components. Two filtering methods are presented that preserve accurate lifetime readings in delicate cellular structures. Further, machine learning is used to segment individual cells and subcellular components to extract their metabolic trends independently. These tools provide a workflow that reveals the metabolic dynamics in different subcellular components. Our workflow is applied to INS-1E cells to analyze subcellular metabolic responses of INS-1E cells during glucose-induced insulin release. This workflow can be easily modified to serve as a template for other similar research projects to accurately analyze cellular metabolism.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Extending multiplexing capabilities with lifetime and hyperspectral fluorescence imaging

PDF

Metabolic profiling of single hematopoietic stem cells for developing novel ex vivo culture strategies

PDF

Multiplexing live 5d imaging with multispectral fluorescence: Advanced unmixing through simulation and machine learning

PDF

Hyperspectral phasor for multiplexed fluorescence microscopy and autofluorescence-based pathologic diagnosis

PDF

Genome-scale modeling of macrophage activity in the colorectal cancer microenvironment

PDF

Neural network integration of multiscale and multimodal cell imaging using semantic parts

PDF

Non-invasive live-cell imaging for monitoring and evaluating pancreatic islet and beta cell metabolism

PDF

Analytical tools for complex multidimensional biological imaging data

PDF

Machine learning based techniques for biomedical image/video analysis

PDF

Coding centric approaches for efficient, scalable, and privacy-preserving machine learning in large-scale distributed systems

PDF

Scaling up temporal graph learning: powerful models, efficient algorithms, and optimized systems

PDF

Development of a quantitative proteomic molecular imaging platform using multiplexed nanoscale Raman scattering contrast agents

PDF

Parylene-based biomems sensors for multiple physiological systems

PDF

Ultrasound neuromodulation and its applications for noninvasive vision restoration

PDF

Development of a toolbox for global functional brain imaging of wake and sleep states in zebrafish

PDF

Development of an integrated biomechanics informatics system (IBIS) with knowledge discovery and decision support tools based on imaging informatics methodology

PDF

The role of the environment around the chromophore in controlling the photophysics of fluorescent proteins

PDF

Simulation and machine learning at exascale

PDF

Electron transfer capability and metabolic processes of the genus Shewanella with applications to the optimization of microbial fuel cells

PDF

Deep learning architectures for characterization and forecasting of fluid flow in subsurface systems

Asset Metadata

Creator Wang, Peiyu (author)

Core Title Machine learning and image processing of fluorescence lifetime imaging microscopy enables tracking and analysis of subcellular metabolism

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Biomedical Engineering

Degree Conferral Date 2022-12

Publication Date 10/05/2022

Defense Date 12/03/2021

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag fluorescence lifetime imaging,image processing,machine learning,OAI-PMH Harvest,subcellular metabolism

Format theses (aat)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Fraser, Scott (committee chair), White, Kate (committee member), Zhou, Qifa (committee member)

Creator Email peiyuwan@usc.edu,pw385@cornell.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC112114346

Unique identifier UC112114346

Identifier etd-WangPeiyu-11262.pdf (filename)

Legacy Identifier etd-WangPeiyu-11262

Document Type Dissertation

Format theses (aat)

Rights Wang, Peiyu

Internet Media Type application/pdf

Type texts

Source 20221017-usctheses-batch-986 (), University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu