University of Southern California Dissertations and Theses

Parametric and non‐parametric modeling of autonomous physiologic systems: applications and multi‐scale modeling of sepsis

(USC Thesis Other)

Parametric and non‐parametric modeling of autonomous physiologic systems: applications and multi‐scale modeling of sepsis

PDF

Download

Open document

Flip pages

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content PARAMETRIC AND NON-PARAMETRIC MODELING OF AUTONOMOUS PHYSIOLOGIC SYSTEMS: APPLICATIONS AND MULTI-SCALE MODELING OF SEPSIS by Steffen E. Eikenberry A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (BIOMEDICAL ENGINEERING) August 2014 Copyright 2014 Steffen E. Eikenberry Acknowledgements I would like to thank my advisor, Vasilis Marmarelis, for many enlightening conversations and unwavering support. He has, to an unusual degree, nurtured intellectual independence and free inquiry. I would also like to thank Dr. Annie Wong-Beringer and Marc Cole for their generous provisions of data. Further thanks are in order for my Qualifier and Defense committee members, Dr. Norberto Grzywacz, Dr. David D’Argenio, Dr. Annie Wong-Beringer, and Dr. Michael Khoo. IwouldberemissifIdidnotacknowledgeMischalgraceDiasantaforhercheerfulhelpandpatience. Finally, I must express my profound gratitude to Lindsey van Sambeek, for her infinite patience and gentle good humour during the completion of this document. i Contents Acknowledgements i List of Tables v List of Figures vi Abstract xi 1 Introduction 1 2 Recursive Volterra Methodology 7 2.1 Introduction to the Volterra series . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Discrete Volterra series and the basis expansion technique . . . . . . . . . . . . . . . 8 2.2.1 Extension to multiple inputs and outputs . . . . . . . . . . . . . . . . . . . . 13 2.3 Recursive generalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.3.1 Auto-recursive (nonlinear autoregressive) extension . . . . . . . . . . . . . . . 15 2.3.2 Extension to multiple recursive outputs . . . . . . . . . . . . . . . . . . . . . 18 2.4 The Principal Dynamic Mode (PDM) concept . . . . . . . . . . . . . . . . . . . . . . 20 2.5 Novel data issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.5.1 Multiple trial/multiple subject data . . . . . . . . . . . . . . . . . . . . . . . 24 2.5.2 Amethodforidentifyingmultipledynamicsub-systemsfrommulti-trial/subject data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.5.3 Sparse and irregular data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.6 Bootstrap statistical procedure for structural variable selection and model comparison 31 2.6.1 The notion of conditional prediction as a foundation . . . . . . . . . . . . . . 31 2.6.2 Bootstrap procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.6.3 Input selection and nonlinear generalization of Granger Causality . . . . . . . 34 2.6.4 Kernel reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 2.6.5 Basis and coefficient reduction . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3 Single-neuron dynamics 43 3.1 Auto-recursive (autoregressive) Volterra modeling of the Hodgkin-Huxley equations . 44 3.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.1.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.1.3 Model parametrization/training . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.1.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 ii 3.1.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 3.2 Principal dynamic mode analysis of the Hodgkin-Huxley equations . . . . . . . . . . 78 3.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 3.2.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 3.2.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 3.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 3.4 Integrated single neuron with point-process inputs/output . . . . . . . . . . . . . . . 113 3.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 3.4.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 3.4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 4 Smooth muscle dynamics 122 4.1 Auto-recursive representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 4.2 Exponential relaxation characteristic . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 4.3 Optimizing force generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 5 The Riccati equation and other growth laws 130 5.0.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 5.0.2 Auto-recursive Volterra representations . . . . . . . . . . . . . . . . . . . . . 134 6 Recursive Volterra model for endotoxemia 140 6.1 Introduction and methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 6.2 Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 6.2.1 Model behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 6.2.2 MIMRO Volterra representation . . . . . . . . . . . . . . . . . . . . . . . . . 143 7 Sepsis 149 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 7.2 Biological and historical background . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 7.2.1 Clinical overview and epidemiology . . . . . . . . . . . . . . . . . . . . . . . . 156 7.2.2 Historical background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 7.2.3 Cytokines for predicting outcome in sepsis . . . . . . . . . . . . . . . . . . . . 163 7.2.4 Pathophysiology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 7.3 Existing models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 7.3.1 Sepsis- or inflammation-specific models . . . . . . . . . . . . . . . . . . . . . 171 7.3.2 A note on parametrization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 7.4 A hierarchy of parametric models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 7.4.1 First echelon: bacteria-phagocyte dynamics in vitro . . . . . . . . . . . . . . 179 7.4.2 Second echelon: basic models for non-specific immunity . . . . . . . . . . . . 184 7.4.3 Third echelon: Extended models of non-specific immunity . . . . . . . . . . . 219 7.4.4 Cytokines in the well-mixed setting . . . . . . . . . . . . . . . . . . . . . . . . 234 7.5 Antibiotic chemotherapy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 7.5.1 PK/PD Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 7.5.2 Antibiotic therapy in the well-mixed setting . . . . . . . . . . . . . . . . . . . 255 7.6 Spatial and geometric models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 7.6.1 Reaction-diffusion framework . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 iii 7.6.2 Cell motility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 7.6.3 Multi-patch model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278 7.6.4 Systemic model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290 7.7 Overall discussion and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 References 318 iv List of Tables 3.1 Hodgkin-Huxley model parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 7.1 All baseline parameters and baseline value ranges for the predator-prey-based model hierarchy up to the baseline neutrophil model.. . . . . . . . . . . . . . . . . . . . . . 218 7.2 Two-compartmentciprofloxacinpharmacokineticparametersfromtwodifferentstud- ies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 7.3 Estimated pharmacodynamic parameters for the effect of ciprofloxacin on three bac- terial species. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254 7.4 Baselineorganweightsaspercentageoftotalbodyweightandperfusionsaspercent- age of cardiac output. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 7.5 Systemic model-specific parameters and baseline values. . . . . . . . . . . . . . . . . 297 v List of Figures 2.1 Modular representation of the SISO modified discrete Volterra model. . . . . . . . . 11 2.2 The two basic functional bases for kernel expansion. . . . . . . . . . . . . . . . . . . 12 2.3 Modular representation for a two-input, two-output (non-recursive) Volterra model. 15 2.4 Example recursive model architectures.. . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.5 Example PDM-based model architecture for a second-order SISO model with two PDM basis functions.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.6 Example of a two-input, single-output PDM model architecture where each input has two associated PDM basis functions. . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.7 Example of concatenating three trials of input-output data. . . . . . . . . . . . . . . 26 2.8 Test system with four sub-systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.9 Identification of four sub-systems via k-means clustering applied to coefficient space. 30 2.10 Illustration of bootstrap procedure for comparing two models. . . . . . . . . . . . . . 35 3.1 Modular representation of the NARV model. . . . . . . . . . . . . . . . . . . . . . . 51 3.2 NARV model performance for a variety of model orders. . . . . . . . . . . . . . . . . 56 3.3 NARV model performance as assessed by the NMSE for a variety of model orders. . 56 3.4 NARVmodelperformanceforthefullysecondandthird-ordermodelsusingdifferent values of α y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 3.5 Voltage tracings giving the second-order NARV model prediction and actual data for the σ 32 and σ 4 testing data-sets. . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.6 Comparisonofthesecond-orderNARVmodel-predictedspiketrainsversustheactual spike trains for all four testing data-sets. . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.7 Predicted and actual interspike time histograms under the second-order NARV model. 61 3.8 NARV model response to 1 ms wide current pulses. . . . . . . . . . . . . . . . . . . . 62 3.9 NARV model response to constant current injections. . . . . . . . . . . . . . . . . . . 64 3.10 AP firing frequency and interspike intervals under the NARV model as a function of injected current magnitude. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 3.11 NumericalexperimentdemonstratingthattheNARVmodelsuccessfullypredictsthe existence of absolute and relative refractory periods. . . . . . . . . . . . . . . . . . . 67 3.12 The first and second-order self-kernels (k 1;0 and k 2;0 ) for the exogenous input, x(m). 68 3.13 The first and second-order autoregressive self-kernels (k 0;1 and k 0;2 ) for the autore- gressive input, ˆ y(m). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 3.14 The second-order cross-kernel, k 1;1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 3.15 NARV model performance on noisy data-sets as assessed by the coincidence factor. . 70 3.16 Voltagetracingsfornoisytestingdata-setsandsecond-orderNARVmodelpredictions. 71 3.17 Example PDM model architecture for two forward and feedback PDMs. . . . . . . . 84 vi 3.18 The five forward and feedback PDMs ordered by significance. . . . . . . . . . . . . . 91 3.19 Pruned 8-P PDM model structure obtained under coefficient pruning of the 27-P PDM model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 3.20 The four surviving terms under direct pruning of the full 65-P PDM model, yielding the maximally reduced 4-P model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 3.21 PerformancemetricsforN-coefficientPDMmodelsunderthefulland2-4PDMbasis sets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 3.22 Out-of-sample membrane potential predictions for the 27-P, 8-P, and 4-P models. . . 97 3.23 The bootstrap distributions of the mean Γ, NRMSE, TPR, and FPR for the full (65-P), 27-P, 8-P, and 4-P models run on the full testing data-set. . . . . . . . . . . 98 3.24 ROC curves for the full (65-P), 27-P, 8-P, and 4-P models. . . . . . . . . . . . . . . . 99 3.25 Coincidence factor and K-statistic as a function of the threshold for spike detection. 100 3.26 The interspike time histograms and CDFs for the full testing data-set and model predictions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 3.27 Selected Volterra kernels reconstructed from full and reduced PDM models. . . . . . 102 3.28 Results from the linear minimal model.. . . . . . . . . . . . . . . . . . . . . . . . . . 103 3.29 PDM coefficient values as a function of the underlying H-H system’s maximum sodium conductance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 3.30 PDMcoefficientvaluesasafunctionoftheunderlyingH-Hsystem’smaximumpotas- sium conductance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 3.31 Structure of the coefficient-pruned NARV model. . . . . . . . . . . . . . . . . . . . . 109 3.32 Conceptualschematicfortwo-stepcascademodelofAPgenerationinamodelneuron in response to four spike-train dendritic inputs. . . . . . . . . . . . . . . . . . . . . . 114 3.33 Given a system with hidden recursion, the two alternative Volterra models for the system are a strict input-output model, or an auto-recursive model.. . . . . . . . . . 118 3.34 Currents generated by point-process inputs in a single-neuron model system. . . . . 119 3.35 Comparative model performance on single-neuron point-process system. . . . . . . . 120 3.36 Out-of-sample performance under different models when one, two, three, or four out of four inputs are known. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 4.1 Segment of input-output smooth muscle data, with input, x(n), being electrical current and output, y(n), being contractile force. . . . . . . . . . . . . . . . . . . . . 123 4.2 Out-of-sample model predictions for mollusk smooth muscle dynamics. . . . . . . . . 124 4.3 Smooth muscle auto-recursive model kernels. . . . . . . . . . . . . . . . . . . . . . . 125 4.4 Auto-recursive PDM-based model architecture for mollusk smooth muscle. . . . . . . 125 4.5 Smooth muscle force response to increasing levels of tonic stimulus, under the PDM- based model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 4.6 Smooth muscle force generation under different duty cycle frequencies. . . . . . . . . 129 4.7 Integratedforce:powerratioasafunctionofpulsewidthforthreeselecteddutycycle frequencies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 5.1 Performance of the three-kernel model for the Riccati system. . . . . . . . . . . . . . 136 5.2 Out-of-sample predictions of the recursive Volterra model for different generalized Riccati systems.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 6.1 Schematic representation of parametric endotoxemia model. . . . . . . . . . . . . . . 142 vii 6.2 Parametric endotoxemia model response to three different LPS infusion schedules. . 143 6.3 Normalized peak TNF and IL-10 levels in response to different bolus infusions of LPS.144 6.4 Endotoxemia model TNF response when IL-10 is blocked. . . . . . . . . . . . . . . . 144 6.5 Bootstrap distributions of the NRMSE for TNF and IL-10 on out-of-sample predic- tions under three classes of Volterra model. . . . . . . . . . . . . . . . . . . . . . . . 145 6.6 Out-of-sample predictions of TNF and IL-10 on three concatenated trials (with SNR dB = 25) for the SIMRO endotoxemia model. . . . . . . . . . . . . . . . . . . . 145 6.7 Functional connectivity structure among the observable variables LPS, TNF, and IL-10 inferred from SIMRO model reduction. . . . . . . . . . . . . . . . . . . . . . . 146 6.8 Results of blocking IL-10 under the recursive Volterra model. . . . . . . . . . . . . . 147 6.9 Example of sampled and interpolated LPS, TNF, and IL-10 time-series. . . . . . . . 148 7.1 Schematic for the overall hierarchical model-building strategy. . . . . . . . . . . . . . 154 7.2 Schematic progression of local response to infectious insult leading to the four car- dinal signs of inflammation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 7.3 Asymptotic outcomes under mass-action and saturable killing terms for phagocyte- bacteria interaction in vitro. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 7.4 Summaryofhowdynamicsvaryunderdifferentformalchoicesforphagocyte-dependent bacteria killing in vitro. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 7.5 The three Holling functional responses. . . . . . . . . . . . . . . . . . . . . . . . . . . 191 7.6 Cartoon for the baseline predator-prey model. . . . . . . . . . . . . . . . . . . . . . . 194 7.7 Thetwoclassesofdynamicsforthebaselinepredator-preymodelwithtypeIIHolling functional response, demonstrated in the phase-plane. . . . . . . . . . . . . . . . . . 195 7.8 Schematicmodelwithpredator-preytypedynamicsandTNF-mediatedpositivefeed- back. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 7.9 Bifurcation diagrams for the TNF-only model. . . . . . . . . . . . . . . . . . . . . . 200 7.10 Phase plots for the TNF-only model. . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 7.11 Phase portraits and bifurcation diagrams for the TNF-only model under varying n. . 202 7.12 Schematic illustration of the TNF+IL-10 system and the hypothesized roles of IL-10.206 7.13 Bifurcation diagrams for the TNF+IL-10 model. . . . . . . . . . . . . . . . . . . . . 208 7.14 Numerical experiment simulating exogenous IL-10 infusion at 72 hours following initial infection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 7.15 The effect of blocking IL-10 under simulated LPS infusion on TNF levels for lung- and liver-specific macrophage reserves. . . . . . . . . . . . . . . . . . . . . . . . . . . 209 7.16 Schematic illustration of the baseline neutrophil model. . . . . . . . . . . . . . . . . 213 7.17 Time-series for infection under the baseline neutrophil model resulting in resolution. 214 7.18 Time-seriesforinfectionunderthebaselineneutrophilmodelresultinginoverwhelm- ing infection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 7.19 Response of model variables to re-infection under the baseline neutrophil model. . . 216 7.20 Simulatedre-infection under the neutrophil+apoptosis model demonstrates a hyper- followed by a hypo-immune phase. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 7.21 Relative peak responses of model variables following re-infection under the neu- trophil+apoptosis model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 7.22 Time-series for infections resulting in resolution and overwhelming infection under the neutrophil+apoptosis model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 viii 7.23 Cytokine time-courses and peak values under variations of the initial infectious load, B 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 7.24 Time-series for the IL-10:TNF ratio under different B 0 values. . . . . . . . . . . . . . 236 7.25 Median TNF, IL-10, and IL-10:TNF values for different B 0 at initial presentation and 72 hours later. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 7.26 Replication of Figure 7.25, but with a logarithmic scale for the y-axes. . . . . . . . . 239 7.27 Median initial and 72-hour TNF, IL-10, and IL-10:TNF values divided into bacterial clearing and persisting classes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240 7.28 Median relative changes in absolute TNF, IL-10, and the IL-10:TNF ratio between presentation and 72 hours for persisting and non-persisting infection. . . . . . . . . . 240 7.29 Schematic representations of the one- and two-compartment pharmacokinetics models.244 7.30 Graphical illustration of the mutant selection window hypothesis. . . . . . . . . . . . 250 7.31 Modified sigmoid E max model fits to the (augmented) data of Hyatt et al. for three bacteria species. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254 7.32 Bacterial populations following 24 hours of ciprofloxacin treatment versus the total dose delivered and 24-hour AUC/MIC under different drug infusion schedules.. . . . 256 7.33 Simulated time-series for serum ciprofloxacin and corresponding S. aureus and P. auruginosa levels under four dose fractionation schedules. . . . . . . . . . . . . . . . 257 7.34 Bacterial load after 24 hours of treatment plotted against total dose and AUC/MIC when PK/PD parameters vary randomly. . . . . . . . . . . . . . . . . . . . . . . . . 258 7.35 Minimum daily ciprofloxacin doses for complete bacterial clearance after three days of treatment under different schedules. . . . . . . . . . . . . . . . . . . . . . . . . . . 260 7.36 The effect of treatment on bacteria differs qualitatively between immunocompetent and compromised patients. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 7.37 Timetobacterialclearance,asafunctionoftotaldoseandfractionationschedule,for immunocompetent versus immunocompromised patients when treatment is initiated at either 24 or 72 hours following infection. . . . . . . . . . . . . . . . . . . . . . . . 263 7.38 Bacterial load after 24 hours of treatment versus total dose and AUC/MIC under different permutations of treatment time and immune status. . . . . . . . . . . . . . 263 7.39 Minimum ciprofloxacin doses needed to suppress resistant mutant overgrowth for immunocompetent and compromised patients, with treatment initiated at either 24 or 72 hours following infection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264 7.40 Minimum ciprofloxacin doses needed to suppress resistant mutant overgrowth for immunocompetent patients only, with treatment initiated at either 24 or 72 hours following infection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 7.41 Exampletime-seriesofsensitiveandresistantcellssubjectedtotreatmentatdifferent times, in the presence of a competent immune response. . . . . . . . . . . . . . . . . 265 7.42 Relative peak population of an opportunistic infection, as compared to the peak when the naive host is challenged by a similar inoculum, as a function of time after treatment initiation for the primary pathogen. . . . . . . . . . . . . . . . . . . . . . 267 7.43 Example 2-D worlds for the agent-based model populated by bacteria and activated macrophages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 7.44 Example 3-D worlds for the agent-based model populated by 100 bacteria and a single activated macrophage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 7.45 Demonstrationoftheeffectofbacterialmotilityonclearanceintheagent-basedmodel.278 ix 7.46 Bacterial survival as a function of the number of activated macrophages, under the agent-based model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 7.47 Bacterial survival as a function of (initial) bacterial density, under the agent-based model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 7.48 PDE diffusion equation solution with projection onto a 21-patch geometry; ϕ is optimized to fit the multi-patch solution to the PDE projection. . . . . . . . . . . . 283 7.49 Fitting surfaces for inter-patch transition rate, ϕ. . . . . . . . . . . . . . . . . . . . . 284 7.50 Bacterialinvasioninaone-dimensionalmulti-patchgeometrythatisultimatelycleared.285 7.51 Overwhelming infection spreading from an initial focus throughout the entire one- dimensional multi-patch geometry. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 7.52 Bacterialinvasionresultinginsustainedspatialoscillationsinthemulti-patchgeometry.286 7.53 Asymptoticoutcomeswheninfectionisinitiatedinacentralpatchwithcompromised immunity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 7.54 Integrated cellular and cytokine totals as a function of time, under the scenario of complete multi-patch geometry invasion. . . . . . . . . . . . . . . . . . . . . . . . . . 289 7.55 Schematic representation of the PBPK-based system-level model. . . . . . . . . . . . 291 7.56 Example time-series for variables specific to the system-scale model when primary lung infection results in bacterial clearance. . . . . . . . . . . . . . . . . . . . . . . . 300 7.57 Example time-series for variables specific to the system-scale model when primary lung infection results in persistent, overwhelming lung infection and bacteremia. . . 301 7.58 Effect of the intrinsic rate of TNF production by activated liver macrophages on lung neutrophil infiltration and bacterial clearance. . . . . . . . . . . . . . . . . . . . 301 7.59 Maximum initial bacterial (lung) inocula that can be cleared by the innate immune response, as a function of the hepatic macrophage TNF production rate. . . . . . . . 302 7.60 Serum TNF, IL-10, and IL-10:TNF time-series under 10 different levels of bacterial shedding into circulation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 7.61 Lung bacterial and neutrophil time-series that result from treating an overwhelming lunginfectioncausingbacteremiawithantibioticsaloneorantibioticsincombination with a TNF inhibitor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304 7.62 Time to bacterial clearance and integrated lung neutrophil load under antibiotic treatment with and without TNF inhibition. . . . . . . . . . . . . . . . . . . . . . . 305 x Abstract Fundamental to physiology is the concept of homeostasis, the dynamic maintenance of an equi- librium that is, in general, in disequilibrium with the larger environment. Physiologic systems must, then, be able to respond to exogenous perturbations in the environment and to their internal state. That is, physiologic systems can be understood as a set of input-to-output transforma- tions between system variables, both exogenous and endogenous. Such systems are quite complex, and knowledge must be summarized in the form of models, either informal or formal. Formal mathematical modeling approaches can be either parametric or non-parametric. Non-parametric modeling represents an inductive approach, wherein the model form is estimated directly from ob- served time-series data. The fundamental advantage of the non-parametric approach is that it can effectively summarize very complex systems when little is known about the internal workings. The Volterra series expansion is a canonical non-parametric methodology for representing the input- output functional relationship between any two variables. However, as classically formulated, the Volterra series cannot be applied to nonlinear oscillators or chaotic systems, nor can it capture autonomous systems behaviors not directly driven by an exogenous input. Moreover, even when a strict input-output formulation is theoretically proper, internal system feedbacks or recursion can give rise to inefficiencies in the input-output functional representation. By relaxing the strict input-output requirement of the classical formulation and making the outputs recursive, we ex- pand dramatically the class of systems that are amenable to Volterra-style modeling, including those that operate strictly autonomously. We have demonstrated the method’s efficacy on several test systems. In the single-output setting, auto-recursive Volterra models can capture autonomous behaviors, such as action potential firing in neural systems or population growth in ecological sys- tems. In the multi-output case, in which outputs of interest interact dynamically, the concept of xi recursive outputs allows “closed” or “nested” loop configurations to be captured in a Volterra-style framework. Bacterial sepsis is characterized by a complex inflammatory response, and there has been sig- nificant interest in modeling this process as a dynamical system. It is believed that the pathology of sepsis is largely due to the immune response itself, which has motivated the clinical strategy of blocking pro-inflammatory cytokines. However, this strategy has uniformly failed in clinical trials, motivatingasystemsapproachtothedisorder. WehaveshownthattherecursiveVolterramethod- ology can describe both autonomous growth, e.g. that of bacteria, and can capture the dynamics of simulated endotoxemia, a simplified model for actual sepsis. Therefore, we were hopeful that it could be applied to clinical time-series data for actual sepsis patients. However, the temporal resolution of available data has proven too coarse, and thus the constraints imposed by the data have necessitated a parametric approach to modeling this system. Parametric modeling represents a deductive, hypothesis-driven methodology, and we build our parametric representation of bacte- rial infection and the immune response in a hierarchical manner. This allows a careful examination of how the incorporation of different mechanisms, and different formal representations for these mechanisms, affects overall model dynamics and predictions. This approach has generated several predictions salient with respect to both future modeling efforts and to the clinical management of sepsis. We find that the interaction of local and systemic macrophage and neutrophil populations, respectively, is essential both for efficiently mobilizing the immune system and for resolving inflam- mation. Antibiotic treatment is modeled and found to interact synergistically with the immune response; however, this synergism is weak in immunocompromised hosts. Finally, we find that an exuberant inflammatory cytokine response is the natural consequence of a severe infection or compromised immune system, and while this may contribute to systems pathology, it is appropri- xii ate with respect to clearing infection. Thus, anti-inflammatory interventions are likely to have a limited clinical role, and the mainstay of treatment must remain treating the cause, principally via antibiotics. xiii Chapter 1 Introduction Amodel,ingeneral,isaformalorinformalframeworkthatsummarizesdataand/orbeliefs. Formal modelsareexpressedinthelanguageofmathematicsandinthecaseofdynamicalsystemsmodeling, theyarebroadlydividedintotwomainmathematicalframeworks: parametricandnon-parametric. Philosophically, these frameworks are based on deductive and inductive logic, respectively. That is, parametric modeling is a hypothesis-driven process, at multiple scales, whereas non-parametric modeling is data-driven, in that the model form may be estimated directly from time-series of interest. However, the latter still requires the imposition of some structure in determining what time-series to account for, which is implicitly a hypothesis or belief, as well as practical choices concerningmethodology. Nevertheless,therequirementforpre-existinghypothesesisquiteminimal comparedtothewhollyhypothesis-drivenprocessofparametricmodeling. Indeed,thefundamental strength of non-parametric models is that they may be estimated directly from time-series data and describe complex systems with minimal knowledge of the physical mechanics of the system. Homeostasis, the ability of an organism or system to auto-regulate in the presence of environ- mental inputs and noise, is central to the concepts of health and disease. Indeed, disease is often 1 understood as a departure from, or at least a threat to, homeostasis per se. Thus, the broad class of systems that we are interested in consists of autonomously operating physiologic systems that are subjected and respond to exogenous forcing. As a practical matter, all systems are understood in terms of specified variables, either hidden or observable, and developing a model consists in specifying the interrelationships between a chosen variable set. When a clear delineation can be madebetweenexogenousforcingvariablesandoutputsendogenoustothesystem, theinput-output paradigm is of great utility. The problem of describing an input-to-output transformation may be cast in the mathematical language of functionals and functional expansions. In general, some functional, F[·], maps an “input” function, x(t ′ ),t ′ ≤t, onto an output scalar, y(t): y(t)=F[x(t ′ ),t ′ ≤t] (1.1) The Volterra series, introduced by Vito Volterra in his 1930 monograph [249], is a powerful and general method for describing a continuous functional. A Volterra series expansion represents the nonlinear dynamic input-output relationship between two variables in hierarchical convolutional form, where each hierarchy represents an order of non-linearity (see Chapter 2 for mathematical details), and has been successfully applied to a number of physiological systems (for a partial review see [164]). Furthermore, the Volterra series may easily be extended to describe the dynamic relation between multiple input and output variables. However, the classical Volterra series has several fundamental limitations that are extremely salient in the context of physiologic systems modeling, including: 1. The system must operate about a stable fixed point; systems exhibiting autonomous oscilla- 2 tions, limit cycles, or chaotic behavior do not admit a classical Volterra representation. 2. Autonomous system behaviors, e.g. growth processes, do not admit an input-output repre- sentation. 3. Thefunctionalrelatingx(t)andy(t)isa“black-box”descriptionoftheunderlyingdynamical system. Therefore, interventions into the internal workings of the system are not easily modeled, contrary to the case of parametric models. 4. In general, non-parametric models are difficult to interpret with respect to the underlying biology. We argue that the class of systems which can be studied using the basic Volterra framework can be dramatically expanded by relaxing the strict delineation between inputs and outputs, and instead allow outputs of interest to be recursive, i.e. act as inputs as well. In the single-output setting, we have the auto-recursive, or nonlinear autoregressive, model framework, in which the past of an output feeds back to act as a second input to the system. We have extensively studied such systems, and have found that highly nonlinear systems with internal feedbacks can be effectively represented by auto-recursive models of relatively low order. We have demonstrated that auto- recursiveVolterramodelscanadmitlimitcyclebehavior,autonomousoscillations,andautonomous growth. In addition to improving the ability to predict system behavior, inclusion of the auto- recursive component can dramatically increase model parsimony and interpretability. Theauto-recursivesettingcapturessingle-outputsystemswithnonlinearfeedbackmechanisms. However, the concept of recursive outputs is most fully realized in the context of autonomously operatingsystems, inwhichmultiplevariablesofinterestinteractsimultaneously. Suchsystemsare the norm in physiology, where maintenance of homeostasis is paramount and control systems are 3 uniformly closed-loop. Closed-loop physiological control systems are often studied experimentally with various techniques designed to “open the loop.” However, if we have data from such a system operating under natural conditions, it makes little sense to delineate between input and output. In this case, we propose that the techniques developed for the input-output Volterra series may be used to identify nonlinear closed-loop systems, and we refer to the variables operating in such a system as “recursive outputs”. Such a data-based description can also incorporate the effects of exogenous inputs imposed by an experimenter or clinician. We have successfully applied the auto-recursive methodology to several systems. In the area of neuroscience, we have derived auto-recursive representations of the Hodgkin-Huxley system and a model for single-neuron dynamics with point-process inputs and output. Working with data from the mollusk smooth muscle, we have shown that the auto-recursive Volterra model yields much better performance and can be better interpreted with respect to the underlying biology. Moving towards autonomous systems, we have shown that several classical growth laws yield an auto-recursive Volterra representation, a class of behavior wholly off-limits to the classical Volterra approach. Furthermore, we have demonstrated the applicability of the multi-recursive output approach to describing synthetic data generated by a simplified parametric model for the innate immune response to exogenous endotoxin injection. On the basis of these successes, we hoped to apply the method to clinical sepsis, but found the available time-series patient data inadequate for the task. Therefore, to study this problem we have taken a parametric approach. Contra the top- down approach of the non-parametric methodology, we take a bottom-up, hierarchical approach to model parametric model building. The problem of sepsis, defined as a bloodstream infection in the presence of a severe systemic inflammatory response, is of interest for several reasons. First, the syndrome is a major public 4 healthproblemofglobalandincreasingimportance. IntheU.S.alone,sepsisistheproximatecause of death in over 200,000 patients yearly, more than die of heart attacks [6], and the incidence is expected to increase with an aging population [6, 172]. The pathology of this disorder is mediated by complex interactions between the host immune system, the invading pathogen, and clinical interventions (“host, bug, and, drug”). Furthermore, it has been believed that an over-exuberant inflammatoryresponsemediatedbycytokines,mostnotablytumornecrosisfactor(TNF),underlies thephysiologicderangementsencounteredinsepsis. Thismotivatedtheclinicalstrategyofblocking TNFandotherinflammatorymediators,butdespiteseveraldozensofclinicaltrials,thishasproven a uniformly disappointing approach. Giventhesefailuresandthecomplex, nonlinearnatureoftheprocess, therehasbeensignificant theoretical interest in sepsis as a dynamical system, with the goal of determining optimal strategies forinterveningintheinflammatoryprocess. Threemainshortcomingssharedbymuch, butnotall, previous work include (1) model construction typically does not proceed with careful attention to the effects of behavior inclusion and exclusion, (2) dynamics are assumed to occur in a single, well- mixed compartment, while sepsis has a geometric component in that a primary infection generally drives the systemic response, and (3) antibiotic treatment, the cornerstone of clinical management, is typically neglected. We address these problems through a hierarchical model building approach, andwemodelbacteria-immunedynamicsatthreespatial/geometricscales: (1)asingle, well-mixed organ compartment, (2) a multi-patch spatial representation of the organ compartment, and (3) a systemic geometry incorporating the lung as the primary infection site generating bacteremia, a central blood compartment with a reactive neutrophil pool, and the liver as the site of bacteremia clearance and source of systemic cytokines. The well-mixed model provides the formal foundation for the second two scales. Finally, we incorporate antibiotic treatment at every spatial scale. 5 This parametric process has yielded several hypotheses. We find that a local-systemic interac- tion between tissue macrophages and short-lived circulating neutrophils is necessary for both an effective immune response and inflammatory resolution. Antibiotic treatment interacts synergisti- cally with treatment, but this synergism is far less pronounced in immunocompromised patients, whoarealsofarmoresusceptibletoresistance-mediatedtreatmentfailure. Finally,ourfundamental conclusion is that a harmful inflammatory response with robust production of both pro- and anti- inflammatorycytokinesisusuallyappropriatewithrespecttopathogencontrolinthecaseofsevere infection,andthereforeanti-inflammatoryagentsarelikelytohavelittleroleinsepsismanagement. Instead, the clinical focus must be on treating the underlying cause (via effective antibiotics) and potentially supporting the immune system through pro-, rather than anti-inflammatory agents. This document is organized as follows. The Volterra modeling methodology, and the recursive extensions and other novel methodological work we have done is presented in Chapter 2. Chapters 3through6giveexamplesofitsapplicationtorealandsimulatedsystems. Inparticular, Chapter3 discusses auto-recursive Volterra representations of single-neuron dynamics and Chapter 4 explores auto-recursive modeling of smooth muscle dynamics. Chapters 5 and 6 focus on truly autonomous systems. Chapter 5 demonstrates that the recursive Volterra method can be applied to basic parametricgrowthlaws;thehistoricalderivationsoftheselawsandtheirmechanisticinterpretations are also discussed. Chapter 6 introduces a simple model for experimental endotoxemia, and it is demonstrated that the recursive Volterra methodology can describe this autonomous synthetic system. Finally, Chapter 7 gives an in-depth discussion of the pathophysiology of clinical sepsis, our parametric modeling strategy, and conclusions. 6 Chapter 2 Recursive Volterra Methodology 2.1 Introduction to the Volterra series The following two sections outline the fundamental mathematics of the Volterra series and are partially adapted from Eikenberry and Marmarelis [65]. In continuous time, the Volterra series expansion gives the general relation between an input function, x(t ′ ),t ′ ≤t, and output,y(t), for a stationary, nonlinear dynamical system with finite memory as an infinite series of functionals that have hierarchical convolutional form [162]: y(t) = k 0 + ∫ ∞ 0 k 1 (τ)x(t−τ)dτ + ∫ ∞ 0 ∫ ∞ 0 k 2 (τ 1 ,τ 2 )x(t−τ 1 )x(t−τ 2 )dτ 1 dτ 2 +...+ ∫ ∞ 0 ... ∫ ∞ 0 k q (τ 1 ,...,τ q )dτ 1 ...dτ q +... (2.1) ThemultipleintegralsontheRHS,referredtoastheVolterrafunctionals,aremultipleconvolutions of the input signal with the Volterra kernels, where k q (τ 1 ,...,τ q ) is the qth-order Volterra kernel. The zeroth-order kernel, k 0 , is simply the system output for null input. The first-order kernel is a 7 weighting pattern which is convolved with the past to give input the first-order contribution to the present output. The second-order kernel is a two-dimensional weighting pattern for the pairwise interaction of the all past input values, and it represents the lowest order of system nonlinearity. Higher order kernels represent higher order nonlinearities in a hierarchical manner by weighing products of multiple values of the input epoch in order to determine the present output value. In practice, only finite-memory systems may be studied, with the integrals taken over a finite interval of the past, from 0 to µ, where µ is the “memory extent” of the system. Also note that, because x(t−τ 1 )x(t−τ 2 ) = x(t−τ 2 )x(t−τ 1 ), the kernels are symmetric with respect to their arguments. 2.2 Discrete Volterra series and the basis expansion technique Since data are collected as discrete time series and the order of the practically estimated Volterra modelisnecessarilyfinite,weconsidertheQth-orderdiscrete-timeVolterraseries,thattheapproach may applied to actual input-output data: y(n) = k 0 +T M ∑ m=0 k 1 (m)x(n−m)+T 2 M ∑ m 1 =0 M ∑ m 2 =0 k 2 (m 1 ,m 2 )x(n−m 1 )x(n−m 2 )+ ...+T Q M ∑ m 1 =0 ... M ∑ m Q =0 k Q (m 1 ,...,m Q )x(n−m 1 )...x(n−m Q ) (2.2) where T is the data sampling interval, n = t/T is the discrete-time index, and m = τ/T is the discrete-time lag. The number of lags is determined as M = µ/T. Given this framework, the system identification task consists of estimating the discrete values of the Volterra kernels. This can be done directly through ordinary least squares (OLS) [164] by casting 2.2 in matrix-vector 8 form as y =Xk, (2.3) where y is the output data vector, k the vector of kernel values to be estimated, and the input matrix X is constructed according to Equation 2.2; k and X are also constructed such that kernel symmetries are enforced. However, such direct estimation quickly becomes intractable for large M or higher-order models. As originally suggested by Wiener [260], the number of parameters that need be estimated can be dramatically reduced by expanding the kernels on a properly chosen orthonormal basis, such as the Laguerre basis, which has a built-in exponential term and, therefore, exhibits the relaxation characteristic typical of finite-memory systems. Watanabe and Stark [258] first implemented this idea, noting that a small set of orthonormal Laguerre functions can act as a basis spanning the subspace of kernels for many physiological systems, including oculomotor control. Note that there is no mathematical requirement that the basis functions be either orthogonal or orthonormal, and any basis appropriate to the system under study may be chosen. Given a set of L basis functions, {b j (m)}, we expand each Volterra kernel as k q (m 1 ,...,m q )= L−1 ∑ j 1 =0 ... L−1 ∑ jq=0 a q (j 1 ,...,j q )b j 1 (m 1 )...b jq (m q ). (2.4) We substitute such kernel expansions into the series in Equation 2.2, and after some rearrangement of terms we arrive at y(n)=k 0 + Q ∑ q=1 L−1 ∑ j 1 =0 ... L−1 ∑ j 1 =0 a q (j 1 ,...,j q )v j 1 (n)...v jq (n), (2.5) 9 where the set of transformed inputs, {v j (n)}, are given by convolution of the input with the basis functions: v j (n)=T M ∑ m=0 b j (m)x(n−m). (2.6) WecanfurtherreducethedimensionalityoftheproblembyexploitingthesymmetryoftheVolterra kernels to arrive at the modified discrete Volterra (MDV) model: y(n)=c 0 + Q ∑ q=1 L−1 ∑ j 1 =0 j 1 ∑ j 2 =0 ... j q1 ∑ jq=0 c q (j 1 ,...,j q )v j 1 (n)...v jq (n) (2.7) where k 0 =c 0 , and c q (j 1 ,...,j q ) = λ q (j 1 ,...,j q )a q (j 1 ,...,j q ). The scaling factor λ q is determined by the multiplicity of the indices (j 1 ,...,j q ). To estimate the expansion coefficients from input-output data, the model is put in matrix form, y =Vc, (2.8) wherey isthevectorofallsamples,V isamatrixoftransformedinputsconstructedfromEquation 2.7 with each row corresponding to a discrete time-point, and c is the coefficient vector. The coefficient vector is then estimated by ordinary least squares or with the pseudoinverse, V + , as ˆ c=V + y. (2.9) Notethatoncetheexpansioncoefficientshavebeendetermined, itisasimplemattertoreconstruct the original Volterra kernels from Equation 2.4. The MDV model can be represented in modular form as passing the input through a linear filterbank where the filters are our basis functions, {b j (m)}, yielding L outputs which are passed through a multi-input static nonlinearity, f[·], that generates the model output. This is depicted 10 Figure 2.1: Modular representation of the SISO modified discrete Volterra model. schematically in Figure 2.1. Under the discrete-time Laguerre expansion technique (LET), proposed in 1993 by Marmarelis [162](seealso[164]),theVolterrakernelsareexpandedonabasisofdiscrete-timeLaguerrefunctions (DLFs), developed by Ogura [197], which can be calculated from the recursive relation b j (m) = √ αb j (m−1)+ √ αb j−1 (m)−b j−1 (m−1), (2.10) b 0 (m) = √ αb 0 (m−1)+T √ 1−αδ i (m), (2.11) where b j (m) is the jth-order DLF, m = 0,...,M is the discrete-time index, T is the sampling interval, δ i is the Kronecker delta function, and α∈ (0,1) is a parameter that determines the rate ofexponentialrelaxation(asmallerα correspondstomorerapiddecay). Wetakeb j (−1)tobezero for all j. The LET has a very important advantage in that α is a trainable parameter that can be optimized for any system, making the discrete Laguerre basis much more flexible than most others. WetypicallyfollowtheLET,anduseaLaguerrebasisforkernelexpansion. However,wesometimes 11 Figure 2.2: The two basic functional bases for kernel expansion. The left panel gives the first four DLFs, forα=0.5 plotted over 50 discrete lags. The right panel shows the discrete-time delta basis over 10 lags. The third-order Laguerre basis spans a subset of the discrete functional space (vector space) spanned by the delta basis, but for a judiciously chosen α, this subspace often includes (at least a near approximation to) the kernel being estimated and requires far fewer basis functions. have occasion to directly estimate the classical Volterra kernels, in which case we expand upon a basis of (discrete-time) delta functions. This seemingly trivial expansion is useful for two reasons: 1. A common mathematical and computational framework may be used regardless of the basis used for expansion. In our framework, an arbitrary number of inputs can be considered with the kernels for each expanded upon a unique basis. 2. Multi-trial or subject data may be concatenated rigorously, as discussed in Section 2.5.1. Figure 2.2 gives examples of the two basic bases that we typically employ, the Laguerre and delta bases. 12 2.2.1 Extension to multiple inputs and outputs The basic single-input single-output (SISO) framework detailed above may be extended to the multi-input single-output (MISO) and multi-input multi-output (MIMO) cases. Given A inputs and B outputs, we label these as x 1 ,...,x A and y 1 ,...,y B , respectively. In the two-input, single- output case, initially studied by [166] and [167], we define a set of “self-kernels” for each input along with a set of “cross-kernels” that represent the nonlinear interactions between the inputs in determining the output. The two-input single-output discrete-time Volterra model is given as y(n)=k 0 +T M1 ∑ m=0 k 1;0 (m)x 1 (n−m)+T M2 ∑ m=0 k 0;1 (m)x 2 (n−m)+ T 2 M1 ∑ m1=0 M1 ∑ m2=0 k 2;0 (m 1 ,m 2 )x 1 (n−m 1 )x 1 (n−m 2 )+T 2 M2 ∑ m1=0 M2 ∑ m2=0 k 0;2 (m 1 ,m 2 )x 2 ((n−m 1 )x 2 (n−m 2 )+ T 2 M1 ∑ m1=0 M2 ∑ m2=0 k 1;1 (m 1 ,m 2 )x 1 (n−m 1 )x 2 (n−m 2 )+ ...+T Q1+Q2 M1 ∑ m1=0 ... M1 ∑ mQ 1 =0 M2 ∑ mQ 1 +1=0 ... M2 ∑ mQ 1 +Q 2 k Qx;Qy (m 1 ,...,m Q1+Q2 )x 1 (n−m 1 )... x 1 (n−m Qx )x 2 (n−m Q1+1 )...x 2 (n−m Q1+Q2 ) (2.12) whereQ i andM i are the order and memory extent of theith input, respectively. Any kernelk a;b is a self-kernel if ab=0 and a cross-kernel otherwise. For a second-order model (Q 1 =Q 2 =2) there is a single cross-kernel. For a third-order model, there are two additional third-order cross-kernels, k 2;1 and k 1;2 . Unlike the self-kernels, the cross-kernels are not symmetric. This expansion may be extended to an arbitrary number of inputs, and for A inputs, the total number of kernels of order j, K j , is given by the following relation: K 1 = A (2.13) K j = j ∑ q=2 ( A+q−2 q ) +A,j ≥2, (2.14) 13 when Q 1 = ... = Q A . This expression is generalized for recursive outputs and Q i not necessarily equal in Section 2.6.4. For any given order, only A kernels are self-kernels; the summation term in Equation 2.14 represents cross-kernels only, and therefore in higher-order models with multiple inputs most free parameters are due to cross-kernels. This is further compounded by the lack of symmetry in cross-kernels. ThekernelsofaMISOmodelmayeachbeexpandeduponafunctionalbasis(whichisgenerally unique to each input), denoted {b x i j (m)}, with L i representing the number of functions. If the Laguerre basis is used, each input also has an associated α i value. As with the SISO case, after substitutionweobtainsetsoftransformedinputs,{v x i j (n)}, givenbyconvolutionoftheinputswith their respective bases as v x i j (n)=T M i ∑ m=0 b x i j (m)x i (n−m), (2.15) which enter into a final polynomial expansion, which is given for two-inputs as: y(n)=c 0 + Q1 ∑ q=1 L11 ∑ j1=0 j1 ∑ j2=0 ... jq1 ∑ jq=0 c q;0 (j 1 ,...,j q )v x1 j1 (n)...v x1 jq (n) + Q2 ∑ q=1 L21 ∑ j1=0 j1 ∑ j2=0 ... jq1 ∑ jq=0 c 0;q (j 1 ,...,j q )v x2 j1 (n)...v x2 jq (n) + Q1 ∑ q1=1 Q2 ∑ q2=1 L11 ∑ j1=0 ... L11 ∑ jq 1 =0 L21 ∑ l1=0 ... L21 ∑ lq 2 =0 c q1;q2 (j 1 ,...,j q1 ,l 1 ,...,l q2 )v x1 j1 (n)...v x1 jq 1 (n)v x2 l1 (n)...v x2 lq 2 (n). SimilartotheSISOMDVmodel, thisoneadmitsamodularrepresentationinwhichthetwoinputs are passed through linear filters, the outputs of which are then fed to a static nonlinearity. In the case of multiple outputs, a MIMO Volterra model may be constructed as a set of parallel MISO models. Themodularrepresentationofatwo-input,two-outputmodelisshowninFigure2.3. Note that the parallel MISO models that make up a MIMO model are estimated and run completely 14 Figure 2.3: Modular representation for a two-input, two-output (non-recursive) Volterra model. independently of each other. 2.3 Recursive generalization 2.3.1 Auto-recursive (nonlinear autoregressive) extension It is straightforward, and often extremely useful, to extend the Volterra modeling paradigm to include the auto-recursive, or nonlinear autoregressive, case. That is, we consider the output, y(t), to not only be a function of some input, x(t ′ ),t ′ ≤ t, but of its own past, y(t ′′ ),t ′′ < t, and our general functional relationship for the “single-input single-recursive-output (SISRO)” case is y(t)=F[x(t ′ ),y(t ′′ );t ′ ≤t,t ′′ <t] (2.16) This functional can be expanded in discrete-time as two-input Volterra model, where the second input is the past output considered over the epoch y(n−1),...,y(n−R). To second-order, the 15 expansion is: y(n)=k 0;0 +T M ∑ m=0 k 1;0 (m)x(n−m)+T R ∑ r=1 k 0;1 (r)y(n−r) +T 2 M ∑ m 1 =0 M ∑ m 2 =0 k 2;0 (m 1 ,m 2 )x(n−m 1 )x(n−m 2 )+T 2 R ∑ r 1 =1 R ∑ r 2 =1 k 0;2 (r 1 ,r 2 )ˆ y(n−r 1 )y(n−r 2 ) +T 2 M ∑ m=0 R ∑ r=1 k 1;1 (m,r)x(n−m)y(n−r). (2.17) Expanding the kernels upon a basis yields a modular representation similar to that of a two-input single-output model, and the auto-recursive framework may similarly be extended to include an arbitrary number of exogenous inputs. Estimation and initial conditions The auto-recursive model can be estimated using the same framework as that used to estimate a two-input, single-output Volterra model, with the second input simply being the output delayed by one step. Initial conditions can pose more of a problem. It is often the case that we wish to make a forecast, but we have only the initial condition at time 0, y(0). To run the auto-recursive model, we must give some reasonable estimate of y(−1),...y(1−R), which we refer to as the initial epoch. The two basic options we consider are: 1. Set the initial epoch equal to y(0). If y(0) = 0, or y(0) is “close” to zero, then this method may work well, as zero is a steady-state. This option also works well with systems lacking sensitivity to initial conditions, which is happily often the case. 2. Using the training data-set, estimate a Volterra model such thaty(n−1) =F[y(n)] to obtain a prediction ofy(−1), ˆ y(−1), as a function ofy(0). To obtain an estimate ofy(−2), estimate 16 the model y(n−2) = F(y(n),y(n−1)], and use y(0) and ˆ y(−1) to get ˆ y(−2). Apply this method successively until an estimate of y(1−R) is obtained. The second method may be practically implemented as follows. Given a training data record, y(1,...,N), construct the “advanced-by-one” record, ˜ y 1 (1,...,N−1), where ˜ y 1 (1)=y(2),...,˜ y 1 (N− 1) = y(N). Then it is a simple matter to use the existing computational framework to estimate y(n) = F[˜ y 1 (n)]; this is equivalent to running the linear regression: y(n)=β 0 +β 1 ˜ y 1 (n)+...+β Q ˜ y 1 (n) Q (2.18) where Q is the order of the Volterra model. Successive steps entail the construction of advanced- by-J records, ˜ y J (1,...,N −J), and estimation of y(n) = F[˜ y J (n ′ ),n−J +1≤n ′ ≤n]. A note on running the model When running an auto-recursive model the exogenous input is, of course, given. Beyond the initial epoch, we typically use the model predicted past for the recursive component. In this case, we say that we are finding the “N-step-ahead” prediction. However,theremaybeinstanceswhereweareinterestedintheone-step-ahead modelprediction, in which case we use the actual output past. This may be the case in many control applications, or if we are trying to determine the causal interrelationships among multiple time-series. Of course, if we wish to make a prediction about a system’s response to a novel input, we have no choice but to use the model predicted past. Unless otherwise stated, all results given in this document are N-step-ahead predictions. 17 2.3.2 Extension to multiple recursive outputs We now expand our operational definition of a recursive output from simply an output in a system with feedback, to an output that itself is rightly considered to be an input as well. This definition encapsulates closed-loop systems which are characterized by “circular causality” [164]. Most phys- iological systems are characterized by highly interconnected recursive variables. This holds under both homeostatic conditions and under departures from homeostasis, or to put it differently, under both health and disease. Anexampleoftheformermightbehomeostaticmaintenanceofbloodpressure. Cardiacoutput is modulated in response to blood pressure, which is of course a function of (among much else) cardiac output. An example of the latter is the collapse of systemic blood pressure seen in septic shock. Either scenario could be (in principle) modeled by a recursive Volterra system. Such a system might also be subject to exogenous inputs, such as treatment with vasopressors, which can have both direct effects upon the recursive outputs or modulate their interrelationships. We propose that a straightforward extension of the (auto-recursive) Volterra framework to multiple recursive outputs allows Volterra-style modeling of autonomously operating physiological systems, and that this framework can incorporate the effect of exogenous inputs, typically in the form of medical interventions. Mathematically, suppose we have A exogenous inputs, x 1 ,...,x A , and B recursive outputs, y 1 ,...,y B . For each y i , we have, in discrete time, y i (n)=F[x 1 (m 1 ),...,x A (m A ),y 1 (r 1 ),...y B (r B );m 1 ,...,m A ≤n,r 1 ,...,r B <n]. (2.19) That is, the value of each y i (n) is taken to be a functional of the values of all past and present exogenousinputs,andthepastsonly ofallrecursiveoutputs. Thus,wehaveasetofparalleloutputs 18 Figure2.4: Theleftgivesanexampleofasingle-input, two-recursive-outputmodelwherethebases are expanded upon Laguerre bases. The right side shows a purely recursive model consisting of two recursive outputs. Running such a model is equivalent to solving the initial value problem. that, unlike in the standard MIMO case, are inextricably linked, and such a set of linked outputs can only be run in parallel. We refer to such a model as a multi-input multi-recursive-output, or MIMRO model, contra the standard MIMO model. Computationally, it is straightforward to estimate a MIMRO model: the model is estimated independently for each output, with all other recursive outputs taken as given. When running the model, at each time-point n, for all i = 1,...,B we estimate ˆ y i (n), using all previously estimated ˆ y j (n−1,...), j =1,...,B values. That is, we find the N-step-ahead prediction for each output in a coupled manner. Note that in a given MIMRO model not all outputs are necessarily recursive. An example of the modular representation of a single-input, two-recursive-output system is shown in Figure 2.4. Some systems may be “purely recursive.” That is, there is no exogenous input and all variables 19 are recursive outputs. In this case, running the multi-recursive-output model is equivalent to solving the initial value problem. An example is an ecological system consisting of a predator and prey species. An exogenous input we might impose on such a system is periodic culling of the prey. Growth laws also represent purely recursive systems in our framework, and are studied in Chapter 5. We study a MIMRO Volterra representation of a simple differential equations model for experimental endotoxemia in Section 6, and it is the ultimate aim of this work to develop a MIMRO representation of S. aureus sepsis. Note that this formulation differs slightly from the closed-loop formulation given in [164], and recently employed by Marmarelis et al. [169] in closed-loop modeling of cerebral hemodynamics. A fundamental issue is the extent to which the MIMRO framework represents the “true” time- series interrelationships vs. being a kind of model short-hand for representing high-order nonlinear transformations. We posit that this is not necessarily a well-posed question, as multiple Volterra- style representations may be equally valid, as demonstrated in a simple example in Section 4.2. In general, we suggest that if a system displays sensitivity, in any reasonable sense, to initial conditions, then the recursive framework is necessary. This is equivalent to stating that a system has memory. But in any case, it is the very nature of non-parametric models to functionally summarize hidden mechanistic dynamics, and our concern is not so much with the “truth” of a model, but its usefulness in a given context. 2.4 The Principal Dynamic Mode (PDM) concept As discussed extensively, Volterra kernel estimation is facilitated by kernel expansion upon a prop- erly chosen functional basis. The principal dynamic mode (PDM) concept, introduced by Mar- marelis in 1997 [163], is in essence a methodology to find an efficient basis that is specific to the 20 Figure2.5: ExampleofaverysimplePDM-basedmodelarchitectureforasecond-orderSISOmodel with two PDM basis functions. system under study. Because the PDM basis is system-specific, its use can dramatically compact and increase the interpretability of the resulting model. Underthemodularmodelform,theinputpassesthroughalinearfilterbankofPDMs,capturing the time-dependent system dynamics, and then to an associated static nonlinearity. Thus, the principaldynamicmodesaresocalledbecausetheyareanefficientbasisforrepresentingthemodel dynamics. The static nonlinearity, which has the general form of a polynomial expansion, can be separated into “associated nonlinear functions” (ANFs), where each PDM output is fed to a single ANF, and a set of cross-terms. The simplest such architecture is shown in Figure 2.5 for a single input and two PDMs. Figure 2.6 shows how the static nonlinearity of a second-order, two-input MDV model with a PDM basis may be conceptually divided into multiple types of self- and cross- terms. We refer the reader to [171] and [66] for further details on these decompositions and how they may be reduced. Several related methods for PDM estimation have been proposed [163, 164, 171]. The method given here is similar to that proposed by [171] and recently used in [66], and follows, for a single- input model: 1. UsingL basis functions (typically Laguerre), estimate the MDV Volterra model, and recover 21 Figure 2.6: Example of a two-input, single-output PDM model architecture where each input has two associated PDM basis functions. The static nonlinearity can be decomposed into self-terms, (a and b), and three types of cross-terms (c, d, and e). Under a second-order architecture, the self terms are quadratic, while the cross terms are bilinear (i.e. a scalar multiple). 22 the first- and second-order Volterra kernels. 2. Performeigendecompositiononthesecond-orderkernel(thisisalwayspossible,asthesecond- order self kernels are symmetric). Retain all eigenvectors with a corresponding eigenvalue at least 1% of the maximum eigenvalue. 3. Construct a rectangular matrix,E, consisting of the first-order kernel and the retained eigen- vectors weighted by their respective eigenvalues as columns. In addition, the eigenvectors are normalized by the root-mean-square of the input record, although we find that this step has a very minimal effect on the obtained PDMs. 4. Perform a singular value decomposition on the rectangular matrix, E =USV ∗ . (2.20) The columns of the U matrix form a set of orthonormal basis vectors, and the diagonal of S gives the associated singular values. We retain the L x most significant columns of U as the principal dynamic modes. The rectangular matrix E can also be constructed by simple concatenation of the first- and second-order Volterra kernels, and we have found this to yield very similar results. We initially retain L PDMs, as this means that the PDM basis spans the same space as the original basis. We then select only the few most significant PDMs; this can either be done on the basis of associated singular values, or through a basis reduction procedure that tests the statistical significance of each PDM [66]. We have recently performed such PDM basis reduction in an auto-recursive description of the Hodgkin-Huxley system, as discussed in Section 3.2. We propose a novel application of the PDM concept in Section 2.5.2, in which a universal 23 PDM basis in conjunction with coefficient cluster analysis is used to identify sub-types of a general system, and we suggest that this method may be useful in identifying patient sub-types in clinical conditions including sepsis. 2.5 Novel data issues There are three major issues with the data that is available for septic patients that prevent the straightforward application of the proposed Volterra-based methods. First, the time-series data is sparse, with insufficient data points from a single patient for model estimation. Therefore, a single model must be estimated using multiple subject data-records concurrently, a problem we address in the following subsection. However, no two of these subjects represent precisely the same dynamical system, and it may be that these subjects cluster into several distinct sub- systems. Therefore,weproposeamethodforidentifyingsub-systemsthatsharethesamestructural architecture. Finally, in the case of sparse data different time-series may not be sampled with the same frequency, and sampling times may be irregular. Therefore, we propose to interpolate such sparse data onto a common, regularly spaced time-grid, allowing model estimation through the least-squares framework. 2.5.1 Multiple trial/multiple subject data Suppose, as is often the case, that we have time-series data from multiple subjects or multiple trials on a single subject. To estimate a single model describing this data, we propose the following method for concatenating multiple trials into a single data-record for model estimation. Consider a SISO model, with x(n) the input and y(n) the output, and suppose we have data from C trials, not necessarily of the same length. We begin by concatenating the trials, which 24 creates a boundary effect: at each trial boundary there is a discontinuity in both x(n) and y(n), and for M time-steps, the transformed inputs {v j (n)} are determined using data from the end of one trial and the beginning of the other. While this boundary effect may have little influence on model estimation for long trials, if there are relatively few data points per trial it could be quite large indeed. However, because the transformed inputs contain all information from the previous M time- steps, we can simply construct a “pruned” data-record consisting of {v j (n)} and y(n) with those time-points occurring in the trial boundary transients omitted. This pruned record of transformed inputs and output is then used to construct the V matrix used for coefficient estimation. The procedure is illustrated in Figure 2.7. Note that this method allows the classical Volterra kernels to be estimated directly from multi-trial data by filtering the input with a delta basis. This basic procedure is easily generalized to the MIMRO case. As far as we are aware, this methodology has not been previously suggested. Under Volterra- based modeling, multiple trials/subjects are often treated as individual case-studies. Marmarelis et al. [171] previously, under the PDM modeling framework, obtained “global PDMs” by “fusing” the PDMs obtained for individual subjects. Under the somewhat related (linear) multi-variate autoregressive (MVAR) paradigm, in which time-series are viewed theoretically as multiple realizations of a stationary random process, it is commontousetheYule-Walkerequationstoestimatemodelcoefficients, whichrequiresestimating the covariance matrix of the process. For multiple trials (each viewed as a single realization of the random process), the covariance matrix is calculated for each, the average obtained, and this average matrix is used in the Yule-Walker equations [58]. This methodology is likely to be most useful for estimating complex Volterra models for which 25 Figure 2.7: Example of concatenating three trials of input-output data. There is a transient of length M at the beginning of each trial which is used to calculate the sets of transformed inputs, {v j (M)} (for each trial). The calculation of any v j (m) with m<M includes input-data from the end of the prior trial and so must be omitted. Therefore, a pruned data-record composed of y(n) and {v j (n)} minus the transients is composed, used to construct the V matrix, and the model is estimated as usual. individual subject data-records have few time-points, e.g. in clinical sepsis. In this case, the full concatenated data-record can give accurate model estimation even when each subject data-record is insufficient for model estimation. A fundamental problem in estimating such an “ensemble average” model is that the underlying dynamical systems will vary between subjects. For example, suppose we have data from two sets of sepsis patients, survivors and non-survivors; survivorship may be due to fundamental differences in patient physiology. In this case, the ensemble average model may be meaningless, and separate models should be estimated for the two cohorts and compared. It may not be obvious, however, in some cases, how to divide trials/subjects into cohorts for model estimation. Determining, from the multiple trial/subject data, how many meaningfully different dynamical systems are represented is crucial future work. In the following section, we present a preliminary method to do just this. 26 2.5.2 Amethodforidentifyingmultipledynamicsub-systemsfrommulti-trial/subject data We propose a preliminary method for identifying, from multi-trial/subject data, multiple dynamic systems sub-types that are similar enough to share a common framework. The basic idea is to use the full concatenated data-record to identify the global structural characteristics of the system, with identification of the PDM basis set particularly important. Then, expansion coefficients for individual subject models, which inherit the PDM basis sets and structure from the global model, are estimated, and clusters of such coefficients are identified to partition the model space into sub-types. The basic steps of the method are: 1. Concatenateallsubjectdata-recordsaspreviouslydescribed,andestimatethe“globalmodel.” 2. Maximally reduce the global model. In particular, identify the minimal PDM basis sets and prune coefficients and/or kernels. 3. Inheriting the PDM bases and other structural variables, estimate the expansion coefficients for each individual subject. If individual subject data-records are of insufficient length for model estimation, subject records may be preliminarily grouped as reasonably and minimally as possible. Coefficients can then be estimated for such subject sub-groups. 4. Use k-means clustering to partition the coefficient space into sub-models. Hierarchical clus- tering methods may also be tried. K-means clustering is a widely used algorithm for dividing M points in N dimensions into K clusters, such that the within-cluster sum-of-squares is minimized [100]. The algorithm requires thatthenumberofclustersbespecifiedbeforehand,anddoesnotgenerallyadmitagloballyoptimal solution. Whilethenumberofclustersmustbeexplicitlyspecified,thisparametercanbeoptimized 27 iteratively. The success of the sub-type model identification results may be tested on out-of-sample data by runningeachsub-modelonagivenout-of-sampletrial,andassessingtheircomparativeperformance. If, for each trial, one sub-model displays superior performance that is statistically significant (as assessed by the bootstrap procedure given in Section 3.2.2), and the proportion of trials best described by a given sub-model roughly correspond to the size of its cluster, then we may have some confidence in the results. We demonstrate the successful application of the method to a simple SISO test system. The system consists of a Gaussian white-noise input, x(n), passed through a linear filter, H(m), the output of which is fed to the static nonlinearity, f(n), where these are defined as H(m) = exp ( −mT 10 ) cos ( mT 4 ) , (2.21) v(n) = x(n)⊗H, (2.22) f(n) = av(n)+bv 2 (n), (2.23) and a and b are taken to be independent random variables with the bimodal distributions shown in Figure 2.8, implying four system sub-types. We generate 100 trials of 128 data points, and, from a second-order Volterra model with L x = 9 and α x = .85, find a single global PDM; we also find c 0 to be unnecessary. This reduces the number of free coefficients to a mere two, and allows coefficient estimation from single trials. As shown in Figure 2.9, the single coefficient distributions showtwopeaks, andK-meansclusteranalysiscorrectlyidentifiestheexistenceoffoursub-systems. The method appears to be quite robust to noise, giving correct results for SNR dB = 1. Note the 28 Figure2.8: Testsystemwithfoursub-systems, asdefinedbytheindependentbimodaldistributions of a and b, displayed in the lower half of the figure. importanceofidentifyingthePDMbasis, whichreducesthedimensionofthecoefficientspacefrom 54 to two. The method may also be applied to in a straightforward manner to multi-input and auto- recursive recursive systems, and we have confirmed that it is capable of discriminating multiple Riccati (logistic growth with exogenous driver) sub-systems defined by different intrinsic growth- rates (results not shown). Extension to the MIMRO case, however, is not entirely trivial, requiring that the coefficient spaces for the different parallel recursive models be considered as one. Thismethodissimilarinspirittothe“physiomarker”conceptrecentlyadvocatedbyMarmarelis et al [170]. However, the cluster-based approach may be more amenable to exploratory analysis, and can be used to determine if differently classified subjects do indeed cluster, e.g. we could test the hypothesis that survivors and non-survivors of sepsis partition into distinct clusters in coefficient space. 29 Figure 2.9: The left panel shows the results of k-means clustering applied to the two-dimensional coefficient space, with the Xs marking the center of each cluster: four sub-models are correctly identified. The right panels give the distributions of the c 1 and c 2 coefficients. Results are shown for the noise-free case, but we have found the method to be robust for SNR dB as low as 1. 2.5.3 Sparse and irregular data It may be the case, particularly with sparse clinical data, that different time-series are sampled at different rates and/or that the interval between samples varies. In this case, we propose interpo- lating all samples onto a single fine temporal grid, and then using these interpolated data records for model estimation. While cubic spline interpolation has previously been used [169], we employ piecewise cubic Hermite polynomial interpolation. An example of this technique is given in Section 6.2.2. 30 2.6 Bootstrap statistical procedure for structural variable selec- tion and model comparison 2.6.1 The notion of conditional prediction as a foundation As a preamble, it is our position that the ultimate purpose of a dynamical model is to predict the behavior of a system given out-of-sample inputs and output initial conditions or epochs. The predictiveperformanceofamodelisassessedwithrespecttosometeststatistic, whichwenormally take to be the normalized root-mean-square-error (NRMSE) for continuous data, or a coincidence factor, Γ, for point-process data (described in Section 3.4.2). We now introduce the notion of conditional prediction. Suppose we wish to predict some output variable, y, and we have a set of putative inputs {x 1 ,...x A }. Given some x i and a set of inputsX, x i / ∈X, we obtain estimates of y as ˆ y 1 (n)=F[X] (2.24) and ˆ y 2 (n)=F[X∪x i ]. (2.25) If ˆ y 2 isasignificantlybetterpredictionthan ˆ y 1 ,thenwesayx i conditionallypredictsy withre- spect toX. Statisticalsignificanceisdeterminedusingabootstrapre-samplingmethod, described in the following section. In the following sections, we use this notion as the basis for selecting the structural characteristics of any model. In general, our goal is to determine the minimal set of ker- nels, basis functions, inputs, etc. such that the exclusion of any decreases predictive performance, while the inclusion of any others fails to improve prediction. We generally extend the concept of conditional prediction to sets of inputs (or other structural 31 variables, e.g. kernels), and often may say that the set X 1 conditionally predicts y with respect to X 2 . Finally, we introduce the concept of unconditional prediction. We say that some input, x, unconditionally predicts the output y if ˆ y(n)=F[x] (2.26) gives a better prediction than that expected by chance alone, which we refer to as the null pre- diction. With continuous data, we can simply define the null prediction as y null ≡k 0 (2.27) where k 0 is the zeroth-order kernel estimated from the data. However, for point-process data this becomes meaningless, as k 0 will either always be above or below threshold. In this case, we divide theoutputrecordintoN segments,temporallyshufflethesegments(say)10,000times,andforeach re-shuffle, y shuffle , we find the value of our test statistic, assuming y shuffle is the actual output, and ˆ y is estimated from Equation 2.26. This gives us a null distribution for the test statistic, and we then determine if ˆ y =F[x] is statistically better than chance at whatever significance level desired. This method can, of course, be applied to continuous data as well. 2.6.2 Bootstrap procedure Inordertoapplytheconceptofconditionalprediction,andtoeffectivelycomparemodelsingeneral, we require a method for estimating the variance of the test statistic of interest, which we label S. One method might be to run two estimated models onN testing data-sets or trials, calculateS for 32 eachtrialandmodel, takethemeanS undereachmodel, anduseaT-testtocomparethemeansto determine if the models give results that vary significantly. An obvious problem with this method is that in practice only a few, or even just one, testing data-sets may be available. Furthermore, for a stationary dynamical system with a strictly defined input and output, we argue that there is no fundamental difference between running a model for, say, 10 trials of 100 time-points each and running a single trial for 1000 time-points. That is, there is no need to do the former to get an estimate of the variance of a test statistic, we can simply divide the model prediction of the latter into 10 segments and calculate the mean of S for each segment. This leads us to the following bootstrap procedure for estimating the bootstrap distribution of the mean of S, and for comparing the means of S predicted by different models. 1. Divide the model prediction, ˆ y for an out-of-sample data-set into N segments. Calculate the statistic of interest for each segment, giving the set S=S 1 ,...,S N . 2. FromS, draw (say) 10,000 re-samples of size N, with replacement. For each re-sample, find the re-sample mean. This gives us the bootstrap distribution of the mean of S, the bootstrap standard error is the standard deviation of this distribution, and we may easily calculate any bootstrap percentile confidence interval. 3. For any two models, we may similarly find the bootstrap distribution of the mean difference inS, i.e. calculate|S 1 i −S 2 i |foreachtrial. Itisthendeterminedfromthebootstrappercentile confidence interval whether the two models vary in their predictive capability at any desired significance level. If it happens that data from many short trials is available, it may be natural to use these trials as segments for bootstrap significance testing. Figure 2.10 illustrates the bootstrap procedure applied 33 to a synthetic system. One complication that may arise in systems with recursive outputs, or even in the simpler case of multiple inputs and a non-recursive output, is that system behavior in short trials may meaningfully differ from its behavior in longer trials. Consider as an example the Riccati system: dy dt =ay−by 2 +cx(t), (2.28) where y(t) is the output, x(t) is an exogenous input, and a and b nonnegative. For t small, the system is dominated by autonomous growth, while for large t, growth has plateaued and the exogenous input is dominant. It is advisable to test such a system with multiple trials that adequately represent both the growth and plateau phases of the dynamics. We must, then, in general ensure either that a single long trial adequately explores the output space of interest, or run multiple trials. Such trials may be rigorously concatenated as described in Section 2.5.1, and the same bootstrap method applied to the concatenated record, with care taken to disregard the transients. 2.6.3 Input selection and nonlinear generalization of Granger Causality For any given system, there is a set of structural variables and relationships that must be deter- mined. At the highest level, we must determine the “connectivity” structure among the observed variables; this is in some sense equivalent to identifying the causal interrelationships among the variables. Given sets of time-series for inputs and (possibly recursive) outputs, designated as such a priori,x 1 (n),...,x A (n)=X all , andy 1 (n),...,y B (n)=Y all , for eachy i we must determine the sets X i and Y i of inputs and recursive outputs, respectively that are both necessary and sufficient for 34 Figure 2.10: Illustration of bootstrap procedure for comparing two models. Data is generated by a second-order synthetic system such that y(n)=F[x 1 ,x 2 ] and the output record is contaminated with noise so that SNR dB = 10. Model 1 takes ˆ y(n) =F[x 1 ,x 2 ] while Model 2 has ˆ y(n) =F[x 1 ]. The out-of-sample model predictions are divided into 16 segments, as shown, and the NRMSE is calculated for each segment. Bootstrap re-sampling is used to give (1) the bootstrap distributions of the mean NRMSE for each model, and (2) the bootstrap distribution of the mean (NRMSE 1 - NRMSE 2 ). The method finds Model 2 to be significantly inferior, and we can conclude that x 2 conditionally predicts y with respect to {x 1 }. 35 maximal predictive performance. More formally, under the functional expansion y i (n)=F i [X i ,Y i ], (2.29) we seek the minimal sets X i and Y i . We define X i to be minimal with respect to F i if there exists no set of inputs ´ X ⊂ X all where ´ X∩X i = , such that ´ X conditionally predicts y i with respect to X i , and if for any input x j ∈ X i , x j conditionally predicts y i with respect to X i \x j , i.e. the exclusion of any x j reduces performance. The minimal set Y i is similarly defined. Note that we require that the addition of no set of inputs not inX i improve performance, as it may be perfectly possible to find a set of inputs such that the addition of anysingle input does not improve performance, and yet is not minimal. Without loss of generality, let us restrict our attention to y 1 , X 1 and Y 1 . To construct the minimal X 1 and Y 1 , we advocate two basic approaches, a “bottom-up” approach, in which we buildX 1 andY 1 up from null sets, and a “top-down” approach. The basic steps of the bottom-up algorithm follow: 1. SetX 1 =ø,Y 1 =ø. 2. IteratethroughallinputsandrecursiveoutputsinX all andY all ,anddeterminewhichofthese unconditionally predict y 1 . Rank these by effect on prediction error. 3. Iteratively add inputs (recursive outputs) by rank to X 1 (Y 1 .) Stop when some criterion is met. Possible stopping criteria include: (1) performance does not differ significantly from that of the full model, (2) addition of more inputs/recursive outputs fails to significantly improve performance, or (3) some threshold for acceptable performance is met. The first is preferred; however, the bottom- 36 up method is best used when data is limited and estimating the full model is not possible. We refer to the iterative addition of inputs as add-one-in (AOI) steps. The top-down algorithm follows a leave-one-out (LOO) approach, where we start with the full model, iterate through each input/recursive output and determine if it conditionally predicts y 1 . Those that do are retained and the others omitted, yielding a reduced model. The LOO approach, however, does not tell us what happens when we leave two (or more) out, and should the reduced model have inferior performance, an add-one-in step may be performed to compensate. It has been our experience that the LOO algorithm is generally superior to the bottom-up construction method, but it has the disadvantage of requiring more data. Asapracticalmatter,wegenerallysettheorderofeachinputto2(Q=2),unlesswehavegood cause a priori to include third-order or higher. Should the length of our data-record be limiting, we can also get a first-approximation of the connectivity structure by setting the order to 1. The optimal approach is expected to be problem-specific, and in general several approaches should be attempted and results compared. Granger Causality GrangerCausality(GC),whichhasbeenwidelyusedineconometricsandneurosciencetodetermine causality relations among time-series, can be considered formally identical to identifying the mini- mal setsY i under a fully recursive linear system and one-step-ahead prediction. The fundamental notion, firstsuggestedbyNorbertWienerin1956[261], isthatiftheinclusionofinformationabout a variabley 1 improves our ability to predict the future ofy 2 , theny 1 can be said to causally related to y 2 . In 1969, Granger [89] framed this notion in the context of linear autoregressive stochas- tic modeling. Geweke [85] made important contributions to the GC concept in the early 1980s, 37 and Bressler and Seth [23] have recently reviewed the method’s historical development and use in neuroscience. Granger causality and non-parametric MIMRO Volterra model estimation both represent at- tempts to inductively determine the dynamic (or causal) interrelationships among observed vari- ables. This is contra the deductive experimental method, in which all factors but one (ideally) are held constant, so that the influence of a single variable may be deduced. An inductive method can only reflect the data it is trained on, and it is possible that spurious relationships may be inferred. Even given ground truth about a system, it may difficult to say whether or not an inferred rela- tionship between two variables is truly “spurious,” as it is the nature of the inductive approach to infer functional relationships that summarize potentially many mechanic relationships. In contrast to our focus on deterministic dynamical systems, GC is founded in the theory of stationary stochastic processes. Granger took the philosophical position that causality is a meaningless concept in a deterministic universe. In such a world, the state of the universe is strictly a function of its past states. Therefore, the argument goes, all the information required to predict y(n) is contained in y(n−1),...y(−∞). This is an important point, but since we cannot practically model the evolution of a deterministic universe from its first cause, whether causality is a meaningful notion in deterministic finite-time systems, where the influence of the distance past converges in some sense to zero, remains (to us) a philosophically open question. Nevertheless, this castsafurthershadowonanyattempttoinductivelyinfer“true”causalormechanisticrelationships among time-series. Ultimately, our motivation for considering recursive models is twofold. First, by considering some structural relationships between variables, we can much better predict system dynamics with lower-orderVolterra-classmodels,asweextensivelydemonstrateinthisdocument. Second,wewish 38 to use the MIMRO framework to model how intervening at different points in an autonomously operating system affects overall behavior. In this case, not only are the issues raised above salient in the general sense, the issue of confounders is of considerable proximate importance. Suppose, as part of a larger system, some variable A is found to predict B well, but only because some third variable C affects both A and B, with no “direct” causal connection between A and B. We then model suppressing A, giving a prediction about the resulting time-course of B. This prediction will, of course, be nonsense. The possible confounding influence of hidden or latent variables on predictions following from the data-derived model structure implies that, as a general rule, such predictions should be regarded as hypothesis generating only. 2.6.4 Kernel reduction Given an input-output structure, any proposed model has self- and cross-kernels up to some max- imum order, Q max . In general, not all of these kernels may be significant, with only of a subset of kernels contributing significantly to the model’s predictive ability. Identifying this subset not only improvestheparsimonyoftheresultingmodel, butmayhelpelucidatetheunderlyingrelationships between inputs and outputs. In this section we discuss methods for identifying statistically signif- icant kernels from input-output data, based on the bootstrap method. The fundamental goal of any such procedure is to identify a set of kernels such that omission of any one results in a signifi- cant degradation of model performance, while the addition of any other kernel fails to significantly improve model performance. Consider a set of (possibly recursive) outputs, y 1;:::;B , and a set of putative inputs, x 1;:::;A . The 39 input-output relation for each output, y i , can be expressed as: y i =F i [x 1 ,...,x A ,y 1 ,...y B ;Q i x 1 ,...,Q i x A ,Q i y 1 ,...,Q i y B ] (2.30) for i = 1,...,B, and where Q i x j is the maximum order for the x j model components, and Q i y j is similarly the maximum order for the y j components. For convenience, we define the set Q as Q={Q i x 1 ,...,Q i x A ,Q i y 1 ,...,Q i y B }. The total number of kernels in F i of order j, K j i , is given by: K 1 i = N 1 (2.31) K j i = j ∑ q=2 ( N j +q−2 q ) +N j ,j ≥2, (2.32) where N j = A ∑ i=1 I i (Q x i )+ B ∑ i=1 I i (Q y i ), (2.33) and I i (x)=        1, x≥i 0, x<i . (2.34) That is,N j is the total number of inputs and recursive outputs such that their order, Q, is greater than or equal to j. The total number of kernels that define F i is obviously given by the sum of K j i s: K total i = Q i max ∑ j=1 K j i (2.35) Q i max = max(Q i x 1 ,...,Q i x A ,Q i y 1 ,...,Q i y B ) (2.36) WedefinethesetK i tobethesetofallkernelsthatdefinethefunctionalexpansionF i ; thesekernels 40 may, for convenience, be re-labeled as K i = { ˇ k i 1 ,..., ˇ k i K i }. We define K as the set of all kernels in allK i . Following Section 2.6.1, we can, in a straightforward manner, define conditional and uncondi- tional prediction for kernel sets. LetH be any set of kernels drawn fromK. Take any kernelk∈H. If the “reduced-by-one” kernel model, F[H\k], gives a prediction of any y i ∈Y significantly worse thanthatofthefullkernelmode,F[H], thenwesaythatkconditionallypredictsYwithrespect toH. This notion may be similarly extended to say that sets of kernels conditionally predict Y. Consider any kernel, ˇ k i j ∈ K i . If a single-kernel model, F[ ˇ k i j ], gives a better prediction of y i than that expected by chance, we say that ˇ k i j unconditionally predicts y i . We define thezero-order kernel set to be the set of all zero-order kernels for a given MIMRO model. We generally take the prediction given by the zero-order kernel set to be that expected through chance alone. Bottom-up AOI and top-down LOO methods may be applied to the kernel set in a manner completelyanalogoustothebottom-upandtop-downmethodsforinput/recursiveoutputselection discussed in Section 2.6.3. For the LOO method, it may often be advisable to specify any kernels that should be omitted a priori based on prior knowledge of the system or in the interest of parsimony. 2.6.5 Basis and coefficient reduction The same basic AOI and LOO procedures proposed for input and kernel reduction may be applied to basis set reduction and static nonlinearity expansion coefficient reduction. In this case, we generally start with full basis and coefficient sets, and apply some variation of the LOO algorithm. We have extensively studied this approach in a PDM-based model for the Hodgkin-Huxley system, 41 as discussed further in Section 3.2. 42 Chapter 3 Single-neuron dynamics The dynamics of both single neurons and ensembles of neurons have been to subject of multiple Volterra and Wiener-style analyses. The basic process of action potential (AP) generation in an excitable neuron is a cascade of two main sub-systems: 1. Somatodendritic integration of pre-synaptic APs acting upon the dendritic tree, yielding a current injection at the level of the axon hillock. 2. Conversion of current injection to membrane potential, leading to AP firing at the axon hillock. Once an AP is generated at the axon hillock, the processes of afterpotential generation and re- fractoriness are initiated. These are recursive phenomena, with past membrane potential strongly influencing the future. We have extensively studied an auto-recursive Volterra representation of the Hodgkin-Huxley equations, a canonical parametric model for AP firing. These results establish that the auto-recursive framework can yield a model with multiple dynamic regimes, give limit cy- cle behavior, and that the auto-recursive framework is amenable to dramatic parameter reduction 43 under PDM-based analysis. This work has recently been published [65], and is under review [66], and Sections 3.1 and 3.2 are adapted directly from the former papers. We follow our work on the excitable membrane by studying an integrated single neuron model with point-process inputs and outputs. Somatodendritic integration of multiple input spike-trains areconvertedtoacurrentinjectionviaa(prescribed)second-orderVolterrasystem; thisinjectionis convertedtooutgoingAPsviatheHodgkin-Huxleymechanism. Weshowthatasecond-orderauto- recursiveVolterra model describes the point-process input-output transformation of this integrated system very well, establishing both that the auto-recursive methodology is appropriate for such data, and that it is useful even when the auto-recursive system component is hidden, or “nested.” 3.1 Auto-recursive(autoregressive)VolterramodelingoftheHodgkin- Huxley equations 3.1.1 Introduction Over the past century, numerous mathematical models for single-neuron firing dynamics have been proposed, ranging from simple parametric integrate-and-fire models to highly detailed biophysical models. Hodgkin and Huxley [105] proposed a four-equation model describing the axonal mem- brane potential in response to injected current and leading to the generation of an action potential (AP). This effort was justly rewarded with a Nobel prize, and the Hodgkin-Huxley (H-H) model remains the canonical model for AP generation by a firing neuron. In this section, we introduce a new method to model the AP generation by a single-neuron based on a Volterra-type model that incorporates a nonlinear auto-regressive (NAR) component. We use data simulated by the H-H model as ground truth for developing and testing our proposed model. 44 The Wiener series methodology (see [260, 142]) , which is closely related to the Volterra series, estimates Wiener kernels from Gaussian white noise (GWN) input and the corresponding output. Beginning with the 1974 work of Guttman et al. [94], a fair number of authors have applied the Wiener methodology to white noise input to both real neurons and the H-H equations, e.g. [94, 93, 27, 95, 28, 137, 29, 148, 231]. Poggio and Torre [208] also derived Volterra representations of the leaky integrator and the integrate-and-fire neuron models. Marmarelis [165] proposed modeling input-output data for single neurons by use of “neuronal modes,” a set of filters which represent the nonlinear dynamics involved in neuronal signal trans- formation. The latter includes the cascaded processes of somatodendritic integration and AP generation at the axon hillock. The outputs of these filters feed into a static nonlinearity followed by a threshold, resulting in a multi-input threshold. In a later work, Marmarelis [168] estimated neuronal modes from synthetic input-output data using Volterra series and the Laguerre expansion technique. Extensiveapplicationsofthemulti-input/multi-outputVolterramethodologyhavebeen made by Berger, Marmarelis, and colleagues, to the study of the functional characteristics of the hippocampal formation [225, 226, 16, 98, 99, 266]. Gerstner and van Hemmen [83] proposed the so-called spike response model (SRM) for single- neuron dynamics (see also [82]). A neuron fires an action potential (AP) whenever an abstract “membrane potential,” h(t), exceeds some threshold θ. Membrane potential is determined as the sum of two functions: one accounts for the refractory period following an AP, while the other integrates the effects of synaptic inputs to the neuron. Kistler et al. [131] proposed a Volterra style SRM for the H-H equations. In this model, the membrane potential output, u(t), is determined from the input current, I(t), according to a 45 modified Volterra expansion: u(t)=η(t−t f )+ ∫ ∞ 0 ε (1) (t−t f ;τ)I(t−τ)dτ +... (3.1) wheret f is the time of the most recent AP, and an AP is considered to occurred if u(t) crosses the threshold θ from below. The kernel η(t−t f ) gives the AP and afterpotential waveforms, and it is imposed. The “response kernels,” ε (i) , determine the potential response to injected current and vary with time since the most recent spike, t−t f . Several more recent works have applied very similar Volterra style SRMs (the principal difference being the inclusion of a dynamic threshold) to real cortical neuron data [122, 120]. The fact that Kistler and colleagues found it necessary to impose the AP waveform as an additional kernel points to the central difficulty in applying the Volterra series approach. Namely, the output of the H-H model is not simply a function of the past input, but it is characterized by two distinct dynamical regimes. Within the subthreshold regime, to a very good approximation, the output is a function of the recent input and can be represented well by a Volterra model. However, upon initiation of an AP, the ionic conductances change dramatically, and the form of theAPispracticallyfixedregardlessoftheparticularvaluesofthecurrentinput. Therefore,during an AP the input and the output are effectively decoupled. AP firing also initiates the processes of an afterpotential and refractoriness, which affect the forms of the subthreshold Volterra kernels describing the membrane response to input following the AP. Note that refractoriness is not a simple binary state, but varies continuously with the time since the most recent AP. Given these difficulties, we advocate a new Volterra-type modeling framework to model a single H-H neuron. We consider the membrane potential, y(t), as a function of both of the past current 46 injection, x(t) and the suprathreshold past membrane potential, ˆ y(t): y(t)=F[x(t ′ ),ˆ y(t ′′ );t ′ ≤t,t ′′ <t] (3.2) where ˆ y =y(t)H(y(t)−θ), (3.3) and H(x) is the Heaviside step function, defined as H(x)=        1, x≥0 0, x<0 . (3.4) As described in Sections 2.2 and 2.3.1, we expandF[·] as a Volterra series and estimate the kernels withtheLaguerreexpansiontechnique. Wefind thatthismodel, unlikea standardVolterramodel, can accurately predict the membrane potential generated by the equation of an H-H neuron in response to white-noise current injection as well as specialized inputs such as a current pulse, current injection, and sinusoidal current. This model differs critically from the SRM of Kistler et al. [131] in that we need not explicitly account for the AP waveform, afterpotential, or refractory period. All these are generated naturally by the modeling process of the proposed framework. Our model has the further advantage that the Volterra kernels can be estimated using ordinary least-squares estimation. While this study applies the proposed Volterra-type model with NAR component to an H-H neuron, this modeling framework is general and can be applied to any firing neuron or ensemble of neurons (simulated or real). 47 3.1.2 Methods Data preparation: H-H model The H-H model considers the neuron membrane as a circuit consisting of four elements in parallel. Three represent the dynamics of sodium, potassium, and “leak” ion channels, and each consist of a battery in series with a resistor. The fourth is a capacitor. The full model follows: C M dV dt = I(t)−¯ g K n 4 (V −V K )− ¯ g Na m 3 h(V −V Na )+¯ g l (V −V l ) (3.5) dn dt = α n (1−n)−β n n (3.6) dm dt = α m (1−m)−β m m (3.7) dh dt = α h (1−h)−β h h (3.8) where α n = :1−:01V exp(1−:1V)−1 , β n =.125exp ( −V 80 ) (3.9) α m = 2:5−:1V exp(2:5−:1V)−1 , β m =.4exp ( −V 18 ) (3.10) α h =.07exp ( −V 20 ) , β h = 1 exp(3−.1V)+1 (3.11) The injected current is given as I(t) and V(t) is membrane potential. We use the parameter values of Hodgkin and Huxley’s original work [105], but modify them so membrane depolarization correspondstoapositivemembranepotential(ratherthannegativeasin[105]);therestingpotential is 0 mV. These are given in Table 3.1. 48 Table 3.1: Hodgkin-Huxley model parameters. Parameter Value C M 1 µF cm −2 V K 115 mV V Na -12 mV V l 10.6 mV ¯ g Na 120 mS cm −2 ¯ g K 36 mS cm −2 ¯ g l 0.3 mS cm −2 We have generated a series of data-sets using broadband white noise as input. Specifically, current values are drawn from a normal distribution with mean (µ) 0 and varianceσ 2 , and current is injected at a rate of 1 kHz (i.e. a new, independent value for I(t) is chosen every 1 ms). The sampling interval, T, is fixed at 0.2 ms. We have generated data series of lengths 8,192, 16,384, and 32,768 ms for σ ∈ {0.5,4,8,16,32} (µA cm −2 ); we denote these as “σ x data-sets,” where x 2 is the variance of the current. We use a 16,384 ms σ 32 data-set for model training, and typically use 8,192 ms σ 32 , σ 16 , σ 8 , and σ 4 data-sets for model testing (validation). When σ = 0.5, the H-H model remains subthreshold. For σ =4, the H-H neuron fires at a relatively low frequency of approximately 36 Hz, while σ =32 gives firing around 78 Hz. Nonlinear Auto-Regressive Volterra-tyoe model of AP generation We propose the Nonlinear Auto-Regressive Volterra-type (NARV) model of AP generation that is shown schematically in Figure 3.1. The exogenous input is the injected current, x(n) (µA cm −2 ), and the output is the membrane potential, y(n) (mV). The thresholded output, ˆ y(n): ˆ y(n)=y(n)H(y(n)−θ) (3.12) 49 is fed back as the autoregressive component of the NARV model, where H(x) is the Heaviside step function, and θ is the membrane threshold potential for AP firing. We have also tried using a simple threshold ˆ y(n) =H(θ−y(n)), and while this option gives reasonable results, it sometimes leads to numerical instability at the beginning and end of the APs and a poorer representation of the AP waveform. The final modified NARV model is given as y(n)=c 0;0 + Qx ∑ q=1 Lx1 ∑ j1=0 j1 ∑ j2=0 ... jq1 ∑ jq=0 c q;0 (j 1 ,...,j q )v (x) j1 (n)...v (x) jq (n) + Qy ∑ q=1 Ly1 ∑ l1=0 l1 ∑ l2=0 ... lq1 ∑ lq=0 c 0;q (l 1 ,...,l q )v (y) l1 (n)...v (y) lq (n) + Qx ∑ qx=1 Qy ∑ qy=1 Lx1 ∑ j1=0 ... Lx1 ∑ jqx =0 Ly1 ∑ l1=0 ... Ly1 ∑ lqy =0 c qx;qy (j 1 ,...,j qx ,l 1 ,...,l qy )v (x) j1 (n)...v (x) jqx (n)v (y) j1 (n)...v (y) jqy (n) (3.13) where the transformed inputs are given as v (x) j (n) = T M ∑ m=0 b (x) j (m)x(n−m), (3.14) v (y) l (n) = T R ∑ r=1 b (y) l (r)ˆ y(n−r), (3.15) withj =0,...,L x −1 andl =0,...,L y −1. The cross-term summation is in Equation 3.13 is subject to q x +q y ≤ min{Q x ,Q y }, and c 0;0 = k 0;0 . Since we expect that no membrane potential should arise in the absence of injected current, we set c 0;0 =k 0;0 = 0. Note that for higher-order models, cross-terms account for the majority of discrete expansion coefficients that must be estimated. [FINISH] 50 Figure 3.1: Modular representation of the NARV model. Since the NARV model has an autoregressive component, we must consider the effect of initial conditions for ˆ y(n), n = 0,...,R. When using an input-output record to train the model, we simply use the known output to initialize ˆ y(n). When predicting the time-course of membrane potential in response to a novel exogenous input, we assume nothing is known about the output and simply set ˆ y(n)=0 forn=0,...,R. Since membrane dynamics within the subthreshold reason are dependent only on exogenous input, there is no sensitivity to initial conditions, although there may be a transient of finite duration which may be disregarded in terms of model estimation and computation of the model prediction error. 3.1.3 Model parametrization/training The basic model has seven structural parameters: Q x , Q y , M, R, L x , L y , α x , α y , and θ. We let Q cross represent the order of the cross-terms, with the obvious constraint Q cross ≤ min{Q x ,Q y }. 51 We defer our discussion on selecting the structural parameters to sections 3.1.4 and 3.1.4 of the Results, wheretheprocessandthechosenparametervaluesarepresentedindetail. Theestimation of the discrete expansion coefficients is discussed in Section 2.3.1. Expansion coefficients are always estimated using the 16,384 msσ 32 training data-set. We have also tried training the model for lower power inputs, but have found that to get good estimates, the model must “see” a large number of APs. This can be accomplished through either a very long data record, or by using large σ for the injected current input. We take the latter approach. We also take this approach because wedesire a model validover a broad dynamic range, and in general the model must be trained over the dynamic range it is to make predictions on. Evaluation of model performance For given structural parameters, we estimate the discrete model expansion coefficients and assess model performance by 3 metrics: (1) the normalized mean square error (NMSE), (2) the ratio N model : N data , where N data is the total number of APs occurring in a data-set and N model is the number of APs predicted, and (3) a coincidence measure Γ, that estimates the degree of similarity between two spike trains. This estimator was introduced by Kistler et al. [131] and has been used in several other works to evaluate spiking neuron models [120]; it is defined as: Γ= ( N coinc −⟨N coinc ⟩ .5(N data +N model ) )( 1 Λ ) (3.16) where ⟨N coinc ⟩ = N data N model K (3.17) Λ = 1− N model K (3.18) 52 andthedatarecordhasbeendividedintoK binsoflength2∆,i.e. K =T record /(2∆). Wechoose∆ = 2 ms as the default value. Using this metric requires some post-processing of the output record. First, we find the time of initiation for every AP for both the data and model output, and an AP is considered to have occurred whenever there is an upward deflection of the membrane voltage greater than 10 mV that is sustained for 1 ms. To help avoid false positives, we also impose a brief refractory period: no additional AP may be counted for 4 ms following the initiation of any detected AP. The total AP counts are N data and N model . Scanning through every AP in the real data (or model output), we consider the model (or real data) to have fired to a coincidental AP if there is a model AP within ±∆ p . This yields N coinc . To avoid artificially inflating the number of coincidences, we do not allow any AP in the data to correspond with more than one predicted AP. The parameter ⟨N coinc ⟩ gives the expected number of coincidences generated by a Poisson process with the same AP frequency as the model, and dividing by Λ normalizes the measure such that Γ = 0 when all coincidences occur by pure chance. The restriction that no AP correspond to more than one other causes this estimator to be equivalent to the coincidence factor “without replacement” used by Gertsner and Naud [84]. This and other measures of spike-train similarity are discussed by Naud et al. [192]. We find the best parameter set using both NMSE and Γ as metrics, but consider Γ as the primary metric of interest. 3.1.4 Results Allresultsaregeneratedusingamodeltrainedwiththe16,384msσ 32 trainingdata-set. Wepresent results for the model predictions on the four 8,192 ms testing data-sets. Unless otherwise stated, all results are generated by either a fully second- or third-order model, i.e. Q x =Q y =Q cross = 2 or Q x =Q y =Q cross =3, and with L x =L y =5, α x =.4, α y =0.7, and θ =4.5 mV. 53 Optimal model order To evaluate model performance we use the coincidence measure, Γ, and the ratio N model : N data , whereN data is the actual number of APs to occur in the data-set and N model is the number of APs predicted. Here, we present results for models of different orders. The model order is determined by Q x , Q y , and Q cross , where Q cross is the order of cross-kernels. We constrain Q y ≤ Q x , and obviously Q cross ≤min{Q x ,Q y }. Results for models of all possible combinations of Q x , Q y , and Q cross up to third-order, subject to these constraints, are given in Figure 3.2. Also included for comparison is a fifth-ordermodelwithnoautoregressivecomponent(Q x =5,Q y =0). FromFigure3.2,themodels cluster into two groups: in the first group all models give comparable, relatively high performance, while performance is quite poor in the second group. The four high performance models are: (1) Q x = Q y = Q cross = 2; (2) Q x = Q y = Q cross = 3; (3) Q x = Q y = 3, Q cross = 2; and (4) Q x = 3, Q y = Q cross = 2. Thus, we conclude that the the model must be at least second-order in both the exogenous and autoregressive input, and inclusion of cross-terms to at least second-order is essential. Model performance is uniformly better for high-power inputs. The low performing group may be subdivided into two clusters, one with mid-low performance. Interestingly, the mid-low group includes all those models first-order in the autoregressive term, while the lowest-performing models are second- or third-order in the autoregressive component but lack cross-terms. The fifth-order MDV (exogenous input only) performs in the mid-low group. We have also examined the NMSE of the estimated models on testing data-sets to determine if this metric, which is used to estimate the model expansion coefficients, agrees with the AP-related performance criteria. As shown in Figure 3.3, the same rough clustering into three groups is seen, 54 but differences in performance are much less dramatic and are not even detectable for the lowest- power input. Interestingly, the weaker models perform worst under the AP-related criteria for the lowest-power input. Of the four high performance models, the weakest is that with Q x = 3, Q y = 3, Q cross = 2, while the other three give essentially identical performance. It is surprising that the second-order model (i.e Q x = Q y = Q cross = 2) is as good as the third-order models. Even more unexpected is the fact that the second-order model out-performs the model third-order in the self-kernels but only second-order in the cross-kernels. Results for the performance of the fully second-order and third-order models for several different α y values are given in Figure 3.4. The third-order model gives slightly better results for the σ 32 data-set, but the second-order model is superior for lower- power inputs. Furthermore, the second-order model performance is less sensitive to the choice of α y . Given the clear parity of the second and third-order models, but the much greater parsimony of the second-order model, we generally favor the second-order model. However, we do find that compared to a fully third-order model with Q x = Q y = Q cross = 3, the second-order model gives markedly inferior performance in response to a constant current injection, as discussed further in Section 3.1.4. Other structural parameter selection and parameter sensitivity In this section we present our method and results for choosing the model structural parameters, M,R,L x ,L y ,α x ,α y , andθ, given the model order (Q x ,Q y , andQ cross ). We haveM =µ x /T and R =µ y /T, whereµ x andµ y are the system memories for the input and autoregressive components. We have T =.2 ms, and we assume a maximum memory of 20 ms, giving M =R =100. 55 Figure 3.2: Model performance for a variety of model orders as assessed by the coincidence factor, Γ (left panel) and the N model : N data ratio (right panel). The four high performing models are those with Q x ≥ 2, Q y ≥ 2, and Q cross ≥ 2. Note that performance is plotted for the four testing data-sets. Figure 3.3: Model performance as assessed by the NMSE for a variety of model orders. Assessment by the NMSE is in general agreement with assessment by the coincidence factor and the N model : N data ratio (see Figure 3.2). Results are for the four testing data-sets. Note that instabilities in the numerical solution for the Q x =Q y =3,Q cross =0 model result in the NMSE being undefined for the σ 16 and σ 32 testing data-sets. 56 Figure 3.4: NARV model performance for the fully second and third-order models using different values of α y . As described in Section 3.1.3, for given structural parameters, we estimate the expansion co- efficients using the 16,384 ms σ 32 training data-set (see Section 3.1.2 for the data preparation), and we use the NMSE and coincidence factor Γ to evaluate performance under the given structural parameters. Weperformthefollowingproceduresforboththefullysecond-andthird-ordermodels. It is our experience that 3–7 DLFs is adequate for most applications, and model parsimony strongly favors as few DLFs as possible. We have performed a Nelder-Mead simplex search for the best α x , α y , θ under L x = L y = 3,5,7 using the σ 32 and σ 4 testing data-sets, and find that model performance under L x =L y =5 is superior to only three DLFs, but is comparable to using L x =L y = 7. We have also observed that for L x =L y ≥ 9 instabilities in the solution sometimes occur, and so we fix L x =L y =5. We now perform a more exhaustive exploration of model performance in the α x , α y , and θ parameter space. To ensure that the estimated model applies over the entire dynamic range of 57 input, we calculate the NMSE and Γ for all four 8,192 ms testing data-sets, where σ =32,16,8,4. Moreover, foreachtestingdata-set, wegenerate3DvolumesofNMSEandΓasfunctionsofα x ,α y , and θ. From inspection of these volumes, we find a range of α x , α y , and θ that give good results over the entire dynamic range of input. We find that NMSE and Γ for all testing data-sets are essentially independent of α x when α x is in the range [0.4,0.9]. However, inspection of the AP waveform shows that larger α x values can give “jitters” in the membrane potential at the AP peak, and so we fix α x =0.4. Such choppiness in the waveform also sometimes occurs for smaller α y . The optimal value of α y varies with θ, but in general the best fit occurs when α y ∈ [.6,.8]. While θ<3 mV can give good results, we consider such values to be physically unlikely, and they give excessive firing for small specialized inputs as discussed below. Therefore, we restrict θ∈[2,8] and examine NMSE and Γ as 2-D functions (surfaces) of θ and α y for the testing data-sets (we do not present these surfaces in the interest of space). In general, a larger θ gives better results for higher power input, while smaller θ values are best for low-power inputs. We also take the average of the four surfaces, and peak values for the average Γ surface occur for α y ∈ [0.675,0.75] and θ∈[3.0,4.5], with the overall maximum at α y =0.725 and θ =3.5 mV for a third-order model. To refine and constrain our estimate of the threshold, θ, we also have examined model response to two specialized inputs: (1) a 1 ms pulse of injected current, and (2) a constant current injection. Wevarythecurrentmagnitudeoverthesubthresholdtosuprathresholdtransitionandweconclude thatθ≥4.5mVgivesthebestresponsetosmallpulsesofcurrent. Unfortunately,themodelappears to predict an overvigorous response to a constant current injection regardless of θ. The values forθ andα y also affect the I-f curves for constant current injection. This is discussed further in Section 3.1.4 (see Figure 3.10). We use α y =0.7 and θ =4.5 mV as default values for all results. 58 The results of this procedure are similar for the second and third-order models, but the param- eter sensitivity is smaller for the second-order model (i.e. a broader range of α y and θ values give good performance), and the range of α y for optimal performance is shifted toward larger values. We also find that the data-record need not be as long for training of a second-order model. Model time-series predictions Using the model estimated for the σ 32 training data-set, we predict the membrane potential time- series for the four 8,192 ms testing data-sets. That is, the model is given the input record and predicts the output; we compare model predications against the observed output. Figure 3.5 shows a 500 ms windows of the membrane potential for the σ 32 and σ 4 testing data-sets. To avoid any bias in what sample window we present, we arbitrarily choose to begin each window at 1000 ms. Similarly,Figure3.6gives1500mssamplesofthepost-processedspiketrains,beginningatt=1000 ms. Predicted versus actual interspike interval distributions We run the model on the 32,768 ms testing data-sets and determine the interspike interval his- togram. Figure 3.7 shows the histograms predicted by a fully second-order model and the actual histograms for all four data-sets. It is clear that the NARV model accurately predicts the distribu- tion of interspike times for the H-H model given a noisy input across a broad range of input-power, even if some individual spike times are in error. Model predictions for specialized inputs We evaluate the ability of the model to predict response to two special types of exogenous input: (1)acurrentpulse, and(2)aconstantcurrentinjection. AsdiscussedinSection3.1.4, theseresults 59 Figure 3.5: Voltage tracings giving the second-order NARV model prediction and actual data for the σ 32 and σ 4 testing data-sets. Figure 3.6: Comparison of the second-order NARV model-predicted spike trains versus the actual spike trains for all four testing data-sets. 60 Figure 3.7: Predicted and actual interspike time histograms for 32,768 msσ 32 ,σ 16 ,σ 8 , andσ 4 data records. The model is fully second-order. have also been used to help determine the appropriate firing threshold, θ. We have also examined the response to a sinusoidal current and have found that the model generally gives good results, but in the interest of space we omit such results. Current pulse We construct an input sequence consisting of 1 ms wide pulses of current. The current amplitude begins sufficiently small such that no AP is fired, and the amplitude is slowly incremented to the point that APs are fired. We find that model predictions for this input type are sensitive to the choice of θ, with θ<4.5 mV giving spurious AP firing. For the fully third-order and second-order models,predictionsmatchthedatawellandthemodelpredictedAPmorphologygenerallymatches the actual APs very nicely; Figure 3.8 gives results for the second-order model. If cross-kernels are 61 Figure 3.8: Data is generated by delivering a sequence of 1 ms pulses of current every 35 ms to the H-H equations. The amplitude of the first pulse is 3 µA / cm −2 and is increased by 1 µA / cm −2 every subsequent pulse. The same input record is given to our estimated model, which gives comparable results. Here, the model is fully second-order. not included, the model fails to predict AP firing for a current pulse of less than 16 µA cm −2 , yet APs are generated for a pulse of only 7 µA cm −2 . Constant current injection In response to a constant current of sufficient magnitude, an indefinite spike train with a constant interspike time occurs. The NARV model successfully predicts this behavior, and several examples of the predicted membrane potential time-series and actual time-series are shown in Figure 3.9. Contrary to our other findings, the third-order model is markedly superior to the second-order model in this setting. We emphasize that a (finite-time) Volterra model that only considers the exogenous input is unable to produce such limit cycle behavior. Since such a model is finite-time, the input epoch is identical at all times, and clearly the model cannot produce different outputs for the same input epoch. However, the NARV model does converge to a limit cycle for a constant exogenous input epoch. This is because while the exogenous epoch is constant, the autoregressive component is non-constant and regularly switches between the suprathreshold and supthreshold regime. We find thatmodelfirst-orderinboththeexogenousandautoregressivecomponentiscapableoflimit-cycle 62 behavior, although the form and timing of the AP train is greatly in error. Second- and third-order models perform better, and we find that for these higher order models cross-kernels to at least second-order must be included for sustained spiking. While the model yields the correct qualitative behavior, there is typically significant quantita- tive error. For the H-H generated data, a constant current below 7 µA cm −2 fails to generate a continuous spike train, while the model predicts a continuous spike train even for very small cur- rent injections. As the current is increased, the model and data come into better agreement, and we observe a range of current values where model and data agree nearly perfectly: the interspike intervals are equal, and the precise timing of the actual and predicted APs coincides. As current is increased further, the interspike times diverge, and most APs are no longer coincidental. Several example time-series are shown in Figure 3.9. We generate a series of I-f curves, which give the frequency of AP firing for 1000 ms of con- stant current injection. The third-order model and data agree reasonably well over most of the current range considered (up to 64 µA cm −2 , as this is roughly the dynamic range over which the model is trained). The second-order model gives qualitatively reasonable results, but dramatically overestimates the number of APs. I-f curves and the interspike intervals are given in Figure 3.10. The refractory period is correctly predicted The NARV model succeeds in predicting the existence of a relative and absolute refractory period following an AP. By “absolute refractory period,” we mean an interval of time following an AP in which even a very large stimulus is incapable of evoking a second AP. In reality, we have observed that a truly massive current injected arbitrarily soon after the first AP can induce an AP-like waveform. Therefore, all refractoriness is essentially relative, and our use of the term “absolute” 63 Figure 3.9: Data is generated by delivering a constant injection of current, beginning at t=20 ms. Comparisons of the model prediction versus H-H data for injections of 10 and 30 µA / cm −2 of current are shown for the second-order (left panels) and third-order model (right panels). While both give qualitatively correct results (sustained spike-trains), the third-order model is somewhat superior. For both models, α y =0.7 and θ =4.5 mV. 64 Figure 3.10: The left panels show I-f curves plotting AP firing frequency as a function of injected current (constant current injection), and the right panels give the interspike intervals. Each panel gives results for the model trained with α y = 0.7, 0.75, and 0.8, along with the actual data. The top panels give results for the second-order model, and the bottom panels show third-order model results. Note that the sudden jump in AP frequency for the second-order model with α=0.7 at a current injection of about 30µA cm −2 represents a transition from proper APs to rapid membrane fluctuations that do not really constitute true APs, but still meet our chosen AP criterion (10 mV positive membrane deflection sustained for 1 ms). Such a transition occurs in the H-H model as well, but at larger current injections. 65 is somewhat loose. By “relative refractory period,” we mean an interval following the absolute refractory period during which large (but realistic) stimuli can evoke an AP, but smaller stimuli which would otherwise trigger an AP still fail to. We perform a numerical experiment where we inject a 1 ms pulse of current of 10 µA cm −2 . After a small interval of time, we inject a much larger pulse of 50 µA cm −2 . As shown in Figure 3.11, for an interval of less than 9 ms, no AP is fired, demonstrating the existence of a (nearly) absoluterefractoryperiod. Ifwefollowtheinitialcurrentpulsewithapulseofthesamemagnitude, no AP is fired until an interval of 15 ms has elapsed, demonstrating the relative refractory period. We find that the fully third-order model matches the data best, although the second-order model is nearly as good. While it does not appear that cross-terms are strictly necessary for the model to yield a refractory period, results are very poor if the cross-kernels are omitted. Shape of the estimated Volterra kernels We graphically present the Volterra kernels estimated for the second-order model with α y = 0.7 and θ = 4.5 mV. First and second-order self-kernels for the exogenous and autoregressive inputs are shown in Figure 3.12 and 3.13, respectively. The second-order cross-kernel is shown in Figure 3.14. Model performance under noisy data To test the robustness of the model estimation process in the presence of noise-contaminated data, we add Gaussian white noise to both the training and testing data-sets. We have generated results forSNR dB =10,3,and1. Figure3.15demonstratesthattheabilitytopredictAPs,asmeasuredby thecoincidencefactorΓ,isonlyappreciablydegradedbyanSNR dB =1. A300mswindow(starting 66 Figure3.11: Numericalexperimentdemonstratingthatthemodelsuccessfullypredictstheexistence ofboth(nearly)absoluteandrelativerefractoryperiods. Thebarsshowthetimingofcurrentpulses, with the area of the bars proportional to the amplitude of the pulse (all pulses are 1 ms in width). For this figure, the model is third-order. The second-order model is nearly as good, but predicts a slightly shorter refractory period (e.g. 14 ms instead of 15 ms for the second experiment). 67 Figure 3.12: The first and second-order self-kernels (k 1;0 and k 2;0 ) for the exogenous input, x(m). Figure 3.13: The first and second-order autoregressive self-kernels (k 0;1 and k 0;2 ) for the autore- gressive input, ˆ y(m). 68 Figure 3.14: The second-order cross-kernel, k 1;1 . at 1000 ms) of the noisy membrane potential time-series and second-order model prediction for the σ 32 testing data-set is given in Figure 3.16. These results give some confidence that the model may be successful applied to noisy recordings of real neurons. Model estimation and predictions under purely subthreshold dynamics We briefly consider the case when membrane dynamics remain purely subthreshold. The autore- gressive component of the NARV model is identically zero, and the model reduces to a single-input MDV model. We find that the NARV model estimated for data that varies between the subthresh- old and suprathreshold regime (i.e the σ 32 data-set) gives satisfactory performance on a purely subthreshold data-set. However, we also estimate a first-order MDV model directly from a sub- thresholddata-series(8,192ms,σ =0.5),andfindthatsuchamodeloutperformstheNARVmodel operating solely in the subthreshold region. The shape of the first-order kernel also varies slightly between the two models, being a positive integrator for the NARV model, while the MDV model yields a first-order kernel that is mostly positive, but briefly becomes negative for large delays. 69 Figure 3.15: NARV model performance on noisy data-sets as assessed by the coincidence factor. The left and right panels give results for the fully second and third-order models, respectively. Note that in each case the model is estimated from the σ 32 training data-set contaminated with the indicated level of noise, and performance is assessed on the indicated training data-set also contaminated with the same level of noise. 70 Figure 3.16: Voltage tracings for noisy testing data-sets and second-order model (estimated using noisy data) predictions. From top to bottom the panels gives results for SNR dB = 10, 3, and 1. 3.1.5 Discussion We have proposed a new methodology for input-output modeling of AP generation at the axon hillock of a neuron that utilizes a nonlinear autoregressive Volterra-type (NARV) framework to represent the causal relationship between the injected/somatic current (input) and the membrane potential (output). The NARV model represents its output, a putative membrane potential at the axon hillock, in terms of the input current and feedback from the generated (suprathreshold) APs. This model takes the general form of a Volterra model with two inputs, namely the actual current input (corresponding to a nonlinear “forward” or “moving-average” component) and the suprathresholdAPs(correspondingtoanonlinear“feedback”or“auto-regressive”component). The latter is active only when the neuron is in the “excited state” of generating an AP and accounts for post-firing processes, such as refractoriness and after-potentials. 71 The NARV model, as a two-input Volterra model, has cross-terms representing the dynamic interactionsbetweenthetwoinputs(inputcurrentandgeneratedAPs)astheyaffectthemembrane potential output. The sequence of generated APs can be predicted by the NARV model for any currentinputwithinthedynamicrangeandbandwidthoftheinputensembleusedforitsestimation, by applying a fixed-threshold operator on the model output (membrane potential). The efficient estimationoftheunknownkernelsoftheNARVmodelisaccomplishedwiththeuseofband-limited Gaussian white-noise current input and Laguerre expansion of the kernels [164]. A schematic of the NARV model configuration is given in Figure 3.1. We have found that 5 Laguerre functions (with appropriate Laguerre parameter α) are adequate for representing each set of kernels for the two inputs. A NARV model of second-order is adequate for predicting the simulated H-H data for random broadband inputs, but the third-order model yields better predictions for constant current inputs. Our main findings are: 1. The proposed NARV model predicts well the simulation outcome of the H-H equations for all input waveforms within the dynamic range and bandwidth of the input ensemble used for its estimation, both in the subthreshold and suprathreshold regions. 2. The predictive performance of a second-order NARV model is satisfactory for all input wave- forms except constant current. A third-order NARV model is found to perform considerably better for constant input current. 3. The performance of the NARV models is better than their Volterra counterparts. In this regard, it is critical to note that, unlike a Volterra model, the NARV model exhibits limit- cycle behavior in response to a constant current input. 4. TheestimationoftheNARVmodelkernelshasbeenfoundtobeaccurateandefficientviathe 72 Laguerre expansion technique over the entire dynamic range of inputs. However, the model predictions are best for high-power random inputs. 5. The NARV model reproduces precisely the after-potential and refractory characteristics of the H-H model for all inputs. 6. Theinclusionofcross-kernelsdescribingthenonlinearinteractionbetweentheexogenouscur- rent input and the autoregressive input (i.e. the feedback of suprathreshold APs) is essential for satisfactory model performance. For higher-order models (Q x ≥2,Q y ≥2), the inclusion of cross-terms appears to be essential for generating limit-cycle behavior in response to a constant current input. 7. The forward component of the NARV model (i.e. the terms that depend only on the input current)isalmostlinear(seeFigure3.12,wherethecontributionofk 2;0 isrelativelyminor)and its first-order kernel exhibits an integrative characteristic. To confirm that the second-order forward kernel is indeed unimportant, we estimated a second-order NARV model lacking this kernel, but retaining the second-order cross-kernel, and found that its predictive capabilities are comparable to the full second-order model (results omitted for space). 8. Accurate waveforms of the AP and the refractory period are obtained directly from the estimatedNARVmodel, andnopriorassumptionsareneededforthatpurpose,asinprevious studies. 9. The only requirement for NARV model estimation is the availability of broadband input- output data that test the system under a variety of input conditions. This makes the data- based NARV model suitable for natural operating conditions. This desirable property and the good performance of the NARV model under noisy simulated data (see Section 3.1.4) 73 suggest that it may be capable of predicting real neuron firing in response to arbitrary input waveforms, but this conclusion must be confirmed by testing the model against real data. 10. The threshold of the NARV model has been determined through a process of successive trials of model estimation and performance evaluation. It is a somewhat curious result that model performance is better for high-power inputs, for which it is trained, than low-power inputs, although we offer a heuristic explanation. Suppose the membrane potential is subthreshold and the input is low-power white-noise. Then each change in the input current has a small effect on the membrane potential. If such a change pushes the membrane over threshold, it shall be only by a very small margin. Thus, the model must track membrane potential very precisely to accurately predict this threshold crossing. This suggests that any model with a threshold for firing may be intrinsically less reliable for low-power versus high-power inputs. The model is also trained much more efficiently by high-power inputs, which is likely due to the fact that the neuron membrane operates in two distinctly different dynamic regimes: subthreshold and suprathreshold. High-power inputs provide adequate data on both dynamic regimes and fa- cilitate accurate estimation of the model for both regimes – something not possible for low-power inputs that do not provide adequate data in the suprathreshold regime. It is possible that the system dynamics are marginally different for low-power compared to high-power inputs. Therefore, we have tried training the model with a mix of input power levels, but this does not improve performance over simply training with only high-power input (results not shown). TheNARVmodelpredictsfiringinresponsetoacurrentpulsequitewell,butforanyreasonable value of θ sustained firing in response to a constant current occurs at current injections much too 74 low (see Figure 3.10). The threshold θ must be increased unreasonably high for this behavior to be avoided. Kistler et al. [131] found that their model similarly overestimated AP firing in response to small injected current. From this, they suggested that there exists no strict membrane potential threshold for firing. Our results corroborate this conclusion. We have also examined the somewhat similar case of anode break excitation, where the membrane potential is hyperpolarized for an extended period. Upon returning to rest, a spontaneous AP occurs as a result of sodium channel activation. The NARV model can only give this behavior if θ is made unreasonably low (results omitted for the sake of space). Thus, we can conclude that an H-H neuron is not strictly a threshold element, but the threshold approximation applies well for most natural inputs (e.g. broadband noise). How to account for this in a Volterra framework is unclear, but to do so in a future iteration is a worthwhile goal. It is unclear whether a second-order or third-order NARV model is preferable for equivalent representation of the H-H equations. The two give comparable results for most input waveforms (including random) and the second-order NARV model has fewer free parameters. In that sense, it appearstobepreferablebecauseitgivessatisfactoryperformancewithgreaterparsimony. However, thethird-ordermodelperformsconsiderablybetterforconstantinputcurrentsandsinusoidalinput currents (results are not presented in the interest of space). The third-order model is also slightly betteratpredictingmembraneresponsetoinputduringtherefractoryperiodfollowinganAP.This suggests that while much of the essential behavior of the H-H model is captured by second-order nonlinearities, third-order nonlinearities do exist and have subtle effects on model performance. Thisfactnotwithstanding,thesecond-ordermodelhastheimportantpracticaladvantagesofgreater parsimony and requiring shorter data-records for reliable estimation. This trade-off has to be examined in each case based on the specific prevailing considerations. 75 Other models of spiking neurons use an autoregressive structure similar to the one proposed here. Pillow and colleagues [205] have proposed the generalized integrate-and-fire (IF) model (or “generalized linear model”) as a successor to the SRM. This model considers a leaky integrate- and-fire spike generator driven by a linearly filtered stimulus, a spike history-dependent feedback current, and a noise current. A spike fires whenever the membrane exceeds some threshold, and the feedback current is given by convolution of the past spike train with a fixed afterpotential current waveform. This model is conceptually similar to a linearized version of the NARV model, with the feedback current analogous to the thresholded feedback of the NARV model. Despite this conceptualsimilarity,thereareseveralcrucialdifferencesbetweenthemodels: thegeneralizedlinear model generates discrete spikes, and the underlying membrane potential is automatically reset to zero whenever a spike occurs. The NARV model continuously tracks membrane potential and the “reset” of the membrane potential following a spike is not imposed. The feedback component in the generalized linear model is a convolution with previous discrete spikes, while the feedback component in the NARV model is a non-linear convolution with the past continuous membrane potential. The generalized linear model has been used in other recent works, such as that by Pillow et al. [206], who considered stimulation of a population of such neurons with the inclusion of couplingfiltersthatallowthespikingactivityofnearbycellstoaffecteachother, andthatofMensi et al. [177]. Volterra models of input-output transformations of spike trains in the hippocampal formation [225, 226, 16, 98, 99] also employ a threshold for firing along with a linear feedback kernel for the thresholded output. This model formulation is similar to the current work, but has the important difference of employing a linear feedback (as opposed to nonlinear feedback for NARV with cross- interaction terms). It is also dissimilar from the NARV model in that it considers point-process, 76 ratherthancontinuous,data. Consideringthemembranepotentialinacontinuousmannerappears to affect the necessary autoregressive model order. In the current work, we have found that non- linear autoregressive components with cross-terms are essential to accurately capturing the precise membrane dynamics during and after AP firing. This has not been the case in the aforementioned works which consider point-process data, where a linear feedback term has proven sufficient. This work adds to a small existing literature on autoregressive Volterra-type modeling. For example, Barahona and Poon [10] proposed a closed-loop version of the discrete Volterra-Wiener- Korenberg series and proposed a methodology by which it can be used to detect the presence of nonlinear determinism in experimentally obtained time-series. Schiff et al. [221], following [247], appliedanonlinearautoregressive(NLAR)modelequivalenttoasecond-orderautoregressive VolterramodeltoanalysisofEEGsignals. However,becauseoftheproblemofparameterexplosion they only estimated NLARs with a single nonlinear term. As previously mentioned, Chon and colleagues [40] proposed applying the Laguerre expansion technique to estimation of autoregressive Volterra models, avoiding the curse of dimensionality that plagued earlier works. We note that an autoregressive term can be equivalent to an infinite series of moving average terms, and previous studies have examined the relation between classical Volterra series and non- linear (autoregressive) difference equations [55, 123, 269, 270]. This relates to the current work in that it suggests that the autoregressive Volterra model may admit a classical Volterra (or moving average) representation, but with many higher order terms. Itisalsousefulforfuturestudiesofthisproblemtosummarizeourunsuccessfulefforts. Wehave found that autoregressive modeling without a threshold does not improve results over a traditional Volterra model and frequently results in numerical instability. Likewise, derivative feedback fails to improve results. Given the practical problems arising from the ”stiffness” of a hard threshold 77 (θ), we experimented with a soft (sigmoidal) threshold, but this typically resulted in somewhat inferior results. Of all our efforts, only those with a thresholded autoregressive component yield the essential limit-cycle behavior of the H-H model for constant current inputs. In conclusion, the proposed NARV model seems to offer a data-based alternative to the H-H model of AP generation with potential implications for real data analysis and large-scale neural integration. 3.2 PrincipaldynamicmodeanalysisoftheHodgkin-Huxleyequa- tions 3.2.1 Introduction Following the development of the NARV model detailed in the previous section and in [65], we now analyse the H-H system using PDMs; this work is currently under review [66]. The concept of principal dynamic modes (PDMs) posits that the (time-dependent) dynamics of a system may be represented by a small set of basis functions, unique to that system [162]. These PDMs may be estimated from the NARV model and used as a basis for a PDM model that is mathematically equivalent in structure to the NARV model, but having the crucial advantage of an equivalent modular representation that can reveal the functional characteristics of the system. In this section, we apply the PDM concept to the H-H system to demonstrate that it allows very significant reduction in model complexity at minimal cost to performance, while much better revealing the underlying system’s structural dynamical relationships among the exogenous and autoregressive components. Ultimately, the second-order NARV model, with 65 free coefficients, can be reduced toaneightparameterPDMmodelwithonlyamarginalperformancecost,ortoonlyfourparameters at somewhat higher cost. Note that this also compares favorably to the 24-parameter system of 78 differential equations that defines the H-H system. The proposed PDM methodology is intended as a general method which may be used to describe any spiking neuron, simulated or real, and the H-H system is used as a well-studied test case. While displaying excellent predictive performance, the NARV model is difficult to interpret, either by examining the reconstructed Volterra kernels, or through its modular representation, which consists of 10 DLFs and 65 coefficients. This motivates the application of the PDM concept, in which we estimate a functional basis for kernel expansion that is specific to the particular system under study. We separately estimate forward and feedback PDM bases from the forward and feedback Volterra kernels, using singular value decomposition. The two PDM bases then take the place of the corresponding Laguerre bases in the forward and feedback filterbanks. The nonlinearity is further decomposed conceptually into forward and feedback self- and cross-terms. Thus, we move from a generic modular representation to a system-specific modular representation, but preserve the basic mathematical framework. ThePDMmodelisalsohighlyamenabletoreduction,becausewemayselectahighlyinfluential subset of PDMs as a basis. The PDMs can be ranked by singular value, which gives a rough indication of their relative importance, and the subset may be selected using this criterion [171]. However, at least in this work, we have found this ranking to be unreliable (with the exception of thehighestrankedPDM),andweusealeave-one-out/add-one-in(LOO/AOI)algorithmtoidentify the reduced PDM bases. This algorithm identifies two forward and four feedback PDMs that are significant, yielding a 27-parameter (27-P) basis-reduced model with performance comparable to that of the NARV model. Given the reduced PDM bases, we perform further model reduction by pruning the coefficients ofthestaticnonlinearity. Weiteratethrougheachcoefficientandleaveitoutofthefullmodel;ifthe 79 omission of a coefficient results in a statistically significant reduction in performance, it is retained, otherwise it is omitted. Several summary statistics are used to quantify model performance, and bootstrap procedure is used to estimate the variance of these statistics. Using these statistics and bootstrap procedure, we thoroughly demonstrate that the basis-reduced and coefficient-pruned PDM models give performance comparable to or only minimally inferior to that of the full model. The principal focus of this paper is on four particular models: the 65-parameter (65-P) full PDM model, the 27-P basis-reduced model, and 8-P and 4-P coefficient-pruned models. We have compared final coefficient-pruned models where the PDM basis reduction step is per- formed first versus using the entire PDM basis and found that reducing the PDM basis set first always results in a better final model. Thus, eliminating coefficients associated with insignifi- cant PDMs before pruning appears to improve the performance of the pruning algorithm by pre- eliminating these spurious coefficients. We have also generated curves plotting the number of retained coefficients versus the perfor- mance metrics, where those coefficients found to have the least influence on predictive ability are removed in succession. We find that the inclusion of a significant number of coefficients actually reduces model performance, probably due to over-fitting. Examining the structure of the reduced and pruned PDM-based models shows that, within the subthreshold regime, the H-H membrane acts as a “leaky integrator” with a memory of approx- imately 5 ms, in accordance with the widely posited leaky integrating characteristic of the H-H membrane[132]. The leaky integrating characteristic inferred from the data-based analysis, how- ever, is not a simple exponential and appears to have three different time-constants. Furthermore, unlike leaky-integrator neurons, the PDM model does not simply reset the membrane potential upon threshold crossing, but accounts for the AP and associated afterpotential and refractory pe- 80 riod through a separable early AP waveform component, representing the invariant AP response to threshold crossing, and bilinear interactions among forward and feedback PDMs to give complex peri-AP phenomena, e.g. refractoriness. The PDM model also differs from the previously pro- posed Spike Response Model (SRM)[131], which may be viewed as a generalized integrate-and-fire model[121, 122]. The SRM convolves the injected input with a linear integrating kernel, while the AP and feedback is accounted for by a linear AP and afterpotential kernel. The SRM lacks the bilinear cross-terms of the PDM model that have proven, in our model framework, essential to accurately representing the AP waveform and accurate model predictions. We have also generated H-H data-sets with the underlying H-H ion channel dynamics altered, specifically the maximum sodium and potassium conductances. We impose a minimal four coef- ficient architecture, using the “universal PDM basis” identified on the standard H-H system, and examine how the coefficients change systematically with changes in the ionic conductances. We find a clear pattern in which increases in sodium conductance diminish coefficients associated with the AP and refractoriness, whereas increased potassium conductance has the opposite effect. The overall effects, however, are not simple inverses of each other. Thus, changes in the underlying biophysics are reflected in an orderly fashion in the highly reduced data-based PDM model. The proposed methodology represents a general method for nonlinear system identification that may, at least in principle, be applied to systems with multiple inputs and outputs, and the outputs may or may not be autoregressive. The method is practically useful in that the same mathematical framework is used at all steps of model identification. That is, the input-output relation is expressed as a family of functional bases that transform input as linear filterbanks, and an associated polynomial nonlinearity. This polynomial may be conceptually subdivided into various self- and feed-back terms, allowing better visualization of the model structure. 81 In sum, we have proposed a methodology for system identification based on the Volterra series and PDM concepts that (1) uses the same basic mathematical framework as the NARV model and modified discrete Volterra (MDV) model that has been widely used in physiological modeling [164], (2) is amenable to dramatic reduction in model complexity, as defined by the number of free model coefficients, by first reducing the PDM basis sets and then pruning the coefficients of the static nonlinearity, (3) has a modular structure specific to the system under study that can be visualized and interpreted much more readily than kernel-based Volterra models, and (4) uses a flexible and computationally straightforward bootstrap procedure for statistical evaluation of all model components (PDMs and expansion coefficients). Application of the method to the H-H system yields an eight-parameter PDM model that is comparable to the full 65 parameter NARV model. 3.2.2 Methods Data preparation As in Section 3.1.2, we have generated time-series data for the Hodgkin-Huxley (H-H) model[105] with injected current, I(t), as input, and membrane potential, V(t), as output. We generate data-sets using white-noise current values drawn from a normal distribution with zero-mean and standard deviation σ = 32 µA cm −2 , with new values chosen at 1 kHz. The sampling interval, T, is 0.2 ms. We also generate several data-sets with altered H-H parameters, namely, the maximum potassium and sodium conductances, ¯ g K and ¯ g Na . Thetrainingdata-sets,fromwhichthemodelcoefficientsareestimated,are16,384msinlength, andwegenerate 10 8,192 ms testingdata-sets for model validation. Werefer to all 10 concatenated as the full testing data-set. 82 Model structure and estimation The general form for both the NARV and PDM models is mathematically identical and is given in Section 3.1.2. This form can be represented modularly as two linear filterbanks, consisting of the chosen bases, receiving exogenous and autoregressive inputs and whose outputs are fed to a static nonlinearity in the form of a polynomial expansion (see Figure 3.17). Any set of basis functions may be chosen, and the fundamental difference between the NARV and PDM models is that in the former a more generic set of discrete Laguerre functions (DLFs) is used, the Laguerre expansion technique [162], while in the latter the basis is specific to the system under study. For the PDM model, the two sets of principal dynamic modes (forward and feedback) makeupthebasissets, andwepresenttheestimationofthesePDMsfromtheNARV/kernelmodel in Section 3.2.2. Under the PDM model, we decompose the polynomial expansion into self- and cross-terms (somewhatanalogoustotheself-andcross-kernels). Furthermore, thecross-termsmaybeforward- forward, feedback-feedback, or forward-feedback. For a second-order expansion, the self-terms are quadratic, while the cross-terms are bilinear in their arguments, i.e. just a constant. An example of such a modular architecture for two forward and two feedback PDMs is given in Figure 3.17. As discussed further in Section 3.2.2, the PDM model is also pruned by testing the significance of each expansion coefficient. PDM estimation The second-order NARV model is estimated as detailed in Section 2.3.1, and the PDMs are esti- mated as described in Section 2.4. 83 Figure 3.17: Example PDM model architecture for two forward and feedback PDMs. Each PDM- defined filter-bank receives its respective input and generates PDM outputs that are transformed by polynomial static nonlinearities and summed to form the model output. The static nonlinearity is decomposed into (a) self-forward, (b) cross-forward, (c) self-feedback, (d) cross-feedback, and (e) cross-forward-feedback terms. For the second-order architecture, the self terms are quadratic, while the cross terms are bilinear (i.e. a scalar multiple). 84 Model reduction: PDM subset selection We reduce model complexity in a two-step process: (1) identify a subset of PDMs that have the strongest influence on the output, (2) using such a reduced subset of PDMs as a basis, prune the expansion coefficients. Here, we discuss the first. In most previous works, the PDMs have been ranked by eigenvalue or singular value, and some subset based on this ranking (e.g. top four, retain those PDMs with eigenvalues at least 1% of the maximum, etc.). However, in this work the four least significant singular values are four to five orders of magnitude smaller than the first singular value, and we have found that for any PDM but the first, the singular value is not a reliable indicator of its importance. We suspect that this is related to NARV model estimation errors. PDM selection We perform a leave-one-out (LOO) process by which 10 reduced PDM models are estimated, each omitting a single (forward or feedback) PDM. The performance of each reduced model is compared to the full model, and if the performance is found to be significantly inferior as measured by the NRMSE, using the bootstrap procedure detailed in Section 3.2.2, the omitted PDM is deemed significant. The LOO procedure yields a subset of significant PDMs that form the bases for the “LOO- reduced PDM model.” We then use the LOO-reduced model as the baseline model and perform an add-one-in (AOI) procedure, whereby each omitted PDM is added in turn, and the performance of theresultingmodeliscomparedtothatoftheLOO-reducedmodel. IfaddinganyPDMsignificantly improves the NRMSE, it is added back to the basis set. The basic rationale for this procedure is that while the LOO step tells us what happens when leaving any one PDM out, is says nothing about leaving two, three, etc. PDMs out, and the AOI step is an attempt to compensate for this. 85 ThisLOO/AOIproceduremaybeperformedanarbitrarynumberoftimestozero-inonanoptimal basis set of PDMs (somewhat analogous to a binary search). We refer to the final model arrived at via this procedure as the “basis-reduced PDM model.” Model coefficient pruning In this section we describe our method for reducing PDM model order by pruning the insignificant expansioncoefficients. Then,usingourdecompositionofthepolynomialintovariousself-andcross- terms(seeFigure3.17),retainingonlythesignificanttermsallowsvisualizationofthemodel’smost important structural characteristics (see, e.g., Figure 3.19 in the Results). Instead of building the list of significant coefficients from the “bottom-up” by adding successive coefficients and determining if they significantly improve prediction accuracy, as has been done in previous works[266], we prune the coefficients list from the “top-down,” using a LOO approach. That is, we iterate through every coefficient and remove it from the full list, and determine if the “reduced-by-one” model’s performance on the testing data-set is significantly diminished. The coefficients of each reduced-by-one model are re-estimated from the training data-record. This approach minimizes the risk of missing important interactions among terms that may be missed in the bottom-up method. Under decomposition of the polynomial expansion, the self-forward and self-feedback terms are quadratic and encompass two coefficients. Therefore, we also assess model performance with these coefficients lumped to determine if the two-coefficient self-terms are significant as a whole even if the two coefficients assessed individually do not attain significance. Finally, given a list of significant coefficients, we re-estimate the coefficients of the “pruned” model and assess model performance. As for PDM basis selection, significance is assessed by the 86 bootstrap procedure detailed in Section 3.2.2, using out-of-sample data. Since coefficient estimation is a multiple linear regression problem, it is also possible to use an F-test to determine if R 2 is significantly reduced by the omission of any coefficient, but we have found that the F-test tends to retain an excessive number of coefficients; the bootstrap method is more flexible and useful. Metrics for performance evaluation We use the normalized root-mean-square-error (NRMSE) of the continuous membrane potential as the the primary metric for model evaluation, PDM basis set reduction, and coefficient pruning. We are also interested in the comparative ability of models to predict APs in a binary (all-or-nothing) sense. Therefore, we convert the continuous membrane potential to a spike-train by imposing a threshold for detection θ D . Anytime membrane potential goes above θ D from below, an AP is recorded. We also impose a 4 ms refractory period, so that no additional AP may be recorded within 4 ms of an AP detection. To compare spike trains, we use two statistics: (1) the coincidence factor without replacement, Γ[131, 84] and (2) the K-statistic for the two-sample Kolmogorov-Smirnov test comparing the interspike time distributions. The coincidence factor is given as Γ= ( N coinc −⟨N coinc ⟩ .5(N data +N model ) )( 1 Λ ) , (3.19) where ⟨N coinc ⟩ = N data N model K (3.20) Λ = 1− N model K . (3.21) 87 The total AP counts are for the data and model output areN data andN model . An AP in the model output and actual data are considered coincidental if within ±∆; we set ∆ = 3 ms. Counting all such coincidences, with the restriction that no predicted AP may correspond with more than one actual AP and vice versa, yields N coinc . The parameter ⟨N coinc ⟩ gives the expected number of coincidences generated by a Poisson process with the same AP frequency as the model, with the data record divided into K bins of length 2∆. Dividing by Λ normalizes the measure such that Γ=0 when all coincidences occur by pure chance. The K-statistic gives the maximum difference between the empirical cumulative distribution functions of the interspike times. For the simulated data, we fix θ D = 50 mV. Choosing θ D for model predictions is somewhat problematic, as variations in predicted AP morphology may result in clearly recognizable APs that failtomeetagivenθ D . Highlyprunedmodels, inparticular, tendtogivelowerpeakAPmembrane potentials. To make our model comparisons as fair as possible, we optimize θ D for each individual model with respect to the statistic of interest (Γ or K-statistic). This allows us to compare the best possible model predictions against each other. We also consider as secondary metrics the true positive rate (TPR) and false positive rate (FPR), which are calculated following binning of spikes into 2 ms wide bins as: TPR= number of true positives Number of actual APs FPR= number of false positives Number of bins lacking an AP A predicted AP is considered a true positive if it occurs within a 1 bin margin of an actual AP. In addition, no predicted AP is allowed to coincide with more than one actual AP, and no actual 88 AP may coincide with more than one predicted AP, i.e. no double-counting is allowed. Any AP which is not deemed a true positive is counted as a false positive. Optimizing Γ tends to result in a TPR and FPR that are slightly lower than when optimizing the K-statistic. That is, Γ has a slight “preference” for decreasing the FPR at the expense of the TPR relative to the K-statistic. Wealsogeneratereceiveroperatingcharacteristic(ROC)curvesforθ D ,whichplotsTPRagainst FPR and each point represents a particular value ofθ D . The ROC curve may also be used to select the optimal θ D , with the point closest to the upper-left corner as the most typical choice[266]. However, becauseFPR isextremely lowforall feasiblevaluesofθ D (asalmost allbins lackan AP), we do not use the ROC curves to chooseθ D , but only to visualize comparative model performance. Using the NRMSE as the test statistic for PDM basis set reduction and coefficient pruning guards against any sensitivity of spike-train similarity metrics on θ D . Finally, note that θ D is not the same parameter as θ, which determines when the model autoregressive input is non-zero and is always fixed at 4.5 mV. Bootstrap procedure for estimating variance To compare model performance, we must estimate the variance of our performance metrics. We calculate the bootstrap distribution of the mean of a statistic on the testing data as follows. A given model is run for the ten 8,192 ms testing data-sets, and the model results are concatenated into the full prediction record. This record is divided into 80 snippets of 1,024 ms each. The metric of interest (e.g. NRMSE) is then calculated for each snippet, giving 80 estimates. Then, 80 samples with replacement are drawn from this set of estimates and averaged to give an estimate of the mean. 10,000 such re-samplings are performed to yield the bootstrap distribution of the mean. To, for example, determine if the NRMSE for a full and reduced model varies at the α = 89 .05 significance level, we calculate the bootstrap distribution of the NRMSE difference. If the 95% confidenceintervaldoesnotincludezero,thenthereducedmodelisconsideredtohaveasignificantly different NRMSE. We assess significance at the α = .01, .05, .10, and .20 significance levels using thetwo-sidedconfidenceinterval. Thisprocedureisperformedonout-of-sample(testing)data, and so determines the variance of the predictive performance. 3.2.3 Results Estimated principal dynamic modes The five estimated forward and feedback PDMs for the baseline H-H model are given in Figure 3.18. In the following section, we find two forward and four feedback PDMs to be significant, and re-label them, in order of significance, as X1, X2, Y1, Y2, Y3, and Y4. The major forward PDM, X1, is a simple integrator, while X2 has the characteristics of both a differentiator and a slow or delayed integrator. We also examine the filter outputs of the Y1 and Y2 in response to an input pulse. Since the autoregressive component is typically only non-zero for brief periods corresponding to APs, a pulse is a much more appropriate interrogator than a step input. Based on the LOO results in Section 3.2.3, Y1 and Y2 are the two most influentialfeedback PDMs; Y1 appears to be principally responsible for the AP waveform, while Y2 has a delayed integrator characteristic and gives the slower afterpotential. The Y3 and Y4 PDMs also have a slightly delayed response to the pulse and appear to contribute to the afterpotential or refractory period, but have rather complex responses. 90 Figure 3.18: The five forward (top row) and feedback (bottom row) PDMs ordered by significance. The two forward (X1, X2) and four feedback (Y1, Y2, Y3, Y4) PDMs found to be significant are bolded. 91 PDM basic reduction We perform the alternating leave-one-out/add-one-in (LOO/AOI) procedure described in Section 3.2.2, using the NRMSE as the test statistic. Using an α = .10 we get rapid convergence to the X1–2, Y1–4 PDM model, which we refer to as the 2-4 basis set. This model has 27 free expansion coefficients, contra the 65 of the full PDM model, and we refer to it as the 27-P model hereafter. Pruned PDM models We apply the LOO coefficient pruning algorithm to several PDM models. The difference between the full and each reduced-by-one model, NRMSE diff , is used as the test statistic, and two methods for deciding to prune each coefficient are proposed: 1. If NRMSE diff is significantly different from 0, then the coefficient is retained. Otherwise, it is pruned. Results are very insensitive to α, being identical for α = .01, .05, and .10, and usually identical for α = .20. 2. An absolute threshold is set, and if NRMSE diff is less than this threshold, the coefficient is retained. The first method yields pruned models with fewer coefficients and performance that is good but slightly inferior to the full PDM model. The second method, for a judiciously chosen threshold, yields pruned models with several more coefficients but performance comparable to the full model, as explored further in Section 3.2.3. Unless otherwise specified, all pruned PDM models are deter- mined using Method 1. We also rank the retained coefficients by NRMSE diff to give an indication of the relative importance of each term. 92 Figure 3.19: Pruned 8-P PDM model structure obtained under coefficient pruning of the 27-P PDM model (i.e. the 2-4 basis set), using Method 1 with α = .10. Each coefficient is ranked by NRMSE diff . Pruned 2-4-basis PDM model: the P-8 model Applying Method 1 with α = .10 to the 27-P PDM model, we obtain the eight-parameter (8-P) model, depicted in Figure 3.19. There is a single self-forward term in X1 and a single self-feedback terminY1,bothofwhicharelinear. SinceX1isintegrative,weconcludethattheforwardseparable component of the model is linear and integrative. The model is dominated by cross-terms, most of which are cross-forward-feedback terms. The X1 and Y1 PDMs are clearly of central importance, together contributing to seven out of eight terms. 93 Figure 3.20: The four surviving terms under direct pruning of the full 65-P PDM model, yielding the maximally reduced 4-P model. Note that these coefficients are the same four identified as most essential in the 8-P model. Pruned full-basis PDM model: the P-4 model Weapply the pruning algorithm to the full 65-P PDM model. Perhapssurprisingly, the full-pruned PDM models are markedly inferior to the 2-4-pruned PDM models. Pruning of the full model leaves four coefficients and three PDMs, X1, Y1, and Y2, as shown in Figure 3.20. This model has linear forward terms in X1 and Y1, and X1-Y2 and Y1-Y2 cross terms. These terms are the same as the top four terms of the 8-P pruned model, as ranked by NRMSE diff . From this, we conclude that direct application of pruning to the full PDM/NARV model re- liably identifies the most essential model components. However, to estimate those components of significant but not overwhelming importance, it is important to reduce the PDM basis sets first. The superior predictive performance obtained when reducing the PDM sets before pruning justifies this conclusion. 94 N-coefficient models To determine how model performance changes with the number of expansion coefficients included, for a given PDM basis set (i.e. either the full or the 2-4 basis sets), we construct a set of “N- coefficient” models, where each coefficient of the model is ranked by its NRMSE diff as determined by the LOO procedure described above, and the first N coefficients are included. We do this for the full basis set (65-P model) and the 2-4 basis set (27-P model); Figure 3.21 plots the bootstrap mean and 95% confidence intervals for the NRMSE, Γ, TPR, and FPR as functions of N. As can be seen from the figure, NRMSE (and Γ) actually improves as the first few coefficients are omitted, i.e. the N = 20 – 26 models perform better than the full N = 27 model under the 2-4 basis set, and under the full basis set, the N = 50 – 64 models are all significantly better than the N = 65 model. This is likely related to over-fitting. As N is further reduced, model performance degrades gradually and slightly until there is a dramatic and abrupt drop in performance at either seven (2-4 basis set) or six (full basis set) coefficients. Weconclude that a small number of model coefficientsare actually detrimentalto performance, the majority have a minor or nonexistence effect on performance, and a small number, between 6 and 10, are responsible for almost all predictive ability. Moreover, the NRMSE performance curve for the 2-4 basis set is always significantly below that of the full basis set (with the sole exception of N = 18), confirming the value of reducing the basis set before coefficient pruning. The location of the 8-P pruned model identified by the LOO pruning procedure on the N- coefficient curve shows that the procedure can identify the (nearly) minimal model that retains good performance. Alternatively, one could simply select the desired model from the N-coefficient model curve, which explicitly displays the trade-off between parsimony and performance. 95 Figure 3.21: Performance metrics for N-coefficient PDM models under the full and 2-4 PDM basis sets. The bold lines give the bootstrap mean, and the shaded regions enclosed by dotting lines indicate the 95% CIs. The asterisks mark the location of the 8-P model. The coefficients of the 8-P model are selected by significance testing on all coefficients of the 2-4 basis set, and are found to be the same as those retained by NRMSE diff ranking. Performance is assessed using the full out-of-sample testing data-set. 96 Figure 3.22: Out-of-sample membrane potential predictions for, from top to bottom, the 27-P, 8-P, and4-Pmodels. Aspredictionsforthefull65-Pand27-Pmodelsarenearlyidenticalinappearance, we omit results for the full model. Predicted time-series membrane potential and spike trains Example 500 ms snippets of the membrane potential tracings for the 27-P, 8-P, and 4-P models are given in Figure 3.22. On visual inspection, the 27-P model predictions are essentially identical to the full 65-P model predictions, which is why we have omitted the latter from the figure. The pruned models (8-P and 4-P) typically capture the AP waveform less precisely and predict APs with a slightly lower peak amplitude. In the case of the 8-P model, this has only a minimal effect on the ability to predict APs in the binary sense. This is not the case with the 4-P model. 97 Figure 3.23: The bootstrap distributions of the mean Γ, NRMSE, TPR, and FPR for the full (65-P), 27-P, 8-P, and 4-P models run on the full testing data-set. An asterisk indicates the mean is significantly different from that of the full model at the α = .05 level, while an “x” indicates significance at the α=.01 level. Comparative performance: coincidence factor and NRMSE Figure 3.23 shows the bootstrap distributions of the mean Γ, NRMSE, TPR, and FPR for the full 65-P, 27-P, 8-P, and 4-P models. These distributions are determined from the full testing data set and represent out-of-sample results. PDM basis set reduction results in a significant increase in the NRMSE(p<.05)andsignificantdecreaseintheTPR(p<.05),butnosignificantdifferenceinthe coincidence factor or FPR. Pruning leads to a reduction in Γ that achieves statistical significance relative to the full model (p < .05), but no reduction in TPR and an increase in FPR (p < .01). Compared directly, the 27-P and 8-P models do not differ significantly in NRMSE or Γ. The 4-P model is clearly inferior in all metrics, and its inclusion in the figure helps demonstrate the very similar performance of the first three models in an absolute sense. 98 Figure 3.24: The four panels on the left show the individual ROC curves for the four models. The marker meanings follow: (1) the red square indicates the point closest to the upper left normalized by the maximum TPR and FPR, (2) the green diamond shows the location of the optimal θ D with respect to Γ, and (3) the black triangle indicates the optimal θ D with respect to the K-statistic. The right panel plots all ROC curves together. Comparative performance: ROC, Γ, and K-statistic curves Figure 3.24 displays ROC curves for the full 65-P, 27-P, 8-P, and 4-P models. There is only a slight degradation in performance, as measured by the area-under-the-curve (AUC), moving from the 65-P to the 8-P model, while the 4-P model is markedly inferior. We also mark, on the ROC curves, the locations of the optimal thresholds for spike detection, θ D , with respect to the coincidence factor and K-statistic. We find that both metrics give similar thresholds, but they favor a higher TPR and FPR compared to the normalized closest point to the upper left. Figure 3.25 gives the coincidence factor, Γ, and K-statistic as function of θ D . Both un-pruned models (65-P and 27-P) are relatively insensitive to θ D over a fairly broad range, while pruning both lowers the optimalθ D and narrows the range of good performance. Results for Figs. 3.24 and 3.25 are for the full testing data-set. 99 Figure 3.25: The left panel gives the coincidence factor, Γ, as a function of the threshold for spike detection, θ D , and the right panel shows how the K-statistic varies with θ D . The ranges of good performance are similar between the two, and the optimal θ D values are also quite close. This figure demonstrates that the predicted spike-trains under the non-pruned models are insensitive to θ D over a broad range, while the pruned models have much narrowerθ D tuning curves. This makes sense, as the principal effect of coefficient pruning is to degrade the model’s ability to recapitulate the fine details of the AP waveform. Interspike time histograms We generate interspike time histograms for the full 65-P, 27-P, and 8-P models on the full testing data-set. These are compared to the actual interspike time histogram, as shown in Figure 3.26. We also determine the empirical cumulative distribution function (CDF) for each such histogram, and we perform a two-sample Kolmogorov-Smirnov (KS) test on each histogram pair, which tests the null hypothesis that the two data-sets are drawn from the same distribution. Under the KS test, the data and the model predictions all are drawn from different distributions, although the data to 65-P model comparison is of borderline significance at p = .0301. The 65-P model and 27-P interspiketimehistogramsarenearlyidentical, andtheKStestfailstorejectthenullhypothesisat p=.6945. The8-Pmodelhistogramshowsthepoorestagreementwiththedataandissignificantly different from all the other histograms (the 4-P model is much worse and is omitted). 100 Figure 3.26: The interspike time histograms for the full testing data-set and those predicted by the full 65-P, 27-P, and 8-P models are given on the left. The empirical CDFs derived from these histograms are plotted on the right. The CDFs for the former three histograms are nearly indistinguishable. Volterra kernels for full and reduced models Comparing the Volterra kernels reconstructed from the full and reduced PDMs allows direct visu- alization of how greatly the models vary. We compare the full 65-P, 27-P, 8-P, and 4-P models. As shown in Figure 3.27, the first-order forward and feedback kernels are essentially the same for all four. The second-order forward self-kernel disappears under both pruned models (8-P and 4-P), while the second-order feedback self-kernels are also all similar, with the only major differences occurring at one or two lags. The major difference between the 4-P model and the others is in the second-order cross-kernel. The high-frequency component, which can be accounted for by the X1-Y1 cross-term, is missing from the cross-kernel of the 4-P model. Therefore, this cross-term is demonstrated to be highly significant with respect to kernel reconstruction. Thegoodagreementinkernelmorphologybetweenthefullmodel,with65expansioncoefficients, 101 Figure 3.27: Selected Volterra kernels reconstructed from full and reduced PDM models. The left part of the figure gives the first-order forward and feedback kernels, and the right portion gives the second-order cross kernels (with the x and y axes having units of delay in ms). and the 8-P model confirms that the eight retained terms capture most of the system structure. Linear minimal model Pruning reveals the PDM model structure to be dominated by an integrative forward PDM with a linear ANF and a feedback PDM that represents the early AP waveform, also with a linear ANF (Figs. 3 or 4). Presumably, the nonlinear cross-terms modulate AP shape and are important con- tributors to the afterpotential and refractoriness. We have tested these assumptions by examining the performance of a model retaining only the first forward and feedback PDMs and the associated linear ANFs, which we refer to as the linear minimal model. On the full testing dataset, the bootstrap means for the coincidence factor, Γ, and the NRMSE, are 0.41 and 0.14, respectively, which are markedly inferior to both the 8-P and 4-P models, 102 Figure 3.28: Results from the linear minimal model. The left side of the figure gives a 400 ms example of out-of-sample membrane potential predictions for the 8-P (top) vs. the linear minimal model(bottom). Therightsidedisplaystheresponsesofthesetwomodelstoa1mspulseof20µA of current. Thus it is demonstrated that the cross-terms excluded from the linear minimal model are essential to accurately representing the AP waveform. demonstrating the importance of cross-terms to overall performance. Sample membrane potential tracing for the 8-P and linear minimal models are given in Figure 3.28. Figure 3.28 also shows the response of these two models to a 1 ms current pulse of 20 µA, showing that the cross-terms are also essential to capturing the AP waveform. We have also replicated the numerical experiment reported in Section 3.1.4, where pairs of 1 ms current pulses are given in sequence, with a progressively wider interval between the two, to determine if the reduced PDM models correctly predict the existence of a relative and absolute refractoryperiod. Whileboththe8-Pand4-PPDMmodelsperformreasonablywell(althoughthey do underestimate the duration of the refractory periods), the linear minimal model fails completely to demonstrate refractoriness. 103 Altered channel dynamics Changes in the underlying system’s ion channel dynamics are necessarily reflected in data-derived Volterra-style models. We have examined how changing the maximum sodium and potassium con- ductances, ¯ g Na and ¯ g K , respectively, affects the estimated PDM morphology and reduced/pruned modelstructure. Anumberofmedicallyusefuldrugs,includinglocalanaestheticsandmanyantiar- rhythmics, and many toxins, most notably tetrodotoxin (TTX), inhibit sodium conductance (per Paracelsus, “all substances are poisons...The right dose differentiates a poison and a remedy”). TTX inhibition of sodium channels in giant squid axon is well-modeled by decreasing ¯ g Na in the Hodgkin-Huxley equations[191, 232], and does not appear to affect the temporal dynamics of the sodium channel, represented by the m and h variables in the H-H equations[232]. Therefore, we have generated training and testing datasets with ¯ g Na and ¯ g K varying from 60 to 240µA cm −2 and from 12 to 84 µA cm −2 , respectively. We have found that, in general, the PDMs of the modified systems resemble those of the orig- inal but the ranks of the singular values of the most similar PDMs are not necessarily equal. To facilitate comparison, for each conductance-modified forward (feedback) PDM we find the correla- tion coefficient between it and the five original PDMs, and the most similar PDMs are paired. In general, the paired PDMs are quite similar. Given this, we choose to impose a common basis set across models so that they can be compared more easily within the coefficient space, and we use the PDM basis set estimated for the original, unaltered H-H system as the “universal PDM basis.” For systems with altered ion conductance, use of the universal basis in lieu of a system-specific PDM basis only slightly decreases performance. Undercoefficientpruning,theresultingcoefficientstructuretendstobesimilartothatobtained for the original H-H system, and in particular, the four coefficients of the original P-4 model tend 104 to be identified as the most essential. We have tried examining the pruned model structures for different conductance-modified model, and while they clearly vary, it is difficult to gain insight when considering seven or eight parameters. Therefore, we impose the basic 4-P model structure, consisting of X1 and Y1 forward terms, and Y1-Y2 and X1-Y2 cross-terms (see Figure 3.20), and determine how (and if) the values of the four coefficients change systematically with ¯ g Na and ¯ g K . As shown in Figures 3.29 and 3.30, each term exhibits a clear trend. Themostsignificanttrendsareinthecross-terms: assodiumconductanceincreasestheseterms diminish, while they are enhanced by increases in potassium conductance. These terms contribute principallytotheafterpotentialandrefractoriness, anditisquitesensiblethatincreasedpotassium conductanceshouldenhancerefractorinessandsodiumconductancediminishit. Indeed, theeffects ofincreasingsodiumandpotassiumconductanceonthefourcoefficientsofthe4-Pmodelarenearly inverse images of each other, but this antisymmetry is broken in that increasing either conductance decreases the magnitude of X1. Model reduction applied to the NARV model with Laguerre basis The NARV model, in which the Volterra kernels are expanded on a Laguerre basis, is mathemati- cally equivalent to the PDM model structure, and the proposed basis set reduction and coefficient pruning methods may be directly applied to the NARV model. We have found that, in contrast to the PDM model, much less parsimony can be achieved through direct reduction of the NARV model, and the pruned NARV models are not interpretable. Basis set reduction of the NARV model results in a 44-coefficient model that has performance comparabletothe27-PPDMmodelthatresultsfromtheun-pruned2-4PDMbasis. Pruningofthe basis-reducedNARVmodelresultsinextremelypoorperformance(notshown),whilepruningofthe 105 Figure 3.29: Values of the estimated X1, Y1, Y1-Y2, and X1-Y2 coefficients under the 4-P model framework as a function of the underlying H-H system’s maximum sodium conductance, ¯ g Na . Increasing ¯ g Na suppresses Y1-Y2 and X1-Y2, diminishing the refractory period and easing AP firing. While the trends in X1 and Y1 are clear, they are relatively small in absolute value. The location of the standard H-H ¯ g Na is marked with an asterisk. 106 Figure 3.30: Values of the estimated X1, Y1, Y1-Y2, and X1-Y2 coefficients under the 4-P model framework as a function of the underlying H-H system’s maximum potassium conductance, ¯ g K . The counter-trends in X1 and Y1 appear to have to have the net effect of enhancing the “late” activity of Y1 in response to current input as ¯ g K increases, while having little effect on the early AP waveform. Increasing in ¯ g K greatly increases the magnitude of the Y1-Y2 and X1-Y2 terms to suppress AP firing; this suppression is active immediately following AP firing, and enhances refractoriness to further firing in response to a current stimulus. The location of the standard H-H ¯ g K is marked with an asterisk. 107 full NARV model gives the 10-coefficient model depicted in Figure 3.31. This model is difficult to interpret,doesnotsuggestthatthemembraneactsprincipallyasaleakyintegratorofcurrentinthe subthreshold regime, and the AP waveform and afterpotential are not represented as in the pruned PDM-based model. This eight-DLF, 10-coefficient model has performance significantly inferior to the six PDM, eight coefficient 8-P model, while it is comparable in performance to the much more interpretable three PDM, four-coefficient 4-P PDM model. Thus, we demonstrate the importance of employing the PDM basis set with respect to both model reduction and interpretation. 3.3 Discussion This section has explored the use of PDM analysis of the system described by the H-H equations to achieve parsimonious PDM-based models of predictive capability comparable to the full Non- linear Autoregressive Volterra model representation that was recently published [65]. We find that a two-step model reduction procedure is advisable for this purpose, whereby a set of PDMs is first identified that efficiently represents the equivalent Volterra model, and subsequent pruning of the PDM-based model terms yields models of dramatically reduced complexity with comparable predictive capability. This procedure identifies an eight-parameter (“8-P”) model, representing an eightfold reduction relative to the full NARV model, and a threefold reduction relative to the differential equations representation of the HH system. Wehavedemonstrated that the proposed methodology greatly reduces PDM-based model com- plexity and may facilitate model interpretation. Reduction to the 2-4 PDM basis gives a model (27-P model) consisting of six PDMs and 27 free parameters, compared with 10 PDMs and 65 free parametersforthefullmodel. Itsperformanceisonlyslightlyinferiortothefullmodelwithrespect to prediction NRMSE, and comparable with respect to most measures of spike-train prediction, 108 Figure 3.31: Structure of the coefficient-pruned NARV model; a minor quadratic term in the Y1 ANFhasbeenomittedfromthefigureforclarity. Ascanbeseen, themodelstructureisdominated by self-terms, rather than cross-terms as in the PDM model, and there is no obvious functional interpretation. 109 such as the coincidence factor, Γ, the ROC curves, the interspike-time histograms, and the curves describing Γ and K-statistic dependence on the threshold for spike detection (see Figure 3.25). Furtherpruning of the 27-P model yields a compact model with only eightfree parameters (8-P model) and whose performance is only slightly inferior to that of the full model. Moreover, with only eight free parameters most kernel structure is preserved upon reconstruction of the Volterra kernels (see Fig 3.27). We conclude that PDM-based modeling with basis reduction is advisable in this system, as it reduces model complexity by one-half to two-thirds at essentially no performance cost. Further pruning also appears advisable, as it leads to greater model parsimony with only a slight degradation of performance. Examination of the form of the obtained forward PDMs for the H-H equations/system reveals that X1 is a “leaky integrator” over an input past-epoch of approximately 5 ms, and X2 is a “finite-bandwith differentiator” (see Figure 2). Marmarelis[165] previously proposed a model for single-neuron operation comprising two “neuronal modes” that are analogous to the X1 and X2 leaky-integrator and slow-differentiator PDMs, followed by a static nonlinearity and a threshold- trigger operator. The present analysis seems to corroborate that postulate, but only with regard to the forward branch of the PDM-based model of the H-H equations (suitable for the sub-threshold operation of this system). However, in the supra-threshold operation of this system, our analysis reveals a feedback PDM (Y1) responsible for the waveform of the generated action potential (AP), and three more feedback PDMs that partake in the supra-threshold dynamics of the system via modulatory influences principally upon the output of the first forward and first feedback PDMs (see Figure 3.19), expressed in the PDM-based model as additive pair-product terms. Further pruning reveals that the most significant of these modulatory influences are the ones exerted by the second feedback PDM (see Figure 3.20) which exhibits integrative characteristics 110 over roughly 15 ms. Moreover, under pruning the X2 PDM is preserved only in a forward-feedback term, indicating that the rate of current injection also influences post-AP dynamics. We note that the fourth feedback PDM exhibits dynamic characteristics of first-order differentiation. Future studies will seek to explore the relation of these PDMs with the specific ion-channel mechanisms of this system. The most highly pruned model, shown in Figure 3.20, captures most of the essential system characteristics, and we give the following interpretation. The dominant forward PDM of the H-H system is X1, which is integrative, with a linear ANF. In the sub-threshold regime, it is the only active model component, and thus the sub-threshold model acts purely as an integrator of recent current input, with a memory of roughly 5 ms, consistent with the widely used integrate-and-fire model. However, X1 is not a simple exponential waveform, but appears to have three different time-constants. The fact that the NARV model retains two separable forward bases (Figure 3.31) undermodelreduction,oneofwhichisanexponential,suggeststhatthisisnotsimplyanartifactof model estimation. In the supra-threshold regime, the model has more complex structure including a linear feedback term and two pair-product terms between the second feedback PDM (Y2) and the first forward (X1) and first feedback (Y1) PDM. The Y1 PDM is responsible for the early features of the AP waveform, and also contributes to the afterpotential. The pair-product terms (cross-terms) of the model contribute to peri-AP phenomena and have an effect on the refractory period. Reconstruction of the Volterra kernels (Figure 3.27) also suggests that the X1-Y1 term is an important contributor to the high frequency characteristics of the Volterra cross-kernel. We have also performed a preliminary analysis of how changes in underlying channel dynamics affect the coefficients of the maximally reduced 4-P model. We have observed that changes in the maximum sodium and potassium conductance affect the form of the PDMs only mildly, which 111 maintain their main functional characteristics (e.g. integrative, differentiating etc.). This, and the difficulty of comparing waveforms versus comparing scalars, motivates the strategy of fixing a universal PDM basis and coefficient set, and examining system variability within coefficient space. Usingthe4-Pmodelastheuniversalframework,changesinthesodiumandpotassiumconductance consistently affect the X1, Y1, Y1-Y2, and X1-Y2 terms of the 4-P model (see Figs. 13 and 14). These changes are most marked in the Y1-Y2 and X1-Y2 cross-terms, both of which increase in magnitude with potassium conductance, but decrease in magnitude with sodium conductance. Application of the two-step model reduction procedure directly to the NARV model demon- strates the importance of using PDMs as a basis. Model reduction under the PDM basis leads to more compact models with better predictive capability. While performing a basis-reduction step before pruning almost uniformly improves model performance when using a PDM basis (see Figure 5), this does not appear to be the case when using the Laguerre basis of the NARV model, and in fact is detrimental to performance (not shown). Moreover, the pruned full NARV model (shown in Figure 15) has no obvious interpretation in terms of the functional characteristics of the H-H system, in stark contrast to the pruned PDM models. As demonstrated in Figure 3.25, the optimal threshold for spike detection is not altered by PDM basis reduction, but is markedly affected by coefficient pruning. It is, therefore, important to optimize metrics of spike-train similarity, such as Γ, for each individual model to get a fair assessment of its performance. We have also found that, for pruned models, the optimal threshold for detection, θ D , changes with input power; namely, lower input power leads to lower detection thresholds for pruned PDM-based models (results not shown). In conclusion, our results on model reduction indicate that many terms of the full PDM-based model of the H-H equations make insignificant contributions to model predictive performance, as 112 assessed by all prediction metrics used (see Figs. 7–11) and the reconstructed equivalent Volterra kernel estimates (see Figure 11). Moreover, Figure 5 suggests overfitting when all coefficients are included, and only 6-10 coefficients are necessary to capture the main H-H dynamics. This finding suggests that PDM-based modeling with appropriate pruning can achieve parsimonious Volterra- equivalent models of the H-H equations with excellent predictive capability. 3.4 Integrated single neuron with point-process inputs/output 3.4.1 Introduction Intheprevioussection,themembranepotentialoftheHodgkin-Huxleysystemfeedsback“directly” on itself. In this section, we consider neural point-process input-output data generated by a model neuron system with a hidden or “nested” auto-recursive component. Data is generated by an underlyingtwo-stepcascademodelforspikingneuronactivityinresponsetomultiplepointprocess inputs. The spike-train inputs are transformed into a continuous current injection at the axon hillock via a prescribed second-order Volterra model, and this current injection is transformed into membranepotentialviatheHodgkin-Huxleyequations;themembranepotentialisthenthresholded to give a spike-train output. This point-process system is much more relevant in a practical sense than the H-H system, as actual data from neural recording experiments using multi-electrode arrays and spike-sorting algorithms is typically given in the form of point-process spike-trains, and such data has been the focus of extensive Volterra-style modeling, including a number of studies by Berger, Marmarelis, andcolleaguesofthefunctionalcharacteristicsofthehippocampalformation,e.g. [226,16,98,266], and recent work on the primate prefrontal cortex [171]. 113 Figure 3.32: Conceptual schematic for two-step cascade model of AP generation in a model neuron in response to four spike-train dendritic inputs. We have estimated input-output Volterra models that are (1) purely forward, (2) have a linear auto-recursive component, and (3) have a nonlinear auto-recursive component. We demonstrate that,whiletheoverallsystemishighlynon-linear, anauto-recursiveVolterramodelthatislinearin the forward and recursive components, and that includes a threshold operator, performs quite well, while a second-order auto-recursive model is even better. Thus, an auto-recursive model that is linearinitskernels, whenaugmentedbyanon-linearthresholdoperator, cangivehighlynon-linear behavior that results from hidden recursion, and this cannot be captured even by a high-order forward model. 114 3.4.2 Methods Data preparation We construct a simulated neuron system that takes as its inputs four binary spike-trains, x i (n), i={1,2,3,4}, and gives as output a spike train,y(n). The four inputs are imagined as the activity of presynaptic neurons that synapse at the dendritic tree of our target neuron. These four inputs are each given a relative weight and summed as x sum (n)= 4 ∑ i=1 w i x i (n) (3.22) where w i is the weight of the ith input. Note that x i ∈ {0,1} and w i ∈ R. We take each x i to be an independent Poisson process with rate parameter λ = .2 ms −1 . These Poisson processes are modified slightly such that no AP is allowed within 2 ms of another, which has the effect of slightly reducing the effective firing rate. The lumped spike-train input is then transformed into a continuous current, I(n), via a second- orderVolterramodel,denotedthe“x sum (n)-to-I(n)”operator,definedbyitsfirst-andsecond-order Volterra kernels, denoted k I 1 and k I 2 , respectively. This current, I(n), is then passed as input to the Hodgkin-Huxley equations, which yield a membrane potential output, V(n), which is in turn passed to a threshold operator, giving the final spike-train output, y(n). Note that only an upward deflection of V(n) is counted, and a brief refractory period is enforced to ensure that each AP waveform is counted only once. Finally, the raw x(n) and y(n) time-series, which have a temporal resolution of 0.2 ms, are binned into 2 ms bins. 115 x sum (n)-to-I(n) Volterra model The bi-exponential mathematical form of the k I 1 kernel is given, in discrete time, as k I 1 (m;ρ)=W(ρ)e −mT() ( 1−e −mT ln:5 ) , (3.23) with τ(ρ)= ln.5 ρ , (3.24) whereT =0.2msisthesamplinginterval,ρdeterminesthememoryextentofthekernel,andW(ρ) is a scaling factor introduced to keep the overall output spike frequency similar between systems with different ρ values. The form of k I 2 is similarly chosen as k I 2 (m 1 ,m 2 ;ρ)=W(ρ)e −m 1 T() e −m 2 T() ( 1−e −m 1 T ln:5 )( 1−e −m 2 T ln:5 ) . (3.25) We have generated multiple sets of x sum (n)-to-I(n) kernels with different W(ρ) scaling factors to studybothdenseandsparsespike-traininputsandoutputs. However,wehavefoundourqualitative results to be similar across such models, and here we report only results for dense inputs with ρ = 1 and W(ρ)≡6. The x sum (n)-to-I(n) kernels act as non-linear filters that give a “synaptic-like” current at the axon hillock in response to AP inputs. The first-order kernel is bi-exponential, with a brief rising phase followed by a mono-exponential decay. Mensi et al. [177] recently used a similar mono- exponential filter to generate a synaptic-like current for current-clamp stimulation of individual neurons. We consider the inclusion of the rising phase more realistic, but we have performed analysis with and without the rising phase and found no significant differences in overall results. 116 The second-order kernel allows the dendritic tree to act as a non-linear coincidence detector. That is, the EPSPs evoked by temporally coincident APs sum non-linearly. It is well-established that excitatory inputs to dendritic trees interact non-linearly: EPSPs adding sublinearly in passive dendrites, while synchronous inputs in active dendrites can greatly amplify the response [153]. Our second-order kernel encodes the latter mechanism. x sum (n)-to-y(n) Input-output Volterra model WeconstructaVolterrarepresentationofthex sum (n)toy(n)transformation. Sincemultipleinputs are summed to give x(n) sum , we may think of this as a quasi-multi-input model; the advantage of summing instead of considering individual inputs is in the much greater model parsimony that comes at no cost to performance (we have confirmed this assertion). From the input-output point process data, a Volterra model is estimated which yields a continuous output, u(n), which is then passedtoathresholdoperatortoyieldy(n). Theparameterθisthethreshold,andweoptimizeθfor eachindividual model to givethe highest coincidence factor, Γ, whichweuse as our primary metric for model assessment. The coincidence factor quantifies the similarity between two spike-trains, and has been used in numerous works [131, 65]; see [65] for the definition. 3.4.3 Results x sum to I(n) transformation Figure 3.34 shows a sample segment of summed point-process input, x sum , superimposed upon the resulting current, I(n), for either a first-order (linear) or second-order (nonlinear) x sum (n)-to-I(n) operator. It is demonstrated that inclusion of the second-order kernel leads x sum (n)-to-I(n) to act as a highly nonlinear coincidence detector, amplifying (near) synchronous inputs. The same figure 117 Figure 3.33: Given a system with hidden recursion, the two alternative Volterra models for the system are a strict input-output model, or an auto-recursive model. In theory, the input-output description should suffice, but we have found that, as a practical matter, the auto-recursive frame- work is much more useful. 118 Figure 3.34: The left side gives the currents generated by either a linear or nonlinear x sum (n)-to- I(n) operator, where the summed input spikes, x sum , are indicated as black tick marks. The right panel shows the H-H membrane response to these injected currents, demonstrating the importance of k I 2 . also shows the H-H membrane response given these I(n) inputs, demonstrating the importance of k I 2 to the overall neural response. Note that the H-H operator is also highly non-linear. Comparative model performance We compare the predictive performance of a family of kernel models. For convenience, we refer to the forward self-kernels (k 1;0 , k 2;0 , ...) as the X1, X2, etc. kernels, the recursive self-kernels as Y1, Y2, ..., and the second-order cross-kernel as the XY kernel. For each model, the threshold value, θ, is uniquely determined as that which gives the best value of the coincidence factor, Γ, to allow the most fair model comparison. We have attempted to perform kernel-reduction on the full second-order model; the LOO algorithm suggests that only the X1, Y1, and XY kernels are significant. However, such a model has performance significantly inferior to that of the full model, suggesting that X2 and Y2 are also of some importance. The bootstrap distributions of the mean Γ for purely forward models up to fifth-order, and auto-recursive models up to second-order, are given in Figure 3.35. While the second-order auto- recursive model performs best, the first-order auto-recursive model also does surprisingly well. 119 Figure 3.35: Bootstrap distributions of Γ for purely forward models of order 1 through 5, and auto-recursive models of first- and second-order. As can be seen, there is little benefit to addi- tional forward kernels, while a single auto-recursive kernel dramatically improves performance, and second-order auto-recursive model does better still. Note that these results are for out-of-sample data. Performance under input subsets We examine model performance when only a fraction of the x i (n) inputs that sum are known. This is a highly relevant scenario, as under micro-electrode recording experiments, only a handful of potentially thousands of input neurons can be recorded. We could imagine that omitting some relevant inputs could cause the auto-recursive model component to harmfully interfere with pre- diction. Fortunately, this turns out not to be the case. Figure 3.36 shows out-of-sample model performance under one, two, three, or four out of four inputs known; the auto-recursive component is clearly helpful in all cases. Themodelneuroncontainsnon-trivialnon-linearitiesinbothofitsinternalcascadedoperators, yet an auto-recursive model with linear forward and recursive kernels performs much better than evenafifth-orderpurelyforwardmodel. Moreover, theauto-recursivesystemcomponentishidden, yet an auto-recursive representation at the level of observable variables is highly efficacious, and 120 Figure 3.36: Out-of-sample performance under different models when one, two, three, or four out of four inputs are known. From top to bottom, results are for the purely forward model, first- orderauto-recursivemodel, andsecond-orderauto-recursivemodel. Underallmodels, performance improves with knowledge of the input, but the auto-recursive models are clearly superior under all input subsets. the model is robust to negative interference in the form of missing inputs. 121 Chapter 4 Smooth muscle dynamics 4.1 Auto-recursive representation We consider the case of mollusk smooth muscle contractile force in response to an electrical current stimulus, using data generously provided by Marc Cole. We have previously examined this system using the classical, i.e. non-recursive, Volterra methodology [45]. The input, x(n), consists of a randomstimulustrainofeither0,.75,or1.50mAofcurrent,andtheoutput,y(n),isthecontractile force. Figure 4.1 shows a snippet of the input-output data-record. We have divided the overall record into a training and testing data-set of equal length, and found that a Volterra model with a single second-order forward kernel and a quasi-exponential first-order recursive kernel is optimal, suggesting that the contractile force relaxes in a quasi-exponential manner to zero following a non-linear response to stimulation. As Figure 4.2 shows, a purely forward second-order Volterra model tends to capture the high-, but not the low-, frequency dynamics well. Examination of Figure 4.1 suggests that the smooth muscle contractile force responds in a non-linear fashion to current input, and then relaxes roughly 122 Figure 4.1: Segment of input-output smooth muscle data, with input, x(n), being electrical current and output, y(n), being contractile force. exponentially to zero. We have found that inclusion of a first-order auto-recursive component, expanded on three DLFs, i.e. L y = 3, significantly improves results. This implies a departure from a purely exponential relaxation characteristic, which requires only a single DLF to represent (see following section). Application of the LOO kernel reduction algorithm to the baseline second-order auto-recursive model (Section 2.6.4) indicates that only k 2;0 , the second-order forward kernel, which we also refer to as the X2 kernel, and the first-order recursive kernel, k 0;1 (“Y1 kernel”) are significant. More- over, a third-order model has no advantage over the second-order model, either with or without the recursive component, and k 0 is unsurprisingly found to be unnecessary as well. The perfor- mance advantage to the X2, Y1 auto-recursive kernel model over the second-order forward model is illustrated in Figure 4.2, and these kernels are displayed in Figure 4.3. Finally, application of the PDM methodology shows that the basis sets can be reduced to a single auto-recursive PDM and a single forward PDM, reducing the free parameters of the forward 123 Figure 4.2: On the left, the top panel gives out-of-sample predictions for the second-order forward VolterramodelwithL x =7,α x =.8;whichwerefoundtobeoptimalthroughacombinedbootstrap and ternary search procedure. The bottom panel shows out-of-sample predictions for the X2, Y1 auto-recursive model, with L x = 5, L y = 3, α x = .4, and α y = .4, which were similarly found to be optimal. The right side of the figure shows the bootstrap distributions (calculated using 256 segments and 10,000 re-samples) of the NRMSE for the two models, which vary at the (two-sided) α = .01 significance level. model from 18 to a mere three, but at absolutely no cost to predictive performance. The PDM model architecture is given in Figure 4.4. 4.2 Exponential relaxation characteristic We briefly study the related differential equation, dy dt =ax(t) 2 −by(t) (4.1) where x(t) is an exogenous input, and a and b are positive real numbers. In this section, we fix a = 1 and b = .05. Multiplying both sides by exp(bt) and integrating by parts gives the analytic solution: y(t)=y(0)e −bt +a ∫ t 0 x(s) 2 e b(s−t) ds (4.2) 124 Figure 4.3: The two smooth muscle model kernels found to be significant under the auto-recursive framework. On the left is the k 2;0 (X2) kernel, and the right shows the non-exponential k 0;1 (Y1) recursive kernel. Figure 4.4: Auto-recursive PDM-based model architecture for mollusk smooth muscle. The input is passed through a single PDM and on to a quadratic ANF. The past output is passed through a single PDM equivalent with a linear ANF. The PDM model is defined by two basis functions and three free polynomial expansion coefficients. 125 Fory(0)=0,thefirsttermdisappears,andthesystemhasapurelyforwardVolterrarepresentation with the continuous-time kernel: k 2 (τ 1 ,τ 2 )=        ae −b 1 , τ 1 =τ 2 0, else . (4.3) Such a kernel is difficult to approximate with the Laguerre basis, requiring a fairly large number for a good approximation. We can also expand on a delta basis (equivalent to the classical Volterra model, see Section 2.2 for details), to estimate the above kernel from simulated data. We have done so, and compare results for a 25-lag (i.e. M = 25) X2 model (i.e. k 2;0 kernel only) with delta basis, which allows a very close approximation of the true system kernel, to those for an X2, Y1 auto-recursive model with Laguerre basis, where L x = 3 and L y = 1. In the noise-free case, the X2, Y1modelwithonlyeightfreeparametersperformssignificantlybetterthantheX2modelwith delta basis, which has 326 free parameters (although both give excellent performance). Note that the latter also requires vastly more data for accurate estimation. We have found the two models have equivalent performance with noise contaminating the output record such that the SNR dB ≤ 20. Thus, it is concluded that the exponential relaxation characteristic is efficiently captured by a first-order auto-recursive kernel expanded on a single exponential Laguerre basis function. The overall auto-recursive model is more compact, and lends itself to a straightforward biological inter- pretation. Comparingresultsfromthesmoothmusclesystemtothoseforthisdifferentialequations systemshastwointerestingimplications: (1)thesmoothmusclerelaxationcharacteristicisnotsim- ply exponential, and (2) the second-order smooth muscle response is similarly not a simple scaled squaring of the input. The second-order forward kernel derived from this data under the auto- 126 recursive framework is similar to that determined from the smooth muscle response to short bursts of stimulation, where the auto-recursive model component can be neglected [45]. We conclude that an auto-recursive model component can function as a “tracking” component that dramatically reduces the required forward kernel memory, the number of free model parame- ters, and may have an interpretable relationship to the underlying system. Moreover, comparison of kernels or PDMs derived for actual systems to those of toy differential equations systems may also be useful. 4.3 Optimizing force generation Our ultimate goal in modeling smooth muscle force generation is to optimize stimulation patterns for implantable gastric devices. To that end, we have tested integrated force generation in response to several simple current injection patterns. We consider power dissipated proportional to x 2 (t), i.e. current squared, and the ratios of peak and integrated force generation to power consumption are our primary performance metrics. Under tonic stimulation, the force response increases linearly with the square of current ampli- tude, as demonstrated in Figure 4.5. The peak and integrated force:power ratios are constant for tonic stimulation, indicating that there is no preferred power level under constant stimulation. In reality, the muscle response must be bounded, but this is not seen in the data range used to train the model. We have tested numerous duty cycles based on a monophasic square-wave and defined by three parameters: (1) pulse amplitude, (2) pulse width (PW), and (3) frequency (λ). As we have seen under simulations of tonic stimulation, altering pulse amplitude does not affect the integrate force:powerratio, andthereforewefocusonPWandλ. Numericalinvestigationshowsadutycycle 127 Figure 4.5: The left panel gives the time-course of the smooth muscle force response to increasing levels of tonic stimulus, under the PDM-based model. The right side of the figure illustrates both the absolute peak and integrated force development and these metrics normalized by the total power dissipated. of about 0.5 Hz frequency with a pulse width of 0.5 s to be optimal, as illustrated in Figures 4.6 and 4.7. Compared to tonic stimulation, such a duty cycle gives about 2.8 times the integrated force development per unit of power expended. 128 Figure 4.6: Smooth muscle force generation under different duty cycle frequencies. The left panels showintegratedforceandtheratioofintegratedforcetopowerdissipatedasafunctionoffrequency for the PW fixed at .5 s. The right panels give these metrics for a variable PW such that PW = 0.5/Hz. Figure 4.7: Integrated force:power ratio as a function of pulse width for three selected duty cycle frequencies. Results are shown, from left to right, for λ of 0.1, 0.5, and 1 Hz. Over all frequencies, the optimal pulse width is roughly 0.5 s. 129 Chapter 5 The Riccati equation and other growth laws 5.0.1 Background Modelsforpopulationgrowthformthebasisformuchofmodernmathematicalecologyandbiology, and population dynamics is the basic theoretical framework for a great deal of modeling of disease processes. Asimpleexamplemightbethatofbacterialpopulationgrowthandresponsetoantibiotic treatment. Growth is also a canonical example of a biological process displaying autonomous behavior, and it would be futile to approach such a process with a strict input-output paradigm. In this chapter, we review several classical growth models, and we show that auto-recursive Volterra representations of such systems may be derived from (sparse) data. We focus in particular on a non-linear Riccati differential equation of the form dy dt =ay−by 2 +cx(t), (5.1) 130 wherey(t) is the output,a,b, andc are real numbers withb nonnegative, andx(t) is an exogenous input. Setting x(t)≡0, the equation reduces to that of logistic growth, which has the form due to Verhulst, dy dt =ay−by 2 , (5.2) proposed in 1838 [246]. Following the celebrated work of Malthus, who posited that, without constraints, populationtendstoincreasegeometrically, therewasgreatinterestinelucidatingthose constraints to growth. On the suggestion of his teacher Quetelet, Verhulst presented his model as a formalization of the proposition that the “sum of obstacles opposing the indefinite development of the population...is proportional to the square of the rate at which the population tends to grow.” The −by 2 term was imagined to be a friction term analogous to that force opposing movement experienced by a body moving through a viscous medium, while the more modern conception is typically that of density-dependent inhibition of growth, with y 2 being a mass action term. As t → ∞, we have y → a/b. The Verhulst equation may be re-arranged to give the more familiar “ecological” form of the logistic model: dy dt =ry ( 1− y K ) (5.3) whereK =a/bistheso-calledcarryingcapacity,andr =aistypicallycalledthe“intrinsic”growth rate in this formulation. This equation admits the closed-form solution y(t)= Ky(0)e rt K−y(0)(1−e rt ) (5.4) The ecological form of the equation is apparently originally due to Lotka in 1925 [130], who devel- 131 oped an interest in the equation following the 1920 work of Raymond Pearl and Lowell Reed [203], who re-introduced the logistic growth curve independently of the earlier and, at the time, largely forgotten work of Verhulst. Pearl and Reed, however, did not derive the equation from underlying principles, but suggested it on purely empirical grounds with the form y(t)= ne mt 1+pe mt , (5.5) where n, m, and p are constants, with no posited relation to the underlying biology. However, differentiating shows this expression to be equivalent to Equation 5.3. The ecological form due to Lotka is typically considered to represent the postulate that the growth rate of a population decreases linearly as the population approaches its carrying capacity [130]. Curiously, this was not the derivation Lotka gave in 1922, which follows [130]. Consider a stable population, such that dy dt =f(y)=0. (5.6) Applying Taylor’s Theorem (and assuming f(0)=0) gives f(y)=ay+by 2 +... (5.7) If we assume that this equation has exactly two roots (corresponding to two population steady- states), namely zero and a/b, we truncate at second-order to arrive at the Verhulst form of the logistic equation. As an aside, consider truncating at third order: dy dt =f(y)=ay+by 2 +cy 3 . (5.8) 132 Forcertain valuesofa,b, andc, this equation has a zero root and twopositive roots, corresponding to three population steady states. The smaller of the positive roots is unstable, while both zero and the larger positive root are stable. This is therefore a model for positive density-dependence in growth at low population densities, a phenomenon that has been observed in real populations and is also known as the Allee effect. We have found that a third-order auto-recursive Volterra model can capture such behavior, but we do not present these results in detail here. Clearly, there are multiple ways to arrive at the same basic logistic/Verhulst framework, and accordingly, the interpretation of the biological meaning of the form and parameters can vary widely; a history of the early work and controversies surrounding the logistic model is given by Kingsland [130]. We prefer the Verhulst formulation; in the ecological form, r and K are often treated as free parameters which can lead to paradoxes, as discussed by Gabriel et al. [81], but which are easily resolved under the Verhulst framework. Moreover, the Verhulst framework leads ustoaclassof“generalized”logisticmodels, thebestknownofwhichisthevonBertalanffymodel. von Bertalanffy was interested in the basic question of organic growth: why does an organism growatall, andwhatcauses thisgrowthtostop? He suggestedthatanimalgrowthisthe“resultof the counteraction of synthesis and destruction: anabolism and catabolism of the building materials of the body” [252], and that this can be expressed with the mass-balance equation dy dt =ay −by (5.9) wherey now represents organism mass, the first term represents anabolic processes and the second catabolic; it is assumed that these processes scale with some power of the mass. Anabolism tends often to scale with surface area, as surface area represents the area available for gas exchange, 133 nutrient absorbtion, etc., suggesting λ = 2/3, but this does not necessarily hold for all species. Catabolism seems to scale with mass, implyingµ = 1, which is the value chosen by von Bertalanffy in his 1938 work [251]. Equation 5.9 with µ = 1 and λ unspecified is often referred to as the “generalized”vonBertalanffygrowthfunction(VBGF),whilefixingλ=2/3givesthe“specialized” VBGF. Equation 5.9 can also be considered a generalization of Verhult’s model. 5.0.2 Auto-recursive Volterra representations Let us assume the Riccati equation describes bacterial growth; therefore we let y(t) represent bacterial cell count, while x(t) is a source/sink term with units of cells min −1 . We have generated time-series data from the Riccati equation above, setting a = .01 min −1 , b = .001 cell −1 min −2 , and c = 1. We use a sampling interval of T = 1 min, and run trials with T max = 1028 min. In addition, as discussed in Section 2.3.1, we assume an initial epoch identically equal to the initial conditions (this allows very sparse sampling of the data, which is addressed below), and evaluate model performance using out-of-sample data. The input x(t) is taken to be mean-zero Gaussian white noise with σ = 0.03, updated every minute. Results We have applied the LOO algorithm for kernel reduction to the base Q x = 2, Q y = 2 model structure to determine that an auto-recursive Volterra model with Q x = 1, Q y = 2, and three kernels, namely the first-order kernel in x, k 1;0 , the first-order kernel in y, k 0;1 , and the second- orderself-kerneliny,k 0;2 ,whichwerefertoastheX 1 ,Y 1 ,andY 2 kernels,respectively,hasexcellent predictive ability. Moreover, L x = L y = 1. That is, a single exponential basis function is all that is needed to describe all kernels. 134 Figure 5.1 compares out-of-sample predictions for the full (six-kernel) and the reduced three- kernel models over a single trial. Model estimation and predictions are also quite robust to noise. It is concluded that the Riccati system admits the following discrete-time Volterra-style represen- tation: y(n)= M ∑ m=0 β 1;0 e −˘ xm x(n−m)+ R ∑ r=1 β 0;1 e −˘ yr y(n−r)+ R ∑ r 1 =1 R ∑ r 2 =1 β 0;2 e −˘ yr 1 e −˘ yr 2 y(n−r 1 )y(n−r 2 ) (5.10) It is convenient to write the analogous continuous time representation: y(t)= ∫ ˘ M 0 β 1;0 e −˘ xm x(t−s)dm+ ∫ ˘ R 0 + β 0;1 e −˘ yr y(t−r)dr+ ∫ ˘ R 0 + ∫ ˘ R 0 + β 0;2 e −˘ yr 1 e −˘ yr 2 y(t−r 1 )y(t−r 2 )dr 1 dr 2 (5.11) Unexpectedly, an auto-recursive model that is linear in both x andy (i.e. Q x =Q y = 1) yields sigmoidal growth and, sometimes, reasonably good out-of-sample predictions. For this model, the zeroth-order kernel, k 0 , plays an essential role, as we demonstrate by the following argument. Asymptotically, the Riccati system fluctuates about the point ˜ y = a/b (when x(t) is zero-mean), and the linear kernel system can only approach such a quasi-steady-state ifk 0 is nonzero. Consider y(t)=k 0 +x⊗k 1;0 +y⊗k 0;1 . (5.12) Since on average x⊗k 1;0 = 0, we search for constant ˜ y such that ˜ y =k 0 + ∫ ˘ R 0 ˜ yβe −˘ r dr (5.13) 135 Figure 5.1: Demonstration that the reduced three-kernel model (X1, Y1, Y2 kernels) has perfor- mance comparable to that of the full six-kernel model. The left panels give the out-of-sample performance on three separate trials, and the right panel gives the bootstrap distributions of the NRMSE. 136 For ˘ R sufficiently large, we arrive at the approximate solution, ˜ y =k 0 ( ˘ α−β ˘ α ) (5.14) and it is clear that unless k 0 ̸= 0 (we disregard the biologically irrelevant case of k 0 < 0), no non-zero ˜ y exists, and in fact the model will give unbounded growth if k 0 =0. We have confirmed this empirically. Thus, it is demonstrated that an auto-recursive Volterra model with only linear kernels can exhibit non-linear behavior, provided k 0 ̸= 0. This is somewhat analogous to the case explored in Section 3.4, where we demonstrate that an auto-recursive model for point-process data linear in its kernels can give highly nonlinear behavior when thresholding of the output is included. Generalized Riccati equation Consider the generalized Riccati equation: dy dt =ay −by +cx(t) (5.15) We have found that the same three-parameter second-order model framework that works for λ=1 and µ = 2, works for any monotonic growth law such that λ < 2, µ <= 2, subject to λ < µ. So that growth curves may be reasonably compared, we fix the asymptotic value, K = ( a b ) 1 , (5.16) to 10. Note that a closed-form solution to the growth law with c = 0 exists, but it is quite complex [174]. Figure 5.2 displays out-of-sample predictions on a single trial for all permutations 137 Figure5.2: Out-of-samplepredictionsforthethree-kernelsecond-ordermodelestimatedforRiccati systems such that λ ∈ {1/3,1/2,2/3,1,3/2}, and µ ∈ {1/2,2/3,1,3/2,2}, subject to λ < µ. To keep time-courses on same rough scale, we setb=.005 =2 ,a is selected to keepK =10, andc=1. The noise-free case is shown, but results are robust to noise. of λ ∈ {1/3,1/2,2/3,1,3/2}, and µ ∈ {1/2,2/3,1,3/2,2}, subject to λ < µ. Note that it is difficult to imagine any biological growth process which could actually give λ> 1, i.e. faster than exponential growth. It is quite surprising to us that non-integer values for λ and µ still yield a system that can be represented by a second-order auto-recursive Volterra model, and we have extensively confirmed this result. In general, we have found that setting Q y =⌈µ⌉ (for µ>1) guarantees good results. Curiously, for µ > 2 (and optionally λ > 2), the second-order framework frequently gives an excellent system approximation, which is often improved by inclusion of k 0 , paralleling our finding that a first-order model with k 0 >0 can give sigmoid growth. 138 Generalized Verhulst equation with per capita source/sink We briefly consider a variation on the generalized Riccati equation, where an exogenous input is assumed to act as a per capita source/sink, rather than an absolute source/sink as in the Riccati equation. We have the general underlying system as dy dt =ay −by +f(x)y, (5.17) with the simplest example of f(x) being f(x) = cx, where c a constant. We have found this per- capita sink can be captured through a second-order cross-kernel between x andy expanded on two first-order (exponential) DLFs. Thus, a very simple auto-recursive framework can capture a range of basic growth laws. Sparse data We briefly note that auto-recursive Volterra models for the basic growth laws considered may be estimated from multi-trial data, where only a few, e.g. four, time-points are sampled per trial. 139 Chapter 6 Recursive Volterra model for endotoxemia 6.1 Introduction and methods In this section we develop a relatively simple “direct-response” model for the early response to systemic endotoxin injection, i.e. experimental endotoxemia. This response is mediated primarily by macrophages of the distributed mononuclear phagocyte system (MPS, previously known as the reticuloendothelial system), with hepatic macrophages (Kupffer cells) of particular importance, as most LPS is rapidly cleared by the liver [175]. TNF has been viewed as the primary early pro-inflammatory mediator in experimental endo- toxemia, and interleukin-10 (IL-10) has been demonstrated to play a crucial negative feedback role [15]. Our primary purpose is to develop a model that faithfully captures the range of qualitative phenomena in early endotoxemia, and to demonstrate that a non-parametric MIMRO model with out-of-sample predictive capability can be derived from only the observable variables. This model 140 is quite limited, and our aim is not to draw conclusions about the immune response, a problem we address in great detail in Chapter 7, but to establish the efficacy of the Volterra-based MIMRO modeling framework in a simplified context. The basic model considers resting macrophages, M R (t), activated macrophages,M A (t), plasma TNF, T(t), plasma IL-10, I(t), and plasma endotoxin/LPS, L(t), and our model construction is similar in its basic style to that of Chow et al. [41]. The differential equations-based model follows: dL dt = f(t)−k L L, (6.1) dM R dt = −Λ(L,T)M R +Ψ(I)M A +k M M A , (6.2) dM A dt = Λ(L,T)M R −Ψ(I)M A −k M M A , (6.3) dT dt = βM A Ψ(I) −1 −δ T T, (6.4) dI dt = γM A −δ I I, (6.5) where Λ(L,T) = ( α(L+α T T) 2 K 2 1 +(L+α T T) 2 ) , (6.6) Ψ(I) = ( I 2 K 2 2 +I 2 ) , (6.7) and f(t) is the imposed endotoxin infusion schedule. In brief, macrophages are activated by both LPS and TNF according to a sigmoid response function, and inactivated by IL-10. Activated macrophagesproduceTNFataratethatisnegativelymodulatedbyIL-10. Activatedmacrophages also produce IL-10 at a constant rate, and LPS, TNF, and IL-10 all degrade via first-order kinetics. The conceptual model framework is shown in Figure 6.1. 141 Figure 6.1: Schematic representation of parametric endotoxemia model. LPS and TNF both in- duce macrophage activation, while IL-10 induces activated macrophages to enter the resting state. Activated macrophages produce TNF and IL-10, with IL-10 directly inhibiting TNF production. First-order clearance rates are determined from the substance half-lives, estimated to be 15 minutes for LPS [175, 253], 30 minutes for TNF [20, 253], and roughly 2 hours for IL-10, although IL-10 clearance appears to exhibit two-compartment kinetics [113]. 6.2 Results 6.2.1 Model behavior Figure 6.2 demonstrates the invariant TNF spike response to the three different LPS infusions schedules considered by Waage et al. [253]: (1) bolus injection, (2) bolus followed by continuous infusion, and (3) continuous infusion alone. The model behavior mirrors that which is seen exper- imentally [253]. Figure 6.3 shows that the TNF peak saturates with bolus LPS dose, while IL-10 peak does not saturate, but increases very slowly with dose beyond the TNF saturation point. Finally, Figure 6.4 gives the TNF response to two specialized scenarios: blocking IL-10 results in a much stronger, sustained elevation in TNF, as demonstrated experimentally by Berg et al. [15], 142 Figure 6.2: Parametric endotoxemia model response to three different LPS infusion schedules, reflectingthosestudiedbyWaageetal. [253]. Fromlefttoright: (1)25unitLPSbolus, (2)25unit bolusfollowedbythreehoursofcontinuousinfusionatrate5hr −1 ,and(3)threehoursofcontinuous infusion at 45 units hr −1 . TNF invariably responds as a burst, while the IL-10 time-course varies more appreciably. and pre-treatment with a very small dose of LPS results in short-term tolerance, a well-known experimental phenomenon [138]. 6.2.2 MIMRO Volterra representation We have generated simulated multi-trial data for multiple endotoxin infusion schedules. The LPS infusionscheduleconsistsofaninitialrandombolusbetween0and50,followed,atrandomtimes,by LPS boluses between 0 and 25. From this data we have estimated three variations of a single-input (LPS), two-output (TNF, IL-10) model: (1) a purely forward SIMO model, (2), two parallel auto- recursive SISRO models, and (3) a “fully connected” SIMRO model. In principal, a classical SIMO frameworkshouldsuffice,sinceallsystembehaviorultimatelyflowsfromtheexogenousLPSdriver. However, in practice, the fully connected SIMRO model not only better reflects the underlying system biology, but gives far better out-of-sample predictive performance, as demonstrated by Figure 6.5. Figure 6.6 shows SIMRO out-of-sample model predictions for three trials. 143 Figure6.3: TheleftpaneldisplaysthenormalizedpeakTNFandIL-10levelsinresponsetodifferent bolus infusions of LPS, while the right panel gives the normalized TNF peak as a function of the logarithm of the dose, reflecting Figure 1 of [253]. Figure 6.4: The left panel shows that, when IL-10 is blocked, TNF levels increase dramatically and stay elevated. The right panel demonstrates that pretreatment with 1 unit of LPS, followed by 25 units at 3 hours results in a significant blunting of the TNF response. 144 Figure6.5: BootstrapdistributionsoftheNRMSEforTNFandIL-10onout-of-samplepredictions under three classes of Volterra model: (1) a purely forward SIMO model, (2), two parallel auto- recursive SISRO models, and (3) a “fully connected” SIMRO model. There is a clear advantage to the SIMRO model over either alternative. Figure 6.6: Out-of-sample predictions of TNF and IL-10 on three concatenated trials (with SNR dB = 25) for the SIMRO endotoxemia model. 145 Figure 6.7: Functional connectivity structure among the observable variables LPS, TNF, and IL- 10 inferred from SIMRO model reduction. The left side gives the mechanistic connectivity for comparison. Note that no functional connection from TNF to IL-10 is inferred. This is expected, as omitting the mechanistic connection from the underlying parametric model has no effect on model behavior. We have found that the SIMRO model estimation and prediction is robust to noise. Note that, since in the SIMRO framework there is no clear delineation between input and output, we contaminate all data-records with noise, not just the “outputs.” Figure 6.7 gives the connectivity structure inferred from model reduction. Note that IL-10 influences TNF, but not visa versa. This is expected, as IL-10 directly inhibits both TNF production by macrophages and inhibits their activation, while TNF has no detectable direct influence on IL-10 (α T may be set to zero without measurable effects on parametric model dynamics). A further advantage to the SIMRO framework is that interventions into the system, such as blocking IL-10, can be tested under the non-parametric model. Figure 6.8 gives the out-of-sample model TNF prediction following a 25 unit LPS bolus, with and without IL-10 blocked (IL-10 blocking is modeled simply by forcing IL-10 to zero when running the SIMRO model). 146 Figure 6.8: Two trials are performed using the SIMRO model trained on regular data. In the first, IL-10 is allowed to evolve normally, while in the second it is forced to zero. The correct qualitative behavior is observed in the TNF dynamics: a rise followed by a sustained plateau. Sparse data We have examined the case when only five, irregularly spaced time-points are available from each trial. Sampling times are 0, 1, 3, 6, and 12 hours for all time series, and piecewise cubic Hermite polynomialinterpolationisusedtore-samplewithasamplingintervalofT =.01(coarsersampling may be used, but we use T = 01 to directly compare to the densely sampled time-series consid- ered above). Figure 6.9 shows five such example interpolated time-series. So long as a sufficient number of trial are used for model training (roughly 25, in this case), we have found out-of-sample predictions to be generally quite good. 147 Figure6.9: ExampleofsampledandinterpolatedLPS,TNF,andIL-10time-series. Samplingtimes are at 0, 1, 3, 6, and 12 hours, and piecewise cubic Hermite polynomial interpolation is used for re-sampling. 148 Chapter 7 Sepsis 7.1 Introduction The pathophysiology of sepsis, a bloodstream infection accompanied by a severe systemic inflam- matory response, has long been considered to be at least as much a function of the host’s response toinfectionasoftheinfectionitself. Initsmoreseveremanifestations, themodernsepsissyndrome is accompanied by the multiple organ failure (MOF) and hypotensive shock, and it is clinically similar to other severe disease states, including trauma, supporting the notion that endogenous, rather than exogenous, factors are the proximate mediators of pathology. The onset of the antibi- otic era and advances in supportive care following the Second World War improved acute survival in these conditions [57]. By the 1970s, the MOF syndrome was described, which is characterized by a temporal lag between the inciting event and a subsequent cascade of failure in multiple organ systems [57]. In parallel, it was discovered that endogenous pro-inflammatory cytokines induce can induce a potentially fatal sepsis-like syndrome when injected into mammals [18], and the view that sepsis pathology is orchestrated by pathologic activation of pro-inflammatory cytokine signaling 149 networks became dominant by the 1990s. Volume resuscitation to support the systemic circulation and renal function, and early and appropriate antibiotic treatment, form the foundation of effective sepsis therapy, but short-term mortality in sepsis still ranges from 20-50% [56]. Such dismal numbers, the paradigm of sepsis as a hyper-inflammatory immune response, and observations that elevated cytokine levels predict a pooroutcome[128]motivateddozensofclinicaltrialstestinginhibitorsofspecificinflammatorycy- tokines, mostnotablytumornecrosisfactor(TNF).Nevertheless, despiteovertwodecadesofwork, no agent modifying the inflammatory response has been conclusively shown to reduce mortality [222]. Sepsis patients are highly heterogeneous, and it has been suggested that the failure of single agents may be due to, among other factors, the existence of sub-populations which may respond in diametrically opposed ways to anti-inflammatories. Recently, the view that sepsis is characterized by an initial hyper-inflammatory state that transitions to an immunosuppressed state character- ized by opportunistic infection and re-activation of latent viruses has gained some support [112]. Furthermore, sepsis typically afflicts those with depressed immune function, especially the aged [6]. This suggests that hypo- rather than hyper-inflammation may actually underly most deaths in the modern era. Fromadynamicalsystemsperspective,wemayviewsepsisasanautonomousphysiologicprocess thatissubjecttoexogenousinputsintheformofclinicalinterventions. Giventhecomplexityofthe disorder, with dynamics governed by an almost impossibly complex immune response (host) that interacts with both the invading bacteria (bug) and treatments (drug), a non-parametric approach is attractive. In particular, the MIMRO methodology we have developed could, in principle, be used to classify patients, improve understanding of the causal interrelationships among observable 150 clinicalvariables,and/orpredicttheefficacyofnoveltreatments. Wehaveattemptedsuchaproject, using clinical time-series data generously provided by Dr. Annie Wong that gives daily vital signs and the white blood cell count (WBC) in patients with Staphylococcus aureus bacteremia, with the major goal to classify patients according to cytokine levels at presentation. However, we have found the temporal resolution of the data (daily) too coarse for the MIMRO method to be of much efficacy. Therefore, in order to gain insight into the system we have taken a parametric, differential equations-based approach. The immune system has been the subject of mathematical modeling as early as 1970 [13], and predator-prey models originally applied to ecological questions, most notably the Lotka-Volterra equations, have formed the basis for many works since [75]. The failure of cytokine inhibitors has been the main impetus for sepsis-specific modeling: the inflammatory response is highly nonlinear, and a formal representation may lead to a better understanding of how clinical interventions affect systems level behavior than the linear reasoning that has, apparently, failed thus far. Ofthosemodelsfocusingoninnateimmunityandinflammationspecifically,theymaybebroadly divided into abstract models that consider interactions among the pathogen and generic pro- and anti-inflammatory factors, e.g. [213, 140, 218, 53], and more complex models that attempt to take into account a large number of known cellular actors and cytokine factors, e.g. [44, 41, 248, 190, 196]. The majority of such models may be classified as “direct-response” models, contra “indirect-response” [79, 78] models that attempt to take into account the cell signaling networks that underly effector cell behavior. Several relatively recent works have examined the immune response to bacteria in the lung [265, 103, 224], the most common site of primary infection in sepsis. Most models are differential equations-based, but some agent-based models have been employed [2, 3, 4]. 151 We have identified three basic shortcomings of the existing literature. First, the dynamics of sepsis are usually assumed to occur within a well-mixed central blood compartment, although Nie- mannetal. [196]recentlyconsideredatwo-compartment—lungandblood—modelofendotoxemia, and Chung and colleagues [42] analyzed the dynamics of Staphylococcus epidermidis clearance by multiple organs in experimental bacteremia. In most cases of sepsis, a primary infection site is identifiable—most often this is the lung, followed in frequency by the abdomen and genitourinary tract [211, 6]—and the Kupffer cells (resident macrophages) of the liver are likely the principal source of serum cytokines, given their prominent role in clearing both bacteria and endotoxin from circulation and the fact that 85–90% of all macrophages in the distributed mononuclear phagocyte system (MPS) are hepatic [175]. Second, how model behavior and predictions change with model complexity and construction has not generally been examined in a systematic fashion, although Smith et al. have performed a careful analysis of the pulmonary immune response in three steps [224]. For more complex models this becomes extremely difficult, but is feasible for simpler models that form the foundation of larger-scale works. The problem of model-building must also be considered at two scales: (1) what behaviors, actors, etc. shall be formalized into the model framework, and (2) how are these behaviors represented formally? Typically, less care is taken with the latter question, but it can be of fundamental importance to model behavior and predictions, as we discuss extensively in the context of phagocyte-mediated killing of bacteria (Section 7.4.1). Third, application of published models to clinically relevant problems, e.g. treatment optimiza- tion, patient classification, etc., has been limited, but again, there exist some notable exceptions. Clermont et al. [44] performed simulated clinical trials on a complex sepsis model, and predicted that a subset of patients with high cytokine levels and virulent infections may benefit from anti- 152 TNFtherapy,butotherpatientscouldbeharmed. An[3],inanagent-basedsetting,testedmultiple anti-cytokine strategies and found none to be of any efficacy. A sizeable literature devoted to modeling antibiotic treatment and dose optimization exists [271, 50, 152, 125, 31], but these works are generally devoid of any complex representation of the immune response; Austin et al. [8] did examine treatment under relatively simple representations ofimmunityinHIVandmalaria. Motivatedbytherecentclinicalinterestincytokineinhibitors,all theinflammation-specificmodelswehaveidentifiedthatstudytreatmentconsidercytokine-specific treatment alone and ignore antibiotic therapy [44, 3]. However, antibiotics would never be omitted in clinical practice and therefore should be included with any novel therapy. Given that sepsis is typically driven by a primary organ-system infection, we model bacteria- immune dynamics at three spatial/geometric scales: (1) a single, well-mixed organ compartment, (2) a multi-patch spatial representation of the organ compartment, and (3) a systemic geometry incorporating the lung as the primary infection site generating bacteremia, a central blood com- partment with a reactive neutrophil pool, and the liver as the site of bacteremia clearance and source of systemic cytokines. The well-mixed model provides the formal foundation for the second two scales. In the well-mixed setting, we perform a detailed hierarchical model-building process, with predator-prey interaction between bacteria and phagocytes as the basis. Positive and negative cytokine-mediatedfeedbackisincorporatedintheformofmacrophage-derivedTNFandinterleukin- 10 (IL-10), respectively. Upon this basis we add endothelial activation with recruitment of circu- lating neutrophils, and active immunosupression driven by neutrophil apoptosis in turn. We also add explicit consideration of tissue damage and monocyte recruitment, but finding that these have littlequalitativeeffect, weomitthemfromfurtherconsideration. Fromthisexercisewehavedrawn 153 Figure 7.1: Schematic representation of the overall hierarchical approach to model-building. At the first spatial/geometric scale, a hierarchy of parametric models are studied, and one of interme- diate complexity is chosen as the basis of further efforts. The first of these is a multi-patch spatial geometry, where dynamics within each patch are governed by the ODE model with the addition of inter-patch transitions. Finally, this informs a model for the systemic geometry, incorporating primary infection in the lung, reactive circulating neutrophils, and the liver as a site for bacterial clearance and cytokine production. A pharmacokinetic/pharmacodynamic-based antibiotic treat- ment module is applied at every spatial scale. 154 several conclusions. First, the widely used mass-action formalism for bacteria-phagocyte interac- tionislikelyinappropriate, whereasaHollingTypeIIformnotonlyisbetterjustifiedonaphysical basis, it has significant effects on model dynamics even in the simplest in vitro setting. Second, we find that tissue-resident macrophages alone cannot initiate a robust immune response that spon- taneously clears, but local-systemic interaction in the form of neutrophil recruitment is necessary. Finally, when neutrophil apoptosis is explicitly considered we observe a transition from a hyper- to a hypo-inflammatory state following infection. We develop a formal representation for antibiotic treatment and incorporate this into our bacteria-immune models at all three spatial/geometric scales, and we examine the effect of treat- ment with and without adjuvant TNF inhibition. We have found that antibiotics interact synergis- tically with the immune system, with this synergism far more pronounced in a basically immuno- competent host versus an immunocompromised host. Moreover, immunocompromised patients are extremely vulnerable to the evolution of antibiotic-resistant mutants, and treatment delay both impairs bacterial clearance and increases the likelihood of resistance. Using the model developed for the well-mixed setting, we adapt it to both spatial and sys- temic representations of infection. We find that the qualitative behavior of the basic model is generally preserved at these higher scales, as are our predictions concerning treatment. Computa- tional investigation of cytokine levels and anti-TNF treatment combined with antibiotics suggests a basic conclusion that is in general accord with previous modeling work and the clinical litera- ture: elevated cytokine levels are a natural consequence of serious infection and are not necessarily inappropriate; early and effective treatment of the causative pathogen must be the clinical focus. Anti-inflammatoriesmayhaveaminorbenefittosomepatients,butattheriskofimpairedbacterial clearance and opportunistic infection. 155 Thus, our overall approach is hierarchical at multiple levels; this is demonstrated schematically inFigure7.1. Thischapterisorganizedasfollows: Section7.2reviewssepsisepidemiology,historical background, and the current understanding of sepsis pathophysiology. Previous theoretical work on the subject is reviewed in Section 7.3. As the foundation of our modeling effort, Section 7.4 presents our hierarchical model-building process in the well-mixed spatial setting. Section 7.5 introduces antibiotic therapy and a combined pharmacokinetic/pharmacodynamic framework for incorporating fluoroquinolone and vancomycin treatment into our model framework. Section 7.6 discusses spatial and geometric model extensions, with our overall conclusions detailed further in Section 7.7. 7.2 Biological and historical background 7.2.1 Clinical overview and epidemiology Bloodstream bacterial infection (bacteremia) can lead to an array of system-wide physiological de- rangements,resultinginalife-threateningdeparturefromhomeostasisthatisgovernedbycomplex, system-level interactions between the host, pathogen, and treatments. The systemic inflammatory response syndrome (SIRS) is defined as a clinical response to a non-specific event including two or more of (1) temperature > 38 ◦ C or < 36 ◦ C, (2) HR > 90 beats/min, (3) RR > 20 breaths/min or PCO2 < 32 mmHg, or (4) WBC > 12.0 × 10 9 /L or < 4.0 × 10 9 /L, or the more than 10% immature neutrophils [216]. Sepsis is defined as SIRS in the presence of a bloodstream infection. Sepsis represents a major public health problem of increasing importance. Sepsis is extremely heterogeneous in its clinical presentation, with comorbidities and immunosuppression common. The respiratory tract is the most common primary site of infection (44% of cases), followed by the 156 genitourinary tract and the abdomen [6]. A large and influential study by Angus et al. [6] estimated that in the United States, there were 751,000 cases of severe sepsis in 1995, with an overall mortality rate of 28.6%, translating into 215,000 deaths. The incidence of sepsis increases dramatically with age (100-fold elevation from children to > 85 years old), as does the case-fatality rate. The latter trend is especially robust in patients without underlying comorbidities. Even with modern critical care management, 20–30% of patients still succumb. Moreover, survival is typically assessed at 28 days, highlighting the acute nature of the condition [216]. The incidence of sepsis in the U.S. increased three-fold from 1979 to 2000 [172], and this trend is widely expected to continue as the population ages. 7.2.2 Historical background Endotoxin, nowalsoknownaslipopolysaccharide(LPS),isacomponentofthebacterialcellwallof gram negative bacteria. When injected into mammals, it evokes a severe shock state characterized by fever, hypotension, and multi-organ dysfunction [239]. Endotoxin was discovered in the late 1800s by Richard Pfieffer, who found that injection of dead Vibrio cholerae into guinea pigs caused the animals to die; Pfieffer deduced that this was due to a component within the cell walls of the bacteria. In the latter part of the twentieth century, it would be discovered that the pathologic effects of endotoxin administration, which so mimic those seen in severe sepsis, are mediated by soluble factors produced by immune cells, leading to the currently dominant paradigm of sepsis as a disease state governed by the cytokine-orchestrated inflammatory response. By the 1940s, it had been discovered that soluble factors extracted from the immune cells of infected patients can induce a body-wide febrile response, and in 1944 Menkin attempted to purify such factors, which he termed “pyrogens” [80]. In the 1970s and 80s, it came to be recognized 157 that a wide variety of cells produce hormone-like mediators that affect immune cell function, and Cohen introduced the general term “cytokine,” for such factors. The cytokines interleukin-1 (IL-1) and tumor necrosis factor (TNF) were first described in 1971 and 1975, respectively, and are now considered canonical cytokines in sepsis biology. Injection of endotoxin has long been known to cause hemorrhagic necrosis in tumors; Carswell and colleagues [33] discovered that this effect could be attributed to a soluble factor produced by macrophages that selectively killed malignant cells, which they dubbed TNF. In 1985, it was demonstratedthatTNFandthemacrophage-derivedsubstance“cachectin,”sonamedforitsability to induce the cachexia syndrome of anorexia, weight loss, and anemia often seen in cancer and chronic disease, were one and the same [17]. In parallel, it had become clear by the 1970s that endotoxin did not act directly as a poison, but, as is typical for substances that exert their effect in minute amounts, likely acted on specific cellular receptors with associated signal-amplifying transduction pathways [18]. TNF/cachectin produced by macrophages emerged as the central link between endotoxin and pathology. Injection of anti-TNF antibodies protects mice from otherwise lethal endotoxin-induced shock, and TNF from endotoxin stimulated macrophages causes a sepsis-like syndrome similar to that caused by endotoxin itself. Through a spontaneous genetic mutation, C3H/HeJ mice have been rendered highly resistant to endotoxin, but TNF injection still induces a sepsis-like syndrome [18]. While protected from direct endotoxin challenge, C3H/HeJ mice are far more likely to succumb to infection by bacteria [18], as the inflammatory response induced by endotoxin is essential to an effective immune defense. Indeed, it is a common theme in sepsis that the poison and the antidote are often the same. But, contra Paracelsus, it is more than simply the dose that differentiates the poison from the remedy, it is the context in which that dose is given. 158 In the 1990s, the view that sepsis was caused by an over-exuberant inflammatory response mediated by pro-inflammatory cytokines, including TNF, IL-1, IL-6, IL-12, and interferon-γ (IFN- γ)becameprevalent,asdidthenotionofthe“cytokinestorm,”anun-checkedpositivefeedbackloop in which cytokines induce effector cells to activate and release more cytokines, further activating the immune cells, and so forth. The sequential appearance of cytokines in the plasma following insult, e.g. by endotoxin, became known as the “cytokine cascade” that orchestrates the body’s response [222]. Various clinical studies also linked cytokine levels to outcome in sepsis (reviewed in Section 7.2.3). Given this view, a number of mediator-specific anti-inflammatories were developed. Initially, multiple agents targeting TNF and IL-1 were tested, but despite promising pre-clinical results, failed to reduce mortality in clinical trials [222]. It was suggested that these failures could be explained on the basis that TNF and IL-1 are “early” cytokines, whose role is to initiate the larger immune response; these cytokines return to near baseline within several hours, and may be down-regulated by the time a diagnosis of clinical sepsis is made, implying the window for therapeutic intervention has usually passed by that point [222]. A number of other agents were tested, all with similarly disappointing results [222, 267]. However, a meta-analysis by Zeni et al. [267] combining 6,429 patients who had received different anti-inflammatories suggested a borderline-significant, roughly 10% reduction in relative risk conferred by mediator-specific anti- inflammatories as a class. Moreover, a meta-analysis by Eichacker et al. [64] also suggests that anti-inflammatories are beneficial to those with a high risk of death, but harmful to those at low risk. Hope also remains with respect to targeting presumptive “late” mediators of sepsis, such as high-mobility-group-box-1 (HMGB-1). Since these failures, an hypothesis has emerged that sepsis proceeds in roughly two stages. The 159 classical, early hyper-inflammatory state appears to be followed by a prolonged period of profound counter-regulatory immunosuppression that increases susceptibility to secondary infections and death [110]. In a murine sepsis model, elevated markers of inflammation predict early, but not late deaths[200]. Underthisview, pro-, ratherthananti-inflammatorystrategies, wouldbeappropriate for many patients, particularly those surviving into later-stage sepsis. Clinical trials and meta-analysis have consistently demonstrated high-dose glucocorticoids to be actively harmful, while low-dose or “physiologic-dose” steroid treatment may have a modest beneficial, although this is controversial and any benefit likely relates to the positive hemodynamic effect of glucocorticoids and the high rate of adrenal insufficiency encountered in sepsis patients, rather than any immunomodulatory effect [202]. In the last decade, awareness of sepsis has increased, due in no small part to the international Surviving Sepsis Campaign (SCC). The story of the SCC is intimately linked to that of the only anti-inflammatory agent ever approved to treat sepsis, recombinant human activated protein C (rhAPC), also called drotrecogin alfa (activated). In 2001, the single randomized clinical trial (RCT) to find rhACP to be beneficial, the PROWESS trial, found an absolute risk reduction of 6.1% for all-cause mortality at 28 days (30.8% control mortality, 24.7% in rhACP arm) [212]. Post-hoc subgroup analysis suggested that only patients at high risk of death benefitted from the drug. Based on this single trial (in contrast to the usual requirement for two positive trials), the FDA approved rhAPC only for patients with a high risk of death (as defined by the APACHE score), a controversial decision that has been strongly criticized, as such subgroup analyses are generally prone to over-interpretation and should only be considered hypothesis-generating unless they are strictly defined with good rationale beforehand [209]. 160 Given the controversy surrounding the PROWESS trial, and lingering doubt concerning the efficacy of APC, sales were disappointing. As part of a three-pronged marketing campaign, the drug’s maker, Eli Lilly, provided over 90% of the funding for the initial phases of the SSC, which was introduced in October of 2002, and its guidelines for sepsis management were published in March of 2004 [63]. Eichacker and colleagues present a thorough discussion of the shortcomings of these guidelines, and of the frankly insidious marketing campaign run by Eli Lilly [63]. Two subsequent clinical trials, ADDRESS and RESOLVE, were terminated early due to likely futility [63]. Finally,attheendof2011,rhAPCdiedaquietdeath. ThePROWESS-SHOCKtrial,performed at the request of the European Medicines Agency (EMA), failed to show a benefit to the drug, and Eli Lilly announced that it would be withdrawn from the market [181]. It may be unrealistic to expect any single targeted anti-inflammatory agent to reduce absolute mortality by more than a few percentage points. If this is the case, then clinical trials have been uniformly under-powered to detect an effect. Despite significant interest in bundling therapies, as in the SCC guidelines, there is little evidence that bundling, per se, improves outcomes, with decreased time to antibiotics and increased appropriateness of antibiotics the only components of bundled care consistently linked to a positive outcome [11]. Indeed, early and effective antibiotic treatment has been consistently shown to dramatically affect survival [56, 11]; in one large study, a delay of only six hours to treatment was associated with a threefold increase in mortality [56]. Very recently, the ProCESS trial found no benefit to either early goal-directed therapy (EGDT) (a sepsis bundle) or a second protocol compared to usual care [211]. As it now stands, there is no effective non-antibiotic agent available known to be effective in sepsis, with the possible exception of low-dose glucocorticoids. Two decades of experience with 161 anti-inflammatoryagentshasbeenuniformlydisappointing, nowthatrhAPChasfallenfromgrace. While the SCC has pushed the adoption of a sepsis bundle for overa decade, there is little evidence that any interventions in the bundle other than rapid and appropriate antibiotic treatment are beneficial. Multiple problems exist with the current practice of clinical trials, including admission criteria, a possible (or even probable) lack of external validity, and it is especially likely that trials are under-powered. As pointed out by Angus et al. [5], sepsis is a disease of the elderly, and frequently affects patients with HIV and malignancy; such patients are typically excluded from clinical trials, yet this may compromise the external validity of such trials. Existing animal models are highly problematic for a variety of reasons. Animals are typically young with no co-morbidities, and antibiotics and supportive care, cornerstones of sepsis treatment in humans, are rarely given [5]. Furthermore, common experimental models for sepsis may have little relation to the actual clinical syndrome. In particular, experimental injection of LPS, while inducing a vigorous, stereotyped, and highly reproducible cytokine response accompanied by a clinical syndrome resembling that seen in acute illness, has little meaningful relation to actual human sepsis. Cytokine levels in animal endotoxemia are orders of magnitude higher than those encountered in clinical sepsis. Itcanhardlybesurprisingthatamodificationofhostresponseinayoung,healthy,unsupported animal that is beneficial with respect to an experimental challenge only peripherally related to human sepsis may not, in fact, be beneficial to an elderly human patient receiving support in a modern critical care unit [5]. 162 7.2.3 Cytokines for predicting outcome in sepsis A number of studies have related outcome to initial and serial cytokine measurements (TNF, IL-1, IL-6, and IL-10 are the most studied), but most have been small, single-center studies. Despite intense interest in the molecule, TNF alone is a poor predictor of outcome, and while on average inflammatory cytokines are significantly elevated in sepsis, a significant portion of septic patients have undetectable levels of TNF, IL-1, IL-6, and IL-10 [34, 128]. In large studies, only a small fraction of patients have detectable IL-1 [129], and at best half have detectable TNF [129, 128]. On a single-mediator basis, IL-6 appears to best predict mortality across multiple studies [51, 34, 128], although “cytokine scoring” with multiple mediators may be superior [34]. Pinsky et al. [207] found that TNF and IL-6 were elevated in septic compared to non-septic shock. Peak cytokine levels, which varied over nearly three orders of magnitude, had no bearing on mortality, but persistent elevation did predict a poor outcome. One small study suggested that an increase in the IL-6 to IL-10 ratio over time is associated with a poor outcome [235]. A more recent large, multi-center study by Kellum et al. [128] measured serial serum TNF, IL-6, and IL-10 levels in sepsis due to community-acquired pneumonia. They found no initial cytokine spike, with a gradual decline observed in all three cytokines, the only exception being a modest late rise in TNF in non-survivors. This general pattern has been observed in multiple studies [235, 207, 51]. Cytokines remained elevated for weeks, consistent with prior studies. This is particularly salient, as persistent cytokine elevation may predict mortality [207], yet most anti- cytokine trials have administered treatment for only three days [128]. While in [128], severe sepsis at day 1 was associated with elevated levels of all markers, many patients with adverse outcomes had normal cytokine levels, while others with positive outcomes had a strong cytokine response. In general, IL-6 and IL-10 levels were congruous, with patients 163 partitioning roughly into low, medium, and high levels of general cytokine activation, with the highest risk of death seen in those with high IL-6/high IL-10 levels. This is contra earlier results that suggested increased anti-inflammatory activity, as measured by the IL-10:TNF ratio, best predicted outcome [87]. However, this is not unexpected under the view that TNF is an early acting cytokine, with IL-6 a better marker for later pro-inflammatory activity (see, e.g. [128, 129]). Despite highly significant associations between cytokines and outcome in cohort studies, cy- tokine levels do not reliably predict outcome on an individual basis. Sepsis patients are highly heterogeneous, with robust cytokine responses often seen in those who do well, suggesting that a one-size-fits-all approach to modulating inflammatory mediators is unlikely to perform well. 7.2.4 Pathophysiology Classically, “inflammation” refers to a stereotyped vascular response to tissue injury or pathogens thatinvolves,atthesiteofinjury,increasedbloodflow,increasedvascularpermeability,andchanges in endothelial surface molecules that attract effector immune cells to the site. These changes cause the cardinal signs of inflammation, as recognized since antiquity: 1. Rubor (redness) 2. Tumor (swelling) 3. Calor (heat) 4. Dolor (pain) 5. Functio laesa (loss of function, added to the original four by Galen of Pergamon) These signs are primarily mediated by activated endothelium at the level of the microvasculature, and at the post-capillary venules in particular [245]. Endothelial “activation” can be induced 164 Figure 7.2: Schematic progression of local response to infectious insult leading to the four cardinal signs of inflammation. Figure partially adapted from Figure 1-8 of [116]. by a variety of stimuli, including cytokines such as TNF and IL-1, TLR-2 and TLR-4 activation, hypoxia, reactiveoxygenspecies(ROS),andmetabolicstress[245]. Thelocalimmuneandvascular responses leading to the four cardinal signs are illustrated schematically in Figure 7.2. Activated endothelium recruits immune cells to the site of insult, and a local and regional im- mune response is initiated. This local response can become systemically activated when bacterial products or markers of tissue damage enter the bloodstream. An unchecked, systemic inflamma- tory response to can lead to widespread tissue damage, cardiac and circulatory compromise, and ultimately irreversible bioenergetic failure and death. There exist a number of severe disease states that share as their common etiology a severe systemic inflammatory response, including both in- fectious and noninfectious states, suchas serious trauma or burns (sometimes referred to as “sterile inflammation”). Pathogen and damage recognition Inflammation is a non-specific response, and as such is considered to be part of the innate im- mune system. When bacteria or other foreign invaders enter into a normally sterile site, resident 165 antigen-presenting cells (APCs), principally macrophages and dendritic cells, recognize their pres- ence through pattern recognition receptors (PRRs), which recognize an array of highly conserved microbial molecules, termed pathogen-associated molecular patterns (PAMPs) [227]. The toll-like receptors (TLRs), discovered in the mid-1990s, are canonical transmembrane PRRs. TLR4 was discovered to be the long sought-after receptor for endotoxin (LPS), and also is a receptor for lipoteichoic acid, expressed by Gram-positive bacteria, and the important endogenous late inflam- matory mediator, HMGB-1. The TLRs, through several transduction pathways, ultimately signal tothetranscriptionfactorsNF-κB,AP-1,andIRF3[223],inducingexpressionofmultiplecytokines, chemokines,andinduciblenitricoxidesynthase(iNOS).NF-κBhasbeencalledthe“masterswitch” of inflammation, and is central to sepsis biology. This transcriptional activity activates the endothelium, recruits neutrophils and monocytes from circulation, and causes activation and enhanced bacteriocidal activity in these effector cells [223]. Activated phagocytes destroy bacteria through multiple mechanisms. Activation of phago- cytes through PRRs also leads to the activation of the adaptive immune response, which is vitally important in clearing serious infections. Tissue damage caused by the inflammatory response may act as a positive feedback, and this feedback loop has received much attention in existing theoretical treatments of sepsis. In 1994, largely on theoretical grounds, Matzinger [176] proposed that dying cells release pro-immunity adjuvants, referred to as danger associated molecular patterns (DAMPs). It has since been shown that dying cells do indeed play a central role in determining immune responses, with apoptosis and necrosis acting as fundamentally different modes of cell death. Necrotic cells are believed to act as danger signals that promote immunity, while apoptotic deaths are either immunologically silent or actively immunosuppresive [135]. 166 Anumberofpre-existinginflammatorymoleculesarereleaseduponnecroticcelldeath,including HMGB1, which stimulates acute inflammation and monocyte activity [135], and has been found to mediate late mortality in murine endotoxemia [256]. Apoptotic cells do not appear to release DAMPs, and massive lymphocyte apoptosis may lead to prolonged immunosuppression in sepsis, as discussed in Section 7.2.4. Reactive oxygen species (ROS) and nitric oxide (NO) ROS and NO both play central roles in vascular homeostasis, inflammatory cell signaling, the microbicidal activity of phagocytes, and in sepsis-induced pathology. Upon activation, phagocytes consumemassivequantitiesofoxygenintheso-calledrespiratorybursttoproduceabatteryofROS, including superoxide, O •− 2 , H 2 O 2 , and HO • in a NADPH-dependent fashion [9]. The respiratory burst is vital to anti-microbial defense, and inherited defects in the respiratory burst cause severe immunodeficiency. While toxic to bacteria, ROS also causes damage and dysfunction in normal tissues. ROS also induces inflammatory cell signaling through NF-κB. Cytokine stimulation also results in phagocyte expression of inducible nitric oxide synthase (iNOS). Nitric oxide (NO) was declared Molecule of the Year by Science in 1992 [48] and with good cause. It causes vasodilation and is vitally important in maintaining the microcirculation. It is also a powerful oxidizing agent with direct bactericidal activity, acts synergistically with ROS to kill pathogens, and rapidly reacts with O •− 2 to form the extremely potent peroxynitrite (ONOO − ) radical[238]. NOcanreversiblyinhibitthemitochondrialelectrontransportchain(ETC),resulting in oxidative stress and depletion of anti-oxidant defenses [238]. Like ROS, NO can damage host tissues, and in excessive amounts causes pathological endothelial cell dysfunction, dysfunction of the microcirculation, and inhibits mitochondrial respiration. This leads to hypoxia, mitochondrial 167 dysfunction, and ultimately cellular respiratory failure. Endotoxinandmultipleinflammatorycytokines(IL-1,IL-6,TNF,etc.) induceiNOSinavariety ofcells,whileanti-inflammatorycytokinessuchasIL-10andTGF-β inhibitiNOS,andexcessiveNO production may be largely responsible for the hemodynamic derangements encountered in sepsis. Hemodynamic changes Hemodynamic changes are prominent in sepsis, driven largely by microvasculature dysfunction that can exist in the face of adequate systemic blood pressure. However, sepsis also affects the macrocirculation, and systemic circulatory failure plays an essential role in septic shock and other conditions (e.g. hemorrhage, anaphylactic shock, etc.). Sepsis causes hypovolemia and cardiac depression [124]. Hypovolemia may be caused by widespread capillary vasodilation and/or leakage and the subsequent redistribution of fluid from the systemic circulation to the peripheral circula- tion and interstitial spaces. Cardiac output is also compromised in septic states, and inflammatory cytokines such as IL-6 have been shown to directly cause myocardial depression; this further con- tributes to hypoperfusion [124]. Even if systemic blood pressure is adequately supported, tissue hypoperfusion may persist, due either to maldistribution at a regional or microvascular level, or to mitochondrial dysfunction, a condition that is sometimes referred to as “cryptic shock.” Immunosuppression and aberrant cell death There is substantial evidence that, following an initially strong inflammatory response, counter- regulatory mechanisms lead to a later, prolonged stage characterized by profound immunosuppres- sion, a state that has been termed “immunoparalysis.” Normally dormant endogenous viruses are 168 frequently re-activated in septic patients, and these patients are also susceptible to opportunistic infections that are rarely problematic in an immunocompetent host [59, 110]. It has been consistently observed that whole blood and leukocytes from septic patients display markedly suppressed inflammatory cytokine production [68]. Boomer et al. [21] recently found splenocytes from patients who died from sepsis to have profoundly impaired cytokine expression and multiple functional impairments. T cell dysfunction is prominent in sepsis, with widespread apoptosis, defective proliferation, cytokine production, and anergy observed [111]. Progressive loss of B cells, CD4 + T cells, and follicular dendritic cells due to apoptosis has been observed, and in animal models, preventing lymphocyte apoptosis improves survival [111]. Impaired monocyte function has also been associated with a poor prognosis, and interferon-γ (IFN-γ) may restore monocyte function and improve prognosis [59]. Aberrant patterns of cell death may be particularly important. Widespread lymphocyte apop- tosis is detrimental to the host, as it not only represents a profound depletion of effector immune cells,buttheapoptoticmodeofcelldeathisanti-inflammatoryandmaybecentraltotheinduction of immunoparalyis. From local to global A common theme in sepsis biology is that inflammatory and immune responses that are beneficial in combating focal infections become detrimental to the host when activated systemically [223]. For example, neutrophils are early responders to infection, and produce ROS and neutrophil extra- cellular traps (NETs) that, while potent antimicrobial defenses, also contribute to tissue damage and organ dysfunction. The endothelial activation that is vital to the inflammatory response can lead to widespread hemodynamic abnormalities when it occurs on a systemic scale. The inflamed 169 endothelium also causes the activation of platelets and the coagulation cascade, which helps to se- quester local infections, but can also contribute to organ dysfunction and even lead to uncontrolled bleeding caused by disseminated intravascular coagulation (DIC) [223]. The central role of signalling cascades The inflammatory response in sepsis can be understood at multiple scales, e.g. cell, tissue, organ, etc., but underlying behavior at all scales is the behavior of individual cells, which is governed by transcriptional activity, which is in turn governed by a relatively small number of interacting signal transduction cascades. The paradigm of many cytokines, PAMPs and DAMPs converging upon a few signalling pathways (characterized by significant cross-talk) that in turn affect the expression ofmanyhundredsofgeneproductsgoverningmyriadcellularbehaviorshasbeentermeda“bow-tie model” [264]. The geometry of sepsis Finally,wenotethatwhilesepsisisdefinedbythepresenceofbacteriainthebloodstream,andmany models treat the process as if it occurred in a well-mixed compartment, this is not at all the case. Thereisgenerallyaprimarysiteofinfection, localcytokinelevelsmaynotreflectplasmalevels, the MPS,andparticularlytheKupffercellsoftheliver,areessentialtoLPSandbacterialclearance,and immune responses are staged in lymphoid tissues. Thus, there is clearly a spatial/compartmental geometry to sepsis that is ignored when we only monitor the plasma. 170 7.3 Existing models 7.3.1 Sepsis- or inflammation-specific models There has been significant interest in modeling sepsis as a dynamical system in recent years, gener- ally through either differential equations or agent-based modeling; we review some of those efforts here. Abstract models Several models that consider the behavior of a highly abstracted inflammatory response to a pathogen have been proposed. Such models can be useful in rigorously determining the dynamical space that general hypotheses concerning inflammation generate, and they can be used to explore general treatment strategies. One of the earliest such attempts was that by Kumar et al. [140], in 2004. The model focuses upon the posited positive feedback loops between early and late initiators of inflammation. While only considered as abstract entities, two possible and important candidates for such early and late mediators are TNF and HMBG-1, respectively. The model considers three variables: some pathogen, p(t), an early pro-inflammatory cytokine, m(t), and a late pro-inflammatory cytokine, l(t). The governing equations are: dp dt = k p p(1−p)−k pm mp (7.1) dm dt = (k m p+l)m(1−m)−m (7.2) dl dt = k lm f(m)−k l (7.3) 171 where f(m)=1+tanh ( m−θ w ) (7.4) In numerical simulations the model allows three behaviors following infection: (1) sustained oscil- lations, taken to represent recurrent infection, (2) persistent inflammation in the presence of a high pathogen level, and (3) persistent inflammation in the absence of infection. The model can also be considered to give pathogen clearance, if clearance is assumed to occur once the pathogen level has dropped below some threshold (mathematically, but not biologically, oscillations are sustained). One of the clearest problems with the model is that l does not actually act as a true late mediator, since it instantaneously responds to the early mediator and in simulations its levels rise sooner and faster than the ostensible early mediator. Based on their results, the authors suggested that clinical sepsis may represent at least two distinct inflammatory states that share a common “wiring diagram,” and that can be expected to respond differently to treatment. This is an intriguing possibility, but without further elaboration, of little direct use. The usefulness of the model is further limited by its somewhat haphazard construction and arbitrary parametrization. Continuing in the vein of abstracted inflammation, Reynolds, Day, and colleagues [218, 53] suggested a model which more completely captures the key features of acute inflammation but is not overly complex. Four key variables are considered: P(t) is the pathogen load,N ∗ (t) represents pro-inflammatory activity in the form of activated phagocytes, D(t) represents tissue damage, and C A (t) is an anti-inflammatory mediator. We omit a detailed presentation of the differential equations,but,inshort,thepathogencausesinflammation;inflammationinhibitsthepathogen,and induces itself both directly and through a positive feedback loop with tissue damage. Inflammation 172 and damage both induce “anti-inflammation.” Reynolds et al. [218] found that their model admits three equilibria, taken to correspond to health, septic death, and aseptic death. The inclusion of the anti-inflammatory, C A , increases the basin of attraction of health compared to the model of Kumar et al. [140]. Interestingly, the model predicts that following infection there is a period of relative immunosuppression, a well-known phenomenon. Furthermore, it was found that simulated treatment with an anti-inflammatory could be beneficial for a modest dose, but that high doses are detrimental. This corresponds to the well-established observation that administration of high-dose gluco- corticoids in sepsis is clearly detrimental to survival, while low-dose steroids may have a modest survival benefit. However, the latter point is controversial and is by no means firmly established [161]. In any case, the relative benefit of anti-inflammatory intervention is predicted to be modest and to depend on the growth characteristics of the pathogen. Unfortunately, the authors did not explore simulated treatments more thoroughly, so conclusions are limited in this respect. Afollow-upworkbyDayetal. [53]examinedthescenarioofrepeatedendotoxinadministration, and experimental endotoxemia has been the focus of many more recent theoretical works. It had beenobservedasearlyas1946,byBeeson[12],thatrepeateddosesofendotoxin(orkilledE.typhosi, in Beeson’s original work) result in a blunted response, a phenomenon termed endotoxin tolerance. Endotoxin tolerance has usually been studied over the time-scale of days to weeks, with a blunted response seen up to a week following initial endotoxin challenge in the rat [253], but tolerance can occur within several hours [79, 138]. Pre-treatment with a sublethal dose of endotoxin can save mice from a subsequent, normally lethal, challenge. Endotoxin tolerance occurs within hours in humans and may persist for two weeks or more [138]. 173 Whiletoleranceistheusualscenario,potentiationcanalsooccur,andmayoccurwithshort-term exposuretoendotoxin. Thatis,thereisanarrowwindowinwhichaprimingdoseofendotoxinmay potentiate the response to a second challenge, but within a few hours tolerance occurs. Moreover, endotoxin tolerance is nonspecific, implying a global alteration in immune function [36]. Under several simulated scenarios mimicking experiments reported in the literature, the model of Day, Reynolds and colleagues successfully reproduced the qualitative outcomes of endotoxin tolerance and potentiation. Direct response models Following, and in parallel to, the more abstract works discussed above [140, 218, 53], Vodovotz and colleagues have published a number of far more detailed models [44, 41, 248, 190, 196], largely based on the 2004 model of Clermont et al. [44] and the 2005 model by Chow et al. [41]. Because this family of models takes cytokine production to be a “direct” function of the level of activated macrophagesandneutrophils(andothercytokines),disregardinganyoftheunderlyingcellsignaling pathways at work in these cells, we refer to them as the “direct response models.” Clermontandcolleagues[44]proposedanin silico clinicaltrialofanti-TNFantibodytreatment, based on a detailed differential equations-based model for the inflammatory response in sepsis, which considered a generic Gram-negative pathogen along with multiple inflammatory mediators. The model was initially calibrated using experimental endotoxemia data, and a study population of 1,000 patients was generated by varying initial pathogen load, growth-rate, time to diagnosis, and certain model parameters were varied ± 25% relative to their baseline values. Nine different anti-TNF dosing strategies were tested, and for the globally optimum one, a statistical model was used to identify those patients most likely to benefit, based on initial cytokine levels. This is a 174 potentially very useful general strategy. The model of Chow et al. [41] is quite similar to that in [44], but it is more clearly the basis for the later models of this group, and like most later models, the model is applied primarily to experimental endotoxemia (rather than a clinical question). This model is a complex system of differential equations, consisting of 16 variables: 1. Endotoxin (LPS) 2. Resting neutrophils (N R ) 3. Activated neutrophils (M R ) 4. Resting macrophages (M R ) 5. Activated macrophages (M A ) 6. Endothelial/constitutive nitric oxide synthase (eNOS) 7. Inducible nitric oxide synthase (iNOSd, iNOS) 8. Stable reaction products of NO: NO − 2 /NO − 3 (NO3) 9. TNF 10. IL-10 11. IL-6 12. Adrenergic inhibitory activity (CA) 13. IL-12 14. Blood pressure (BP) 15. Tissue damage (D) The model is formulated based on known relationships between the variables, with activated neutrophils and macrophages the sources for all cytokines. The full model is extremely complex, with 99 free parameters, not including initial conditions. We give one example governing equation, that for activated macrophages, M A : (7.5) dM A dt =       LPS 2 1+ ( LPS xMLPS ) 2 +k MD D 4 x 4 MD +D 4    ( TNF 2 x 2 MTNF +TNF 2 +k M6 IL6 2 x 2 MS +IL6 2 ) +k MTR TR(t) +k MB f B (B)    1 1+ ( IL10+CA xM10 ) 2 M R −k MA M A . 175 In brief, sigmoid terms in LPS and damage sum to give activation at a rate that is positively modulated by TNF and IL-6. Trauma, represented by TR(t), and blood pressure, through the functionf(B), also add to activation. All activation is directly countered by IL-10 and CA. While apparently reasonable, the different functional forms are phenomenological and do not have any direct correlate with underlying biological mechanisms. The requirement that TNF or IL-6 levels be non-zero for either LPS or damage to induce macrophage activation is unrealistic, and various other aspects of the model may be questioned. For example, damage (equation not shown) is a function only of IL-6, trauma, blood pressure, and NO. The model can be tuned such that it reproduces time-series of endotoxemia data for mice reasonably well, both quantitatively and qualitatively. However, given the degrees of freedom of the model, it is hard to imagine how any remotely reasonable formulation could fail to. Ithaslongbeenrecognizedthatwithsuchcomplexnonlinearmodels,thereexistmanydisparate parameter sets that give a good fit to data, i.e. many local minima. Daun et al. [52] proposed a method for identifying clusters of acceptable parameters, applied to an eight-state model for the inflammatory response similar to [44]. The method entails first identifying highly-sensitive parameters via a finite-difference method, and fixing less-sensitive parameters strongly correlated withhighly-sensitiveparameters. Areducedsetofsensitiveparametersthusidentified,thisreduced set is optimized for a multitude of initial conditions, and the resulting ensemble of parameter estimates is subjected to a clustering algorithm. Different clusters of acceptable parameters give verydifferentestimatesofthehiddendamagevariabletime-course,suggestingthatsimilarcytokine profiles may be associated with very different mortality outcomes. Recently, Nieman et al. [196] extended Chow et al.’s model [41] to include a lung compartment, inadditiontothewell-mixedsystemiccompartment,andcalibratedthemodeltoswineendotoxemia 176 data. This model has 200 parameters, but using the algorithm of Daun et al. [52], a subset of roughly 20 highly sensitive parameters was identified, and these parameters were varied to fit the model to individual pig time-series. They also employed a method based on principal component analysis (PCA), originally proposed by Mi et al. [179], to identify the most influential cytokines; the results of such an analysis may then help inform parametric model construction. In brief, for each the leading principal components, the weight assigned to a given cytokine was multiplied by the principal component’s eigenvalue; for each cytokine, an overall score was calculated as the sum of such products. While Nieman et al. claim that these results were used to inform model construction, it unclear how and to what extent this was actually done, as the model appears to be directly adapted from Chow et al. [41]. In sum, these models have become increasingly complex, and more sophisticated methods for reducing the parameter space and identifying influential variables have been incorporated. While theycanbefittodescribeendotoxemiatime-courses,thecentralroleofthehiddendamagevariable is, in our view, problematic. Nieman et al. [196] found D(t) to match, in a very rough qualitative sense, the time-course of the oxygen index in three of four pigs. However, there was no relation between standard markers of physiologic dysfunction such as AST, ALT, pH, or base excess, and theabsolutevalueofD(t)doesnotappeartopredictmortality(seeFigure8of[196]). Furthermore, as the authors themselves point out, there is often an apparent mismatch between the degree of clinical dysfunction and histological organ pathology. Indirect response models In two recent works, Foteinou and colleagues [79, 78] developed indirect response models for sepsis dynamics. The basic model [79] considers a highly simplified transduction cascade from the LPS 177 receptor (TLR4) to activate anti-inflammatory, pro-inflammatory, and energetic transcriptional responses. The model produces the qualitative range of behaviors observed in experimental en- dotoxemia, and also admits a persistent sterile inflammatory state in response to large levels of LPS. However, the latter behavior can be attributed to the inclusion of a positive feedback term in one of the governing equations explicitly designed to give such bi-stability; the term has no other biological meaning. The second work [78] considers a more complex model of LPS and glucocorticoid signalling to NF-κB, and the modulating effect of steroid administration on the system response to LPS was explored. Agent-based models The inflammatory response has also been investigated by An in a series of spatially explicit agent- basedmodels[2,3,4]. Thesemodelsconsideralargenumberofcelltypes,receptors,andcytokines, and agent behavior is governed through relatively simple arithmetic rules. Multiple anti-cytokine treatment strategies have been tested using this model framework [3]. Signal transduction models Many mathematical models have been developed to describe individual inflammatory signal trans- duction pathways, for example TNF signaling to NF-κB [106, 151]. In general, however, these models have not been incorporated into a description of a larger immune response, and they typ- ically examine dynamics over a relatively short time-window. Nevertheless, this literature may be quite valuable in constructing an agent-based model for sepsis where agent behavior is governed at the level of such signaling cascades. 178 7.3.2 A note on parametrization For very simple (differential equations-based) models, asymptotic behavior can be characterized in terms of parameters, see e.g. [7], and phase-plane analysis and bifurcation diagrams are also very usefultoolsforunderstandingmodelsofrelativelylowdimension. However,eveninrelativelysimple models there can be many free parameters, and wholly characterizing possible model behavior over all parameter space is a daunting task. Moreover, certain mathematically possible behaviors may not be relevant if they occur under biologically implausible parameter regimes. Thus, it is vitally importanttoconstrain,atleastwithinareasonableorderofmagnitudeestimate,allowedparameter values. Thisalsohasanimportantimplicationwithrespecttomodeldevelopment: functionalforms mustbedirectlyinterpretableintermsofbiology,andtheparametersneedtohaveastraightforward biological meaning. 7.4 A hierarchy of parametric models 7.4.1 First echelon: bacteria-phagocyte dynamics in vitro The simplest predator-prey context that has been considered with respect to phagocytes and bac- teria is that of a fixed phagocyte population preying upon bacteria in an in vitro context. We have identified several waves of interest in this scenario in the literature, with the first efforts at quan- tifying phagocytosis occurring three to four decades ago. Leijh et al. [145] studied the kinetics of granulocyte phagocytosis of S. aureus and Escherichia coli, finding a maximal rate of phagocytosis on the order of one bacterium per granulocyte per minute, and observed that each granulocyte was capable of phagocytosing 40–50 bacteria. Figure 3 of [145] is broadly compatible with Michaelis- Menten killing kinetics with K m ≈ 5×10 7 bacteria and k cat = 1 bacteria granulocyte −1 min −1 . 179 Thisgroupalsostudiedphagocytosisbymonocytes[144]andintracellularkilling(ofalreadyphago- cytosed bacteria) by both cell types [143, 144]. We note that each CFU corresponds to about 1.47 cells for S. aureus and 1.54 cells for E. coli [144], and therefore we use a conversion factor of 1.5 bacteria per CFU, although with respect to order of magnitude parameter estimates, CFU and cell number units are essentially interchangeable. Li et al. [149] more recently suggested that infection control is a function only of neutrophil concentration, and the so-called critical neutrophil concentration (CNC) was calculated to be on the order of 3− 4× 10 5 PMNs ml −1 , for cells in suspension. Observing that 10 5 PMNs ml −1 failed to reduce the population of only 10 3 cfus ml −1 , a neutrophil:bacteria ratio of 100:1, Li et al. concluded that this ratio is not the principal determinant of infection control, and their subsequent mathematical analysis suggests dynamics are completely independent of the ratio. Li et al. modeled bacterial growth as an exponential process, while neutrophil-mediated killing was taken to be a second-order collision, i.e. mass-action, process (without any handling time), giving dB dt =rB−γNB =(r−γN)B, (7.6) where B and N are the bacterial and neutrophil (PMN) concentrations, respectively. The natural bacterialgrowthrateisr, andγ isthemass-actionconstantforkilling. Itisclearfromthesolution, B(t)=B(0)e (r− N)t , (7.7) that unbounded growth occurs when N < r γ , (7.8) 180 giving the CNC as a function of r and γ. Now, estimated parameter values of γ = 1.02×10 −6 ml PMN −1 hr −1 and r = .444 hr −1 , suggest a CNC of 4.3×10 5 PMNs ml −1 , about an order of magnitude below the normal absolute neutrophil count in peripheral blood, and a value considered to be severe neutropenia [97]. Thus, it was concluded that clinical neutropenia renders patients vulnerable to infection due to a drop below the CNC in peripheral blood [149]. This interpretation, in our view, misunderstands the role of peripheral neutrophils in bacteremia and infection more generally, as the blood serves only as a transportation corridor for neutrophils on their way to inflamed sites [97], and neutropenia can be a normal finding in many ethnic groups [97]. AsecondworkbyLiandcolleagues[150]appliedtheCNCconcepttoinfectionintissue,amuch more reasonable setting. However, the basic prediction that control of an exponentially growing bacterial population is completely independent of the initial inoculum seems biologically unlikely. As the following analysis shows, this prediction is partly an artifact of the mass-action assumption. Let us replace the mass-action killing term with a Michaelis-Menten (or Holling type II) term, dB dt =rB−γN B θ+B . (7.9) Now, for b nonnegative, we have the nullcline (curve for dB/dt=0) of        B = N r −θ, N ≥ r B =0, N < r (7.10) The nullcline divides regimes of unbounded bacterial growth and clearance, as illustrated in Fig- ure 7.3. Under this model formulation, there does still exist a CNC below which an arbitrarily small bacteria inoculum escapes control. Above this critical concentration, however, large bacterial 181 Figure 7.3: The y-axes give the initial bacteria concentration, B 0 (cells ml −1 ), and the x-axes give the fixed neutrophil population, N (cells ml −1 ); both axes are on a logarithmic scale. To the left/above the nullcline, there is unbounded growth (red region), while in the blue region the bacteria is eliminated. The left panel demonstrates that, under a mass-action formulation for bacterial killing, bacterial escape is strictly a function of N, with clearance above a CNC. The right panel shows the richer dynamics that result from saturable killing. A CNC does exists, but above it escape/clearance is mutually dependent upon B(0) and N. The bacteria:neutrophil ratio is given at several points along the nullcline, and approaches 125:1 for large N, demonstrating quasi-ratio-dependence for clearance. populations can still escape the immune system, and control is a function of both neutrophil and bacteria concentration. Interestingly, for large neutrophil populations, we approach a “quasi-ratio- dependence,” i.e. bacterial escape occurs above a threshold B 0 :N ratio, while clearance occurs below it. The dynamical differences resulting from mass-action versus saturable killing terms are summarized in Figure 7.4. Exponential growth of bacteria is, of course, unrealistic, and in vitro bacterial growth may be reasonably approximated by the logistic equation: dB dt =rB ( 1− B Λ ) −γN B θ+B . (7.11) Suchsaturablebacterialgrowthleadstobistabledynamics. Thatis,failureofimmunecontrolleads 182 Figure 7.4: The dynamics of in vitro killing of an exponentially growing bacteria inhibited by a fixed phagocyte, e.g. neutrophil, population vary markedly with the formal choice for phagocyte- dependent bacteria killing. 183 to the bacterial population approaching its carrying capacity, and we refer to this and unbounded growth, depending on model formulation, as “immune escape.” The switch from exponential to lo- gistic growth does not otherwise affect the qualitative dynamics. Bi-stability in neutrophil-bacteria dynamics has been explored extensively by Malka and colleagues [159, 160], who employed an axiomatic formulation for in vitro bacteria-phagocyte dynamics of the form: dB dt = ρB 1+βB +λ−δB−γ NB 1+αB+ηN . (7.12) ThismodelvariesfromEquation7.30intheformfornaturalbacterialgrowth,ithasaninfluxterm, λ, and it also has killing saturate with neutrophil as well as bacteria concentration. However, the dynamics of Malka et al.’s model do not differ qualitatively from the simpler one given in Equation 7.30, indicating that the departure from mass-action to a Michaelis-Menten style formulation for killing is the key formal difference between this family of models and that of Li et al. [149]. Finally, the existence of a threshold phagocyte concentration for passive control of arbitrarily small bacterial inocula suggests that immune reactivity is vitally important. In un-inflamed tissue, neutrophil populations are very low, although macrophages are a significant component of many tissues, especially the liver. 7.4.2 Second echelon: basic models for non-specific immunity Now, let us consider the non-specific response to an infection in some tissue. Macrophages are activated by contact with bacteria or bacterial products, release inflammatory factors that activate the endothelium, recruiting neutrophils and monocytes to tissue. Already we are faced with sev- eral modeling choices. First, we must account for how contact with bacteria increases phagocyte 184 biomass. We must decide whether to distinguish between resident macrophage and infiltrating neutrophil populations, and whether to explicitly include resting macrophages along with acti- vated macrophages. These basic choices are in addition to the particular functional forms used to formalize them. Lotka-Volterra equations Onecommonapproachtothisproblemhasbeentoviewbacteria-immuneinteractionasapredator- prey process, and the classical predator-prey model is that due to Lotka [154] and Volterra [250], who independently derived the basic model form in 1920 and 1928 (reprinted in translation, 1931 [250]),respectively. Briefly,ifB(t)(forbacteria)istheprey,andM(t)(formacrophage)ispredator, the Lotka-Volterra model is given as dB dt = αB−γMB, (7.13) dM dt = βMB−δM. (7.14) Prey and predator interact according to the law of mass-action from chemical kinetics. This inter- action results in loss of prey and is converted into predator biomass with efficiency β/γ ≤ 1. In the absence of predation, the prey proliferate exponentially, while the predator population declines exponentially. It can be shown that all positive initial conditions yield limit cycle behavior, i.e. sustained oscillations, except for a trivial unstable steady-state [14]. Bell, in 1973 [14], was the first author to explicitly explore a Lotka-Volterra style system in the context of immune-pathogen dynamics, and in this he has many successors [75]. 185 Generalized Lotka-Volterra system and functional responses Consider the generalized Lotka-Volterra system dB dt = αB−Mf(B), (7.15) dM dt = βMf(B)−δM. (7.16) The nature of the functional response,f(B), determines the nature of the dynamics, and this basic model form has received considerable interest in the context of immune-parasite interactions, see, for example, Fenton and Perkins [75] and references therein. Three basic functional responses, Holling’s Types I, II, and III, based on the seminal work of Holling [107], are widely used. The latter two are often cast in the form of either Holling’s disc equation [108] or Michaelis-Menten enzyme kinetics. We review the derivations and physical interpretations of the three functional forms, and examine their appropriateness to bacteria-phagocyte dynamics. Holling type I response. A Type I response simply states that prey attrition is directly proportional to prey, f(B)=γB, (7.17) and thus we have mass-action kinetics, and Equations 7.15 and 7.16 reduce to the classical Lotka- Volterra system. It should be noted that, while Equation 7.17 is typically used in newer sources, Holling’soriginalformulation[107]statesthatutilizationofpreyincreaseslinearlyuntilasaturation level, which can be represented mathematically, for example, as f(B)=max{γB,f max }. (7.18) 186 Holling Type II response. A Type II response is one where predation saturates with prey density, due to a handling time for predators to consume and/or digest their prey. The classical ecological formulation is given by Holling’s disc equation [108]: f(B)= aB 1+aT h B , (7.19) where a (cells −1 ml hr −1 ) is the attack rate or searching efficiency, and T h (hr) is time is takes for a predator to handle a single prey item (“handling time”); the derivation can be found in [108] or [215]. The disc equation is closely related to the famous Michaelis-Menten equation for enzyme kinetics, first derived in 1913 by its namesakes [180] (see also the English translation by Johnson and Goody [119]). The equation is derived as follows: consider an enzyme E that catalyzes the conversion of a single substrate S to a product, P. Essentially, it is assumed that the substrate and enzyme combine to form an enzyme-substrate (ES) complex; this complex then dissociates to yield the product and free enzyme. Schematically, S+E k 1 −−−→ ←−−−− k 1 C k 2 −−−→ P +E (7.20) The Michaelis-Menten equation describes the rate, V, of this reaction as V = d[P] dt =− d[S] dt = V max [S] K m +[S] , (7.21) where the brackets ([]) indicate concentration. The Michaelis constant, K m , can be related to the elementary rate constants via a derivation based on the quasi-steady-state-assumption (QSSA), 187 first proposed by Briggs and Haldane in 1925 [24], which gives K m = k −1 +k 2 k 1 , (7.22) and we have that V max =k 2 [E t ], (7.23) where [E t ] is the total enzyme concentration in the system, i.e. the sum of bound and free enzyme. Now, let us assume that the phagocyte acts as the enzyme, converting the bacterial substrate to internalized (and un-modeled) debris. In that case, the predation functional is given as: f(B)=k 2 B K m +B . (7.24) The disc equation can be manipulated into this form: f(B)=T −1 h B (aT h ) −1 +B , (7.25) and thus the disc equation has an equivalent enzyme kinetic interpretation such that T −1 h = k 2 , a = k 1 , and k −1 = 0. This makes perfect physical sense, as the inverse of k 2 is the expected life-time of the enzyme-substrate (or predator-prey) complex, when k −1 = 0. We also see that Holling’s search efficiency constant a is equivalent to the second-order collision rate-constant, k 1 . The ecological basis of the disc equation is consistent with the idea that k −1 = 0, i.e. it assumes that once predator attacks prey, there is no escape. Under either derivation, we have that for large prey populations, predation is limited by the handling time (T h or k −1 2 ), while the half-maximal prey concentration is increased in proportion to handling time, and decreased in proportion to 188 search efficiency. Thus, we have multiple physical derivations for predator-prey interaction leading to a Type II response where handling time is limiting. We argue that this is appropriate for phagocyte- mediated killing, as multiple lines of experimental evidence indicate an upper limit of about one bacterium phagocytosed per neutrophil per minute [144], and simple intuition dictates that the time for phagocyte-bacteria interaction must be non-zero. Furthermore, we have determined from agent-based simulations (Section 7.6.2) that handling time is likely to be limiting for reasonable values of bacterial motility. We believe this is a highly non-trivial point, as most existing works use the Holling Type I/mass-action formulation, and as we have shown in Section 7.4.1, Type I and Type II responses give very different system dynamics even in the very simplified setting of in vitro phagocytosis. Holling Type III response. The Type III response is similar to Type II, but is characterized by a sigmoidal, rather than hyperbolic, relationship between predation and prey density. In the ecological interpretation, this represents a foraging strategy where predation is weak when prey are scare,butstrongwhenpreyabundant. Thisisduetolearningresponsesonthepartofthepredator, who will not invest significant energy in hunting rare game, but will focus on more abundant prey. This learning-based ecological process has as its chemical kinetics analogue the process of coop- erative ligand binding; this analogy was formally studied by Real [215]. The canonical example of such cooperativity is the binding of oxygen (O 2 ) to hemoglobin (Hb), first studied by Hill in 1910 [104], who gave us yet another eponymous equation. Each hemoglobin molecule exists as an aggregate of four oxygen-binding subunits that act cooperatively. That is, oxygen binding to a single subunit alters the overall molecule conformation such that the other free subunits bind oxygen much more readily. If we make the simplifying 189 assumption that each molecule is either free of oxygen (Hb) or saturated (Hb(O 2 ) 4 ), we have the basic chemical process Hb+4O 2 ka −−−→ ←−−− k d Hb(O 2 ) 4 (7.26) Taking total hemoglobin to be constant, we have the fraction of ligand-bound hemoglobin, F, as F = [O 2 ] n [O 2 ] n +K d = [O 2 ] n [O 2 ] n +(K 1=2 ) n (7.27) where [O 2 ] is the free oxygen concentratiton, K d = k d /k a is the dissociation constant for the reaction,andK 1=2 isthefreeoxygenconcentrationwherehalfofthehemoglobinisbound. Equation 7.27, with n = 4, was originally proposed by Hill in 1910 [104] as an empirical description of oxygen-dissociation curves. Mechanistically, this implies that binding cooperation among subunits is perfect, meaning all subunits are simultaneously bound or unbound. In practice, cooperation is not perfect, and K d and n are determined empirically from experimental oxygen-dissociation curves, and the Hill coefficient, n≤4, approximates the degree of binding cooperativity. By direct analogy, n can be taken to represent the number of prey encounters required for a predator to learn to consume the prey with maximal efficiency [215]. Note that this interpretation requires the predator to be “forgetful,” as only the encounter rate with the prey population at the current time counts. The Type III response can be formalized in quasi-disc equation form [75], f(B)= aB n 1+aT h B n , (7.28) 190 Figure 7.5: The three Holling functional responses. The left panel gives a Type I response, ei- ther without saturation, as is typically used in modeling studies, or with saturation, as originally proposed by Holling [107]. The right shows Type II and III responses, with the inset giving the response over a wider range of prey densities. or in a quasi-Michaelis-Menten form, f(B)=k 2 B n K m +B n . (7.29) The three Holling functional responses are illustrated in Figure 7.5. Holling Type III response is unlikely to apply to phagocyte-bacteria dynamics. In Section 7.4.1, we demonstrate that Holling Type II functional response for in vitro killing gives richer dynamics than a Type I response. Suppose now that the bacteriocidal term is Holling Type III (with n>1): dB dt =rB−γN B n θ n +B n . (7.30) This model admits a non-zero steady state, which must be, for arbitrary n, determined itera- 191 tively,althoughforn=2wehavecanobtainaclosed-formexpression. Computationalinvestigation shows that, for sufficiently large N, and anyn marginally greater than 1, i.e. n'1.2, all bacterial initial conditions below a threshold converge to the same non-zero steady state. That is, we see control but not clearance. As with the Type II response, we still observe a CNC, and mutual de- pendence on B(0) and N beyond the CNC. However, the prediction that all bacterial populations that do not diverge converge to the same steady-state is quite incompatible with observed time-kill curves, e.g. [145, 149]. Therefore, we hereafter consider a Type II/Michaelis-Menten killing term to be decidedly optimal. Antia and Koella model Antia and Koella [7] proposed a model where activated macrophages, M(t), are recruited from a fixedpoolofsizeJ atarateproportionaltopathogenload,andthebacteria,B(t),andmacrophage interact according to mass-action kinetics, giving the model: dB dt = rB−γBM, (7.31) dM dt = λ−µM +σB(J−M). (7.32) Note that macrophage activation is a mass-action process, as (J −M) = M R , where M R is the population of resting macrophages. This system displays no sensitivity to initial conditions, and infection is eliminated if r<R 1 = γλ µ , (7.33) 192 and infection is controlled, i.e. a steady state of chronic infection is reached, if r<R 2 =γJ. (7.34) Otherwise, unbounded bacterial growth occurs. In the absence of infection, we have M ≡ λ/µ, which we refer to as the “sentinel” population; the sentinel population is analogous to the CNC for non-reactive phagocytes, discussed in Section 7.4.1. As in Section 7.4.1, a mass-action model formulation implies clearance is independent of initial bacterial load, although the new dynamic of controlisnowadmittedifthemacrophagereserve,J,isgreatenough. Thedynamicsofrecruitment represented by σ play no role in the asymptotic behavior of the system. Baseline predator-prey model Forourbaselinepredator-prey-stylemodel,wealtertheAntiaandKoellamodeltoemployaHolling type II functional response for bacteria-macrophage interaction: dB dt = rB−γM B θ+B , (7.35) dM dt = λ−µM +σ(J−M) B θ A +B . (7.36) Note that the physiological meanings (and units) of the γ andσ parameters have changed relative to Equations 7.31 and 7.32; the model is illustrated schematically in Figure 7.6. The effect of substituting the Type II functional response in this model of reactive immunity response mirrors its effect when the immune response is non-reactive, by dividing dynamics into two basic regimes. For a sufficiently large baseline level of activated macrophages, λ/µ, the bacteria-free steady-state becomes (locally) stable. However, for smaller λ/µ, the chronically infected steady-state is locally 193 Figure 7.6: Cartoon for the baseline predator-prey model, where resting macrophages are activated via contact with bacteria and the activated macrophages prey upon bacteria. Both interactions occur via a Type II Holling functional response. stable. In either case, a sufficiently large bacterial initial conditions gives unbounded bacterial growth. These two dynamical pictures are shown in the phase plane in Figure 7.7. Further mir- roring our findings for a fixed phagocyte concentration, substituting logistic bacterial growth for exponential has little essential effect on dynamics, other than to introduce bistability in place of divergence for large B(0). In the above analysis, we assume the θ for activation and bacterial killing to be the same, followingaclassicalpredator-preytypeinteraction. However, macrophagesareactivatedbysoluble bacteria products, such as LPS: no direct contact is necessary. Thus, the effective θ for activation may be much different than that for killing. Indeed, from Zhang et al. [268], we have 7 µg ml −1 of LPS per 6×10 8 cfu ml −1 for plateau growth-phase E. coli. Assuming a molecular weight of 10 kDa for LPS, this gives the conversion factor of 1.17×10 −18 moles LPS cfu −1 . The dissociation constantforLPS-(TLR-4-MD-2)complexeswasestimatedat3nM[1]. Ifweassumethatactivation is directly proportional to the fraction of TLR-4 receptors that are ligand bound, this gives an equivalent θ for activation of θ A =2.56×10 6 cfu ml −1 ≈3.84×10 6 cells ml −1 , about an order of magnitude lower than that estimated previously for killing. Using this lower value for activation 194 Figure 7.7: The two classes of dynamics for the baseline predator-prey model with type II Holling functional response demonstrated in the log-log phase plane. The left panel shows bi-stability between bacterial control and escape (overwhelming infection), which occurs when the sentinel macrophagepopulationsmall. Therightdemonstratesthebi-stabilitybetweenclearanceandescape that occurs for a sufficiently large sentinel macrophage population. does not change the essential dynamics of the system, but it lowers the bacterial population in the chronic steady state by over an order of magnitude. Finally, if the site of infection is the lung, we let θ lung =θ 2=3 and θ lung A =θ 2=3 A , as the alveolar macrophages and invading bacteria act in a quasi-two-dimensional space. Therefore, we expect bacteria-macrophage contact to be greater, due to the reduced search dimensionality. Basal macrophage dynamics The constant flux of resting to activated macrophages, λ, is problematic, since for sufficiently large λ, the number of resting macrophages (implicitly) becomes negative. Moreover, incorporating this into a more complex model where resting macrophages act independently could introduce problems. Therefore, a better formulation, in our view, is per-capita transitions between these two 195 compartments: dM dt =ρ(J−M)−µM, (7.37) where we have omitted the bacteria-dependent activation term. This change does not affect any of the basic dynamics discussed so far, but it makes the model framework more flexible. Cytokine production: TNF (pro-inflammatory) Modeldevelopment. Wenowconsidertheinfluenceofcytokineproduction. Thepreviousmodels have macrophage activation as a direct function of the contact between pathogens and resting macrophages. Suppose now that macrophage-bacteria contact, via PAMP signalling to TRL-4 and then NF-κB (see Section 7.2.4), induces macrophage activation and cytokine expression. First, we consider TNF alone, represented asT(t), and we assume that it is produced at a constant rate,α T , by activated macrophages, and we assume it degrades via first-order kinetics. If we assume simple receptor-ligand binding kinetics, we have C =C T T K T +T (7.38) where C represents the concentration of TNF-TNFR1 complexes, C T is the total concentration of TNFR1receptors,andK T isthedissociationconstant. TNFisatrimericmolecule,andhistorically, ligand-mediatedtrimerizationwasregardedasthekeyeventinreceptoractivation,butmorerecent evidence indicates that TNFRs exist as pre-assembled oligomers [37, 255]. Therefore, we need not explicitly model receptor trimerization, as done by Werner et al. [259]. If we assume macrophage activation is directly proportional to the fraction of ligand-bound TNFR1 receptors, we arrive at 196 the model: dB dt = rB ( 1− B Λ ) −γM B θ+B , (7.39) dM dt = ρ(J−M)−µM +σ 1 (J−M) B θ A +B +σ 2 (J−M) T K T +T , (7.40) dT dt = α T M −δT. (7.41) To ease bifurcation analysis (and for biological realism), we now consider natural bacterial growth to be logistic rather than exponential. Now,Tsujimotoandcolleagues[240]foundthehalf-maximalcellularresponsetoTNF(inTNF- sensitive cells) to occur with less than 1% of TNF receptors bound. Let us suppose the activation responseisproportionaltothefractionofreceptorsbound,withf d ≈.01thefractioncorresponding to the half-maximal response. We use a widely used sigmoid functional form for activation: f n f n d +f n , (7.42) where f = T K T +T . (7.43) Thisfunctionalformispurelyphenomenological,althoughinthecontextofreceptor-ligandbinding dynamics, n approximates the degree of cooperativity in ligand-binding, as discussed in Section 7.4.2. As signal transduction cascades are characterized by amplification of small signals and can lead to all or nothing responses in individual cells, n > 1 may be justified. For n = 1, Equation 7.42 simplifies to 1 1+f d T f d 1+f d K T +T . (7.44) 197 Figure 7.8: Schematic model with predator-prey type dynamics and TNF-mediated positive feed- back. TNF-mediated positive feedback on macrophage activation is modeled through basic ligand- receptor binding dynamics. This is equivalent to a Michaelis-Menten term, but with the maximum response reduced by the factor 1/(1+f d ) and an effective Michaelis constant of K d f d /(1+f d ). Forf d small, the activation function approximates the standard Michaelis-Menten or Holling type II function, but with f d K d as the effective Michaelis constant. Thus, the principal effect is to reduce the effective K d . For clarity, the final model considered in this section follows: dB dt = rB ( 1− B Λ ) −γM B θ+B , (7.45) dM dt = ρ(J−M)−µM +σ 1 (J−M) B θ A +B +σ 2 (J−M) f n f n d +f n , (7.46) dT dt = α T M −δ T T, (7.47) f = T K T +T . (7.48) If we assume that TNF dynamics are fast compared to the cellular dynamics, we can take it to be in quasi-steady-state and fix T =M α T δ T , (7.49) reducing our system to two dimensions. 198 Modelbehaviorandresults. Thissystemhasthreepossibleasymptoticbehaviors: bacterial clearance with chronic inflammation, chronic sub-maximal infection (“control” or “sub-saturated infection”), and chronic infection that approaches the bacterial carrying capacity (“escape” or “saturated infection”). Unlike the previous models considered, even if ρ = 0 (or equivalently, λ=0), the immune response is capable of clearing infection, due to the inclusion of TNF-mediated positive feedback in macrophage activation. Generally,bacterialescapeisalwayslocallystable,andoneofthetwounsaturatedsteady-states is locally stable: both lead to chronic inflammation, but in one bacteria are cleared. Interestingly, while altering most model parameters can induce a switch in stability from the sub-saturated chronic steady-state to the bacteria-free state, σ 1 and θ A , which govern macrophage activation in response to the pathogen per se have no influence on these states. A sufficiently weak immune response or virulent pathogen also leads to overwhelming infection regardless of initial conditions, i.e. bacterial escape becomes globally stable. Figure 7.10 gives phase portraits, and Figure 7.9 shows bifurcation diagrams. While usually only a single unsaturated state is stable, when n> 1, we observe that sustained oscillationsandtri-stabilityexistinsmallregionsofparameterspace. Inthetri-stableregion, small initial bacterial populations fail to induce strong positive feedback via TNF, but are sufficiently controlled by “directly activated” macrophages for either sustained oscillations or chronic sub- saturated infection to develop. Large initial inocula are sufficiently activating for the TNF-based positive feedback loop to become dominant, and bacteria are cleared with sustained inflammation. This is illustrated by phase diagrams and bifurcation plots in Figure 7.11. Finally, we suggest comparing this model to a mass-action style formulation of the process. In such a setting we have multiple mass-action kinetic parameters with no straightforward interpre- 199 Figure 7.9: Bifurcation diagrams (on a logarithmic scale) for the TNF-only model, for bifurcation parameters σ 1 , θ A , σ 2 , θ, γ, and r. The dotted lines show the saturated chronic state, which is always locally stable, and the solid lines show the other locally stable state. The two sets of lines converge when the sub-saturated chronic state loses stability. Baseline parameters (suppressing units) areσ 1 =1,θ A =3.84×10 6 ,σ 2 =.22,θ =5×10 7 ,γ =10,µ=1,ρ=10 −6 ,K d =5×10 −10 , f d =.5, α T =2.93× −20 , δ T =2.29, r =.48, n=1, and J =10 8 . Figure 7.10: Phase plots for the TNF-only model. The left panel demonstrates the first class of dynamics,seenforσ 2 small,whereinthereisbistabilitybetweenbacterialcontrolandescape. When σ 2 is sufficiently large, we have bi-stability between bacterial escape and sterile inflammation, as seen in the right panel. Parameter values other than σ 2 are as in Figure 7.9. 200 tation, and we have no way to estimate them from experimental data other than through naive curve-fitting. Parametrization. Watanabe et al. [257] determined K d = 5.15×10 −10 M for TNF binding to cell surface receptors in a human myosarcoma cell line, and estimated a total of 1.5×10 4 surface receptors per cell. Similar values were obtained by Tsujimoto et al. [240], who determined K d s of 6.1×10 −10 and 3.2×10 −10 M, and 2200 and 7500 binding sites per cell, for murine L929 and human FS-4 cells, respectively. Toobtainanestimateforα T ,weusedatafromMunozetal. [186],whostudiedin vitro cytokine expression by human monocytes from septic patients. Briefly, 10 6 cells ml −1 were cultured for 24 hours with 2 µg ml −1 of Neisseria meningitidis LPS, a robust inducer of monocyte activation, yielding TNF concentrations of 1.76 ng ml −1 and 4.85 ng ml −1 and for septic patients and controls, respectively. Assuming TNF was produced at a constant rate and that none degraded over the course of the assay, we have, for controls, α T = 2.021×10 −7 ng cell −1 hr −1 = 3.96×10 −21 moles cell −1 hr −1 . If, on the other hand, we assume that TNF decays with first-order kinetics and use δ T = 2.29 hr −1 , based on an in vitro study using whole blood [199], we getα T = 1.11×10 −5 ng cell −1 hr −1 = 2.17×10 −19 moles cell −1 hr −1 . Given the uncertainty in decay, we use the geometric mean of these two values, 2.93×10 −20 moles cell −1 hr −1 , as our baseline α T . Using Figure 1 of [158], and assumingδ T = 2.29 hr −1 , we obtain a similar estimate ofα T =7.93×10 −20 moles cell −1 hr −1 . EstimatesforTNFhalf-lifeinhumansrangefrom10to80minutes,withmostclusteringaround 15–30 minutes [20]; we use 18 minutes [199] as our default value, giving δ T = 2.29 hr −1 . 201 Figure 7.11: The left gives phase portraits for the TNF-only model with n = 2.2 and n = 3; in the former, some initial conditions converge to a limit cycle, while others lead either to bacterial clearance (with chronic inflammation) or saturated bacterial growth. With the latter, the limit cycle is replaced by a stable fixed point. The right half of the figure gives bifurcation diagrams with n as the parameter. Baseline parameter values are as in Figure 7.9, except f d = 0.1 and γ =20. 202 Cytokine production: IL-10 (anti-inflammatory) IL-10 is produced by activated macrophages and strongly suppresses expression of a wide array of pro-inflammatory cytokines, including TNF. It was our initial hypothesis that the addition of IL-10 production by activated macrophages to the model would counterbalance the TNF-mediated positive feedback and allow resolution of inflammation following bacterial clearance, as suggested by other models, e.g. [218]. Experimental studies have also indicated a central role for IL-10 in shutting off pro-inflammatory cytokine production following endotoxin challenge [15]. While our model suggests that, in the case of challenge with a non-proliferating pathogen analogous to an LPS or dead bacteria infusion, IL-10 plays an important role in limiting TNF expression, IL-10 cannot lead to inflammatory resolution following bacterial clearance. Over-expression of IL-10 can, however, cause a switch in stability from chronic sterile inflammation to chronic infection. Modeldevelopment. IL-10broadlysuppressescytokineproductionbyactivatedmacrophages: inhibited cytokines include TNF, IL-1, IL-6, IL-12, G-CSF, GM-CSF, iNOS, and IL-10 itself [60, 158]. It is well-established that IL-10 plays a non-redundant role in limiting both chronic and acute inflammation [188]. IL-10 expression in macrophages is induced by a variety of stimuli, e.g. LPS,andIL-10productionisdelayedrelativetoclassicalpro-inflammatoryfactorssuchasTNF and IL-1 [60]; therefore, IL-10 is generally viewed as a negative feedback on cytokine production [60]. Molecularly, IL-10:IL-10R binding signals through the JAK-STAT (Janus-associated kinase- signal transducer and activator of transcription) pathway; the transcription factor STAT3 induces SOCS-3 (suppressor of cytokine signalling-3) synthesis, which in turn inhibits endotoxin-inducible cytokine expression [60]. A central question, especially within our model framework, is does IL-10 simply suppress TNF production by activated macrophages, or does it affect the underlying macrophage activation state, 203 either through enhancing de-activation or preventing activation? We may be fairly confident that it does not, in fact, affect activation. The influence of IL-10 on NF-κB and the MAPKs JNK and p38, central executors of macrophage activation in response to TLR and TNFR signalling, is controversial but is very likely minimal [60, 188, 43]. IL-10 may play a small role in inhibiting NF-κB activity, but this activity appears to be independent of its ability to suppress TNF [43]. Furthermore, IL-10 does not inhibit TNF synthesis in JAK1-deficient mice [60] and STAT3 has been found essential to IL-10 activity [188]. IL-10 suppresses 20–25% of TLR-induced genes, implying that IL-10 does not affect the basic process of macrophage activation [188], and cytokine inhibition requires the continuing presence of IL-10 [60]. These results all strongly support a mathematical formulation where the influence of IL-10 is expressed solely through α T =α T (I) and α I =α I (I). We consider three hypotheses for the effect of IL-10 on the system: 1. IL-10 inhibits TNF production by activated macrophages in proportion to the fraction of bound receptors. 2. IL-10 similarly inhibits IL-10 production by activated macrophages. 3. IL-10 inhibits macrophage activation. Asdiscussed,thefinaloneisunlikelybiologically,butwestillwishtoexploreitspotentialeffectson system dynamics. We also assume an explicit delay in IL-10 production by activated macrophages. Incorporating all three hypotheses yields the following system, with the mathematical formalism 204 corresponding to each hypothesis indicated in the under-braces: dB(t) dt = rB ( 1− B Λ ) −γM B θ+B , (7.50) dM(t) dt = ρ(J−M)−µM + ( σ 1 (J−M) B θ A +B +σ 2 (J−M) f n f n d +f n ) (1−F I ϕ i ) | {z } H-3 ,(7.51) dT(t) dt = α T M(1−F I ϕ I ) | {z } H-1 −δ T T, (7.52) dI(t) dt = α I M(t−τ)e − (1−F I ϕ I ) | {z } H-2 −δ I I, (7.53) ϕ I = I K I +I (7.54) f = T K d +T . (7.55) The dependence on time for most variables is suppressed and preserved only for the delay term, M(t−τ), where τ is the time-delay to IL-10 production following activation; the system and hy- potheses are illustrated schematically in Figure7.12. Il-10 production, omitting autocrine feedback, is given by α I M(t−τ)exp(µ(t−τ)); the exp(µ(t−τ)) term is included to account for loss of activated macrophages in the τ hours between activation and IL-10 expression. We assume no delay between IL-10 binding and its effect on cytokine production (either TNF or IL-10). The effect of IL-10 is assumed to be proportional to the fraction of receptors bound, and F I gives the degree to which a given behavior is suppressed. We assume F I to be about 0.8−1.0. We setF I =1fortheremainderofthissection, toensurethatsettingF I <1doesnotundulyinfluence our conclusions, but in subsequent sections we consider a value of 0.8 or 0.9 to be more realistic: IL-10 reduced TNF expression by LPS-activated monocytes by 88% in [158]. Results. We have computationally explored all three hypotheses, alone and in combination (with a total of seven possible combinations, although Hypothesis 2 alone is a trivial case), and 205 Figure 7.12: Schematic illustration of the TNF+IL-10 system and the hypothesized roles of IL-10. find that under no combination do the basic dynamics of the TNF-only model change. That is, the inclusion of IL-10 can alter the basins of attraction for the fixed-points, but there remain three basic asymptotic behaviors: saturated infection, sub-saturated infection, and bacterial clearance with chronic inflammation. Altering τ to delay IL-10-mediated feedback until the infection is controlled does not lead inflammation to resolve, although large values of τ combined with large α I can lead to sustained oscillations in the activated macrophage population following bacterial clearance. These observations run contrary to our previous working hypothesis, which had been that, as we successively add TNF and IL-10 to the model framework, we should observe TNF to act as an essential positive feedback leading to bacterial clearance, while IL-10 would then in turn act as a negative feedback to give inflammatory resolution. The inclusion of IL-10 does reduce both the number of activated macrophages and the TNF concentration of the sterile inflammation steady-state, and sufficiently large IL-10 production can cause a switch in stability from sterile inflammation to sub-saturated infection. When the third hypothesis (IL-10 inhibition of macrophage activation) is included, very large α I values can cause 206 saturated infection to become globally stable. This is illustrated in Figure 7.13. We have also numerically simulated exogenous IL-10 infusion, finding that exogenous IL-10, unlike endogenous IL-10, can eliminate sterile inflammation following bacterial clearance. This is demonstratedinFigure7.14. In sum, IL-10 produces by activated macrophages clearly dampens the immune response, but cannot cause spontaneous resolution, as defined by a reduction in activated macrophages to their pre-infection level, within the model framework thus far. How can the observation that IL-10 has little apparent role in clearing inflammation be squared with experimental observations, such as those by Berg et al. [15], that IL-10 plays a central role in limiting TNF and other pro-inflammatory cytokines following experimental endotoxin infusion? As demonstrated in Figure 7.15, the inclusion of IL-10 clearly limits TNF following simulated LPS infusion when the macrophage reserve population, as in the liver, is large. However, the effect of IL-10 is almost nil when the resting macrophage population is smaller, as we might expect in the lung. These results are consistent with [15], and also suggest the hypothesis that IL-10 plays an important role in preventing the systemic spread of sterile inflammatory activity, but plays a limited role at the local tissue level. A similar hypothesis was suggested by Cox [46] on the basis of a rat model for LPS-induced lung inflammation. Parametrization. Tan et al. [233], working with human JY and mouse MC/9 cells, found IL-10 to bind a single class of receptor as a homodimer, with K d =50−200 pM, and an estimated 100–300 receptors per cells. Carson et al. [32] similarly estimated a K d of 1 nM and 90 receptors per cell for IL-10 binding in human natural killer (NK) cells. Malefyt et al. [158] measured in vitro cytokine production by human monocytes (4 × 10 6 cells ml −1 ); using a maximum IL-10 concentration of 55 ng ml −1 (Figure 1 of [158]) and assuming a half-life of two hours and molecular weight of 37 kDa for IL-10 homodimers, we estimate α I = 1.288×10 −19 moles cell −1 hr −1 , which 207 Figure7.13: Bifurcationdiagramswithα I asthebifurcationparameterundermodelsincorporating hypotheses 1 and 2 (left panel), or all three hypotheses (right panel). Figure7.14: NumericalexperimentsimulatingexogenousIL-10infusionat72hoursfollowinginitial infection; exogenous IL-10 successfully forces resolution of sterile inflammation. Note that it also suppressesendogenousIL-10andTNF.Forclarityofpresentation,thebacterialpopulationisforced to zero once it falls below .001 cells ml −1 . 208 Figure 7.15: The effect of blocking IL-10 under simulated LPS infusion on TNF levels is demon- strated for resting macrophage compartments of size 10 6 and 10 8 cells ml −1 , which are reasonable for the lung and liver, respectively. The upper panels show results for τ =.5 hrs, and the lower are for τ = 2 hrs, demonstrating the sensitivity of peak TNF to the delay in IL-10 production. LPS infusion is simulated by replacing the governing equation forB(t) with a first-order clearance term, −δ B B, with δ B =3 hr −1 . 209 is similar to our previous estimate of the TNF production rate, α T . Huhn et al. [113] studied IL-10 pharmacokinetics in healthy humans. Elimination from serum following bolus infusion appeared to follow two-compartment kinetics; the apparent elimination half-life increased with dose and ranged from 2.3±.5−3.7±.8 hr −1 . For simplicity, we neglect the two-compartment aspect and assume a single half-life towards the bottom of this range (2 hours) to compensate for omitting the distribution phase. Activated endothelium with neutrophil recruitment Thepreviousmodelsandanalysisconsideredrecruitmentfromapresumablylocalsupplyofresting macrophages. However, thesystemicneutrophilandmonocytepoolthatcanberecruitedtoalocal infection is essentially infinite for small infections. We first consider endothelial activation by TNF and subsequent recruitment of circulating neutrophils. Uponcytokinestimulation(TNF,IL-1),endothelialcellsundergoabroadprogramofactivation, largely mediated by NF-κB signaling, that leads to increased blood flow and vascular permeability, activation of the coagulation cascade, and adhesion factor expression that attracts effector immune cells to the site [245]. Early in inflammation, neutrophils are the predominant cell type attracted, and continuing cytokine exposure is necessary for sustained endothelial adhesion factor expression [126]. Neutrophils are major effector cells in acute inflammation, severe neutropenia is associated with profound vulnerability to bacterial infection, and patients with genetic defects in neutrophil function also have an increased propensity for bacterial and fungal infections [228]. Nawroth et al. [193] have characterized TNF receptor binding dynamics in endothelial cells, finding an K d = 105± 40 pM, with 1500± 500 binding sites per cell. The half-maximal TNF concentrations for endothelial IL-1 expression, 40–80 pM [193], and pro-coagulant activity, 30–40 210 pM [194], are on the same order of magnitude. Therefore, we assume endothelial activation to be a function of TNF receptor occupancy. We have the total endothelial concentration as E T , which we normalize to unity, andE A represents the activated endothelial population. Given the requirement of ongoing cytokine stimulation [126], we assume that in the absence of TNF, endothelial cells de-activate relatively rapidly with first-order kinetics. IL-10appearstobroadlyinhibitpro-inflammatoryactivityinactivatedendothelialcells. Knolle et al. [133] found IL-10 to downregulate IL-6 expression in LPS-activated endothelium by up to 75%, and IL-10 has been found to inhibit leukocyte adhesion to IL-1-activated endothelium via both decreased ICAM-1 and VCAM-1 endothelial expression and a direct effect on leukocytes [139]. Activated endothelium also produces inflammatory cytokines, including TNF, IL-1, and IL- 6,althoughunlikemacrophages,itdoesnotproduceIL-10followingactivation[133]. However,since macrophages and other effector cells are generally believed to be the primary sources of cytokines in inflamed tissue, we omit this behavior from our model. 211 We have the following basic system for endothelial activation and neutrophil recruitment: dB(t) dt = λ B (t)+rB ( 1− B Λ ) −γ(M +N) B θ+B , (7.56) dM(t) dt = ρ(J−M)−µM + ( σ 1 (J−M) B θ A +B +σ 2 (J−M) f n f n d +f n ) , (7.57) dT(t) dt = α T M(1−F I ϕ I )−δ T T, (7.58) dI(t) dt = α I M(t−τ)e − (1−F I ϕ I )−δ I I, (7.59) dE(t) dt = ν(E T −E) ( T K E +T ) −µ E E, (7.60) dN(t) dt = ω QN B V ( E ne θ ne E +E ne ) (1−ϕ I )−δ N N, (7.61) ϕ I = I K I +I , (7.62) f = T K d +T , (7.63) where Q (ml hr −1 ) is the blood flow to the organ, V is the organ volume, N B is the circulating neutrophil concentration, θ E and n e describe the rate of neutrophil infiltration into tissue as a function of E, and ω ≤ 1 gives the maximum proportion of neutrophils flowing through the organ that can extravasate into tissue. The λ B (t) term represents imposed bacterial infusion, and it is used to simulate re-infections (see below). As usual, a simple schematic illustration of the model is given in Figure 7.16. We impose B(t) =B(t)H(B(t)−.001), where H(·) is the Heaviside step function to avoid the “atto-fox” problem: we expect a population of less than a single bacterium per liter to die off via stochastic extinction, rather than form the seed for a recurring infection. This model allows mobilization of an immune response that both clears the infection and spontaneously resolves, and we have five possible asymptotic behaviors: (1) resolution of infection and inflammation (“reso- lution”), (2) chronic infection at bacterial carrying capacity (“saturated” or “overwhelming” in- 212 Figure 7.16: Schematic illustration of the baseline neutrophil model. fection), (3) bacterial clearance with chronic inflammation (“sterile inflammation”), (4) sustained oscillations with chronic infection and inflammation (“sub-saturated oscillations”), and (5) chronic sub-saturatedinfectionandinflammation. Themajornewdynamic, ofcourse, isthatofresolution. This model retains dependence on bacterial initial conditions, with small inocula typically lead- ing to clearance or oscillations, and sufficiently large inocula overwhelming the immune response. Figures 7.17 and 7.18 give example time-series for all variables under the scenarios of resolution and overwhelming infection, respectively. When the immune response is relatively weak, very small bacterial initial conditions, on the orderoflessthan100bacteriaml −1 ,canleadtosustainedoscillationsinthebacteriapopulationand inflammatory response, while larger initial conditions typically lead to either clearance or escape. Furthermore, we have observed that, when TNF-mediated positive feedback is sufficiently large, only one of two steady-states is stable: sterile inflammation or overwhelming infection. 213 Figure 7.17: Time-series of all model variables over 24 hours following infection with 10 6 bacteria ml −1 , which results in resolution. The top left gives B, M, and N on a log-scale, while the top right gives these variables on a linear scale. Figure 7.18: Time-series of all model variables over 24 hours following infection with 10 8 bacteria ml −1 , leading to overwhelming infection. 214 We have found that the inclusion of neutrophil recruitment lowers the local macrophage popu- lation, J, necessary for bacterial clearance. Under the TNF-only and TNF+IL-10 models, J must be on the order of 10 7 −10 8 cellsml −1 for bacterial clearance to occur under biologically reasonable parameter values, while with neutrophil recruitment, J is easily as low as 10 6 cells ml −1 . While the former values are reasonable for hepatic tissue, the latter is appropriate for lung. The size of the local macrophage reserve, J, and σ 2 and K T , which together determine the magnitudeoftheautocrinemacrophageresponsetoTNF,playacrucialroleindeterminingwhether inflammation is resolved following bacterial elimination. Under the TNF-only and TNF+IL-10 models, bacterial clearance requires the TNF-mediated positive activation feedback loop; when neutrophil recruitment is included, this loop becomes potentially pathological. Increases in J or σ 2 , or decreases in K T , improve macrophage mobilization and clearance, even with the added neutrophils, but such changes simultaneously increase the chance of sterile inflammation. Reinfection. We have performed a numerical experiment where an initial infection, fated to endinresolution,isfollowedbyre-infectionbyanequivalentlysizedinoculumatvaryingtimesafter theinitialinfection. AsdemonstratedinFigure7.19, thereisatransientlastingseveraldaysduring which re-infection is strongly suppressed. This hyperimmune state is accompanied by a relative blunting in the cytokine response, endothelial activation, and in macrophage activation. There is, however, little change in the peak neutrophil response, and there is minimal dependence on the inoculumsize. Theseresultsarequitecontrarytowhatweobservewhenactiveimmunosuppression induced by apoptotic cells is considered in Section 7.4.3. Conclusion. Our basic conclusion is this: recruitment of short-lived neutrophils from the systemic compartment by the local immune response leads to infection clearance and resolution of inflammation. Without neutrophil recruitment, positive feedback at the local level is necessary for 215 Figure 7.19: Under the baseline neutrophil model, re-infection following an initial bacterial insult is suppressed for several days, via non-specific activation of the immune system. The cytokine response to re-infection is blunted, but peak neutrophil recruitment is little changed. It can be inferred that strong cytokine and macrophage responses are unnecessary, as the system is already primed with activated enothelium and a neutrophilic infiltrate. robust bacterial clearance, but this dynamic also prevents resolution. While exogenously imposed IL-10 (or a similar anti-inflammatory) can eliminate sterile inflammation, it is not possible for IL-10 produced by local macrophages to do so. Therefore, neutrophils have a crucial, if passive, “anti-inflammatory” effect at the systems level. Neutrophil cytokine production and activation Thebaselineneutrophilmodelregardsneutrophilsasquiescentwithrespecttocytokineproduction, which was the traditional view. However, it has become clear in recent years that neutrophils can producemultipleinflammatorycytokinesinresponsetostimulation. Experimentalresultscompiled by Cassatella [35] suggest that per cell cytokine production by mononuclear cells is typically 10- to 100-fold greater than that of activated neutrophils, although neutrophilic expression is occasionally 216 comparable. Under the baseline neutrophil model, introducing TNF production by neutrophils, no matter how small, leads to a positive feedback loop between endothelial activation and neutrophil recruit- ment, resulting in chronic inflammation. However, if we distinguish between resting or primed neu- trophils and activated neutrophils that produce TNF, this pro-inflammatory trap can be avoided. To model this, we replace the singleN(t) variable with resting/primed neutrophils that enter from the circulation, N 0 (t), and activated neutrophils, N A (t), which produce TNF at per-capita rate α N . The governing equations for these two variables follow: dN 0 (t) dt = ω QN B V ( E ne θ ne E +E ne ) (1−ϕ I )−N 0 ( B θ A +B + f n f n d +f n ) +µ N N A −δ N N, (7.64) dN A (t) dt = N 0 ( B θ A +B + f n f n d +f n ) −µ N N A −δ N N, (7.65) and all other model equations are essentially as in Equations 7.56–7.60, but with appropriate modifications to account for the additional neutrophil population. Neutrophil activation is induced viacontactwithTNFandbacteria,similartotheactivationofmacrophagesbytheseproducts. For roughly α N < .1α T , a fraction consistent with the literature [35], sterile inflammation is avoided. Active immunosuppression, considered in subsequent sections, also prevents a neutrophil-mediated positive feedback trap, even if all neutrophils produce TNF. Parameter values Table 7.1 gives the range of parameter values for the models considered in this section; more detaileddiscussionoftheirderivationandestimationmaybefoundinthemodelcomponent-specific subsections. 217 Table 7.1: All baseline parameters and baseline value ranges for the predator-prey-based model hierarchy up to the baseline neutrophil model detailed in Section 7.4.2. Parameter Description Range (default) r Intrinsic bacterial growth rate 0−1.5 hr 1 (.48) Λ Bacteria carrying capacity 10 10 −10 11 cells ml 1 (10 10 ) γ Max phagocyte bacteria killing 0−60 hr 1 (10) θ Half-max bacteria conc. for killing 1.35×10 5 −5×10 7 cells ml 1 (5×10 7 ) λ Basal absolute macrophage act. rate 0−10 5 cells ml 1 hr 1 (0) ρ Basal per-capita macrophage act. rate 0−1 hr 1 (0) µ Macrophage de-activation rate 0.25−1 hr 1 (0.5) J Macrophage reserve 10 6 −10 8 cells ml 1 (10 6 lung, 10 8 liver) σ 1 Bacteria-induced macrophage act. rate 0.1−1 hr 1 (0.5) θ A Half-max bacteria conc. for mac. act. 2.45×10 4 −3.86×10 6 cells ml 1 (3.86×10 6 ) σ 2 TNF-induced macrophage act. rate 0.1−1 hr 1 (0.5) K T Half-max TNF for macrophage act. 3.2×10 10 −6.1×10 10 M (5×10 10 ) α T Act. macrophage TNF production rate 3.96×10 21 −2.17×10 19 moles cell 1 hr 1 (2.93×10 20 ) δ T First-order TNF decay constant 0.5−4 hr 1 (2.29) n Hill coefficient for TNF 1−3 (1) f d TNF half-max receptor occupancy .01−1 (.5) τ Delay to IL-10 production 1−8 hr (2.0) K I Half-max IL-10 for effect 5×10 11 −2×10 10 M (2×10 10 ) α I Act. macrophage IL-10 production rate 1.288×10 19 moles cell 1 hr 1 δ I First-order TNF decay constant 0.15−0.39 hr 1 (2.0) F I Max IL-10-mediation suppression 0.8−1.0 (0.8) Q Organ blood flow 3×10 5 ml hr 1 (lung), 6.8×10 4 (liver) N B Neutrophil blood concentration 1.5−9×10 6 cells ml 1 (5×10 6 ) ω Max neutrophil extravasation .01−.1 (.1) BW Body weight 70 kg V Organ volume 8×BW ml (lung), 26×BW (liver) δ N First-order neutrophil death rate .05 hr 1 θ E Half-max act. endothelium for PMN entry .5E T −.9E T (.9E T ) K E Half-max TNF for endothelial act. 0.75−1.45×10 10 M (10 10 M) n e Hill coefficient for neutrophil extravasation 1 ν Max endothelial act. rate 1 hr 1 µ E Endothelial deactivation rate 0.5−1 hr 1 (1) 218 7.4.3 Third echelon: Extended models of non-specific immunity In the previous section we established that, under a predator-prey style formulation of phagocyte- bacteria interaction with (1) activation of a resting macrophage compartment and functional re- sponses encoded as Holling’s Type II, and (2) cytokine-mediated positive and negative feedback upon macrophages encoded through basic receptor-ligand dynamics, the inclusion of a short-lived neutrophil population is necessary for robust bacterial clearance and resolution. In this section, we extend this foundation to take into account known active immunosuppressive effects of apoptotic neutrophils. We find that this leads to a dynamic of an initial hyper-inflammatory phase, followed by a more prolonged hypo-inflammatory phase, where the magnitude of immunsuppression in the latterphaseisproportionaltothemagnitudeoftheinitialbacterialinoculum. Thisextendedmodel forms the foundation for all our subsequent work. We also have explored two more model extensions: (1) inclusion of an explicit measure of tissue damageresultingprimarilyfromactivatedneutrophils,and(2)recruitmentofcirculatingmonocytes to the inflamed site. Neither extension, alone or in combination, significantlyaffects the qualitative model dynamics. Since they add significant complexity and a number of parameters difficult to estimatefromtheunderlyingbiology,wechoosetoomitthesefromfurtherwork. Thisalsoincreases our confidence in using the “neutrophil+apoptosis” model somewhat, as it demonstrates that the basicdynamics atthat modeling levelhavestabilizedin theface offurther modelingcomplications. Extension to neutrophil apoptosis Here we extend the model framework to explicitly account for neutrophil apoptosis. We include both the effects of TNF and IL-10 on the rate of apoptosis and the effect of apoptotic neutrophil phagocytosis by macrophages. The latter induces an anti-inflammatory macrophage phenotype, 219 which is maintained by autocrine production of TGF-β, an additional cytokine added to the model at this level. Cytokines and neutrophil apoptosis. Aberrent patterns of apoptosis have been implicated in the pathophysiology of sepsis, and a central role for neutrophil apoptosis in the resolution of inflammation has recently been recognized. Cytokines strongly affect neutrophil survival, and LPS stimulation and multiple pro-inflammatory cytokines, including IFN-γ, GM-CSF, and TNF, prolong the functional neutrophil lifespan [244, 127]. IFN-γ and GM-CSF are clearly neutrophil survival factors, and GM-CSF may be the principal factor related to prolonged neutrophil survival in sepsis [72], but the role of TNF has been somewhat controversial; the preponderance of the evidencesupportsaprotectiverole,yetsomeauthorshavereportedTNFtoinduceapoptosis. These conflicting observations have been elegantly synthesized by van den Berg and colleagues [244], who determined that while low-dose TNF is anti-apoptotic, high-doses indirectly promote apoptosis via induction of the respiratory burst and toxic oxygen metabolites; blocking the respiratory burst restores the protective role of high-dose TNF. IL-10 has been shown to counteract cytokine and LPS-induced prolongation of neutrophil sur- vival in a dose-dependent manner [127, 46]. In a rat model of LPS-induced lung inflammation [46], IL-10enhancedneutrophilclearancebypromotingapoptosis. ExogenousIL-10didnotsignificantly alter early neutrophil infiltration in the lung, either by prolonging time to peak neutrophilia, six hours, or by diminishing the peak neutrophil population. This was in spite of diminished TNF production by hour four, leading the author to suggest that the well-known cytokine-suppressing activity of IL-10 is of little importance in localized inflammation and serves primarily to limit inflammation in models of systemic infection [46]. Notably, IL-10 does not alter apoptosis in un- stimulated cells, and thus the pro-apoptotic effect is purely counter-regulatory [127, 46]. 220 Multiple studies have found spontaneous apoptosis in neutrophils from SIRS patients to be profoundly suppressed [117, 127, 72, 25], and the idea that neutrophil apoptosis is impaired in sepsis is broadly accepted [25, 112]. However, Martins et al. [173] observed slightly increased apoptosis in septic neutrophils and associated this with increased ROS production, corresponding to the findings that TNF-induced ROS can be pro-apoptotic [244] and that GM-CSF counters neutrophil apoptosis through inhibition of ROS production [72]. Keel et al. [127] found IL-10 to block the anti-apoptotic effect of LPS-stimulation in septic neutrophils, but IL-10 treatment alone did not restore normal apoptosis in such cells. This is somewhat at odds with the findings of Fanning et al. [72], who showed that SIRS plasma added to normal neutrophils inhibits apoptosis, but this is reversible by the addition of IL-10. Role of apoptotic cells in resolution. While apoptosis contributes passively to resolution by removing effector cells, it has become clear in recent years that apoptotic neutrophils, via their effects on engulfing macrophages, also play a central role in actively suppressing inflammation. In a widely cited work, Fadok and colleagues [71] showed that cytokine expression in macrophages is broadly suppressed following phagocytosis of apoptotic neutrophils, with the major exception of TGF-β, the expression of which is markedly up-regulated. Curiously, phagocytosis inhibited not only classical pro-inflammatory cytokines, but IL-10 as well. TGF-β, actinginanautocrine/paracrinemanner, isanimportantexecutorofcytokinesuppres- sion and macrophage reprogramming. TGF-β, prostaglandin E2 (PGE2), and platelet activating factor (PAF) all are induced upon phagocytosis and contribute to cytokine suppression, but the latter two may act at least partly by TGF-β induction [71]. TGF-β is a pleiotropic factor that is infamousforhavingvaryingeffectsdependingontheenvironment,anditplaysanimportantrolein inflammatoryresolution,endotoxintolerance,woundhealing,fibrosis,aswellascancerprogression, 221 a context in which it plays both Jekyll and Hyde [19]. TGF-β also strongly inhibits lymphocyte function and proliferation [219]. TGF-β receptors are extremely high affinity, with K d estimated at 1 pM in natural killer cells [219], and 1–10 pM [254] in peripheral blood monocytes. TGF-β also is a potent chemoattractant for monocytes at femtomolar concentrations, with the optimal range 4–40 fM [254]. Recognition of phosphatidylserine (PS) on the outer membrane leaflet is crucial to recognition of apoptotic cells [102], and contact alone, without phagocytosis, can induce profound changes in target immune cells. Contact with apoptotic cells appears to significantly alter intracellular signalling, which represents a fundamental reprogramming of the cellular machinery. It is also clear that contact per se, and not, say, some soluble mediator released by dying cells, is necessary [49, 102]. Apoptotic neutrophils affect not only macrophages, but neutrophils and monocytes as well. It has recently been shown [69] that neutrophils themselves, once primed by inflammatory stimuli, canefficientlyphagocytoseotherneutrophilsthatundergoneapoptosis; thisinhibitstherespiratory burst and TNF expression by neutrophils, but enhances IL-8 [69], a potent chemokine and inducer of angiogenesis [134]. Monocytes are unable to phagocytose apoptotic cells, but simple contact with apoptotic cells alters the monocyte cytokine expression pattern to an anti-inflammatory one, characterized by enhanced TGF-β and IL-10 production [30]. In sum, several major phagocytes involved in innate immunity switch to a broadly anti-inflammatory phenotype upon contact with apoptotic cells, and apoptosis is increasingly being viewed as the modulator of a ubiquitous and profoundly anti-inflammatory branch of the innate immune system [49]. Model. We extend our model to consider the following behaviors: 1. TNF and bacteria, as a proxy for LPS and other bacteria products, inhibit neutrophil apop- 222 tosis. This inhibition is directly counteracted by IL-10. 2. Neutrophils that undergo spontaneous apoptotis add to an apoptotic cell compartment, A(t). 3. Macrophages, and neutrophils clear apoptotic cell bodies. Contact with apoptotic bodies induces these cells to enter an alternative pattern of activation. For macrophages, this is as- sumed to resemble alternative activation (M2), with suppressed TNF and IL-10 production, reduced bacterial killing, but increased clearance of apoptotic debris. Alternatively acti- vated neutrophils have suppressed bacterial killing. Classically and alternatively activated macrophages are represented as M 1 (t) and M 2 (t), respectively. Similarly, neutrophils are divided into N 1 (t) and N 2 (t) compartments. 4. Alternatively activated macrophages produce TGF-β, represented as G(t), which has sev- eral autocrine and paracrine effects on macrophages. TGF-β enhances IL-10 production by macrophages [156], inhibits macrophage activation [241] and TNF expression by activated macrophages. The full ODE version of the system follows: 223 dB(t) dt = λ B (t)+rB ( 1− B Λ ) −γ(M 1 + ´ M 1 +φM 2 +N 1 +φN 2 ) ( B θ+B ) , (7.66) dM 1 (t) dt = ρM R −µ 1 M 1 +M R (1−F G ϕ G ) ( σ 1 B θ A +B +σ 2 f n f n d +f n ) − γ A M 1 ( A θ Ap +A ) − M 1 τ , (7.67) d ´ M 1 (t) dt = M 1 τ −µ 1 ´ M 1 −γ A ´ M 1 ( A θ Ap +A ) , (7.68) dM 2 (t) dt = γ A (M 1 + ´ M 1 ) ( A θ Ap +A ) −µ 2 M 2 , (7.69) dN 1 (t) dt = QN B V ( E ne θ ne E +E ne ) (1−F I ϕ I )−γ NA N 1 ( A θ Ap +A ) − ˆ δ N N 1 , (7.70) dN 2 (t) dt = γ NA N 1 ( A θ Ap +A ) − ˆ δ N N 2 , (7.71) dA(t) dt = ˆ δ N (N 1 +N 2 )−(γ A (M 1 + ´ M 1 +2M 2 )+γ NA (N 1 +N 2 )) ( A θ Ap +A ) , (7.72) dE(t) dt = ν(E T −E) ( T K E +T ) −µ E E, (7.73) dT(t) dt = α T (M 1 + ´ M 1 +.1M 2 )(1−F I ϕ I )(1−F G ϕ G )−δ T T, (7.74) dI(t) dt = α I ( ´ M 1 +.1M 2 )(1−F I ϕ I )(1+ϕ G )−δ I I, (7.75) dG(t) dt = α G M 2 −δ G G, (7.76) ϕ I = I K I +I ,ϕ G = G K G +G ,f = T K d +T , (7.77) ˆ δ n = δ N ( 1−.5(1−ϕ I ) ( T K N +T + B θ A +B )) , (7.78) M R = J−M 1 − ´ M 1 −M 2 . (7.79) Since macrophage activation dynamics are much more complicated in this model than in previ- ous iterations, we have switched from an explicit delay differential formulation to an ODE for- mulation with a “pseudo-delay” in IL-10 production by newly activated macrophages. These macrophages, M 1 , transfer to the ´ M 1 compartment at rate 1/τ; only ´ M 1 and M 2 macrophages 224 produce IL-10. Apoptotic body clearance by phagocytes occurs via a Holling Type II functional response, and induces a transition from classical to alternative (immunosuppressive) activation. Results. The set of possible asymptotic behaviors is not essentially altered by the addition of apoptosis and alternatively activated effector cells, relative to the baseline neutrophil model. Chronic sub-saturated infection and sterile inflammation can both occur, but the most typical outcomes are bacterial clearance with resolution, and chronic saturated infection, i.e. bacterial escape. As expected, the probability of sterile inflammation is reduced, but not eliminated, under this model extension. The major new dynamic is a period of relative immunosuppression following clearance of the initial infection. The degree of immunosuppression is proportional to the severity of the infection, andsecondarybacterialinoculaintroducedduringthishypoimmunephasethatarenormallycleared can escape immune control. This is demonstrated by several numerical experiments shown in Figures 7.20 and 7.21. Interestingly, immediately after the initial infection, secondary infections arestronglysuppressed,despiteadiminishedcytokineresponse. Thus,weobserveahyper-tohypo- inflammatory transition, similar to that which has been hypothesized [112] (see Section 7.2.4). We also observe that, should the bacteria escape control and grow to its carrying capacity, the immuneresponsebecomesrelativelysuppressed: sustainedIL-10suppressesTNF,andthemajority of macrophages and neutrophils switch to an anti-inflammatory phenotype. Examples of bacterial clearance and escape are given in Figure 7.22. Further extensions in toto We consider several further extensions to the model framework; since the model has, at this point, become rather elaborate, we give here the fully extended model in toto, and in subsequent sections 225 Figure 7.20: Following an initial infectious insult of 2×10 7 cells ml −1 , reinfection with the same total (infused over 1 hour) is imposed at either the 24 or 72 hour mark (indicated by arrows), as shown on the left and right sides of the figure, respectively. When reinfection occurs early, bacterial suppression is stronger than during the initial infection, despite a blunted cytokine re- sponse. However, followingthis initial transient, there is a second transientduring whichimmunity is suppressed, as demonstrated on the left hand side. 226 Figure 7.21: Under the neutrophil+apoptosis model, at time zero, infection with either 10 5 or 10 6 ml −1 cellsoccurs,andre-infectionofthesametotalisimposedaftersomedelay. Thepeakbacteria, TNF,IL-10,andneutrophillevelsfollowingre-infection,relativetothepeakfortheinitialinfection, are displayed. Not shown is infection/re-infection by 10 7 cells ml −1 , which leads to overwhelming infection over a delay to re-infection of roughly 1.5 to 4 days. As can be seen, the larger the infection, the greater the degree of immunosuppression. Note that the suppressive transient is not characterized by increased IL-10, but is a globally suppressed state. 227 Figure7.22: Theupperpanels giveexample time-seriesforcellular populations(ona logscale)and cytokine concentrations (linear scale) under bacterial escape. The lower panels give these time- series in the case of successful bacterial clearance. At this time-scale it is not obvious that the inflammatory response resolves, but viewed over several weeks, as in the bottom left inset, we see that all activated immune cell populations (and cytokine levels) go to zero. comment on the individual sub-extensions. The following 17 variables are considered: 1. B(t), bacteria. 2. B 2 (t), representing a possible second strain of bacteria. 3. M R (t), resting macrophages. 4. M 1 (t), classically activated macrophages that do not express IL-10. 5. ´ M 1 (t), classically activated macrophages that now express IL-10 (pseudo-delay formulation). 6. M 2 (t), alternatively activated macrophages. 7. N 1 (t), activated neutrophils. 8. N 2 (t), alternatively activated neutrophils. 9. U 1 (t), monocytes. 10. U 2 (t), alteratively activated monocytes. 11. A(t), apoptotic cell bodies. 12. D(t), damage, or necrotic cell bodies. 13. E E (t), early activated endothelium. 14. E L (t), late activated endothelium. 15. T(t), TNF. 16. I(t), IL-10. 228 17. G(t), TGF-β. 229 Thefullsystem,includingtwoseparatebacterialstrainsthatinteractcompetitivelywithrespect to growth and phagocyte-mediated killing, follows: dB(t) dt = λ B (t)+rB ( 1− B+c 12 B 2 Λ ) − γ(M 1 + ´ M 1 +N 1 +U 1 +φ(M 2 +N 2 +U 2 ))   B B+θ ( 1+ B2 2 )   , (7.80) dB 2 (t) dt = λ B2 (t)+r 2 B ( 1− c 21 B+B 2 Λ 2 ) − γ 2 (M 1 + ´ M 1 +N 1 +U 1 +φ(M 2 +N 2 +U 2 )) ( B 2 B 2 +θ 2 ( 1+ B ) ) , (7.81) dM R (t) dt = µ 1 (M 1 + ´ M 1 )+µ 2 M 2 −ρM R −g(M R ,B,B 2 ,T,I,D), (7.82) dM 1 (t) dt = ρM R −µ 1 M 1 +g(M R ,B,B 2 ,T,I,D)−γ A M 1 ( A θ Ap +A ) − M 1 τ , (7.83) d ´ M 1 (t) dt = M 1 τ −µ 1 ´ M 1 −γ A ´ M 1 ( A θ Ap +A ) , (7.84) dM 2 (t) dt = γ A (M 1 + ´ M 1 ) ( A θ Ap +A ) −µ 2 M 2 , (7.85) dN 1 (t) dt = ω QN B V ( E ne E θ ne E;N +E ne E ) (1−F I ϕ I )−γ NA N 1 ( A θ Ap +A ) − ˆ δ N N 1 , (7.86) dN 2 (t) dt = γ NA N 1 ( A θ Ap +A ) − ˆ δ N N 2 , (7.87) dU 1 (t) dt = ω QU B V ( E ne L θ ne E;U +E ne L ) (1−F I ϕ I )ϕ G −U 1 ( A θ Ap +A ) −δ U U 1 , (7.88) dU 2 (t) dt = U 1 ( A θ Ap +A ) −δ U U 2 , (7.89) dA(t) dt = ˆ δ N (N 1 +N 2 )−(γ A (M 1 + ´ M 1 +2M 2 )+γ NA (N 1 +N 2 )) ( A θ Ap +A ) −δ A A, (7.90) dD(t) dt = δ A A+r D (N 1 +.1(M 1 + ´ M 1 )+.01(B+B 2 )) (7.91) −γ D (M 2 +U 2 +φ R M R ) ( D θ DC +D ) −δ D D, (7.92) dE E (t) dt = ν(E T −E E −E L ) ( T K E +T ) −µ EE E E −.025(1+40ϕ G )E E , (7.93) dE L (t) dt = .025(1+40ϕ G )E E −µ EL E L , (7.94) dT(t) dt = α T (M 1 + ´ M 1 +.1(M 2 +U 1 ))(1−F I ϕ I )(1−F G ϕ G )−δ T T, (7.95) dI(t) dt = α I ( ´ M 1 +.1(M 2 +U 2 ))(1−F I ϕ I )(1+ϕ G )−δ I I, (7.96) dG(t) dt = α G (M 2 +U 2 )−δ G G, (7.97) 230 where ϕ I = I K I +I ,ϕ G = G K G +G ,f = T K T +T , (7.98) ˆ δ n = δ N ( 1−.5∗(1−ϕ I ) ( T K N +T + B θ A +B )) , (7.99) and g(M R ,B,B 2 ,T,I,D)=(1−F G ϕ G )M R ( σ 1 ( B B+θ A + B 2 B 2 +θ A2 ) +σ 2 f n f n d +f n +σ 3 D θ D +D ) . (7.100) Extension to tissue damage We explore the effect of a generic damage variable, D(t), on model dynamics. This is an abstract summation, representing the sum total of oxidative damange, necrotic cell bodies, and late pro- inflammatorymediatorssuchasHMGB-1thatarereleasedbybothnecroticcellsandmacrophages. Forconcreteness, wetakeD(t)tobenecroticdebriswithunitsofnecroticcellbodies, makingitthe pro-inflammatory analog of apoptotic cell bodies, A(t), considered previously. Apoptotic bodies thatarenotclearedinatimelymannerbecomenecrotic,andweassumethatneutrophils,activated M1 macrophages, and bacteria cause damage. Neutrophils have been identified as key sources of tissuedamage,andthusweweightthemmostheavilyincausingdamage. Whilemuchtissuedamage is attributable to host immune cells, bacteria can also cause direct tissue damage, e.g. through exotoxin expression, and therefore we include bacteria as a lightly weighted damage source. Necrotic debris is cleared by M2 and resting macrophages, with the former much more efficient. Damage is taken to have an activating influence upon macrophages and endothelial cells similar to TNF. Together, these considerations lead to the neutrophil+apoptosis+damage model extension, 231 consisting of the model detailed in Equations 7.66–7.79, plus damage-specific terms as cataloged in Equations 7.80–7.97. Damage is somewhat similar to TNF in its effect on model dynamics: it effectively mobilizes the immune system, but this mobilization can also lead to sterile inflammation following bacterial clearance. The inclusion of active immunosuppression and damage clearance by alteratively acti- vated macrophages can dramatically improve resolution following the acute bacterial insult when damage is explicitly modeled. We can conclude from this that active immunosuppression may be important in vivo for preventing a pathologic feedback loop between tissue damage and sterile inflammatory activity. However, the basic dynamics of the model are not essentially altered by the inclusion of damage. Extension to monocyte recruitment The final possible model extension that we consider is monocyte recruitment from the circulation. It is well-established that the neutophilic infiltrate of early acute inflammation switches to a pre- dominantlymononuclearinfiltratewithin12–24hours. Thisswitchappearstobemediatedbyboth evolving cytokine signalling and internal endothelial signal transduction that alters the endothelial adhesion factor profile [126]. Modelconstruction. CytokinestimulationcausesendothelialE-selectin,ICAM-1,andVCAM- 1 expression to follow a stereotyped course. Karmann et al. [126] found that, following TNF exposure, E-selectin is detectable in 1-2 hours, peaks at 4 hours, and is reduced to 20% at 24-48 hours; VCAM-1appearsin3-4hours,peaksbetween8and24hours,anddeclinesslowlythereafter; ICAM-1increasessteadilyfor3–24hoursandremainssteadyatitspeakforatleast72hoursunder continuing cytokine stimulation. 232 The basic dynamic of transient E-selectin but sustained VCAM-1 and ICAM-1 expression ap- pears to directly mediate a switch from predominantly neutrophil to monocyte recruitment. E- selectin induces tethering and rolling by neutrophils, while VCAM-1 induces adhesion by T-cells and mononuclear cells, but not neutrophils. ICAM-1 induces firm adhesion and transmigration by both neutrophils and T-cells [126]. Therefore, we distinguish between “early-activated” endothelium, E E (t), which induces neu- trophil diapedesis, and “late-activated” endothelium, E L (t), which preferentially attracts mono- cytes. Since our model construction imposes rapid baseline endothelial cell de-activation that leads tonetde-activationintheabsenceofTNF,itisdifficulttodirectlycapturethetransitionfromearly to late endothelium. Therefore, we have the transition rate depend strongly upon TGF-β. TGF-β becomes prominent at roughly the same time after infection that we expect monocytic infiltrate, and TGF-β is also a potent monocyte chemoattractant [254]. In simulations, this construction yields the expected transition from predominantly early to late endothelium; this is formalized in Equations 7.93–7.94 and 7.88, and in the modification to the neutrophil recruitment term in Equation 7.86. Mirroring macrophages, monocytes are assumed to have either an inflammatory phenotype, represented as U 1 (t), or a suppressive phenotype, U 2 (t), the expression of which is induced by apoptotic cell contact (see the discussion in Section 7.4.3). We assume that monocytes have an inflammatory (U 1 ) phenotype upon recruitment from circulation. The monocyte recruitment term, given in Equation 7.88, mirrors the recruitment term for neutrophils, but with monocyte-specific parameters. Results. The inclusion of monocytes both enhances the early response to pathogens, making bacterial clearance more likely for fixed parameters, and extends the immunosuppressed phase that 233 follows. Much like the damage extension, the basic qualitative behavior of the model is not altered by monocyte inclusion, either alone or in combination with the damage extension. Therefore, as previously stated, all subsequent results are generated on the neutrophil+apoptosis model, with appropriate modifications for a second bacteria strain as needed. 7.4.4 Cytokines in the well-mixed setting As discussed more extensively in Section 7.2.3, multiple clinical studies have examined cytokine levelsinsepsisandbacteriaandtheirrelationtooutcome. Suchstudiesbroadlysuggestthatinitial pro-inflammatory (e.g. TNF, IL-6) and anti-inflammatory (IL-10) markers are elevated in more severediseaseandpersistentelevationisassociatedwithapooroutcome. However, thereismarked heterogeneity among patients, and cytokine levels are not useful predictors in individual patients. Some previous studies, e.g. [87], and forthcoming work by Wong and colleagues (Wong, personal communication), suggest that increased anti-inflammatory activity, as measured by the IL-10:TNF ratio, best predicts outcome. Wong et al. have, furthermore, measured initial cytokines and those after 72 hours of effective antibiotic therapy and found that a persistent elevation in IL-10:TNF is associated with bacteremia persistence and mortality. Basic cytokine dynamics In the basic well-mixed setting, TNF and IL-10 both rise early in infection, with the magnitude of rise proportional to that of the initial infection; this is shown in Figure 7.23. The peak TNF and IL-10 values also both monotonically increasing functions of B 0 , the initial infectious load. Figure 7.24 shows the time-series of the IL-10:TNF ratio, again for differing values of B 0 . When the infection is ultimately cleared, we have a pattern of an early spike in this ratio, followed by 234 an extended decline. When infection persists, however, the dynamic is slower, with more gradual increase that is sustained. Similar results are obtained when varying r, the intrinsic bacterial growth rate, although peak IL-10 varies more markedly than peak TNF. If instead of varying B 0 or r, parameters that directly describe infection severity, we fix these values and vary parameters relating to the immune response, we still see largely similar cytokine patterns. That is, the relative severity of the infection, whether that means a larger bacterial inoculum or a weaker immune response, is consistently associated with the same general cytokine pattern. The general exception to this rule occurs under variations of α T and α I , the activated macrophageproductionratesofTNFandIL-10, respectively. Loweringα T yieldsaweakerimmune response, lower peak TNF, and higher peak IL-10 values. Similarly, increasing α I impairs the response and is associated with lower TNF and higher IL-10 values. In sum, TNF and IL-10 values increase in tandem early in infection, with the magnitude of increase proportional to the absolute infection severity, as measured by r and B 0 in our model. Thus, high anti- and pro-inflammatory cytokine values can represent an appropriate response to seriousinfection. Asrelative infectionseverityincreases,IL-10alwaysincreasesinproportion,while TNF usually increases in proportion. The intrinsic tendency of macrophages to produce pro- and anti-inflammatory cytokines can alter the immune response such that TNF is suppressed and the relative infection severity is enhanced, suggesting that cytokine mismatch may indicate an immune response tuned to be overly immunosuppressive. Initial, and 72 hour cytokines in relation to infection severity Itremainsunclearwhethervariationsincytokinelevelsatclinicalpresentationreflectabnormalities of the host response that lead to a poor outcome, or whether these variations reflect appropriate 235 Figure 7.23: Cytokine time-courses and peak values for B 0 , the initial infectious load, varied from 10 −3 −10 −9 cells ml −1 . The top panels give the TNF and IL-10 time-courses, while the bottom panels give the peaks as a logarithmic function of B 0 . Figure 7.24: Time-series for the IL-10:TNF ratio under different B 0 values. When the ultimate outcomeisbacterialclearancethereisanearlypeakinthisratiothatthentapersoff;themagnitude of the peak is directly proportional to B 0 . When bacterial escape occurs, there is a qualitatively different pattern of a relatively slow rise that continues late in time. Early in time, the IL-10:TNF ratio does not reliably distinguish between these two outcomes, but it does so by 72 hours. 236 responses to infections of varying severity. In this section we examine initial and 72-hour cytokine levels in the absence of treatment as a function of the infection severity, as measured by B 0 ; basic immune competence, as measured byγ; and the intrinsic pro-/anti-inflammatory orientation of the immune response, as measured by α T . Figure 7.25 shows the cytokine metrics as a function of B 0 , where time of initial measurement varies uniformly between one and 24 hours; Figure 7.26 gives the same information but on a logarithmic scale. We have run several simulated clinical trials, where 1,000 patients are diagnosed between one and 24 hours following infection; cytokine levels and ratios are measured at diagnosis and 72 hours later. We also randomly vary either B 0 (10 3 −10 9 cells ml −1 ),γ (1−10 hr −1 ), orα T (10 −21 −10 −19 moles cell −1 hr −1 ). Figure 7.27 shows the median values of the cytokine metrics for patients who clear and do not clear the insult, when differences in B 0 lead to outcome variation; results are qualitatively similar whenγ andα T are the independent variables. We find that as the relative severity of the infection increases, TNF and IL-10 levels increase at both initial presentation and at 72 hours. The picture of the IL-10:TNF ratio is more nuanced, with bacterial clearance associated with a slightly higher initialIL-10:TNFratio,butpersistenceleadstoanelevatedIL-10:TNFratioat72hours. Moreover, variation in this ratio is very large initially, but much less so at 72 hours. We also examine how relative changes in cytokine levels in individual patients relate to bac- terial persistence or clearance. We find that the relative decrease in IL-10 is smaller in subjects where infection persists. Statistically, this difference is highly significant, but the magnitude of the difference is slight. On the other hand, the change in IL-10:TNF differs qualitatively among the two patient groups. Bacterial clearance always leads to a significant reduction in this ratio, while persistencetypicallyleadstoanincrease,althoughthereismorevariationinthiscase; thisisshown 237 in Figure 7.28. These results suggest that an elevation in IL-10:TNF at 72 hours may be a highly sensitive and reasonably specific marker of bacterial persistence. While results are presented for B 0 as the independent variable, our findings are similar when γ, representing the ability of phagocytes to clear bacteria, is varied with a constant B 0 . Somewhat surprisingly, varying α T with a constant B 0 also gives qualitatively indistinguishable results. As discussed in the previous section, a low α T can lead to a low peak TNF value and overwhelming infection. However, even though the peak TNF value is lower when immune escape occurs, this peakoccursveryearlyandthenTNFproductionissustainedinthefaceofongoinginfection. Thus, we find that even when infection persistence is related to insufficient TNF production, higher TNF values at diagnosis and at 72 hours are still positively associated with persistence. We have also run a simulated trial where both B 0 and α T are allowed to vary, with 10,000 “patients.” Results are again similar, except that the initial IL-10:TNF ratios do not vary significantly. It should be emphasized that these results reflect the well-mixed scenario, absent any spatial or geometric extensions or therapeutic intervention. 7.5 Antibiotic chemotherapy Appropriateandpromptantibiotictherapyremainsthecornerstoneofeffectivesepsistherapy. Our primaryfocusisonthefluoroquinolones,apowerfulclassofantibioticswithbroadspectrumactivity that enjoy widespread use in sepsis. However, enthusiasm for these agents has been dampened somewhat by the high frequency of treatment failure due to super-infection by resistant strains. Hence, we may consider two primary endpoints in designing an antibiotic regimen: (1) suppression of the overall bacterial population, and (2) suppression of resistance. Fluoroquinolones are attractive for modeling because serum concentrations are an excellent 238 Figure 7.25: Median TNF, IL-10, and IL-10:TNF values for differentB 0 at initial presentation and 72 hours later. Time to presentation is varied randomly between one and 24 hours. Each point represents the median of 50 simulations, and error bars give the interquartile range (IQR). Figure 7.26: Replication of Figure 7.25, but with a logarithmic scale for the y-axes. 239 Figure 7.27: Median initial and 72-hour TNF, IL-10, and IL-10:TNF values divided into bacterial clearing and persisting classes. The initial bacterial load varies uniformly in logarithmic space, and the range ofB 0 is 10 3 −10 9 cells ml −1 . Results are for 1,000 simulations, with time to presentation varying from one to 24 hours. The error bars give the IQR; all metrics vary significantly (p<.05) between persisters and clearers, using a Wilcoxon rank sum test. Figure 7.28: From left to right, the median relative changes in absolute TNF, IL-10, and the IL- 10:TNF ratio between presentation and 72 hours for persisting and non-persisting infection; the error bars give the IQR. While the absolute changes in TNF and IL-10 are lesser when persistence occurs, the sign of the IL-10:TNF difference is positive in these patients, but negative in those that clear infection. 240 surrogate for those in tissue [77], and their effect on bacteria is relatively easy to characterize through a dose-dependent formalism based on theE max model (see Section 7.5.1). Since S. aureus, whichisfrequentlytreatedwithvancomycin, isacommoncauseofbacteremiaandsepsisandisthe focus of ongoing study by Wong and colleagues (Wong, personal communication), we also briefly consider treatment with this agent. 7.5.1 PK/PD Background Broadly speaking, antibiotic action is described mathematically by pharmacokinetics (PK) and pharmacodynamics (PD), where the former is concerned with the effects of the body on the drug, and the latter describes the effect of the drug on its target cells, or on the body more generally. Taken together, we may speak of PK/PD modeling. It is not our purpose to exhaustively review these very extensive fields here, but to review the basic concepts and mathematical machinery relevant to a comprehensive model of sepsis. Pharmacokinetics Pharmacokineticsdescribeshowdrugsaredistributedinandeliminatedfromthebody. Threebroad approaches have traditionally been used in this field. The first is compartmental modeling, where the body is divided into one or more discrete compartments (often aggregations of multiple tissue types)intowhichdrugdistributes. Typically,exchangeoccursbetweenthecompartmentsaccording to first-order kinetics. Physiological modeling employs anatomical compartments (plasma, liver, kidney, etc.) and blood flow rates between the compartments govern drug distribution. Model- independent pharmacokinetics are used to characterize the distribution and elimination of drug in plasma without using any complex underlying model. Simple quantities such as plasma area under 241 the curve (AUC), volume of distribution (V D ), and elimination half-life (t 1=2 ) are measured under this approach. One-compartmentmodel. One-andtwo-compartmentmodelsareillustratedinFigure7.29. Under a one-compartment model the plasma drug concentration, C, is related to the amount of drug in the body, A, as C = A V D , (7.101) where V D is the apparent volume of distribution, which is defined as V D = A C . (7.102) The first order plasma elimination rate k el depends upon the distribution of drug between plasma and tissues, and thus depends upon the volume of distribution. The plasma clearance Cl p , unlike k el , is independent of V D . Plasma clearance has units of volume / time (e.g. L / hr), and reflects the volume of plasma from which drug is completely removed in a unit of time. Mathematically, we have dA dt =−Cl p C =−k el A=−k el V D C, (7.103) and thus Cl p =k el V D . (7.104) Theplasmaareaunderthecurve(AUC)issimplytheintegralofdrugconcentrationC(t)overtime AUC = ∫ ∞ 0 C(t)dt. (7.105) 242 The AUC has units concentration × time (e.g mg hr L −1 ). Interestingly, dose, plasma clearance, and plasma AUC are all related. For bolus of size D, the initial drug concentration is C 0 =D/V D , and C(t)=C 0 exp(−k el t). Integrating gives: AUC = ∫ ∞ 0 D V D e −k el t dt= D V D k el = D Cl p (7.106) This relationship also holds for any infusion schedule as long as the total dose delivered is D. Finally, usingthese conceptswecaneasilyframe thedynamics oftheone-compartmentmodelwith an initial bolus of size D in many ways: dA dt =−k e lA=−k e lV D C =−Cl p C =−Cl p A V D =− D AUC C, (7.107) where dC dt = 1 V D dA dt , (7.108) and A(0) = D, (7.109) C(0) = D V D . (7.110) For a continuous infusion at rate k a from time 0 to T the equations are similar, but with a “+k a H(T −t)” term on the right side. Two-compartment model. The two-compartmental pharmacokinetic model explicitly mod- els the distribution of drug from the central compartment (plasma) to the peripheral compartment 243 Figure 7.29: The left side of the figure is a schematic representation of the one-compartment pharmacokinetic model with either continuous drug infusion at rate k a for time T, or a bolus of total dose D. The right side illustrates the two-compartment model. (tissue), asshownschematicallyinFigure7.29. Thetimecourseofplasmadrugconcentrationfora two-compartmentmodelismarkedlydifferentfromtheone-compartmentmodel. Itischaracterized by two phases: the initial distribution (α) phase where plasma drug concentration rapidly drops and as it is sequestered in tissue, and a much slower elimination (β) phase where plasma and tissue drug are essentially at equilibrium and drug clearance (i.e. k el ) is slower. Using the so-called microscopic rate constants, the differential equations describing the amount of drug in the two compartments are: dA 1 dt = k a +k 21 A 2 −k 12 A 1 −k el A 1 , (7.111) dA 2 dt = k 12 A 1 −k 21 A 2 , (7.112) , where A 1 is the amount of drug in the central compartment (plasma) and A 2 is the amount in 244 the peripheral compartment (tissue). The drug concentrations are given by C 1 (t)= A 1 (t) V 1 , (7.113) C 2 (t)= A 2 (t) V 2 . (7.114) The apparent volumes of distributions for the two compartments are V 1 and V 2 , respectively, and we define distributional, Cl D , and total, Cl T , clearances as Cl T = V 1 k el , (7.115) Cl D = V 1 k 12 =V 2 k 21 . (7.116) Typically,thedrugconcentrationinthecentralcompartment(plasma)iswhatisofinterestandcan be easily measured. For a bolus dose of size D, the plasma concentration, C(t), is often reported using the following model: C(t)=Ae −t +Be −t . (7.117) Thismodelispopularbecauseofitssimplicityandbecausetheso-calledmacroscopicrateconstants A, α, B, β can be estimated graphically from a plot of plasma concentration vs. time. In general A ≫ B and α ≫ β. Heuristically, this model can be thought of as the superposition of two separate models. Because A ≫ B, in early time Aexp(−αt) ≫ Bexp(−βt) and during this time the drug is being distributing from the plasma to peripheral tissues, and C(t)=Aexp(−αt) is the model for the distribution phase. The concentration in this model quickly goes to zero, as α is large. Following distribution, C(t)=Bexp(−βt) is the model for plasma concentration during the elimination phase. The macroscopic rate constants can be expressed in terms of the microscopic 245 rate constants as follows: α = k el +k 12 +k 21 2 + √ ( (k el +k 12 +k 21 ) 2 4 −k el k 21 ) , (7.118) β = k el +k 12 +k 21 2 + √ ( (k el +k 12 +k 21 ) 2 4 −k el k 21 ) , (7.119) A = D(α−k 21 ) V 1 (α−β) , (7.120) B = D(k 21 −β) V 1 (α−β) . (7.121) A more thorough mathematical treatment of these relations can be found in Metzler [178]. The two-compartment model above can be extended to consider an arbitrary number of compartments, whichmayhavevariousphysicalinterpretations; theusualnotationforathree-compartmentmodel is C(t)=Ae −t +Be −t +Ce − t . (7.122) Another useful extension is to consider infusing a total dose D over the time interval (0,T), con- sidering the two-compartment model as the explicit summation of two independent compartments: dC 1 (t) dt = A T H(T −t)−αC 1 , (7.123) dC 2 (t) dt = B T H(T −t)−βC 2 , (7.124) C(t) = C 1 (t)+C 2 (t). (7.125) Pharmacodynamics The most widely used metric for anti-bacterial efficacy is the minimum inhibitory concentration (MIC),whichisusuallydefinedasthelowestconcentrationthatcompletelyinhibitsvisiblebacterial 246 growth in vitro after a 24-hour incubation period [185]. The efficacy of an in vivo course of therapy is often estimated using the serum concentration of the drug and the MIC as determined in vitro. The most commonly used metrics are the time above the MIC (T > MIC), the ratio of peak serum concentration to MIC (C peak / MIC), and the serum AUC over the MIC (AUC/MIC) [185]. Antibioticscanbebroadlyclassifiedaseitherconcentration-ortime-dependent[62]. Theformer demonstrate bacterial killing in proportion to drug concentration over a broad range, whereas the latter exhibit a plateau in drug efficacy for relatively low drug concentrations (as a factor of MIC). Thus, the AUC/MIC is generally a better predictor of efficacy for concentration-dependent drugs, while time above MIC, T > MIC, is preferred for time-dependent agents. Fluoroquinolones are concentration-dependent; vancomycin is not concentration-dependent, yet AUC/MIC is the best predictor of effect. This is explained on the basis that vancomycin is only weakly bacteriocidal but hasaprofoundpost-antibioticeffect(PAE):bacterialregrowthisinhibitedfollowingdrugclearance due to sub-lethal damage. The PAE is proportional to the AUC/MIC ratio [62]. The fairly commonly used (sigmoid) E max model (sometimes referred to as a Zhi model) for drug effect quantifies the plateau in cytotoxic response with drug concentration, and was initially used for the time-dependent beta-lactams [271], but has since been applied to many concentration- dependent drugs as well [50]. For exponentially growing bacteria, B(t), drug induced death is modeled by a Hill function and we have the simple equation: dB dt =B ( r−k max C t EC 50 +C t ) (7.126) where C t is the drug concentration at a given time, r is the unperturbed growth rate, k max is 247 the maximum killing rate, and EC 50 is the drug concentration of half-maximal activity. The Hill coefficient,κ is unity for the standardE max model and is a free parameter under the sigmoidE max model. Combined PK/PD modeling Beginning with the 1988 work of Zhi and colleagues [271], who combined a linear one-compartment pharmacokinetic description for piperacillin kinetics with an E max description of the bacteria and druginteraction,asmallnumberofauthorshaveusedsimilarcombinedPK/PDmodelingtopredict optimal dosing for both bacterial eradication and prevention of resistance, e.g. [152, 125, 31]. Suppose there exists a sensitive population, S(t), and a resistant population R(t). The general model form typically employed follows dS dt = r S S ( 1− S+R Λ ) −k S ( (C S EC S 50;S +C S ) −m(S), (7.127) dR dt = r R R ( 1− S+R Λ ) −k R ( (C R EC R 50;R +C R ) +m(S), (7.128) whereC(t)isdeterminedbyanadjuvantpharmacokineticsmodel, theE max arestrain-specific, and m(S)isagenericrepresentationofmutationfromthesensitivetoresistantclass. Thelattercantake the form of a constant fraction of cell divisions, a Poisson process whereby each new sensitive cell becomes resistant with a specified probability (necessitating a semi-discrete, stochastic formulation of the ODE model), or it may simply be assumed that resistant cells are present initially with no mutations occurring. Different dosing schedules can be simulated to determine those most likely to suppress resistance. Parameterestimatesforthe(sigmoid)E max modelfromanumberofstudieshavebeencompiled 248 by Czock and Keller [50]. One point of concern is that these estimates seem to vary rather widely within the same agent and bacteria species. Mutant selection window hypothesis Whilemultipleworkshaveexaminedspecificdosingstrategiestoavoidresistance,thebasicstrategic approach can be understood through the mutant selection window (MSW) hypothesis. The MSW posits that, for any drug-pathogen combination, there exists a drug concentration range in which thegrowthofsingle-stepdrug-resistantmutantsisselectivelyamplified,i.e. theeponymous“mutant selection window” (MSW) [61]. The lower boundary of the MSW is the minimum concentration needed to exert selective pressure for resistance. This is generally taken to be approximately equal to the MIC of wild-type cells, and the upper boundary, called the mutant prevention concentration (MPC), can be defined as the MIC of the most drug-resistant subpopulation. Thus, between the MICofsensitivecellsandtheMICofresistantcells(theMPC),positivegrowthofresistantmutants occurs, greatly amplifying resistance, as illustrated in Figure 7.30. The validity of the MIC as a lower bound on amplification is questionable, as we expect that even below this threshold the presence of antibiotics is likely to exert weak selective pressure [217]. Fluoroquinolones The pharmacokinetics of IV ciprofloxacin have been characterized by several authors using a two- compartment model [263, 76]. Crump et al. [47] studied its oral pharmacokinetics, and found the absorption rate constant, K a , to be 2.7±1.22 hr −1 , consistent with results in [262]. Ciprofloxacin distributesrapidly,andhasaneliminationhalf-lifeofroughlyfourhours. Wiseetal. [263]compared serum ciprofloxacin to that in blister fluid and determined that the drug rapidly penetrates blister 249 Figure 7.30: Graphical illustration of the mutant selection window hypothesis, which posits that drugconcentrationsbetweentheMICsofsensitiveandthemostresistantcells(theMPC)selectively amplify resistance. fluid, and that the elimination half-life and 24-hour AUC are similar between serum and blister fluid. Ithasbeenconsistentlyseenthat, followinganinitialtransientduringwhichthedrugenters, thetime-coursesofserumandblisterciprofloxacinareverysimilar,withblisterlevelsslightlyhigher [262]. Theseresultsareallinaccordwiththecommonassertionthatserumciprofloxacinconcentration is an effective surrogate for that at the infection site [77]. Table 7.2 gives the pharmacokinetic rate constants from two studies [263, 76]; the study of Forrest et al. [76] was performed on acutely ill patients and is thus the more relevant of the two. The coefficient of variation for most parameters is on the order of 25–50%. The fluoroquinolones display concentration-dependent cytotoxicity across a broad range of con- centrations, explaining why the 24-hour AUC/MIC metric has been the best predictor of efficacy for these drugs. Because the ciprofloxacin MICs for S. aureus and P. auruginosa are relatively high, typically in the .125–1 µg ml −1 range for either species, selection for resistant organisms is a common cause of treatment failure, and several mathematical models have addressed this problem 250 Table 7.2: Two-compartment ciprofloxacin pharmacokinetic parameters from two different studies [263, 76]. The standard deviation and range are given in parentheses for parameters estimated in the original studies; an asterisk indicates derived parameters. Parameter Wise et al. [263] Forrest et al. [76] Body weight N/A 70 (16.8; 24–91) kg V 1 8.48 ∗ L .69 (.18; .2–1.2) L kg −1 V 2 25.36 ∗ L .51 (.17; .2–2) Cl T 34 (5.28; 29.4–49.9) L hr −1 17 (7.48; 4.4–37) L hr −1 1.73 m −2 Cl D 24.09 ∗ L hr −1 38 (9.12; 16–64) L hr −1 1.73 m −2 k el 4.01 (1.6; 1.0–5.5) hr −1 .35 ∗ hr −1 (for 70 kg BW) k 12 2.84 (.52; .4–3.3) hr −1 .79 ∗ hr −1 (for 70 kg BW) k 21 .95 (.27; .5–1.3) hr −1 1.06 ∗ hr −1 (for 70 kg BW) t 1=2; .24 (0.21; .66–.126) hr 0.3436 ∗ hr α 2.8881 ∗ hr −1 2.0174 ∗ hr −1 t 1=2; 4.0 (0.9; 2.48–4.8) hr 6.5 (3.24; 1.6–22) hr; 3.73 ∗ hr β 0.1733 ∗ hr −1 0.1857 ∗ hr −1 A/D 0.0841 ∗ L −1 0.0108 ∗ L −1 B/D 0.0337 ∗ L −1 0.0099 ∗ L −1 [152, 125]. The MIC for Enterobacter and Klebsiella species is typically an order of magnitude lower [115], and therefore resistance is less of an issue in these bacteria. Werequiresomeexplicitmodelforcytotoxicitytoincorporateantibioticsintoourmodelframe- work. The E max and sigmoid E max , while originally applied to the beta-lactam anitbiotics, have also been applied to the fluoroquinolones. For example, Jumbe et al. [125] fit a sigmoid E max model to levofloxacin in the mouse, but the model parameters clearly can not apply in man (e.g., the estimated EC 50 is far too high). Hyattandcolleagues[114]directlyestimatedgrowthandkillratesin S. pneumoniae, S. aureus, and P. aeruginosa, and we use this data to fit modified sigmoid E max models of the form: dB dt =rB−k max B ( (C/M) CM 50 +(C/M) ) , (7.129) where C(t) is the drug concentration and CM 50 is the drug:MIC ratio of half-maximal killing; r is 251 the growth rate M is the MIC, both of which were estimated by Hyatt et al. [114]. Casting the killing rate in terms of the drug:MIC ratio is useful because it allows predictions to be made about theeffectoftreatmentonresistantorganismswithouthavingtoestimatestrain-specificparameters; this is explored in the following section. This model can be proposed on empirical grounds, i.e. it fits the data reasonably well, or it may be derived from the standard sigmoid E max model by assuming antibiotic-mediated killing and natural growth to be balanced at the MIC, as in several works, e.g. [152], and solving for EC 50 , giving EC 50 =M ( k max −r r ) 1= . (7.130) Now, substituting this expression into the standard sigmoid E max model, we obtain dB dt =rB−k max B ( (C/M) kmax−r r +(C/M) ) , (7.131) and thus we have the hidden dependency, CM 50 = ( k max −r r ) 1= . (7.132) Therefore,givenr,k max ,andκ,wehaveaderivedestimateofCM 50 . Wehaveempiricallyestimated k max , κ, and CM 50 from the data of Hyatt et al., but augment these data-sets with a point where the dose equals the MIC and the kill rate equals the growth rate. Hyatt et al. [114] observed a paradoxical decrease in kill rate for very high ciprofloxacin doses (35–60 times the MIC); this may be due to inhibition of RNA synthesis at very high doses. However, while this effect has been observed in vitro, it has not been described in vivo [114], and in any case, such high factors of the 252 MICcouldonlyrarelyandverytransientlybeachievedclinically. Therefore, weexcludethelargest few doses from parameter estimation for S. aureus and P. auruginosa. This is indicated in Figure 7.31, which shows the augmented Hyatt et al. data and modified sigmoid E max model fits. Estimated modified sigmoid E max parameters are reported in Table 7.3. Under out fits, the near maximal drug effect is achieved at about 40 times the MIC for S. pneumoniae and S. aureus, and at about 25 times the MIC for P. auruginosa, consistent with ciprofloxacin being a largely dose-dependent drug. We also note that κ ≈ 1 for the former two bacteria, but κ ≈ 2 for P. auruginosa. The derived CM 50 values given by Equation 7.132 and the empirical CM 50 values reported in Table 7.3 agree extremely well for S. aureus and S. pneumoniae at 2.53 vs 2.45 and 1.65 vs. 1.64, respectively, and reasonably well for S. pneumoniae, at 2.44 vs 3.07, supporting the given derivation as a reasonable physical basis for our modified E max model. HyattandcolleaguesalsoestimatedsigmoidE max modelsforseveralbacteriaandseveraldrugs, including ciprofloxacin, in a following work [115]. Unfortunately, details on most models were not published,andthustheworkisoflimitedusefulnesstous. E max modelsfortheeffectofciprofloxacin on P. auruginosa and S. aureus were estimated by Delacher et al. [54] and Campion et al. [31], respectively. Inthelatterwork,separateparameterestimatesforsensitiveandresistantpopulations are given, and the original works may be consulted for more details. Gr´ egoire and colleagues [91] also studied ciproflaxin and resistance in P. auruginosa using the sigmoid E max model; parameter estimates between this work [91], Campion et al. [31], and those we have determined from [114] all vary appreciably. 253 Figure 7.31: Modified sigmoid E max model fits (red lines) to the (augmented) data of Hyatt et al. [114] (blue dots) for three bacteria species. Drug concentration is given as a factor of the MIC for each species. Data points marked by a black “x” are excluded from the fit, as they occur at very high drug concentrations where a paradoxical decrease in killing occurs that is unlikely to be important in vivo (see text). Data fitting employs a Nelder-Mead simplex search to minimize the mean-square-error. Table 7.3: Estimated k max , CM 50 , and κ for ciprofloxacin killing of the three species studied in [114]. Also given are the growth rates, r, and MICs as determined in [114]. Species r (hr −1 ) MIC (µg ml −1 ) k max (hr −1 ) CM 50 (× MIC) κ S. pneumoniae .74 .41 2.8952 3.0681 1.1984 S. aureus .71 .38 2.6651 2.5279 1.1315 P. auruginosa .79 .41 2.8921 1.6542 1.9819 254 7.5.2 Antibiotic therapy in the well-mixed setting Fluoroquinolone treatment under “complete immunodeficiency” As a prerequisite to studying antibiotic treatment accompanied by an active immune system, we examine how dosing affects both overall population control and emergence of resistance in the ab- senceofanimmuneresponse. Weuseatwo-compartmentpharmacokineticsmodelforciproflaxacin distribution and a sigmoid E max pharmacodynamics model. The pharmacokinetics portion uses parameter values derived from Forrest et al. [77] as reported in Table 7.2, and bacterial killing parameters are determined from Hyatt et al. [114] as discussed in Section 7.5.1. The 24-hour AUC/MIC metric has been widely considered the best predictor of efficacy. As demonstrated in Figure 7.32, our model confirms the metric’s usefulness, but for constant pharmacokinetic param- eters, AUC is principally related to total dose and not to the infusion schedule. For a fixed total dose, more frequent dosing gives better bacterial control, as shown in Figure 7.33. Now, we allow pharmacokinetic parameters to vary randomly, letting body weight, V 1 ,V 2 ,Cl T , and Cl D vary uniformly over the mean plus and minus one standard deviation; we also let the MIC vary by plus/minus 50%. As shown in Figure 7.34, running 25 realizations for each dose (and for the infusion schedule fixed at q12h), demonstrates the 24-hour AUC/MIC to be a far better predictor of efficacy than total dose under modest PKPD parameter variation. We have performed a second experiment, also displayed in Figure 7.34, where we vary the MIC to be one, two, four, and eight times the baseline MIC for each dose, and then compare bacterial killing against total dose and the AUC/MIC. It is also apparent from Figure 7.34 that even moderate resistance, as measured by the MIC, dramatically reduces treatment efficacy and necessitates much higher doses for bacterial control. 255 Figure7.32: Shownisthebacterialpopulation(onalogscale)following24hoursoftreatmentversus the total dose delivered (lower panels) and 24-hour AUC/MIC (upper panels). Pharmacokinetic parameters and the MIC are constant; results are shown for infusion schedules of q24h, q12h, q8h, and q4h (total dose delivered is constant). Points with the same shade represent the same total dose. The least efficacious schedule is q24h, while the others are nearly comparable. The left and right sides of the figure are results for S. aureus and P. auruginosa, respectively. 256 Figure 7.33: Simulated serum ciprofloxacin time-courses for a total 800 mg dose fractionated into a single, two, three, or six 15 minute infusions. Each ciprofloxacin time-course is accompanied by the resulting S. aureus and P. auruginosa levels. Now, weexaminetheemergenceofresistance. Thefrequencyofmutationsconferringresistance is roughly between 10 −6 and 10 −10 mutations per cell division, with 10 −6 −10 −8 the more typically reported range [125]. Campion et al. [31] found ciprofloxacin resistance in MRSA S. aureus (MIC = 0.5 µg ml −1 ) to occur at a lower frequency in the presence of 2 µg ml −1 than 1 µg ml −1 , and this is consistent with our discussion of the MSW hypothesis. We employ the following model: dS dt = r S S ( 1− S+R Λ ) −k S ( (C/M S ) EC 50 +(C/M S ) ) , (7.133) dR dt = r R R ( 1− S+R Λ ) −k R ( (C/M R ) EC 50 +(C/M R ) ) , (7.134) where M S and M R are the MICs for sensitive and resistant bacteria, respectively, and, similar to [152], sensitive bacteria stochastically transfer to the resistant compartment according to a Poisson processwithrateµS(1−S/Λ). Thistwo-populationmodelmaybeextendedtoconsideranarbitrary 257 Figure7.34: Bacterialloadafter24hoursoftreatmentisplottedagainsttotaldoseandAUC/MIC. The left panels give results when PKPD parameters vary randomly, as detailed in the text. The rightpanelsshowresultsforconstantpharmacokineticparamaters,butwithfourdifferentbacterial strains lumped together, with MICs of one, two, four, and eight times the baseline MIC. It is apparent that AUC/MIC is a consistent predictor when multiple strains are considered and that even modest differences in MIC dramatically affect treatment efficacy. The infusion schedule is alwaysq12, andresultsareshownonlyfor S. aureus specificpharmacodynamicparameters(results for P. auruginosa are extremely similar). 258 number of resistant sub-populations, R 1 ,...,R N , where M i > M j for i > j, and sensitive bacteria transfer from the sensitive compartment to each resistant compartment R i with stochastic rate µ i S(1−S/Λ), where µ i <µ j for i>j. When there is no fitness cost to resistance, i.e. r R =r S , then treatment always creates selective pressure that favors the resistant strain, and suppressing resistance depends purely on choosing a dosing regimen that can suppress the resistant strain. Suppose now, as sometimes observed, e.g. [31], that there is a modest fitness cost to resistance, and let r R = ϵr S , ϵ < 1. Somewhat surprisingly, this alters but little the basic dynamics of the system. Even if there is a steep cost for very little resistance such that r R = .5r S and M R = 2M S , a total daily dose as low as 25 mg amplifies resistance and eventually leads to complete invasion by the resistant strain. Over a small range of intermediate doses insufficient to eradicate either strain, we observe coexistence, a behavior not admitted when r R =r S (and non-zero drug treatment). At larger doses the resistant strain can invade, while sufficiently large doses eliminate all bacteria. In short, the only effect of a fitness cost to resistance is to introduce coexistent over a narrow parameter range. RecallingtheMSWhypothesis, thisconfirmsthepredictionsbysomeauthors, e.g. Lipsitchand Levin [152], that intermediate antibiotic levels maximally amplify resistance. However, it suggests that the lower bound of the MSW is extremely low even when the fitness cost to resistance is quite high, and it is in fact zero if there is no fitness cost. Fluoroguinolone treatment in the presence of an immune response The parameter set for the immune response module can be divided into two broad categories: im- munocompromised and immunocompetent. Under the former, all initial bacteria inocula lead to overwhelming infection, while under the latter only large inocula cause overwhelming infection and 259 Figure 7.35: Minimum daily doses for complete bacterial clearance after three days of treatment under different schedules, using baseline pharmacokinetic parameters. The left graph gives results for S. aureus and the right is for P. auruginosa. In the immunocompetent case, once-daily dosing is best, while in the immunocompromised state more frequent infusions are preferred. smaller populations are spontaneously cleared. We find treatment to interact with the immune response in a qualitatively different manner under these two categories of response. Furthermore, treatment efficacy is time-dependent, even in the well-mixed setting considered here. Early treat- ment is better, with the relative benefit to early treatment greatest in the immunocompetent host. Basicresults. Sincetreatmentisonly(strictlyspeaking)necessarywhentheimmuneresponse fails, we study model dynamics for an initial infection of 10ˆ8 organism ml −1 , which leads to overwhelming infection regardless of the immune status. Treatment is initiated at either 24 or 72 hours, and continued for several days. We first observe that, unsurprisingly, the immune response synergizes with treatment such that the stronger the immune response, the lower the daily dose required to clear infection. This is demonstrated in Figure 7.35, which gives the minimum dose to clear infection within three days for three levels of immune response. 260 Figure 7.36: The effect of treatment on bacteria differs qualitatively between immunocompetent and compromised patients. Results are shown for S. aureus pharmacodynamic parameters; results are similar for P. auruginosa. Qualitative differences between immune response types. The effect of treatment on the bacterial load varies qualitatively depending upon whether the immune response is competent, compromised, or absent. Let us initially restrict our discussion to treatment initiated at 24 hours following infection. In the case of an immunocompetent host, if treatment is sufficient to push the bacterial population below a certain threshold (around 10 7 cells ml −1 ), the immune response rapidly clears the remaining organisms. Large, infrequent dosing best accomplishes this goal, and, beyond a threshold dose, there is almost no dependence on total daily dose and time to bacterial clearance. For the immunocompromised host, dynamics more closely resemble those seen when there is no immune response at all than when there is a competent response. Bacterial clearance is much more gradual, with no sharp, sudden clearance dynamic. However, even the compromised response hastens killing somewhat below a threshold bacteria population. More frequent dosing is preferred 261 in this case, and time to clearance is clearly related to the daily dose delivered. Figure 7.36 demonstrates time-series of the bacterial population under the three levels of immune-response considered under nine different dosing regimens. The time to complete clearance as a function of daily dose and dosing interval for immunocompetent and compromised responses is given in Figure 7.37. Time of treatment initiation. We have previously observed our model to predict a rela- tively immunosuppressed state to occur 24 to 48 hours after the initial infection that persists for several days; we now show that this affects treatment efficacy. Compared to initiation at 24 hours, treatment initiated 72 hours following initial infection is associated with a significantly longer time to bacterial clearance, as demonstrated in Figure 7.37. However, the minimum daily dose to ulti- mately clear an infection is only very slightly increased, with the most significant increase seen for infrequent dosing in the immunocompetent host. The relative advantage of infrequent dosing in the competent host is also reduced. As discussed in a moment, time-to-treatment also affects the emergence of antibiotic-resistant strains. AUC/MIC with immunity. We have observed the 24-hour AUC/MIC to be a reasonably good predictor of efficacy in the absence of an immune response, as shown in Figures 7.32 and 7.34. We have replicated the first experiment shown in Figure 7.34 (left side) but in the presence of an immune system. As demonstrated in Figure 7.38, when the immune response is strong and treatment is initiated early, there is a sharp division between very effective and ineffective dosing regimens; if treatment is initiated later, this division is less sharp. When the host immunity is compromised, therelationbetweenAUC/MICandoutcomeisessentiallythesameaswhenthereis no immunity. In sum, this supports the idea that there is a threshold effect in treatment response for immunocompetent, but not immunocompromised patients. 262 Figure 7.37: Time to bacterial clearance, as a function of total dose and fractionation schedule, for immunocompetent(leftpanels)versusimmunocompromised(rightpanels)patientswhentreatment is initiated at either 24 (top) or 72 hours (bottom) following infection. When treatment is initiated early, there is a clear advantage to infrequent dosing in the immunocompetent patient, while the reverse is generally true when immunity is compromised. A delay in treatment increases the time to clearance; the change is more marked in immunocompetents hosts, in whom delay also mostly abolishes the advantage to infrequent dosing. Figure7.38: Bacterialloadafter24hoursoftreatmentisplottedagainsttotaldoseandAUC/MIC. The left two sets of results are for an immunocompetent patient with treatment initiated at 24 hours and 72 hours, while the rightmost is for an immunocompromised patient treated at 24 hours (results when treating at 72 hours are very similar). The earlier treatment is initiated in the immunocompetent host, the greater the AUC / MIC threshold effect. 263 Figure 7.39: Minimum ciprofloxacin doses needed to suppress resistant mutant overgrowth for immunocompetent and compromised patients, with treatment initiated at either 24 or 72 hours followinginfection. ResultsareshownforasingleresistantstrainwiththeMIC,M R ,somefactorof theMICforsensitivecells,M S ,termedthe“relativeMIC.”ArelativeMICof1indicatesnoresistant strain, andgivestheminimumdoseneededtosuppresssensitivecellsonlyforcomparison. Immune status is profoundly influential, with immune compromise associated with extreme vulnerability to highly resistant strains, and the minimum dose increases linearly with relative MIC. Time to treatment also increases the necessary dose, but to a much lesser extent. Resistance suppression. When no immune response is present and resistant mutants are relatively frequent, dosing must be tailored to suppress the resistant strain. We have performed a setofnumericalexperimentstodeterminehowtheimmuneresponseaffectsthedailydosenecessary tosuppressmutantswithvariousrelativeMICs. Ingeneral, acompetentimmuneresponsestrongly suppresses resistance, while the immunocompromised case is similar to the immune absent case. The earlier treatment is initiated, the lower the dose needed to suppress resistance. These results are summarized in Figures 7.39 and 7.40. Figure 7.41 also gives sample time-series of the sensitive and resistance populations when treatment is initiated at different points. Opportunistic infection. We consider superinfection with an opportunistic pathogen, i.e., 264 Figure 7.40: Shown is the minimum ciprofloxacin dose needed to suppress resistance at 24 and 72 hours for immunocompetent patients only. That is, the top row of Figure 7.39 is replicated here for better spatial resolution. Figure 7.41: Example time-series of sensitive (B) and resistant (R) cells (relative MIC of four) sub- jectedtotreatment(250mgday −1 ,q12h)atdifferenttimes,inthepresenceofacompetentimmune response. Treatment at 24 hours handily eliminates both strains. At 60 hours, an intermediate phase is reached, where resistant cells are nearly eliminated, but manage to recover their numbers in the face of ongoing immunosuppression. By 72 hours, the fall in sensitive cells is mirrored by a commensurate rise in resistant cells. 265 one that the host is normally competent to clear. This is modeled through the addition of a second bacterium,B 2 (t), which is killed by phagocytes at maximum rate γ 2 ;B andB 2 act as competitive inhibitors of phagocyte-mediated killing. Macrophage activation by B(t) and B 2 (t), however, is assumed to be synergistic, i.e. we have additive Michaelis-Menten activation terms. Furthermore, weassumethatthetwospeciesmayhavedifferent,butnotentirelydissimilarresourcerequirements that they compete over, and so we have the natural growth of each species affect the other via competition coefficients c 12 and c 21 , in the style of the competitive Lotka-Volterra equations. We perform the following numerical experiment: 1. 24 hours following an initial infection that leads to bacterial saturation, treatment destined to be curative is initiated. 2. Every six hours thereafter, the system is challenged with a small (10 4 −10 6 cells ml −1 ) B 2 inoculum that would easily be cleared in the resting host. The opportunistic infection is assumed to be insensitive to treatment. 3. The peak opportunistic pathogen load resulting at each challenge time-point is recorded. As shown in Figure 7.42, during treatment and before clearance of the primary pathogen, the host is extremely vulnerable to opportunistic infection. The immunocompetent host is, as predicted, less vulnerable, but only marginally. However, we must be careful with these terms, as competent and compromised, as used thus far, only refer to the host’s ability to fight the primary infection. If, on the other hand, we make the immunocompromised patient globally immunocompromised, i.e. less able to fight opportunistic infections (but still capable in the naive host), then we find that immunocompromise dramatically increases vulnerability to opportunistic pathogens. 266 Figure 7.42: Relative peak population of an opportunistic infection, as compared to the peak when the naive host is challenged by a similar inoculum, as a function of time after treatment initiation for the primary pathogen. Points outside the visible range correspond to overwhelming infection. Results are shown for an immunocompromised host; results in the immunocompetent host are qualitatively similar, but with somewhat lessened vulnerability. 267 Vancomycin treatment Model. Given its importance in treating S. aureus, we briefly consider a simplified model for vancomycin’seffectonbacteria,whereinweconsidervancomycintoactstrictlyasatime-dependent drug. Vancomycin’s pharmacokinetics may be described by a two-compartment model; Thomson et al. [237] give V 1 = 0.675 L kg −1 , V 2 = 0.732 L kg −1 , Cl T = 2.99 L hr −1 , and Cl D = 2.28 L hr −1 , from which the microscopic rate parameters may be derived according to Equations 7.115 and 7.116. Nielsen and colleagues [195] fit a sigmoid E max model for the effect of vancomycin upon S. pyogenes in vitro, estimating r = 1.2 hr−1, k max = 1.4 hr−1, and EC 50 = .38 µg ml −1 under κ = 20. This is consistent with a weak bacteriocidal effect [62], and the extremely high sigmoidicity factor approaches an all-or-nothing effect; in fact, κ = 20 was the lowest value consistent with the data, and Nielsen et al. considered κ > 50 more likely. Thus, an all-or-nothing model fits in vitro data over the course of 24 hours of treatment with a constant dose, but cannot describe vancomycin’s post-antibiotic effect, known to be important and related to the AUC/MIC [62]. We use the following simple model in the absence of an immune response: dB dt =rB ( 1− B Λ ) −k max H(C−M), (7.135) whereH(·)istheHeavisidestepfunction,M istheMIC,anddrugconcentrationC isdeterminedby a two-compartment model. It is straightforward to extend this basic model to consider resistance, as in Equations 7.133 and 7.134. Vancomycin also binds to serum proteins at about a 50% level, and has variable penetration into tissue, with poor penetration into the lung in particular [220]. These properties make serum 268 drug level a poor surrogate for that in tissue, but we use it anyway as a qualitative approximation that clearly underestimates the absolute dose value needed for bacterial clearance. Results. The time-dependent model for vancomycin yields several predictions that vary from thosefortheconcentration-dependentfluoroquinolones. Differencesandsimilaritiesaresummarized as follows: 1. It is self-evident from the model construction that of classical pharmacodynamics indices, T > MIC is the best predictor of efficacy, rather than AUC/MIC. 2. Infrequent vancomycin dosing is not predicted to be superior when the host is immunocom- petent, contra our results for fluoroquinolones, and more frequent dosing is uniformly better at minimizing total dose, minimizing time to clearance, and resistance suppression. However, the absolute differences in these metrics between dosing schedules are minimal. 3. Resistance is far more likely to emerge in the immunocomromised host, as with the fluoro- quinolones. 4. Early vancomycin treatment decreases time to bacterial clearance and better suppresses re- sistance; this is also seen with the fluoroquinolones. In sum, the optimal dosing strategy for vancomycin is predicted to vary somewhat relative to the fluoroquinolones, but the basic dynamics of synergism between treatment and the immune responseandthedramaticdifferenceseeninresistancesuppressionbetweenimmunocompetentand compromised hosts is preserved. The prediction that early treatment is superior is also preserved. 269 7.6 Spatial and geometric models A fundamental limitation of the previous analyses is that they treat the infected organ as a single unit, with infection uniformly spread throughout its entirety. This is clearly unrealistic, and it is essential to determine how explicit consideration of spatial geometry might change our previous predictions. To that end, we develop several models with explicit spatial structure. Overall, we have found that the spatial aspect does not significantly affect the qualitative model dynamics, indicating that the well-mixed approximation is a reasonable one. 7.6.1 Reaction-diffusion framework ThemostnaturalandcommonmethodforextendingODEmodelstothespatialdomainiswithpar- tialdifferentialequations(PDE)basedreaction-diffusionmodeling. AlltheODEmodelspreviously considered, excepting the delay differential formulations, can be translated directly into PDE form, but with the additions of diffusive and chemotactic cell migration and cytokine diffusion. PDEs have been fairly widely used in ecological applications [109] and mathematical modeling of tumor growth (see Nagy [189] for a partial review), beginning with Greenspan’s classical model of tumor spheroid growth [90], and there also exist several PDE models of tumor-immune interaction, e.g. [67]. Numerous biomedical applications can also be found in Murray [187] and references therein. There exists, however, a paucity of PDE models for immune-bacteria interaction in general, and sepsis in particular. The general conservation equation for some species u(x,t) is ∂u ∂t =∇·J +σ(x,t), (7.136) 270 where J is the flux, and σ represents any source or sink terms. If the flux is proportional to the species gradient, i.e. linear diffusion with J = D∇u, where D is the diffusion constant, we have the reaction-diffusion equation ∂u ∂t =D∇ 2 u+σ(x,t), (7.137) which in one dimension reduces to ∂u ∂t =D ∂ 2 u ∂t 2 +σ(x,t). (7.138) The linear diffusion term can be considered either a consequence of Fick’s first law, or may be derived from random walk theory. While extremely useful, the random walk derivation is slightly problematic mathematically, as u(x,t) becomes instantaneously non-zero at all points for t > 0. Cellularmotilityinphagocytesandbacteriahasabiasedrandomwalkcharacteristic[39],wherethe bias is induced by a chemotactic gradient; this may be modeled in the continuum approximation as a linear diffusion term summed with a chemotactic term: ∂u ∂t =D∇ 2 u+∇·(χ∇c)+σ(x,t), (7.139) where c is the chemotactic factor and χ is the chemotaxis coefficient. While very useful, the PDE formulation must be implemented on a fairly fine spatial grid for numerical accuracy and stability. Thisisextremelycomputationallyintensiveand,forourpurposes,afinedegreeofspatialresolution is unnecessary. Therefore, we borrow the idea of a multi-patch spatial geometry widely used in ecology and epidemiology, but use the diffusion equation and cell motility parameters to determine transition rates; this is presented in Section 7.6.3. 271 As discussed in the following section, the diffusion coefficient can be expressed in terms of physical cellular motility parameters, under a random walk interpretation. Employing the random walk idea, some authors have suggested that search efficiency, rather than handling time, is the limiting factor in bacteria-phagocyte dynamics. This could have significant implications in the choices of functional form that form the foundation for our ODE-based models (see Sections 7.4.1 and 7.4.2). Therefore, we have implemented an agent-based representation of bacteria-phagocyte interaction and find that a Holling Type II functional form for phagocyte bacteriocidal activity is strongly supported by the basic physics of the system. 7.6.2 Cell motility Asignificantliteraturefromthe1980sand1990sisdevotedtoexperimentalandtheoreticalestima- tion of bacteria, macrophage, and neutrophil diffusion and chemotactic coefficients. The random motility coefficient, µ, analogous to the diffusion coefficient, can be expressed in terms of basic cellular motility parameters [204], namely the single cell speed, ρ, persistence time, τ, and the index of directional persistence, ψ d , as µ= τρ 2 n d (1−ψ d ) , (7.140) where n d is the dimensionality of the system. The persistence time is the time before the cell switches directions, and in bacterial sometimes the reciprocal λ, referred to as the tumbling fre- quency, is used [204]. The index of directional persistence, ψ d , measures the correlation between the previous direction of travel and the new one, and we typically set it to zero. The chemotactic index (CI) is defined as the ratio of net distance traveled up a chemotactic gradient to the total 272 distance moved, and has been found to be on the order of 0.6–0.9 for neutrophils and macrophages responding to markers of bacterial infection [74]. AseriesofworksbyFisher,Lauffenburger,andCharnick[73,74,39,38]modeledtheinteraction ofmacrophagesandbacteriainthetwo-dimensionalgeometryofthealveolus[73,74]andneutrophil- bacteria interaction in three-dimensions [39]. These works generally concluded that the CI must be greater than 0 for efficient bacterial clearance, i.e. purely diffusive motility does not give high enough macrophage-bacteria encounter rates. Furthermore, they predict that time to encounter, rather than handling time, is the limiting factor. However, these works generally considered the bacterialpopulationtobefixed,andextrapolatedfromsingle-phagocytesingle-bacteriumdynamics to multi-phagocyte multi-bacteria dynamics. Agent-based model for motile cells Wehavedevelopedanagent-basedmodeltoindependentlytestthepredictionsofLauffenburgerand colleagues. In the limited scenario of (randomly) uniformly distributed bacteria and macrophage populations, our major findings are 1. Bacterial motility dramatically affects clearance: clearance of even a minimally motile popu- lation is orders of magnitude faster than clearance of a fixed population. 2. Single-cell single-target dynamics cannot be taken as a surrogate for either single-cell multi- targetormulti-cellmulti-targetdynamics. Contacttimeinthesingle-cellsingle-targetsystem is not informative of contact time in the multi-agent scenarios. 3. When bacteria are motile, chemotaxis has little effect in the spatially well-mixed scenario. This is expected, as bacterial persistence time is about two orders of magnitude lower than that of phagocytes, and any change in phagocyte direction will increase the probability of 273 contact only very transiently. 4. Contact time is likely not the limiting factor for bacterial clearance, contra the results of Lauffenburger’s group. Rather, handling time clearly limits bacterial clearance above small bacterialdensities. ThishelpslendcredencetoourpreviousHollingtypeII/Michaelis-Menten kinetic models. 5. It can be inferred that the major importance of chemotaxis is to attract immune cells to the siteofinfection,ratherthanguidecellstoparticulartargetswithintheinfectedsite(assuming a spatially well-mixed site). Modelconstruction. Themodelplacesindividualagentsonaworld-gridofU×V orU×V×W grid-points for a two- and three-dimensional geometry, respectively, with a spatial resolution of 1 µm. The agents considered in the base model are activated macrophages and bacteria. We later extend the model to consider resting and activated macrophages as distinct populations. Each agent has a center point and a circular (spherical) mask, which indicates how much space is filled by the agents. We assume all cells are circular (spherical); macrophages have a mask radius of 10 µm, and bacteria have a radius of 1 µm. If, at any time-point, the masks of a macrophage and bacterium overlap, the macrophage ingests the bacterium with probability p I . Eachagenttravelsinoneoffour(six)directionsinthetwo-(three-)dimensionalgeometry,atrate ρ x (µm s −1 ), where the subscript denotes the cell type. Changes in direction are assumed to occur viaaPoissonprocess, andsotheinter-changetimeisexponentiallydistributedwithrateparameter λ x =1/τ x . Eachtime-step, theprobabilityofachangeindirectionisgivenby1−exp(−hλ), where h is the step-size. Each time-step, every agent moves d=hρ x µm. If d is non-integer valued, then d− mod (d,1) µm are traversed, and an additional µm is traveled with probability mod (d,1). 274 World boundaries are reflecting, with agents reflected back into the world-grid by the amount they would have overshot the boundary, and their direction of travel is reversed. Ingeneral, everybacteriumthatcontactsamacrophageshouldnotbeinstantaneouslyingested, but we expect some handling time on the order of one to several minutes. In addition, there is some upper limit to the number of bacteria that can be processed by a single macrophage within a given time period. A simple approximation of the former behavior is to introduce a refractory time, r T , following each phagocytic event, within which further phagocytosis is impossible. The latter behavior can be subsumed into the refractory time, based on the experiments of Leijh et al. [144], who found maximal phagocytosis of bacteria by monocytes to be 60 and 83 cells per hour, for S. aureus and E. coli, respectively. Bacteria are assumed to proliferate at per-capita rate r, and the probability of cell division for any given bacterium at each time-point is given by 1−exp(−hr). The new bacterium is placed at the closest grid-point to its parent as is possible. Figure 7.43 gives examples of 2-D worlds, and Figure 7.44 shows a 3-D world. Parametrization. Phillips et al. [204] have collated multiple experimental estimates for the random motility coefficient, µ, with most estimates on the order of 1×10 −6 −10× −5 cm 2 s −1 . Most estimates are for E. coli species, but estimates for Salmonella and Pseudomonas species (µ≈6×10 −5 ) are on the same order of magnitude. These diffusion coefficients are quite high and are comparable to those of small molecules in tissue. Phillips et al. [204] estimated µ = 2.6±.4×10 −6 cm 2 s −1 , with a cell speed of ρ = 24.1±6.8 µm s −1 at approximately 25C. Persistence times were roughly exponentially distributed, with a meanτ =.81s. Theseestimatesareconsistentwiththoseformultiplebacterialspeciesbymultiple authors, with ρ ranging from about 15 to 40 µm s −1 , and τ typically on the order of 1 second. 275 Figure 7.43: The left shows a 1000×1000µm 2-D world with 10 activated macrophages (blue) and an initial population of 1000 bacteria (red). The left shows the scaled-down geometry of a single macrophage and 100 initial bacteria. Figure 7.44: Example of the 3-D model on a 100× 100× 100 µm grid with a single activated macrophage and 100 bacteria. 276 Lowe et al. [155] have also observed cell speed to increase linearly with temperature. In in vitro assays, alveolar and peritoneal macrophages have a cell speed of about 2–3µ min −1 , and Fisher et al. [74] measured a persistence time for alveolar macrophages of 30–40 minutes, althoughCharnickandLauffenburger[39], drawingonseveralsources, givearangeof1–10minutes for alveolar macrophages and a range of 1–5 minutes for tissue neutrophils. Neutrophils are also somewhat faster, with a cell speed of 2.5–7.5 µ min −1 reported in [39], and stimulated neutrophils have been reported to achieve speeds of 10–20µ min −1 [86, 182] . These values are consistent with estimatesofmacrophageandneutrophildiffusioncoefficients, whichareontheorderof10 −9 −10 −8 cm 2 s −1 [201, 182, 86]. It should also be noted that undirected cell motility clearly changes with the composition of the tissue matrix, and with the concentrations of various cytokines and small molecules [201, 182, 86]. For example, bacterial speed increases with serine concentration [204], and the random motility coefficient for neutrophils in three-dimensional gels has been found to vary from 1.6 to 13.3×10 −9 cm 2 s −1 withgelcollagenconcentration[201]; motilitypeaksatmid-lowcollagendensity. Mogheet al. [182] similarly found biphasic dependence of random neutrophil motility on IL-8 concentration in a fibrin gel assay, with a peak µ of 11×10 −9 cm 2 s −1 at 5×10 −8 M IL-8. Results. We present all results with a non-proliferating bacterial population, i.e. r = 0, as a proliferating population does not qualitatively affect any of our conclusions. Figure 7.45 demonstrates the strong dependence on bacterial motility, with clearance shown as a function of bacterial speed,ρ B . For bacterial motility within the experimentally determined range, we find the magnitude of macrophage motility to have little effect on clearance. Time to contact is the limiting factor with respect to bacteria killing for very small pathogen densities,buthandlingtimeismuchmoretypicallythelimitingfactor. Wehaveexaminedbacterial 277 Figure 7.45: The left panel gives curves showing the number of surviving bacteria as a function of time, for different values of ρ B , the bacterial movement speed. Each curve is the mean of 10 simulations. The right panel gives mean bacterial population (plus/minus the standard deviation) aftertwohoursas afunctionofρ B . Resultsaregeneratedonthereduced (315×315)2-Dgeometry with a single activated macrophage and r =0. killing under different initial bacterial and activated macrophage populations, denoted B 0 andM 0 , respectively. The initial rate of killing is a linear function of M 0 and a hyperbolic function of B 0 , as demonstrated in Figures 7.46 and 7.47. These results, obtained with well-mixed but spatially explicitpopulations,supporttheuseofMichaelis-Mentenstylefunctionalformsforbacterialkilling in our ODE models. 7.6.3 Multi-patch model We develop a multi-patch model, where each patch represents a fixed volume, V P , of tissue, this volume is assumed to be a w×w×w cube, and these patches may be interconnected in a one-, two-, orthree-dimensionalsense. Furthermore, wemayonlywishtoconsideraportionofanorgan, In a homogenous patch geometry, each receives blood flow equal to Q/V B , although alterations in local blood flow may be incorporated. For example, endothelial activation increases flow, and 278 Figure 7.46: The left panel gives the number of surviving bacteria as a function of time, for different numbers of activated macrophages, M 0 ; the number of initial bacteria is fixed at 1000, and the 1000×1000 2-D geometry is used. The early part of each curve, highlighted in red, is used to determine the initial killing rate. As shown in the right panel, overall and per-macrophage killing increases linearly with M 0 , consistent with any Holling type functional response for the overall predator-prey dynamics. Figure7.47: Theleftpanelgivesthenumberofsurvivingbacteriaasafunctionoftimefordifferent bacterial initial conditions, B 0 . The number of macrophages is fixed at 10, and the 1000×1000 2-D geometry is used. The initial killing rate is determined from the early part of each curved highlightedin red. Asshownon theleft, theinitialoverallkilling rateincreases hyperbolically with B 0 , consistent with an overall Type II functional response. Note that the clear hyperbolic shape of the curve and the fact that bacteria are driven to extinction rule out a Type III response. 279 injured tissue with a super-imposed infection may have reduced flow. A major advantage of the multi-patch framework is that it allows spatially local alterations in model parameters that yield a persistent nidus of infection. Model framework GivenasetofR modelvariablesunderanODEformulation,C={c i } i=R i=1 , letthesetC j denotethe variables in patch j, where j ∈{1,...,P}; elements are labelled c j i . The general model framework under purely diffusive agent motility follows: dc j i dt =g(C j )− P ∑ k=1 ϕ jk i c j i | {z } emigration + P ∑ k=1 ϕ kj i c k i | {z } immigration , (7.141) where g(·) is a source/sink term that depends only on local variables, and ϕ jk i gives first-order transport from patchj to patchk. For simple diffusion, we haveϕ jk i =ϕ kj i andϕ jk i =ϕ i if patches j and k border each other, and ϕ jk i = 0 otherwise. Patches have two, four, and six neighbors in one, two, and three-dimensions, respectively. It is possible to incorporate anisotropic diffusion by making ϕ jk i a function of the relative positions of patches j and k. We now extend the model to consider chemotaxis. For chemotactic motion only, suppose we have species c i in patch j, i.e. c j i , move in response to a gradient of factor c q . Then, dc j i dt =− ∑ k bordering j ψ i;q c j i ( |∆ jkq | |∆ jkq |+χ i;q ) H(∆ jkq )+ ∑ k bordering j ψ i;q c k i ( |∆ jkq | |∆ jkq |+χ i;q ) H(−∆ jkq ), (7.142) where ∆ jkq =c k q −c j q , (7.143) 280 and ψ i;q gives the maximum first-order transport rate of cell type i out of patch j in response to a gradient of factor c q ; H(·) is the Heaviside step function. The χ i;q term gives the half-maximal concentration of c q for the chemotactic response of species c i , and we assume that χ i;q is perhaps one or two orders of magnitude lower than the Michaelis constant for ligand-receptor binding of cytokine q on cell type i. We determine max{ψ i;q }= s i w , (7.144) where s i is the maximum speed of cell type i, by the following physical argument. If an initial population ofC 0 cells are uniformly distributed across a one-dimensional patch of width w, and all travel to the right at speed s i , then ⌊(C 0 δx)/w⌋ cells exit the patch during time interval δt, where δx=s i δ t . It follows that the absolute change in cell count is approximated by dC dt =− s i w C 0 ≈− s i w C, when C ≈C 0 . (7.145) Asimilarargumentholdsintwoorthreedimensions. Forcellsthatundergochemotaxis,theoverall governing equation is the sum of Equations 7.141 and 7.142. Transport parameter estimation Estimation of the inter-patch transport parameters from the random-walk or diffusion characteri- zation of cell motility is somewhat problematic. We calibrate ϕ for each species using the following method: 1. A 1-D multi-patch geometry is generated with P patches, each patch has a width of w µm, and the total width of the domain is w T = Pw. A single species is considered, with initial 281 conditions of zero everywhere but the center patch. 2. A 1-D PDE diffusion equation is numerically solved (over 24 hours), with constant initial conditions on the interval w T /2±w, corresponding to the uniform population of the center patch. 3. The PDE solution is projected onto the corresponding patch centers, and a Nelder-Mead simplex method is used to optimize ϕ such that the multi-patch solution best approximates the projected PDE solution. 4. The above process is repeated for a range ofw andD values, and a multiple linear regression model is then used to get an empirical estimate of ϕ from log(w) and log(D). Figure 7.48 gives an example PDE solution at 24 hours, its projection onto the patch geometry, and the fitted multi-patch model solution. Figure 7.49 shows the directly estimated ϕ values and thelinearregressionestimates(“fittofit”)asfunctionsofw andD. Theestimatedlinearregression model, with w and D in units of µm and µm 2 s −1 , respectively, is given as log(ϕ)=1.825−4.076log(w)+0.802log(w) 2 −0.074log(w) 3 +0.631log(D)−0.044log(D) 2 +0.012log(D) 3 , (7.146) where logarithms are base-10. Diffusion coefficients for the cellular actors are determined from the motilityparametersinSection7.6.2,anddiffusioncoefficientsforcytokinesintissuearedetermined using the empirical relation [229] D tiss =1.778×10 −4 (MW) −:75 , (7.147) 282 Figure7.48: TheleftpanelgivesthesolutionofthePDEdiffusionequationat24hoursforD =900 µm 2 s −1 over a 10 5 µm domain with no-flux boundary conditions. This solution is projected onto a 21-patch geometry, and ϕ is optimized to fit the multi-patch solution at 24 hours to the PDE- projection, as shown on the left. where 32< MW< 69,000 is the solute molecular weight and D tiss has units cm 2 s −1 . We assume that TGF-β acts solely locally. Results Asymptotic dynamics. The basic dynamics of the multi-patch model do not differ dramatically from the single-path, ODE formulation; we present all results for lung-specific parameters. We still have dependence on bacterial initial conditions: in the immunocompetent host, large initial infections in a single patch can either be limited to that patch or lead to overwhelming infection of the entire lung; small initial conditions lead either to bacterial clearance or low-level chronic infection with oscillations. Examples of the basic dynamics of escape, clearance, and oscillations, in the one-dimensional setting, are given in Figures 7.50, 7.51, and 7.52, respectively. It is not surprising that the results on the multi-patch architecture should often mirror those obtained with 283 Figure 7.49: The left surface gives the directly estimated (i.e. estimated from the procedure shown in Figure 7.48) log(ϕ) as a function of w and D. The right surface shows log(ϕ) calculated from Equation 7.146, determined by linear regression on the first surface, hence the “fit to fit” title. a single patch, as the same dynamical rules are replicated at all points. When overwhelming infection occurs, the rate of spread is determined by the bacteria diffusion rate. Moreover, bacterial motility is identified as a key parameter in determining asymptotic outcome in the immunocompetent host. When a large bacterial inoculum is introduced to a single patch, high motility can lead to unchecked spread throughout the lung, whereas with low motility infection is limited to the initial and a few surrounding patches. Patch-specificimmunosuppressioncanalsohavelung-wideconsequences. Thatis,iftheimmune response cannot efficiently attack bacteria in the initiating patch due to, say, some tissue injury that compromises blood flow, a highly motile bacterium may escape immune control and spread throughout the lung, even when parameters reflect immune competence in the other patches. A less motile bacterium, however, can be controlled and sequestered to the affected patch (along with several immediate patches). We also observe sensitivity to initial conditions in this case, with 284 Figure7.50: Bacterialinvasioninaone-dimensionalmulti-patchgeometrythatisultimatelycleared. Bacteria initially infect the central patch at a density of 10 6 cells ml −1 . The left set of panels gives the spatial profile of bacteria, total activated macrophages, and total neutrophil profiles on a logarithmic scale, at nine different points in time. The right panels give TNF and IL-10 profiles on a logarithmic scale. The initial focus of infection is rapidly cleared, but symmetric waves of bacteria propagate away from it; these too are eventually cleared Figure 7.51: Overwhelming infection spreading from an initial focus throughout the entire one- dimensional multi-patch geometry. The left and right panel sets give logarithmic spatial profiles of the cellular (bacteria, macrophages, neutrophils) and cytokine (TNF and IL-10) profiles. TNF is greater than IL-10 at the leading edge of invasion, but within the infected core IL-10 is dominant. 285 Figure7.52: Ahighlymotilebacterialinfectioncanescapeeradicationbyarelativelystrongimmune responseandspreadthroughoutthelungwithoscillatorydynamics. Spatialprofilesofthisdynamic for cellular actor (left panels) and cytokines (right panels) are given at nine time-points. dynamics partitioning based on initial condition and bacterial motility as follows: 1. Small I.C., low motility: control of infection at low level in initiating patch with spatial sequestration. There is also a low-frequency dynamic of very small, symmetric waves of bacteria propagating from the infectious nidus. These are cleared or held to negligible levels. 2. Large I.C., low motility: bacterial escape to carrying capacity within initiating patch, but spatial sequestration. 3. Small I.C., high motility: same as small I.C., low motility outcome. 4. Large I.C., high motility: bacterial escape both within initiating patch and overwhelming invasion throughout the lung. The three possible outcomes are demonstrated in Figure 7.53. Dynamics are simpler under a globally immunocompromised host: an infection initially limited to a single patch will inevitably spread throughout the entire multi-patch domain. In this case, bacterial motility still determines the rate at which infection spreads. 286 We have also explored antibiotic treatment under the multi-patch formulation. The basic dy- namics mirror those seen in the well-mixed case, with the immune system synergies with treatment such that once the bacterial population is reduced below a critical density, the immune system rapidly clears the residual bacteria. This threshold effect is far stronger in the basically immuno- competent host than in an immunocompromised one. Time-scale. In addition to the possible outcome of spatially localized infection that is con- trolled but not eliminated, the second major difference we observe in the multi-patch versus well- mixed setting is the time-scale that dynamics act over. Since bacteria take time to spread through the geometry, the overall time-course is significantly extended. Figure 7.54 shows the total cellular populations and cytokine levels integrated over the entire lung geometry as a function of time, when bacterial escape occurs, for two different levels of bacterial motility. Cytokines and cellular populations change on the time-scale of days, rather than hours, which is more realistic for in vivo infections. Nevertheless, the basic patterns in total cellular population are similar, if not identical, to those observed under the well-mixed approximation. Conclusions In sum, our exploration of spatial models does not significantly affect our predictions or under- standing of the early innate immune response to infection. Prior work examining cellular motility in lung infection [73, 74, 39, 38] suggested that encounter time is the limiting factor in phagocyte- mediated clearance of bacteria. We have developed a relatively simple micro-scale agent-based model of the process of phagocyte-bacteria interaction which, contrary to these earlier results, in- dicates handling-time as the limiting factor for most bacterial densities. More importantly, this investigation supports the use of a Holling Type II functional response (see Figures 7.46 and 7.47) 287 Figure 7.53: Asymptotic outcomes when infection is initiated in a central patch with compromised immunity: blood flow is set to zero, and γ, the maximal phagocyte killing rate, is 1 hr −1 . We have normal blood flow and γ =10 hr −1 in all other patches. The left panels give the final spatial profiles (after three days) of the cellular actors, bacteria, neutrophils, and macrophages, and the right panels give the integrated TNF and IL-10 time-courses. The top panels give the outcome for a small, low motility (ρ = 1 µm s −1 ) bacterial I.C. that is held to a low level and sequestered; the middle panel shows complete invasion of the initial patch but with spatial sequestration resulting from a large I.C but with low motility, and the bottom panels give a large bacterial I.C. with high motility(ρ=10µms −1 )thatspreadsthroughoutthelung. Moreseriousinfectiousoutcomesresult in both larger absolute cytokine levels (note the scales), and larger final IL-10:TNF ratios. 288 Figure7.54: Theleftandrightpanelsetsgivecellularandcytokinetotalsintegratedoverallpatches as a function of time, under the scenario of complete invasion. The top panel shows the overall time-course of invasion when bacterial is relatively low, with ρ B =1µm s −1 ; the lower panel gives invasion for a more motile bacterium, with ρ B =10 µm s −1 . 289 for phagocyte predation of bacteria; this is a foundational aspect of our model hierarchy. We have also examined bacteria infection on a multi-patch geometry. The major challenge in implementingsuchanarchitectureinthebiologicalsettingisappropriatelydetermininginter-patch transition rates. To that end, have developed a quasi-empirical method to link cellular motility parametersandlineardiffusioncoefficientstointer-patchtransitionrates, givenbyEquation 7.146, that covers the range of diffusion coefficients for both cellular actors and macromolecules, e.g. cytokines. Results on the multi-patch geometry do not generally differ from those generated by the well- mixed ODE geometry. While persistent infection can be restricted to one part of space when one (or several) patches have parameter values corresponding to a compromised immune response, this tends to occur only over a narrow parameter range. Typically, the same asymptotic behavior is approached organ-wide. Interestingly, we have found that when immunity in one patch is compro- mised this can lead to infection spreading through the entire organ, even when the other patches are characterized by normal immunity. The basic agent-based framework we have developed could be used in the future, perhaps in conjunction with a PDE-based description of cytokine diffusion, to evaluate the predictions generated by our predator-prey based hierarchy of models (see Section 7.4), and confirm or refute the prediction that neutrophil recruitment is essential to clearing the reactive immune response. 7.6.4 Systemic model Allmodelingintheprevioussectionsconsidersinfectionatthetissue/organscale. Wenowconsider asimplePBPKframeworkthatconsidersflowthroughcentralblood,lung,andlivercompartments, augmentedbytwo“pseudocompartments,”namely,bonemarrowandmarginatedneutrophilpopu- 290 Figure 7.55: Schematic representation of the PBPK-based system-level model. Bacteria and cy- tokines transfer between liver, lung, and blood compartments via blood flow. Circulating neu- trophils are generated in the bone marrow and transfer between the freeling circulating and marginated pools (independently of blood flow), and extravasate irreversibly into tissue at a rate governed by organ blood flow and the local activated endothelium. The antibiotic treatment mod- ule may also be imposed, but the compartments of antibiotic PK module are unrelated to the systemic PBPK model. lations. Attheorganscale,dynamicsarerepresentedbythemulti-patchmodelpresentedinSection 7.6.3, while the blood, bone marrow, and marginated neutrophils are well-mixed compartments; this architecture is illustrated schematically in Figure 7.55. The spleen is often included in PBPK models, as in [42], and it may be salient in modeling sepsis, as Swirski et al. [230] recently demonstrated that the spleen contains a large reservoir of true, undifferentiated monocytes that are rapidly mobilized in response to myocardial infarction (MI) and presumably other major inflammatory stimuli. Furthermore, splenic monocytopoiesis is a major, and perhaps the dominant, source of monocytes throughout acute inflammation [147]. However, since our ODE model hierarchy suggests little change to the basic dynamics of infection when monocytes are included, we omit monocytes and the spleen from the systemic model for the time being. We consider the following variables within the non-organ compartments: 1. Polymorphonuclear neutrophils (N), denoted N B (t)) within the blood compartment. We 291 also consider two “pseudo-compartments” of bone marrow post-mitotic neutrophils, N PM (t) (cells), and intravascular marginated neutrophils, N M (t) (cells ml −1 ). Note that we consider the absolute cell count, rather than concentration, for post-mitotic neutrophils, N PM (t). 2. Blood-borne bacteria, denoted B B (t) i . 3. Serum TNF, T B (t), and IL-10, I B (t). The blood compartment is connected to multi-patch or single-patch representations of the liver and lung. Within these two organs, any of the previous local-scale models may be employed, and parameter values may be organ-specific. The chief difference between lung and liver is the resting macrophage reserve, J, set at 10 6 cells ml −1 in the lung and 10 8 cells ml −1 in the liver. For the single-patchsetting, wedenoteorgan-specific variableswiththe superscriptsL andH, for lungand liver (hepatic tissue), respectively, and the governing equations for the systemic model components follow dB B dt = Q L V B ( p L B B L − ˆ p L B B B ) + Q H V B ( p H B B H − ˆ p H B B B ) −δ B B, (7.148) dN B dt = k PM (T B ) V B N PM −δ B N N B +k de-marg (T B )N M −k marg N B −N B Q L V B (E L ) ne (E L ) ne +(θ E ) ne −N B Q H V B (E H ) ne (E H ) ne +(θ E ) ne , (7.149) dN PM dt = λ P (T B )−k PM (T B )N PM , (7.150) dN M dt = k marg N B −k de-marg (T)N M −δ B N N M , (7.151) T B dt = T L Q L p L T V B +T H Q H p H T V B −T B Q L +Q H V B , (7.152) I B dt = I L Q L p L I V B +I H Q H p H I V B −I B Q L +Q H V B , (7.153) 292 where λ P (T) = λ P;basal ( 1+5 T K T;P +T ) , (7.154) k PM (T) = k PM;basal ( 1+ T K T;P +T ) , (7.155) k de-marg (T) = 0.7 ( 1+10 T K T;P +T ) , (7.156) k marg = 0.8, (7.157) and the organ-level equations are modified to take into account ingress from and egress to the systemiccomponents. Theparametersp i T andp i I arethecytokinepartitioncoefficientsforTNFand IL-10,respectively,incompartmenti. Theparametersp i B and ˆ p i B havesomewhatdifferentmeanings andgivetheflow-dependentratesofbacterialintravasationandextravasation,respectively,inorgan i. If the liver or lung are given a multi-patch representation, it is straightforward to alter the flow-dependent terms to consider summations of the individual patches instead of the organ as a whole. Beyond simple flow-based cytokine and cell transitions between compartments, neutrophil trafficking is the major model addition at this level, and this aspect of the model is discussed extensively in the following section. Systemic neutrophils The production and trafficking of neutrophils is significantly more complex than as depicted in our model. Physiologically, neutrophils are divided into several physiological compartments: freely circulating, intravascular marginated, tissue, and bone marrow. The marginated pool consists of neutrophils slowly transiting through organ capillary beds, namely the liver, spleen, bone marrow, 293 andpossiblythelung[228]. Thecirculatingandmarginatedneutrophilcompartmentsareofnearly equal size at rest [228]. The half-life of circulating PMNs is on the order of 6–10 hours [272, 157], mediated by neutrophil emigration into tissue and subsequent destruction one to two days later in the mononuclear phagocyte system [228, 157]. Bone marrow dynamics. Within the bone marrow, circulating neutrophil precursors are divided into stem cell, mitotic, and post-mitotic pools [228], and the dynamics of these populations havebeenthefocusofmultiplemathematicalmodel, e.g. [272]andreferencestherein. Transittime within the post-mitotic pool, a reservoir of roughly 6×10 11 fully differentiated mature neutrophils, is 4–6 days [228]. Total daily PMN turnover is estimated at approximately 1.7×10 9 cells kg −1 [228]. Post-mitotic transit time is reduced markedly by G-CSF [210], implying rapid mobilization of this reserve into circulation. G-CSF also expands the mitotic pool and is regarded as the principle regulatorofgranulopoiesis[228]. Whileprofoundlyaffectinggranulopoiesis,G-CSFdoesnotappear to affect circulating PMN half-life or margination [210]. Price et al. [210] found G-CSF treatment in healthy humans of 300 µg day −1 to increase PMN turnover sixfold, and to reduce post-mitotic transit time by over 50%. These dynamics are modeled by assuming a basal rate of most-mitotic neutrophil production, λ P;basal , that may be amplified up to sixfold by TNF receptor binding, which we use as a surrogate for granulocyte colony-stimulating factor (G-CSF), as discussed below. We also assume that post- mitotic cells transition to the blood compartment according to first-order kinetics, with the rate basal rate increased up to two times by TNF binding. Note that the assumption of first-order kinetics, which we use for simplicity, implies an exponential distribution of waiting times, although a Weibull-type distribution would be more realistic (see, for example, Figure 2 of [210]). These 294 considerationsgiveEquations7.150,7.154,and7.155;thebasaldeathrateofcirculatingneutrophils in Equation 7.149, δ B N N B , accounts for background tissue emigration and destruction. TNF and IL-1 both induce G-CSF production in vitro within 6 to 12 hours [146], and Tanaka et al. [234] found serum levels of G-CSF, IL-6, and IL-8 to mirror each other in trauma patients, but the G-CSF time-course did not correspond to that of TNF. It was suggested that these two groups of cytokines, TNF and IL-1 and Il-6, IL-8, and G-CSF, and appear sequentially in sepsis withtheformerinducingthelatter. Sinceitwouldrequiresignificantfurthermodeldevelopmentto explicitly include G-CSF, making λ P andk PM delay functions of TNF would be most appropriate for our current context, but for simplicity we simply use an ODE formulation. This is not likely a major issue, since under the model margination/de-margination dynamics are dominant over the first 12 hours, and only later does bone marrow production drive the neutrophilia, even without an explicit delay. Since IL-10 strongly inhibits G-CSF (along with TNF, IL-1, IL-6, IL-8, and GM-CSF) expression in LPS-activated monocytes [158], it may also be reasonable to have IL-10 inhibit bone marrow neutrophil production. However, since IL-10 injection has also been observed [113] to induce neutrophilia, we omit any consideration of IL-10 at this time. Margination/de-margination dynamics. In contrast to G-CSF, the inflammatory cy- tokines IL-8, TNF, and IL-1, along with LPS, appear to mainly affect neutrophil margination, without directly affecting granulopoiesis per se [236, 243, 242]. Terashima et al. [236] found injec- tion of IL-8 in rabbits to induce a transient neutropenia that passed within 15 minutes, followed by a neutrophilia at 30 minutes. The neutropenia was considered likely due to sequestration of PMNs in the lung, while the neutrophilia was related to mobilization of marginated PMNs within the bone marrow. The neutrophilia peaked at 60 minutes with PMN levels roughly 75% higher than baseline, and was associated with release of PMNs from the bone marrow, but did not affect 295 mitotic or post-mitotic transition times [236]. Similarly, Ulich and colleagues [243, 242], working in the rat, found TNF administration to induce a transient neutropenia (roughly 30 minutes) followed by a more sustained neutrophilia that resolves by 24 hours. The initial neutropenia was likely due to transient margination, pos- sibly within the lung, mediated by a direct and immediate effect of TNF on neutrophil adhesion factors. The later neutrophilia was related to release of bone marrow neutrophils into circulation. Furthermore, TNF induces a biphasic neutrophilia, with the later peak likely due to IL-1 induced itself by TNF [243]. Therefore, in summary we assume the existence of a large marginated PMN population within the bone marrow venous sinusoids in equilibrium with the circulating pool at rest. TNF causes this equilibrium to shift towards the circulating pool. We do not directly address the probable increase in PMN lung margination other than through the activation of pulmonary endothelium by TNF. Since at rest the marginated and circulating pools are nearly equal in size, we have the basal ratesofmarginationandde-marginationat.8and.7hr−1,respectively,togiveaslightbiastowards margination [228]. We assume that the shift to the circulation induced by TNF is driven by an increased rate of de-margination rather than a reduced rate of margination, yielding Equations 7.156 and 7.157. PBPK parameters Most baseline organ weight and perfusion as a fraction of cardiac output (CO) received are de- termined from a review by Brown and colleagues [26], and these values are summarized in Table 7.4. Molina and DiMajo [183] recently assessed normal organ weights in men aged 18 to 35 years, and found a mean spleen weight of 139 g. Using the mean total body weight of 76.4 kg [183], 168 296 Table 7.4: Baseline organ weights as percentage of total body weight and perfusions as percentage of CO. Most values are determined from [26], with the determination of other values, as marked by an asterisk, discussed in the text. Organ Weight (% body wt.) Perfusion (% CO) Blood 7.9 - Gastrointestinal tract 1.7 13.6* Kidneys .4 17.5 Liver 2.6 22.7 Portal flow - 18.1 Hepatic flow - 4.6 Lungs .8 100 Red bone marrow Spleen .18* 4.5* Table 7.5: Systemic model-specific parameters and baseline values. Parameter Description Default value ˆ p L B Flow-dep. rate of bacterial intravasation from lung 10 3 p L B Flow-dep. rate of bacterial extravasation into lung 10 5 ˆ p H B Flow-dep. rate of bacterial intravasation from liver 10 5 p H B Flow-dep. rate of bacterial extravasation into liver 1 δ B Bacterial death rate within the blood 0 hr 1 δ B N Basal death rate of circulating neutrophils .1155 hr 1 p L T TNF partition coefficient between lung and blood 1 p H T TNF partition coefficient between liver and blood 1 p L I IL-10 partition coefficient between lung and blood 1 p H I IL-10 partition coefficient between liver and blood 1 λ P;basal Basal post-mitotic neutrophil production rate 4.375×10 9 cells hr 1 k MP;basal Basal transition rate, post-mitotic to circulating neutrophils .0058 hr 1 K T;P Half-maximal TNF for circ. neutrophil responses 2.93×10 12 M ml/min/100 g for splenic blood flow as determined by Oguro et al. [198], and a cardiac output of 5.2 L/min [26], we arrive at a baseline splenic blood flow of 4.5% of CO. Organ/compartment weightisconvertedtovolumeassumingthedensityofwater, thatis, kilogramsaresimplyswitched to liters. Other parameter values specific to the systemic model are given in Table 7.5. 297 Results We study the case of bacteremia generated by a primary lung infection, with single-patch repre- sentations for both liver and lung. Blood-borne bacteria are assumed to be efficiently taken up by the liver, where they induce a local immune reaction and cytokine production. We assume the liver to be highly competent at eliminating bacteria, so any ongoing bacteremia is the result of immune escape within the lung. Figures 7.56 and 7.57 give time-series for the different bacterial compartmentconcentrations,serumcytokines,andsystemicneutrophilpopulationsunderbacterial clearance and escape, respectively. The liver is by far the dominant source of circulating cytokines; in the absence of liver cy- tokine production, serum cytokines are two orders of magnitude lower and infection induces only a transient and mild neutrophilia. When bacteria escape the immune system, we observe a biphasic pattern to serum cytokine levels (see Figure 7.57). Instead of a single TNF peak followed by an IL-10 peak, we observe two TNF peaks in succession, with the magnitude of the second peak much greater. These are driven by the lung and liver, in turn, and this is confirmed computationally by setting cytokine production in either organ to zero. Hepatic cytokine production. Cytokines produced by the liver affect the bacteria-immune dynamics in the lung, the site of primary infection. We investigate how variations in α H T and α H I affect neutrophilic infiltration in the lung, time to bacterial clearance, and the threshold B 0 for bacterial clearance (versus persistence). Increasing α H T , the intrinsic rate of TNF production by liver macrophages, leads to increased neutrophilic infiltration of the lung and to a corresponding increase in the threshold B 0 for clearance and decrease in the time to bacterial clearance (for fixed B 0 ); the effect of α H T on neutrophil infiltration and threshold B 0 is demonstrated in Figure 7.58. Moreover, hepatic TNF production augments the immune response within the lung regardless of 298 that local response’s “basal competency,” as measured by the maximum phagocyte killing rate, γ (see Figure 7.59). The effects of increasing α H I are generally opposite those of α H T , as expected, but smaller in magnitude. We also find that increasing α H I can damp the peak and integrated serum TNF. Excessive increases in α H I can lead to a paradoxical increase in lung neutrophils and serum TNF, should the excess IL-10 production be sufficient to shift the system from bacterial clearance to bacterial escape. An intrinsically pro-inflammatory orientation of liver macrophages can be considered both ben- eficial and potentially harmful. It is beneficial in that it supports the ability of the lung to recruit neutrophilsandclearinfection. However, theinflammatoryresponseistunedtobemoreexuberant regardless of the infectious insult. That is, the response may be excessive beyond that needed in some cases. However, since for a given α H T the pro- and anti-inflammatory response increases in proportiontoinfectionseriousness,itisnoteasytodeterminewhetheragivenrobustinflammatory response is excessive relative to infection severity. BacterialsheddingandTNFblockade. Weexaminetheeffectofp L B , whichdeterminesthe rate at which lung bacteria are shed into circulation, on cytokine and bacterial clearance dynamics. As the cytokine time-series in Figure 7.60 demonstrate, greater p L B values are associated with increased TNF, IL-10, and a higher IL-10:TNF ratio. This increase in systemic cytokines drives neutrophilicinfiltrationinthelung;theseadditionalneutrophilsfailtocleartheinfection,butcould cause additional tissue damage. This systemic feedback, which is not accounted for in our earlier models, suggests an increased role for anti-TNF treatment when bacteremia is present. Therefore, we have simulated therapeutically blocking TNF in conjunction with antibiotic co-treatment; we do not seriously consider TNF inhibition alone, as this would never be done clinically. 299 Figure 7.56: Example time-series for variables specific to the system-scale model when primary lung infection results in bacterial clearance. The top left panel gives the bacterial concentrations in lung, liver, and serum, and the top right is the blood levels of TNF and IL-10. The bottom half of the figure demonstrates systemic neutrophil dynamics, with the bottom left showing how the freely circulating and marginated neutrophil concentrations evolve; the right panel shows the absolute neutrophil population within the bone marrow post-mitotic neutrophil pool. A very transient bacteremia is observed, and cytokine time-series resemble those observed without the systemic model component. This is accompanied by a brief neutrophilia driven by de-margination of vascular neutrophils. 300 Figure 7.57: Example time-series for variables specific to the system-scale model when primary lung infection results in persistent, overwhelming lung infection and bacteremia. Following a series of two TNF peaks, TNF and IL-10 are both persistently elevated, and neutrophilia is sustained by increased bone marrow production. Figure 7.58: The intrinsic rate of TNF production by activated liver macrophages, α H T , affects lung neutrophil infiltration and bacterial clearance. The left panel gives, as a function of α H T , the peak lung neutrophil load and the maximum initial bacterial population, B 0 , that can be cleared by the innate response. It is clear that these two curves are essentially scalings of each other. The right panel demonstrates that the time-integrated lung neutrophil load responds to α H T in a similar manner. 301 Figure 7.59: Shown are the maximum initial bacterial (lung) inocula that can be cleared by the innate immune response as a function of the hepatic macrophage TNF production rate, α H T , for two different levels of lung phagocyte killing efficacy, measured by the parameter γ. The general pattern is identical between the two, with the hepatic inflammatory response simply shifting the curves upward. The left and right panels gives results for α H T plotted on linear and log scales, respectively. To model TNF blockade, we simply impose a 10-fold decrease in the serum TNF half-life, un- der the assumption that monoclonal anti-TNF antibodies penetrate minimally into tissue. This is obviously crude, but we are only interested in the qualitative effect of blocking TNF. A more realistic mathematical representation of the action of three specific TNF inhibitors based on chem- ical kinetics has been studied by Jit and colleagues [118]. Figure 7.61 shows the lung bacterial and neutrophilloadsasafunctionoftimeforfivedifferentlevelsofp L B ,withandwithoutTNFblockade, when antibiotic treatment is initiated at 24 hours. The neutrophil load is reduced significantly by TNF blockade, but this also impairs bacterial clearance. Integrating the total neutrophil load from the time of treatment initiation to bacterial clearance, we see that TNF blockade has no significant effect on this metric (see Figure 7.62). It may be concluded that, if there is an acute benefit to reducing TNF and neutrophil levels then TNF blockade could be advisable in the sickest patients, but the time-integrated effect on neutrophil levels will not be affected, as this treatment will actually increase the time it takes to clear infection. Note that, while TNF inhibition in the absence of antibiotic therapy reduces the 302 Figure 7.60: Serum TNF, IL-10, and IL-10:TNF time-series under 10 different levels of bacterial shedding into circulation, as measured by p L B , from a primary lung infection resulting in bacterial escape. Thep L B values are logarithmically distributed between 10 −10 and 10 −6 , and it is clear that peak and final values of all metrics increase monotonically with p L B . neutrophilload,bacterialinfectionremainsoverwhelmingandsuchatreatmentstrategyistherefore futile and irrelevant. Itmustbekeptinmindthatin vivo,p L B islikelystronglycorrelatedwithotherparametervalues indicating a weak immune response, and the prediction that higher p L B values improve bacterial clearance only holds when p L B varies independently. 7.7 Overall discussion and conclusions We have followed a hierarchical model building process that represents bacterial infection and the local and systemic immune response at three distinct scales: (1) local and well-mixed, (2) local and spatially explicit, and (3) systemic dynamics that take into account local infection, modeled in either a spatial or non-spatial context, and the systemic inflammatory response. We use a 303 Figure 7.61: Lung bacterial and neutrophil time-series that result from treating an overwhelming lung infection causing bacteremia with antibiotics alone (top panels) or antibiotics in combination with a TNF inhibitor (bottom panels). Treatment with 150 mg day −1 of ciprofloxacin (q8h) is initiated at 24 hours, and TNF inhibition is modeled via a 10-fold increase in the serum TNF elimination rate. Results are demonstrated for five different levels of bacteremia, with darker lines indicating greater bacterial shedding. The more significant the bacteremia, the more rapidly the primary infection is cleared, as bacteria cleared by the liver induce an adjuvant inflammatory re- sponse. TNFblockadedoesreduceneutrophilinfiltrationintothelung, butatthecostofdecreased bacterial clearance. 304 Figure 7.62: The left panel gives times to bacterial clearance as a function of p L B , the rate at which bacteria intravasate into blood, for antibiotic treatment alone and with a TNF inhibitor; results are shown for three different levels of immune competence as represented by γ, the maximal phagocytekillingrateTherightpanelgivestheintegratedneutrophilloadfromtreatmentinitiation to bacterial clearance. TNF blockade has less effect for intrinsically stronger immune responses, but significantly increases the time required to clear bacteria for weak responses. TNF blockade does not significantly affect the integrated neutrophil load. 305 generalized predator-prey formulation for phagocyte-bacteria interaction as the foundation for all models;uponthisfoundationwehavelayeredbasicpro-andanti-inflammatorycytokineproduction by local macrophages, as well as recruitment of circulating neutrophils. We have also studied antibiotic treatment, as modeled via a combined pharmacokinetic/ phar- macodynamic approach, at all three scales. We have found that most of the qualitative conclusions madeatthesimplewell-mixedscalearepreservedatthespatialandsystemiclevels,indicatingthat itisalargelyreasonableapproximationforfuturework. Ouroverallconclusionscanbesummarized as follows: 1. Thebasicformalismforphagocyte-bacteriainteractionfundamentallyaffectsdynamicsatthe in vitro and simple predator-prey levels. Specifically, a Holling Type II term for phagocyte- mediated predation is concluded to give the most realistic dynamics, and it can be derived either from Michaelis-Menten enzyme-kinetics or from first ecological principles as encapsu- lated in Holling’s disc equation. This is contra the widely used mass-action or Holling Type I formalism widely used in the mathematical biology literature. 2. TNF-mediated positive feedback can induce a robust local macrophage-mediated immune response that activates resting macrophages to clear bacterial infection. However, the se- quel is sterile inflammation, and the inclusion of macrophage-produced IL-10 cannot lead to resolution. 3. Positive feedback that amplifies the immune response by recruiting short-lived neutrophils fromcirculationcanleadtobacterialclearancewithinflammationresolution. Thus,thislocal- systemic phagocyte interaction avoids the positive feedback trap encountered when positive and negative feedback is mediated solely by locally produced cytokines. 306 4. Whenactivereprogrammingofphagocytesbyapoptoticneutrophilsisincludedinthemodel, weobserveabriefhyper-inflammatorystatefollowedbyamoreprolongedhypo-inflammatory state. The former is characterized by suppression of re-infection, while during the latter phase we see increased vulnerability to infection. Moreover, the magnitude of late immune suppression is proportional to the severity of the initial infection. 5. Immunosuppression,ratherthanhyper-immunity,isthedominantdynamicwhenthebacterial infection is serious enough to escape immune control. 6. Antibiotictreatmentbytheconcentration-dependentfluoroquinolones, whicharewidelyused as empiric therapy for sepsis, acts synergistically with the immune response. That is, we observe a threshold effect where once treatment has sufficiently reduced the bacterial load the immune response becomes capable of rapidly clearing the residual infection. This effect is far more pronounced in the basically immunocompetent host, but it is observed even in the immunocompromised host to some degree. 7. Immunocompromised patients are far more to vulnerable treatment failure via resistance. In immunocompetent patients, the immune response strongly suppresses resistant strains. 8. Antibiotic treatment is more effective the earlier it is initiated. The time to bacterial clear- ance and dose required are both increased by delayed treatment. This is related to reduced synergism with the immune response as immunosuppression becomes dominant. Antibiotic resistance also becomes more likely with treatment delays. 9. Most qualitative results are preserved as we scale from the well-mixed setting to the spatially explicit setting. However, bacterial motility emerges as a key determinant of outcome, the time-courseofbacterialinvasionandcytokinelevelsissignificantlyextendedcomparedtothe 307 well-mixed approximation, and spatial heterogeneity yields spatial sequestration of a chronic infection as a new and distinct dynamical possibility. 10. Our agent-based spatial model supports the use of a Holling Type II functional term for phagocyte predation on bacteria. 11. In the multi-patch setting, a spatial focus of impaired immunity, due to (say) impaired blood flow can give rise to a sequestered focus of infection if bacterial motility is low, whereas high bacterial motility can lead to organ-wide infection, even if the immune parameters in other patches are competent to clear infection in isolation. 12. Atthesystemicscale,whenaprimarylunginfectionleadstobacteremia,hepaticmacrophages are the dominant source of serum cytokines. These cytokines cause a circulating neutrophilia as well increased lung neutrophil infiltration. The hepatic/systemic response increases the ability of the lung-specific immune response to clear infection, but may also lead to excessive TNF and neutrophil infiltration. 13. The magnitude of systemic cytokine response, as measured by serum TNF and IL-10, is directly proportional to the rate at which bacteria shed into circulation from the primary infection site. Moreover, the IL-10:TNF ratio also increases in proportion to bacteremia. 14. Anti-TNF treatment in conjunction with antibiotics acutely decreases serum TNF and lung neutrophils, but also increases the time to bacterial clearance without affecting the time- integrated lung neutrophil burden from treatment to clearance. We have employed a hierarchical model-building process, with predator-prey-style interaction between bacteria and phagocytes as the foundation. This is a paradigm widely used in the math- ematical modeling literature, both for ecological applications and within-host disease dynamics. 308 We have examined how differences in the formalization of this interaction affects dynamics most thoroughly at the in vitro and simple predator-prey scales (Sections 7.4.1 and 7.4.2, respectively). In the former setting, a Holling Type II functional response describing phagocyte predation of bacteria gives much richer and more biologically plausible dynamics than a Holling Type I, or mass-action, functional response. In particular, we find that there is a critical phagocyte concen- tration below which bacteria escape immune control; above this threshold there is bi-dependence on the bacterial initial condition and the phagocyte density that approaches ratio-dependence for large populations. This unifies observations suggesting a critical neutrophil concentration on the one hand [149, 150], and ratio-dependent clearance on the other [145]. Our results are similar to the axiomatic formulation of bacteria-phagocyte dynamics proposed by Malka et al. [159, 160], indicating that the Holling Type II functional response term is the key to these dynamics. The mass-action function has also been criticized in the context of viral infection dynamics as leading to biologically implausible predictions [88]. Incorporating cytokines into the basic predator-prey framework shows that TNF-mediated pos- itivefeedbackcanrobustlyactivatethelocalmacrophagepopulation, leadingtobacterialclearance but with sterile inflammation as the unfortunate sequel. Under the “TNF-only” model, when the sentinel population of activated macrophages is small, dynamics are divided into two regimes. In the first, there is bi-stability between bacterial control at a sub-threshold level and escape; in the second we have bi-stability between bacterial escape and elimination accompanied by sterile inflammation. It was our initial hypothesis that the addition of IL-10 produced by activated macrophages would cause the inflammatory response to resolve following successful elimination of the bacteria. However,wehavefoundthatnocombinationofourthreehypothesesfortheactionofIL-10,namely 309 inhibition of TNF expression, auto-inhibition of IL-10, and inhibition of macrophage activation, leads to resolution. We have also incorporated an explicit delay in IL-10 production following activation, but this has no effect on our conclusion. IL-10 does, however, damp the inflammatory response, and higher production rates can induce a switch in stability from bacterial clearance to control. IL-10 also significantly limits TNF production by large macrophage reservoirs as might be encountered in the liver, under simulated endotoxemia, consistent with experimental observations [15]. We have confirmed this in our systemic sepsis model. Thus, it may be that the primary role of IL-10 is to prevent the pathologic spread of inflammation rather than resolution of the primary inflammatory focus; Cox [46] proposed a similar hypothesis based on a rat model of LPS-induced lung inflammation. The addition of activated endothelium and neutrophils has proven to be the key, at least in our model framework, to allowing both robust activation of a quiescent immune response and res- olution of this response following pathogen elimination. It is the local versus systemic source of the macrophage and neutrophil populations that underlies this conclusion. Local macrophages are susceptible to being trapped in a TNF-mediated positive feedback trap, but short-lived neu- trophils, after performing their work, escape this trap through death itself. Thus, an ostensibly pro-inflammatorycelltype,theneutrophil,appearstoplayananti-inflammatoryroleatthesystems level. Thefinalmajoradditiontothebasicmodelframeworkthatwehaveconsideredisactivephago- cyte re-programming via contact with apoptotic neutrophils. This yields a progression from hyper- to hypo-inflammation following bacterial infection, where the magnitude of the latter phase is pro- portional to the magnitude of the initial infection severity. This corresponds to the progression recently hypothesized to occur in clinical sepsis [112]. The further addition of an abstract damage 310 variable and monocyte recruitment do not affect the qualitative model behavior, and thus the dy- namics are observed to stabilize at the “neutrophil+apoptosis” model stage, which we have used as the foundation for our spatial and systemic model extensions. At the multi-patch spatial scale, we find bacterial motility to be a major determinant of vir- ulence, and the overall time-course of infection is prolonged compared to the well-mixed setting. While chronic spatial sequestration of infection is a new asymptotic possibility at this scale, the major qualitative model predictions are not altered. The major new dynamic we observe at the systemic model scale is that the greater the propensity for bacteria to enter the blood-stream, the greater the systemic cytokine reaction. This systemic reaction augments immunity at the local infection site, and thus can be considered beneficial, but it likely also has (un-modeled) pathologic effects on the host physiology. Our modeling framework has significant limitations. We have considered a very limited reper- toire of cytokines and cell types. In particular, TNF is the only pro-inflammatory cytokine in our model, eventhoughanalphabetsoupofsuchfactorsisknowntoplayaroleinsepsisandimmunity ingeneral[116]. TNFisgenerallyconsideredtobeanearly-actingcytokine, andthereforethemost significant drawback of our framework is the failure to include late-acting mediators. HMGB-1, for example, is an important DAMP released by necrotic cells [135], it has been found to be produced by macrophages cultured with TNF or IL-1, and mediates late mortality in murine endotoxemia [256]. In our consideration of an abstract damage variable (Section 7.4.3), which may be taken as a partial surrogate for late mediators, we did not observe significant qualitative consequences, but this is only a very crude approximation. However, tracking cell histories under the fixed activation states we employ to allow late mediator production is very difficult. We have completely omitted any consideration of the adaptive immune response. Sepsis is an 311 acute disorder, and it has been considered primarily a disorder of the innate immune response. However, the adaptive response is generally necessary to eliminate serious infections [116], and defects in adaptive immunity have been linked to sepsis pathophysiology. Both T and B cell depletion has been observed, and marked T cell dysfunction is a prominent feature of sepsis [111]. Therefore, future efforts may need to consider antigen-specific adaptive immunity. Wehavealsoconsideredallcytokinestobefreeandtakenthefractionofreceptorsitesoccupied onvariouscellstobeafunctionofthefreecytokineconcentration. Thisisreasonablewhencytokine levels are high, but ignoring receptor-binding as a sink for free ligand could be misleading when cytokine levels are low, particularly during the resolution phase of inflammation; it may also be problematic when immune cell concentrations are very high. Thus, the model may be improved either through explicitly modeling receptor-ligand binding or via a cytokine:cell ratio-dependent formulation. Furthermore,TNFreceptor-ligandbindingischaracterizedbyreceptor-ligandcomplex internalization and degradation, without recycling to the cell-surface [257]. Wehaveusedadirect-responseparadigm,wherecytokines,bacterial,andapoptoticcellcontact all directly alter phagocyte activation state. Activation states are imagined as lumped, uniform compartments and individual cells within such compartments have no memory of their previous exposures. Macrophages, for example, are divided into M1 and M2 phenotypes within our model. Whilethisiscongruentwiththecommonbiologicalclassificationofmacrophages, inrealitythereis aricherspectrumofactivationstatesthatdonotformstrictpartitions,butaremoreofacontinuum [184]. There are also many feedbacks that occur at the level of internal cell signaling cascades that cannot be directly captured by our framework. For example, TNF signaling to the inflammatory transcription factor NF-κB is characterized by two distinct negative feedback mechanisms. In the 312 resting cell, NF-κB dimers are sequestered in the cytosol byIκB proteins; a number of signal trans- duction pathways converge to phosphorylate and activate IκB kinase (IKK), which frees NF-κB dimers to translocate to the nucleus where they bind to κB promoter sites to modulate the expres- sion of many hundreds of target gene products [101]. IκB-α gene expression is strongly induced by NF-κB, representing a potent negative feedback mechanism that globally inhibits cytokine sig- nallingtoNF-κB.TheproteinA20isalsoNF-κBresponsiveandactsasaspecificnegativefeedback on TNF-induced NF-κ activity. A20 acts upstream of IKK and specifically inhibits TNF-mediated stimulation: IL-1β-induced NF-κB activity, for example, is unaffected by A20. Mice deficient in A20 develop widespread chronic inflammation [141]. These observations relate to the bow-tie model of cell signalling [264], where many cytokines convergeuponarelativelysmallnumberofcentralsignalingpathwaysthatinturnactivatehundreds of gene products. Clearly, it is practically impossible to account for this in the context of defined activation states that ignore these pathways and individual cell histories. However, incorporating such pathways in a meaningful sense could lead to an impossibly over-parameterized model, and indirect response modeling attempting to take these pathways into account have done so only in a very abstract sense [79, 78]. We use the concept of relative severity to indicate the overall seriousness of the infection with respect to both bacteria- and host-specific parameters. That is, larger values of bacteria-related parametersincludingtheinitialinoculum,naturalgrowthrate,cellspeedinthemulti-patchsetting, and, for the systemic model, the rate at which bacteria enter the bloodstream, all indicate a relativelymoresevereinfection. Immune-specificparametersthatcorrespondtoaweakerresponse, most notably those that quantify phagocyte-mediated killing, also imply greater relative severity. In our model, relatively more serious infections are almost universally associated with higher 313 peak levels of both TNF and IL-10, as well as higher levels at diagnosis. Bacteremia in particular invokesaverystrongcytokine responsefrom theliver. Bacterialpersistence, comparedto bacterial clearance, is associated with persistent elevation of both cytokines, although IL-10 is probably the better marker of the two. Furthermore, in the well-mixed setting, while the initial IL-10:TNF ratio is not very informative, clearance is associated with a relative decrease in this ratio, while persistence typically yields an increase in the ratio, suggesting persistent elevation in IL-10:TNF as a specific marker for ongoing infection. When we move the systemic geometry, we find that the initial IL-10:TNF is correlated with the degree of bacteremia, and hence overall infection severity. Thus, elevated cytokine levels represent a natural response to severe infection, whether the severity is related to a particularly virulent pathogen or a compromised immune system. These findings compare favorably to those of several clinical studies. A large study by Kellum and colleagues [128] suggested that pro- (IL-6) and anti-inflammatory (TNF) cytokines in septic patients are generally congruent, and patients may be roughly partitioned into groups with low, medium, and high levels of general cytokine activation. Most notably, those with high IL-6/high IL-10 levels were at highest risk for death. Persistent elevations in either the absolute TNF or IL-6 levels [207] or in the IL-10:TNF ratio [87] (Wong, personal communication) have also been associated with poor outcomes. While TNF and IL-6 do not have identical effects in vivo and have different temporal dynamics, with TNF considered an early cytokine and IL-6 typically appearing somewhat later, they are both broadly pro-inflammatory, and since our model framework only explicitly considers TNF, we use it as a surrogate for general pro-inflammatory cytokine activity. It has been speculated that an elevated IL-10:TNF ratio indicates an immune response that is tunedtobeexcessivelyimmunosuppressive,andhenceyieldsapooreroutcome. Wehavefoundthat increasedlevelsofbothIL-10andthisratioareobservednomatterwhattheunderlyingmechanism 314 forthepooroutcomeis, andimpairedimmunefunctionisoneofthetwobroadmechanismsleading to a poor outcome. However, it is not so much that IL-10 is produced in excessive amounts per se, but that global immune effectiveness is impaired, e.g. via impaired phagoycte killing of bacteria. That is, IL-10, along with TNF, and IL-10:TNF are non-specific (but probably sensitive) markers of impaired immunity. Whenseriousinfectionisestablished, i.e. bacterialescapehasoccurred, theintrinsictendencies of both local and hepatic (in the case of the systemic model) macrophages to produce TNF and IL- 10 affect the resulting cytokine levels and neutrophil infiltration into the lung. Since the infection has overwhelmed the immune response, we may consider excess neutrophils a futile response that onlyincreasesbystandertissuedamage. ThiswouldstronglysuggestTNFblockadeasatherapeutic strategy. However,whenweexamineTNFblockadeincombinationwitheffectiveantibiotictherapy, we see that while inhibiting TNF acutely decreases neutrophil and TNF levels, the time-integrated neutrophil response is not altered. This is because blocking TNF decreases immune-antibiotic synergism and thus increases the time until complete bacterial clearance occurs. Moreover, we have observed TNF inhibition to increase vulnerability to resistance-mediated treatment failure and opportunistic infection somewhat (in the systemic model). TNF inhibitors are widely used for rheumatoid arthritis and other autoimmune disorders, and in these settings they are well-known to increase the risk of both serious and non-serious infections, as well as opportunistic infections, and to promote re-activation of latent fungal and tuberculosis infections [96]. Our model suggests little universal benefit to treatment with TNF inhibitors. It may be that inhibition of TNF or other pro-inflammatory cytokines is worth some impaired immunity if it acutely reduces organ dysfunction related to excessive neutrophils or endothelial activation. This is plausible given that meta-analysis has suggested a minor benefit to anti-inflammatory agents in 315 thosepatientswiththegreatestriskofdeath. Wecannot,underourmodel,directlytestthis,andit remainsaquestionthatmustbedecidedempirically. Previousmodelingworksthathaveaddressed thisquestionsuggestedeithersomebenefittoTNFblockadeinthesickestpatients[44]ornobenefit at all [3], but these did not consider co-treatment with antibiotics. We do note that, given that any potentially beneficial effect of TNF inhibition would be most pronounced early in therapy, the associated increased risk of opportunistic infection, and the overall state of immunosuppression expected in late sepsis, if cytokine inhibition is attempted it should probably be brief and employ an inhibitor with a short half-life, i.e. etanercept. Because a larger cytokine response is needed in those (simulated) patients with diminished phagocytefunction,immunocompromisedpatientsmayactuallybeatgreaterriskforinappropriate sterileinflammation,ashighlevelsofTNFcouldleadtoapositivefeedbacktrap. Thisrelatestothe seeminglyparadoxicalandlong-standingclinicalobservationthatabroadspectrumofpatientswith inherited immunodeficiencies frequently suffer from autoimmune disease as well [70]. It is believed that this is largely related to the chronic inability of these patients to properly eradicate infection, leading to a (necessarily) exaggerated inflammatory response that causes tissue damage and leads to autoimmunity through an accidental danger association with host antigens [70]. In particular, chronic granulomatous disease (CGD), which is caused by an inherited defect in phagocyte ROS production and hence impaired phagocyte killing of bacterial and fungal pathogens, is associated with a variety of autoimmunities, most notable a severe Crohn’s-like inflammatory bowel disease [214]. Early and appropriate antibiotic treatment has, in clinical studies, been consistently shown to be crucially important to a good outcome. A recent review found such treatment to be the only component of sepsis care bundles consistently linked to a positive outcome [11], and delays to 316 effective antibiotic therapy on the order of hours can dramatically affect survival [56]. Our results areinkeepingwiththese, aswehavefoundearlyantibiotictherapytobemoreeffective,and, inthe case of our spatial models, intervention before complete domain invasion has occurred can reduce maximum cytokine levels and reduce time to bacterial clearance. In sum, the insight gained from our models and review of the literature suggests the following pathologic basis for sepsis. Sepsis is primarily a disease of the aged and immunosuppressed [6]. Immunodeficient patients require a relatively greater inflammatory/cytokine response to mobilize sufficient effector immune cells. Thus, an exuberant cytokine response is likely appropriate with respecttothepatient’sglobalimmunestatus. Thiscytokineresponsecancausesystematicderange- ments in the host physiology, but it is not necessarily wise to inhibit it, as doing so also impairs clearance of the underlying infection. The fundamental aim of treatment must then be to eradicate the ultimate cause, i.e. the infection. Furthermore, following an early hyperinflammatory phase, there is a prolonged immunosuppressed phase during which the host is vulnerable to re-infection by opportunistic pathogens or the emergence of resistant mutants. Thus, even in the face of what appears to be a hyperimmune state (based on cytokine measurements), adjuvant treatment that augments immunity, such as interferons, may be advisable. The central paradox of sepsis seems to be that the harmful inflammatory response is usually necessary with respect to pathogen control, and therefore anti-inflammatory agents are likely to have a limited role in therapy, if any. In fact, pro-immunity adjuvants may prove a better option. In any case, the basic clinical strategy must remain rapid and aggressive treatment of the cause. 317 References [1] Akashi, S., Saitoh, S. I., Wakabayashi, Y., Kikuchi, T., Takamura, N., Nagai, Y., ... & Miyake, K. (2003). Lipopolysaccharide Interaction with Cell Surface Toll-like Receptor 4-MD-2 Higher Affinity than That with MD-2 or CD14. The Journal of experimental medicine, 198(7), 1035–1042. [2] An, G. (2001). Agent-based computer simulation and sirs: building a bridge between basic science and clinical trials. Shock, 16(4), 266–273. [3] An, G. (2004). In silico experiments of existing and hypothetical cytokine-directed clinical trials using agent-based modeling. Critical care medicine, 32(10), 2050–2060. [4] An, G. (2008). Introduction of an agent-based multi-scale modular architecture for dynamic knowledge representation of acute inflammation. Theoretical Biology and Medical Modelling, 5:11. [5] Angus, D. C. (2011). The Search for Effective Therapy for Sepsis. JAMA: The Journal of the American Medical Association, 306(23), 2614–2615. [6] Angus, D. C., Linde-Zwirble, W. T., Lidicker, J., Clermont, G., Carcillo, J., & Pinsky, M. R. (2001). Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Critical care medicine, 29(7), 1303–1310. [7] Antia, R., & Koella, J. C. (1994). A model of non-specific immunity. Journal of theoretical biology, 168(2), 141–150. [8] Austin, D. J., White, N. J., & Anderson, R. M. (1998). The dynamics of drug action on the within-host population growth of infectious agents: melding pharmacokinetics with pathogen population dynamics. Journal of theoretical biology, 194(3), 313–339. [9] Babior, B. M. (1984). The respiratory burst of phagocytes. Journal of Clinical Investigation, 73(3), 599. [10] Barahona, M, and Poon, C. (1996). Detection of nonlinear dynamics in short, noisy time series. Nature, 381(6579), 215–17 [11] Barochia, A. V., Cui, X., Vitberg, D., Suffredini, A. F., OGrady, N. P., Banks, S. M., ... & Eichacker, P. Q. (2010). Bundled care for septic shock: an analysis of clinical trials. Critical care medicine, 38(2), 668–678. [12] Beeson,P.B.(1946).Developmentoftolerancetotyphoidbacterialpyrogenanditsabolitionbyreticulo- endothelial blockade. In: Proceedings of the Society for Experimental Biology and Medicine. Society for Experimental Biology and Medicine (New York, NY). Royal Society of Medicine. 61(3), 248–250. [13] Bell, G. I. (1970). Mathematical model of clonal selection and antibody production. Journal of theoret- ical biology, 29(2), 191–232. 318 [14] Bell, G. I. (1973). Predator-prey equations simulating an immune response. Mathematical Biosciences, 16(3), 291–314. [15] Berg, D. J., K¨ uhn, R., Rajewsky, K., Mller, W., Menon, S., Davidson, N., ... & Rennick, D. (1995). Interleukin-10 is a central regulator of the response to LPS in murine models of endotoxic shock and the Shwartzman reaction but not endotoxin tolerance. Journal of Clinical Investigation, 96(5), 2339–2347. [16] Berger,T.W.,Song,D.,Chan,R.,andMarmarelis,V.Z.(2010).Theneurobiologicalbasisofcognition: identificationbymulti-input,multi-outputnonlineardynamicmodeling.ProceedingsofIEEE,98,356–74. [17] Beutler, B., D. Greenwald, J. D. Hulmes, M. Chang, Y. C. Pan, J. Mathison, R. Ulevitch, & A. Cerami. (1985). Identity of tumour necrosis factor and the macrophage-secreted factor cachectin. Nature, 316(6028), 552–554. [18] Beutler, B., & Rietschel, E. T. (2003). Innate immune sensing and its roots: the story of endotoxin. Nature Reviews Immunology, 3(2), 169–176. [19] Bierie, B., & Moses, H. L. (2006). Tumour microenvironment: TGFβ: the molecular Jekyll and Hyde of cancer. Nature Reviews Cancer, 6(7), 506–520. [20] Bocci, V. (1991). Interleukins. Clinical pharmacokinetics and practical implications. Clinical pharma- cokinetics, 21(4), 274–284. [21] Boomer, J. S., To, K., Chang, K. C., Takasu, O., Osborne, D. F., Walton, A. H., ... & Hotchkiss, R. S. (2011). Immunosuppression in patients who die of sepsis and multiple organ failure. JAMA: the journal of the American Medical Association, 306(23), 2594–2605. [22] Bradley, P. P., Priebat, D. A., Christensen, R. D., & Rothstein, G. (1982). Measurement of cuta- neous inflammation: estimation of neutrophil content with an enzyme marker. Journal of Investigative Dermatology, 78(3), 206–209. [23] Bressler, S. L., & Seth, A. K. (2011). WienerGranger causality: a well established methodology. Neu- roimage, 58(2), 323–329. [24] Briggs, G. E., & Haldane, J. B. S. (1925). A note on the kinetics of enzyme action. Biochemical journal, 19(2), 338–339. [25] Brown, K. A., Brain, S. D., Pearson, J. D., Edgeworth, J. D., Lewis, S. M., & Treacher, D. F. (2006). Neutrophils in development of multiple organ failure in sepsis. The Lancet, 368(9530), 157–169. [26] Brown, R. P., Delp, M. D., Lindstedt, S. L., Rhomberg, L. R., & Beliles, R. P. (1997). Physiological parameter values for physiologically based pharmacokinetic models. Toxicology and industrial health, 13(4), 407–484. [27] Bryant, H. L., and Segundo, J. P. (1976). Spike initiation by transmembrane current: a white-noise analysis. The Journal of Physiology, 260(2), 279–314. [28] Bu˜ no, W., Bustamante, J., and Fuentes, J. (1984). White noise analysis of pace-maker-response inter- actionsand non-linearitiesin slowlyadapting crayfishstretchreceptor. TheJournal of Physiology, 350(1), 55–80. [29] Bustamante, J., and Bu˜ no, W. (1992). Signal transduction and nonlinearities revealed by white noise inputs in the fast adapting crayfish stretch receptor. Experimental Brain Research, 88(2), 303–12. [30] Byrne, A., & Reen, D. J. (2002). Lipopolysaccharide induces rapid production of IL-10 by monocytes in the presence of apoptotic neutrophils. The Journal of Immunology, 168(4), 1968–1977. 319 [31] Campion, J. J., McNamara, P. J., & Evans, M. E. (2005). Pharmacodynamic modeling of ciprofloxacin resistance in Staphylococcus aureus. Antimicrobial agents and chemotherapy, 49(1), 209–219. [32] Carson, W. E., Lindemann, M. J., Baiocchi, R., Linett, M., Tan, J. C., Chou, C. C., ... & Caligiuri, M. A. (1995). The functional characterization of interleukin-10 receptor expression on human natural killer cells. Blood, 85(12), 3577–3585. [33] Carswell, E. A., Old, L. J., Kassel, R. L., Green, S., Fiore, N., & Williamson, B. (1975). An endotoxin- induced serum factor that causes necrosis of tumors. Proceedings of the National Academy of Sciences, 72(9), 3666–3670. [34] Casey, L. C., Balk, R. A., & Bone, R. C. (1993). Plasma cytokine and endotoxin levels correlate with survival in patients with the sepsis syndrome. Annals of Internal Medicine, 119(8), 771–778. [35] Cassatella, M. A. (1995). The production of cytokines by polymorphonuclear neutrophils. Immunology today, 16(1), 21–26. [36] Cavaillon, J. M. (1995). The nonspecific nature of endotoxin tolerance. Trends in Microbiology, 3(8), 320–324. [37] Chan, F. K. M. (2007). Three is better than one: pre-ligand receptor assembly in the regulation of TNF receptor signaling. Cytokine, 37(2), 101–107. [38] Charnick, S. B., Fisher, E. S., & Lauffenburger, D. A. (1991). Computer simulations of cell-target encounter including biased cell motion toward targets: Single and multiple cell-target simulations in two dimensions. Bulletin of mathematical biology, 53(4), 591–621. [39] Charnick, S. B., & Lauffenburger, D. A. (1990). Mathematical analysis of cell-target encounter rates in three dimensions. Effect of chemotaxis. Biophysical journal, 57(5), 1009–1023. [40] Chon,K.H.,Richard,C.J.,Holstein-Rathlou,N.-H.(1997).Compactandaccuratelinearandnonlinear autoregressivemovingaveragemodelparameterestimationusingLaguerrefunctions.AnnalsofBiomedical Engineering. 25(4), 731–38. [41] Chow, C. C., Clermont, G., Kumar, R., Lagoa, C., Tawadrous, Z., Gallo, D., ... & Vodovotz, Y. (2005). The acute inflammatory response in diverse shock states. Shock, 24(1), 74–84. [42] Chung, H. M., Cartwright, M. M., Bortz, D. M., Jackson, T. L., & Younger, J. G. (2008). Dynamical system analysis of Staphylococcus epidermidis bloodstream infection. Shock, 30(5), 518–526. [43] Clarke, C. J., Hales, A., Hunt, A., & Foxwell, B. M. (1998). IL-10-mediated suppression of TNF-alpha production is independent of its ability to inhibit NFκB activity. European journal of immunology, 28(5), 1719–1726. [44] Clermont, G., Bartels, J., Kumar, R., Constantine, G., Vodovotz, Y., & Chow, C. (2004). In silico design of clinical trials: a method coming of age. Critical care medicine, 32(10), 2061–2070. [45] Cole, M., Eikenberry, S., Marmarelis, V., &Yamashiro, S.Nonparametricmodelofsmoothmuscleforce production during electrical stimulation. Under review. [46] Cox, G.(1996).IL-10enhancesresolutionofpulmonaryinflammationinvivobypromotingapoptosisof neutrophils.AmericanJournalofPhysiology-LungCellularandMolecularPhysiology,271(4),L566–L571. [47] Crump, B. R., Wise, R., & Dent, J. (1983). Pharmacokinetics and tissue penetration of ciprofloxacin. Antimicrobial agents and chemotherapy, 24(5), 784–786. 320 [48] Culotta, E., & Koshland Jr, D. E. (1992). NO news is good news. Science, 258(5090), 1862–1865. [49] Cvetanovic, M., Mitchell, J. E., Patel, V., Avner, B. S., Su, Y., Van der Saag, P. T., ... & Ucker, D. S. (2006). Specific recognition of apoptotic cells reveals a ubiquitous and unconventional innate immunity. Journal of Biological Chemistry, 281(29), 20055–20067. [50] Czock, D., & Keller, F. (2007). Mechanism-based pharmacokineticpharmacodynamic modeling of an- timicrobial drug effects. Journal of pharmacokinetics and pharmacodynamics, 34(6), 727–751. [51] Damas, P., Ledoux, D. I. D. I. E. R., Nys, M., Vrindts, Y. V. O. N. N. E., De Groote, D. O. N. A. T., Franchimont, P., & Lamy, M. (1992). Cytokine serum level during severe sepsis in human IL-6 as a marker of severity. Annals of surgery, 215(4), 356–362. [52] Daun, S., Rubin, J., Vodovotz, Y., Roy, A., Parker, R., & Clermont, G. (2008). An ensemble of models of the acute inflammatory response to bacterial lipopolysaccharide in rats: results from parameter space reduction. Journal of theoretical biology, 253(4), 843–853. [53] Day, J., Rubin, J., Vodovotz, Y., Chow, C. C., Reynolds, A., & Clermont, G. (2006). A reduced mathematical model of the acute inflammatory response II. Capturing scenarios of repeated endotoxin administration. Journal of theoretical biology, 242(1), 237–256. [54] Delacher, S., Derendorf, H., Hollenstein, U., Brunner, M., Joukhadar, C., Hofmann, S., ... & M¨ uller, M. (2000). A combined in vivo pharmacokineticin vitro pharmacodynamic approach to simulate target site pharmacodynamics of antibiotics in humans. Journal of Antimicrobial Chemotherapy, 46(5), 733–739. [55] Diaz, H., and Desrochers, A. A. (1988). Modeling of nonlinear discrete-time systems from input-output data. Automatica, 24(5), 629–41. [56] Dickinson, J. D., & Kollef, M. H. (2011). Early and adequate antibiotic therapy in the treatment of severe sepsis and septic shock. Current infectious disease reports, 13(5), 399–405. [57] Deitch, E. A. (1992). Multiple organ failure. Pathophysiology and potential future therapy. Annals of surgery, 216(2), 117–134. [58] Ding,M.,Bressler,S.L.,Yang,W.,&Liang,H.(2000).Short-windowspectralanalysisofcorticalevent- relatedpotentialsbyadaptivemultivariateautoregressivemodeling: datapreprocessing,modelvalidation, and variability assessment. Biological cybernetics, 83(1), 35–45. [59] D¨ ocke, W. D., Randow, F., Syrbe, U., Krausch, D., Asadullah, K., Reinke, P., ... & Kox, W. (1997). Monocytedeactivationinsepticpatients: restorationbyIFN-γ treatment.Naturemedicine,3(6),678–681. [60] Donnelly, R. P., Dickensheets, H., & Finbloom, D. S. (1999). The interleukin-10 signal transduction pathway and regulation of gene expression in mononuclear phagocytes. Journal of interferon & cytokine research, 19(6), 563–573. [61] Drlica,K.,&Zhao,X.(2007).Mutantselectionwindowhypothesisupdated.ClinicalInfectiousDiseases, 44(5), 681–688. [62] Drusano, G.L.(2004).Antimicrobialpharmacodynamics: criticalinteractionsof’buganddrug’.Nature Reviews Microbiology, 2(4), 289–300. [63] Eichacker, P. Q., Natanson, C., & Danner, R. L. (2006). Surviving sepsis-practice guidelines, marketing campaigns, and Eli Lilly. New England Journal of Medicine, 355(16), 1640–1642. [64] Eichacker, P. Q., Parent, C., Kalil, A., Esposito, C., Cui, X., Banks, S. M., ... & Natanson, C. (2002). Risk and the efficacy of antiinflammatory agents: retrospective and confirmatory studies of sepsis. Amer- ican journal of respiratory and critical care medicine, 166(9), 1197–1205. 321 [65] Eikenberry, S. E., & Marmarelis, V. Z. (2013). A nonlinear autoregressive Volterra model of the Hodgk- inHuxley equations. Journal of computational neuroscience, 34(1), 163–183. [66] Eikenberry, S. E., & Marmarelis, V. Z. Principal dynamic mode analysis of a nonlinear autoregressive Volterra model for the HodgkinHuxley equations. Under review. [67] Eikenberry,S.,Thalhauser,C.,&Kuang,Y.(2009).Tumor-immuneinteraction,surgicaltreatment,and cancer recurrence in a mathematical model of melanoma. PLoS computational biology, 5(4), e1000362. [68] Ertel,W.,Kremer,J.P.,Kenney,J.,Steckholzer,U.,Jarrar,D.,Trentz,O.,&Schildberg,F.W.(1995). Downregulation of proinflammatory cytokine release in whole blood from septic patients. Blood, 85(5), 1341–1347. [69] Esmann, L., Idel, C., Sarkar, A., Hellberg, L., Behnen, M., Mller, S., ... & Laskay, T. (2010). Phagocy- tosis of apoptotic cells by neutrophil granulocytes: diminished proinflammatory neutrophil functions in the presence of apoptotic cells. The journal of immunology, 184(1), 391–400. [70] Etzioni, A. (2003). Immune deficiency and autoimmunity. Autoimmunity reviews, 2(6), 364–369. [71] Fadok, V. A., Bratton, D. L., Konowal, A., Freed, P. W., Westcott, J. Y., & Henson, P. M. (1998). Macrophages that have ingested apoptotic cells in vitro inhibit proinflammatory cytokine production through autocrine/paracrine mechanisms involving TGF-beta, PGE2, and PAF. Journal of Clinical In- vestigation, 101(4), 890–898. [72] Fanning, N. F., Kell, M. R., Shorten, G. D., Kirwan, W. O., Bouchier-Hayes, D., Cotter, T. G., & Redmond, H. P. (1999). Circulating granulocyte macrophage colony-stimulating factor in plasma of patientswiththesystemicinflammatoryresponsesyndromedelaysneutrophilapoptosisthroughinhibition of spontaneous reactive oxygen species generation. Shock, 11(3), 167–174. [73] Fisher, E. S., & Lauffenburger, D. A. (1987). Mathematical analysis of cell-target encounter rates in two dimensions. The effect of chemotaxis. Biophysical journal, 51(5), 705–716. [74] Fisher, E. S., Lauffenburger, D. A., & Daniele, R. P. (1988). The effect of alveolar macrophage chemo- taxis on bacterial clearance from the lung surface. The American review of respiratory disease, 137(5), 1129–1134. [75] Fenton, A., & Perkins, S. E. (2010). Applying predator-prey theory to modelling immune-mediated, within-host interspecific parasite interactions. Parasitology, 137(06), 1027–1038. [76] Forrest, A., Ballow, C. H., Nix, D. E., Birmingham, M. C., & Schentag, J. J. (1993). Development of a population pharmacokinetic model and optimal sampling strategies for intravenous ciprofloxacin. Antimicrobial agents and chemotherapy, 37(5), 1065-1072. [77] Forrest, A. L. A. N., Nix, D. E., Ballow, C. H., Goss, T. F., Birmingham, M. C., & Schentag, J. J. (1993). Pharmacodynamics of intravenous ciprofloxacin in seriously ill patients. Antimicrobial agents and chemotherapy, 37(5), 1073-1081. [78] Foteinou, P. T., Calvano, S. E., Lowry, S. F., & Androulakis, I. P. (2009). In silico simulation of corticosteroidseffectonanNFkB-dependentphysicochemicalmodelofsystemicinflammation.PLoSOne, 4(3), e4706. [79] Foteinou, P. T., Calvano, S. E., Lowry, S. F., & Androulakis, I. P. (2009). Modeling endotoxin-induced systemic inflammation using an indirect response approach. Mathematical biosciences, 217(1), 27–42. [80] Funk, D. J., Parrillo, J. E., & Kumar, A. (2009). Sepsis and septic shock: a history. Critical care clinics, 25(1), 83–101. 322 [81] Gabriel, J. P., Saucy, F., & Bersier, L. F. (2005). Paradoxes in the logistic equation?. Ecological Mod- elling, 185(1), 147–151. [82] Gerstner,W.(1995).Timestructureoftheactivityinneuralnetworkmodels.PhysicalReviewE,51(1), 738–58. [83] Gerstner, W., and van Hemmen, J., L. (1992). Associative memory in a network of ‘spiking’ neurons. Network, 3(2), 139–64. [84] Gerstner, W., and Naud, R. (2009). How good are neuron models? Science, 326(5951), 379–80. [85] Geweke, J. F. (1984). Measures of conditional linear dependence and feedback between time series. Journal of the American Statistical Association, 79(388), 907–915. [86] Glasgow, J. E., Farrell, B. E., Fisher, E. S., Lauffenburger, D. A., & Daniele, R. P. (1989). The motile response of alveolar macro-phages. Am. Rev. Respir. Dis, 139, 320–329. [87] Gogos,C.A.,Drosou,E.,Bassaris,H.P.,&Skoutelis,A.(2000).Pro-versusanti-inflammatorycytokine profile in patients with severe sepsis: a marker for prognosis and future therapeutic options. Journal of Infectious Diseases, 181(1), 176–180. [88] Gourley, S. A., Kuang, Y., & Nagy, J. D. (2008). Dynamics of a delay differential equation model of hepatitis B virus infection. Journal of Biological Dynamics, 2(2), 140–153. [89] Granger,C.W.(1969).Investigatingcausalrelationsbyeconometricmodelsandcross-spectralmethods. Econometrica: Journal of the Econometric Society, 37(3), 424–438. [90] Greenspan, H. P. (1972). Models for the growth of a solid tumor by diffusion. Stud. Appl. Math, 51(4), 317–340. [91] Gr´ egoire, N., Raherison, S., Grignon, C., Comets, E., Marliat, M., Ploy, M. C., & Couet, W. (2010). Semimechanistic pharmacokinetic-pharmacodynamic model with adaptation development for time-kill experiments of ciprofloxacin against Pseudomonas aeruginosa. Antimicrobial agents and chemotherapy, 54(6), 2379–2384. [92] Gregory, S. H., & Wing, E. J. (2002). Neutrophil-Kupffer cell interaction: a critical component of host defenses to systemic bacterial infections. Journal of leukocyte biology, 72(2), 239–248. [93] Guttman, R., and Feldman, L. (1975). White noise measurement of squid axon membrane impedance. Biochemical and Biophysical Research Communications, 67(1), 427–32. [94] Guttman, R., Feldman, L., and Lecar, H. (1974). Squid Axon Membrane Response to White Noise Stimulation. Biophysical Journal, 14(12), 941–55. [95] Guttman, R., Grisell, R., and Feldman, L. (1977). Strength-frequency relationship for white noise stimulation of squid axons. Mathematical Biosciences, 33(3), 335–43. [96] Hadam, J., Aoun, E., Clarke, K., &Wasko, M.C.(2014).ManagingrisksofTNFinhibitors: Anupdate for the internist. Cleveland Clinic journal of medicine, 81(2), 115–127. [97] Haddy, T. B., Rana, S. R., & Castro, O. (1999). Benign ethnic neutropenia: what is a normal absolute neutrophil count?. Journal of Laboratory and Clinical Medicine, 133(1), 15–22. [98] Hampson R. E., Song, D., Chan, R. H. M., Sweatt, A. J., Fuqua, J., Gerhardt, G. A., Shin, D., Marmarelis, V. Z., Berger, T. W., Deadwyler, S. A. (2012). A nonlinear model for hippocampal cognitive prostheses: Memory facilitation by hippocampal ensemble stimulation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 20(2), 184–97. 323 [99] Hampson, R. E., Song, D., Chan, R. H. M., Sweatt, A. J., Riley, M. R., Goonawardena, A. V., Mar- marelis, V. Z., Gerhardt, G. A., Berger, T. W., Deadwyler, S. A. (2012). Closing the loop for memory prosthesis: Detectingtheroleofhippocampalneuralensemblesusingnonlinearmodels.IEEETransactions on Neural Systems and Rehabilitation Engineering, 20(4), 510–525. [100] Hartigan, J. A., & Wong, M. A. (1979). Algorithm AS 136: A k-means clustering algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics), 28(1), 100–108. [101] Hayden, M. S., & Ghosh, S. (2004). Signaling to NF-κB. Genes & development, 18(18), 2195–2224. [102] Henson, P. M., & Bratton, D. L. (2013). Antiinflammatory effects of apoptotic cells. The Journal of clinical investigation, 123(7), 2773–2774. [103] Herald,M.C.(2010).Generalmodelofinflammation.Bulletinofmathematicalbiology,72(4),765–779. [104] Hill, A. V. (1910). The possible effects of the aggregation of the molecules of haemoglobin on its dissociation curves. J physiol, 40(4), iv–vii. [105] Hodgkin, A. L., and Huxley, A., L. (1952). A quantitative description of membrane current and its application to conduction and excitation in nerve. Journal of Physiolology, 117(4), 500-44. [106] Hoffmann,A.,Levchenko,A.,Scott,M.L.,&Baltimore,D.(2002).TheIkappaB-NF-kappaBsignaling module: temporal control and selective gene activation. Science, 298(5596), 1189–90. [107] Holling, C. S. (1959). The components of predation as revealed by a study of small-mammal predation of the European pine sawfly. The Canadian Entomologist, 91(5), 293–320. [108] Holling, C. S. (1965). The functional response of predators to prey density and its role in mimicry and population regulation. Memoirs of the Entomological Society of Canada, 97(S45), 5–60. [109] Holmes, E. E., Lewis, M. A., Banks, J. E., & Veit, R. R. (1994). Partial differential equations in ecology: spatial interactions and population dynamics. Ecology, 17–29. [110] Hotchkiss, R. S., Coopersmith, C. M., McDunn, J. E., & Ferguson, T. A. (2009). The sepsis seesaw: tilting toward immunosuppression. Nature medicine, 15(5), 496–497. [111] Hotchkiss, R. S., & Karl, I. E. (2003). The pathophysiology and treatment of sepsis. New England Journal of Medicine, 348(2), 138–150. [112] Hotchkiss, R. S., Monneret, G., & Payen, D. (2013). Sepsis-induced immunosuppression: from cellular dysfunctions to immunotherapy. Nature Reviews Immunology, 13(12), 862–874. [113] Huhn, R. D., Radwanski, E., O’Connell, S. M., Sturgill, M. G., Clarke, L., Cody, R. P., ... & Cutler, D. L. (1996). Pharmacokinetics and immunomodulatory properties of intravenously administered recom- binant human interleukin-10 in healthy volunteers. Blood, 87(2), 699–705. [114] Hyatt, J. M., Nix, D. E., & Schentag, J. J. (1994). Pharmacokinetic and pharmacodynamic activities of ciprofloxacin against strains of Streptococcus pneumoniae, Staphylococcus aureus, and Pseudomonas aeruginosa for which MICs are similar. Antimicrobial agents and chemotherapy, 38(12), 2730–2737. [115] Hyatt, J. M., Nix, D. E., Stratton, C. W., & Schentag, J. J. (1995). In vitro pharmacodynamics of piperacillin, piperacillin-tazobactam, and ciprofloxacin alone and in combination against Staphylococcus aureus, Klebsiella pneumoniae, Enterobacter cloacae, and Pseudomonas aeruginosa. Antimicrobial agents and chemotherapy, 39(8), 1711–1716. [116] Kenneth, M., Travers, P., Walport, M. (2008). Janeway’s Immunobiology - 7th ed. New York, NY: Garland Science, Taylor & Francis Group, LLC. 324 [117] Jimenez, M. F., Watson, R. W. G., Parodo, J., Evans, D., Foster, D., Steinberg, M., ... & Marshall, J. C. (1997). Dysregulated expression of neutrophil apoptosis in the systemic inflammatory response syndrome. Archives of Surgery, 132(12), 1263–1270. [118] Jit, M., Henderson, B., Stevens, M., & Seymour, R. M. (2005). TNF-alpha neutralization in cytokine- driven diseases: a mathematical model to account for therapeutic success in rheumatoid arthritis but therapeutic failure in systemic inflammatory response syndrome. Rheumatology (Oxford), 44(3):323–331. [119] Johnson, K. A., & Goody, R. S. (2011). The original Michaelis constant: translation of the 1913 MichaelisMenten paper. Biochemistry, 50(39), 8264–8269. [120] Jolivet, R., Kobayashi, R., Rauch, A., Naud, R., Shinomoto, S., & Gerstner, W. (2008). A benchmark test for a quantitative assessment of simple neuron models. Journal of Neuroscience Methods, 169(2), 417-24. [121] Jolivet, R., Lewis, T. J., & Gerstner, W. (2004). Generalized integrate-and-fire models of neuronal activity approximate spike trains of a detailed model to a high degree of accuracy. Journal of neurophys- iology, 92(2), 959–976. [122] Jolivet R, Rauch A, Lscher HR, Gerstner W. (2006). Predicting spike timing of neocortical pyramidal neurons by simple threshold models. Journal of Computational Neuroscience, 21(1), 35–49. [123] Jones, J. C. P., & Billings, S. A. (1989). Recursive algorithm for computing the frequency response of a class of non-linear difference equation models. International Journal of Control, 50(5), 1925–40. [124] Jones, A. E., & Puskarich, M. A. (2009). Sepsis-induced tissue hypoperfusion. Critical care clinics, 25(4), 769–779. [125] Jumbe, N., Louie, A., Leary, R., Liu, W., Deziel, M. R., Tam, V. H., ... & Drusano, G. L. (2003). Application of a mathematical model to prevent in vivo amplification of antibiotic-resistant bacterial populations during therapy. Journal of Clinical Investigation, 112(2), 275–285. [126] Karmann, K., Min, W., Fanslow, W. C., & Pober, J. S. (1996). Activation and homologous desensiti- zation of human endothelial cells by CD40 ligand, tumor necrosis factor, and interleukin 1. The Journal of experimental medicine, 184(1), 173–182. [127] Keel, M., Ungeth¨ um, U., Steckholzer, U., Niederer, E., Hartung, T., Trentz, O., & Ertel, W. (1997). Interleukin-10 counterregulates proinflammatory cytokine-induced inhibition of neutrophil apoptosis dur- ing severe sepsis. Blood, 90(9), 3356–3363. [128] Kellum, J. A., Kong, L., Fink, M. P., Weissfeld, L. A., Yealy, D. M., Pinsky, M. R., ... & Angus, D. C. (2007). Understanding the inflammatory cytokine response in pneumonia and sepsis: results of the Genetic and Inflammatory Markers of Sepsis (GenIMS) Study. Archives of internal medicine, 167(15), 1655–1663. [129] Kinasewitz, G. T., Yan, S. B., Basson, B., Russell, J. A., Cariou, A., Um, S. L., ... & Dhainaut, J. F. (2004). Universal changes in biomarkers of coagulation and inflammation occur in patients with severe sepsis, regardless of causative micro-organism. Critical Care, 8(2), R82–R90. [130] Kingsland, S. (1982). The refractory model: The logistic curve and the history of population ecology. Quarterly Review of Biology, 57(1), 29–52. [131] Kistler, W. M., Gerstner, W., van Hemmen, J. L. (1997). Reduction of Hodgkin-Huxley equations to a single-variable threshold model. Neural Computation, 9(5), 1015–45. 325 [132] Knight, B. W. (1972). Dynamics of encoding in a population of neurons. The Journal of general physiology, 59(6), 734–766. [133] Knolle, P. A., L¨ oser, E., Protzer, U., Duchmann, R., Schmitt, E., zum B¨ uschenfelde, K. H., ... & Gerken, G. (1997). Regulation of endotoxin-induced IL-6 production in liver sinusoidal endothelial cells and Kupffer cells by IL-10. Clinical and experimental immunology, 107(3), 555–561. [134] Koch, A. E., Polverini, P. J., Kunkel, S. L., Harlow, L. A., DiPietro, L. A., Elner, V. M., ... & Strieter, R. M. (1992). Interleukin-8 as a macrophage-derived mediator of angiogenesis. Science, 258(5089), 1798– 1801. [135] Kono, H., & Rock, K. L. (2008). How dying cells alert the immune system to danger. Nature Reviews Immunology, 8(4), 279–289. [136] Koo,D.J.,Chaudry,I.H.,&Wang,P.(1999).Kupffercellsareresponsibleforproducinginflammatory cytokinesandhepatocellulardysfunctionduringearlysepsis.JournalofSurgicalResearch,83(2),151–157. [137] Korenberg,M.,J.,French,A.,S.,andVoo,S.,K.,L.(1988).White-noiseanalysisofnonlinearbehavior in an insect sensory neuron: kernel and cascade approaches. Biological Cybernetics, 58(5), 313–20. [138] Kox, M., de Kleijn, S., Pompe, J. C., Ramakers, B. P., Netea, M. G., van der Hoeven, J. G., ... & Pickkers, P. (2011). Differential ex vivo and in vivo endotoxin tolerance kinetics following human endotoxemia. Critical care medicine, 39(8), 1866–1870. [139] Krakauer,T.(1995).IL-10inhibitstheadhesionofleukocyticcellstoIL-1-activatedhumanendothelial cells. Immunology letters, 45(1), 61–65. [140] Kumar, R., Clermont, G., Vodovotz, Y., & Chow, C. C. (2004). The dynamics of acute inflammation. Journal of Theoretical Biology, 230(2), 145–155. [141] Lee, E. G., Boone, D. L., Chai, S., Libby, S. L., Chien, M., Lodolce, J. P., & Ma, A. (2000). Failure to regulate TNF-induced NF-kappa B and cell death responses in A20-deficient mice. Science, 289(5488): 2350-2354. [142] Lee, Y. W., & Schetzen, M. (1965). Measurement of the Wiener kernels of a nonlinear system by cross-correlation. International Journal of Control, 2(3), 237–54. [143] Leijh, P. C., van den Barselaar, M. T., Dubbeldeman-Rempt, I., & van Furth, R. (1980). Kinetics of intracellular killing of Staphylococcus aureus and Escherichia coli by human granulocytes. European journal of immunology, 10(10), 750–757. [144] Leijh, P. C., van den Barselaar, M. T., & van Furth, R. (1981). Kinetics of phagocytosis and intracel- lular killing of Staphylococcus aureus and Escherichia coli by human monocytes. Scandinavian journal of immunology, 13(2), 159–174. [145] Leijh, P. C., Van den Barselaar, M. T., Van Zwet, T. L., Dubbeldeman-Rempt, I., & Van Furth, R. (1979). Kinetics of phagocytosis of Staphylococcus aureus and Escherichia coli by human granulocytes. Immunology, 37(2), 453–465. [146] Leizer, T., Cebon, J., Layton, J. E., & Hamilton, J. A. (1990). Cytokine regulation of colony- stimulatingfactorproductioninculturedhumansynovialfibroblasts: I.InductionofGM-CSFandG-CSF production by interleukin-1 and tumor necrosis factor. Blood, 76(10), 1989–1996. [147] Leuschner, F., Rauch, P. J., Ueno, T., Gorbatov, R., Marinelli, B., Lee, W. W., ... & Nahrendorf, M. (2012). Rapid monocyte kinetics in acute myocardial infarction are sustained by extramedullary monocy- topoiesis. The Journal of experimental medicine, 209(1), 123–137. 326 [148] Lewis, E. R., Henry, K. R., and Yamada, W. M. (2000). Essential roles of noise in neural coding and in studies of neural coding. BioSystems, 58(1), 109–15. [149] Li, Y., Karlin, A., Loike, J. D., & Silverstein, S. C. (2002). A critical concentration of neutrophils is required for effective bacterial killing in suspension. Proceedings of the National Academy of Sciences, 99(12), 8289–8294. [150] Li, Y., Karlin, A., Loike, J. D., & Silverstein, S. C. (2004). Determination of the critical concentration ofneutrophilsrequiredtoblockbacterialgrowthintissues.TheJournalofexperimentalmedicine, 200(5), 613–622. [151] Lipniacki, T., Paszek, P., Brasier, A. R., Luxon, B., & Kimmel, M. (2004). Mathematical model of NF-kappaB regulatory module. Journal of theoretical biology, 228(2), 195–215. [152] Lipsitch, M., & Levin, B. R. (1997). The population dynamics of antimicrobial chemotherapy. Antimi- crobial agents and chemotherapy, 41(2), 363–373. [153] London, M., & H¨ ausser, M. (2005). Dendritic computation. Annu. Rev. Neurosci., 28, 503–532. [154] Lotka, A. J. (1920). Analytical note on certain rhythmic relations in organic systems. Proceedings of the National Academy of Sciences of the United States of America, 6(7), 410–415. [155] Lowe, G., Meister, M., & Berg, H. C. (1987). Rapid rotation of flagellar bundles in swimming bacteria. Nature, 325, 637–640. [156] Maeda, H., Kuwahara, H., Ichimura, Y., Ohtsuki, M., Kurakata, S., & Shiraishi, A. (1995). TGF- beta enhances macrophage ability to produce IL-10 in normal and tumor-bearing mice. The Journal of Immunology, 155(10), 4926–4932. [157] Maianski, N. A., Maianski, A. N., Kuijpers, T. W., & Roos, D. (2004). Apoptosis of neutrophils. Acta haematologica, 111(1–2), 56–66. [158] de Waal Malefyt, R., Abrams, J., Bennett, B., Figdor, C. G., & de Vries, J. E. (1991). Interleukin 10 (IL-10) inhibits cytokine synthesis by human monocytes: an autoregulatory role of IL-10 produced by monocytes. The Journal of experimental medicine, 174(5), 1209–1220. [159] Malka, R., Shochat, E., & Rom-Kedar, V. (2010). Bistability and bacterial infections. PloS one, 5(5), e10010. [160] Malka, R., Wolach, B., Gavrieli, R., Shochat, E., & Rom-Kedar, V. (2012). Evidence for bistable bacteria-neutrophil interaction and its clinical implications. The Journal of clinical investigation, 122(8), 3002–3011. [161] Marik, P. E. (2011). Glucocorticoids in sepsis: dissecting facts from fiction. Critical Care, 15:158. [162] Marmarelis, V. Z. (1993). Identification of nonlinear biological systems using laguerre expansions of kernels. Annals of Biomedical Engineering, 21(6), 573–89. [163] Marmarelis, V. Z. (1997). Modeling methodology for nonlinear physiological systems. Annals of Biomedical Engineering, 25(2), 239–51. [164] Marmarelis, V.Z. (2004). Nonlinear Dynamic Modeling of Physiological Systems. Hoboken: Wiley- IEEE Press. [165] Marmarelis, V. Z. (1989). Signal transformation and coding in neural systems. IEEE Transactions on Biomedical Engineering, 36(1), 15–24. 327 [166] Marmarelis, P. Z., & McCann, G. D. (1973). Development and application of white-noise modeling techniques for studies of insect visual nervous system. Biological Cybernetics, 12(2), 74–89. [167] Marmarelis, P.Z., & Naka, K. I. (1973). Nonlinear analysis and synthesis of receptive-field responses in the catfish retina. 3. Two-input white-noise analysis. Journal of Neurophysiology, 36(4), 634–48. [168] Marmarelis, V.Z., &Orme, M.E.(1993).Modelingofneuralsystemsbyuseofneuronalmodes.IEEE Transactions on Biomedical Engineering, 40(11), 1149–58. [169] Marmarelis, V. Z., Shin, D. C., Orme, M. E., & Zhang, R. (2013). Closed-loop dynamic modeling of cerebral hemodynamics. Annals of biomedical engineering, 1–20. [170] Marmarelis, V. Z., Shin, D. C., Orme, M. E., & Zhang, R. (2013). Model-based Quantification of Cerebral Hemodynamics as a Physiomarker for Alzheimers Disease?. Annals of Biomedical Engineering, 41(11), 2296–2317. [171] Marmarelis, V. Z., Shin, D. C., Song, D., Hampson, R. E., Deadwyler, S. A., & Berger, T. W. (2013). Nonlinear modeling of dynamic interactions within neuronal ensembles using Principal Dynamic Modes. Journal of computational neuroscience, 34(1), 73–87. [172] Martin,G.S.,Mannino,D.M.,Eaton,S.,&Moss,M.(2003).TheepidemiologyofsepsisintheUnited States from 1979 through 2000. New England Journal of Medicine, 348(16), 1546–1554. [173] Martins, P. S., Kallas, E. G., Neto, M. C., Dalboni, M. A., Blecher, S., & Salomao, R. (2003). Upregulation of reactive oxygen species generation and phagocytosis, and increased apoptosis in human neutrophils during severe sepsis and septic shock. Shock, 20(3), 208–212. [174] Marusic, M., & Bajzer, Z. (1993). Generalized two-parameter equation of growth. Journal of mathe- matical analysis and applications, 179(2), 446–462. [175] Mathison, J. C., & Ulevitch, R. J. (1979). The clearance, tissue distribution, and cellular localization of intravenously injected lipopolysaccharide in rabbits. The Journal of Immunology, 123(5), 2133–2143. [176] Matzinger,P.(1994).Tolerance,danger,andtheextendedfamily.Annualreviewofimmunology,12(1), 991–1045. [177] Mensi, S., Naud, R., Pozzorini, C., Avermann, M., Petersen, C. C., Gerstner, W. (2012). Parameter extraction and classification of three cortical neuron types reveals two distinct adaptation mechanisms. Journal of Neurophysiology, 107(6), 1756–75. [178] Metzler, C. M. (1971). Usefulness of the two-compartment open model in pharmacokinetics. Journal of the American Statistical Association, 66(333), 49–53. [179] Mi, Q., Constantine, G., Ziraldo, C., Solovyev, A., Torres, A., Namas, R., ... & Vodovotz, Y. (2011). A dynamic view of trauma/hemorrhage-induced inflammation in mice: principal drivers and networks. PloS one, 6(5), e19424. [180] Michaelis, L., & Menten, M. L. (1913). Die kinetik der invertinwirkung. Biochemische Zeitschrift, 49, 333–369. [181] Mitka, M. (2011). Drug for severe sepsis is withdrawn from market, fails to reduce mortality. JAMA: The Journal of the American Medical Association, 306(22), 2439–2440. [182] Moghe, P. V., Nelson, R. D., & Tranquillo, R. T. (1995). Cytokine-stimulated chemotaxis of human neutrophils in a 3-D conjoined fibrin gel assay. Journal of immunological methods, 180(2), 193–211. 328 [183] Molina, D. K., & DiMaio, V. J. (2012). Normal organ weights in men: part IIthe brain, lungs, liver, spleen, and kidneys. The American Journal of Forensic Medicine and Pathology, 33(4), 368–372. [184] Mosser, D.M., &Edwards, J.P.(2008).Exploringthefullspectrumofmacrophageactivation.Nature reviews. Immunology, 8(12), 958–969. [185] Mueller,M.,delaPena,A.,&Derendorf,H.(2004).Issuesinpharmacokineticsandpharmacodynamics of anti-infective agents: kill curves versus MIC. Antimicrobial agents and chemotherapy, 48(2), 369–377. [186] Munoz, C., Carlet, J., Fitting, C., Misset, B., Bleriot, J. P., & Cavaillon, J. M. (1991). Dysregulation of in vitro cytokine production by monocytes during sepsis. Journal of Clinical Investigation, 88(5), 1747– 1754. [187] Murray, J.D. (2003). Mathematical biology II: spatial models and biomedical applications. New York, NY: Springer. [188] Murray, P. J. (2005). The primary mechanism of the IL-10-regulated antiinflammatory response is to selectively inhibit transcription. Proceedings of the National Academy of Sciences of the United States of America, 102(24), 8686–8691. [189] Nagy, J. D. (2005). The ecology and evolutionary biology of cancer: a review of mathematical models of necrosis and tumor cell diversity. Mathematical biosciences and engineering: MBE, 2(2), 381–418. [190] Namas, R., Zamora, R., Namas, R., An, G., Doyle, J., Dick, T. E., ... & Vodovotz, Y. (2012). Sepsis: Something old, something new, and a systems view. Journal of critical care, 27(3), 314-e1–314.e11. [191] Narahashi, T., Moore, J. W., & Scott, W. R. (1964). Tetrodotoxin blockage of sodium conductance increase in lobster giant axons. The Journal of general physiology, 47(5), 965–974. [192] Naud, R., Gerhard, F., Mensi, S., Gerstner, W. (2011). Improved similarity measures for small sets of spike trains. Neural Computation, 23(12), 3016–69. [193] Nawroth, P. P., Bank, I., Handley, D., Cassimeris, J., Chess, L., & Stern, D. (1986). Tumor necrosis factor/cachectin interacts with endothelial cell receptors to induce release of interleukin 1. The Journal of experimental medicine, 163(6), 1363–1375. [194] Nawroth, P. P., & Stern, D. M. (1986). Modulation of endothelial cell hemostatic properties by tumor necrosis factor. The Journal of experimental medicine, 163(3), 740–745. [195] Nielsen, E. I., Viberg, A., Lwdin, E., Cars, O., Karlsson, M. O., & Sandstrm, M. (2007). Semimech- anistic pharmacokinetic/pharmacodynamic model for assessment of activity of antibacterial agents from time-kill curve experiments. Antimicrobial agents and chemotherapy, 51(1), 128–136. [196] Nieman, G., Brown, D., Sarkar, J., Kubiak, B., Ziraldo, C., Dutta-Moscato, J., ... & Vodovotz, Y. (2012). A two-compartment mathematical model of endotoxin-induced inflammatory and physiologic al- terations in swine. Critical care medicine, 40(4), 1052–1063. [197] Ogura,H.(1985).EstimationofWienerkernelsofanonlinearsystemandafastalgorithmusingdigital Laguerre filters. 15th NIBB Conference, Okazaki, Japan. [198] Oguro, A., Taniguchi, H., Koyama, H., Tanaka, H., Miyata, K., Takeuchi, K., ... & Takahashi, T. (1993). Quantification of human splenic blood flow (Quantitative measurement of splenic blood flow with H2 15O and a dynamic state method: 1). Annals of nuclear medicine, 7(4), 245–250. [199] Oliver, J. C., Bland, L. A., Oettinger, C. W., Arduino, M. J., McAllister, S. K., Aguero, S. M., & Favero, M.S.(1993).Cytokinekineticsinaninvitrowholebloodmodelfollowinganendotoxinchallenge. Lymphokine and cytokine research, 12(2), 115–120. 329 [200] Osuchowski, M. F., Welch, K., Siddiqui, J., & Remick, D. G. (2006). Circulating cytokine/inhibitor profiles reshape the understanding of the SIRS/CARS continuum in sepsis and predict mortality. The Journal of Immunology, 177(3), 1967–1974. [201] Parkhurst, M. R., & Saltzman, W. M. (1992). Quantification of human neutrophil motility in three- dimensional collagen gels. Effect of collagen concentration. Biophysical journal, 61(2), 306–315. [202] Patel, G. P., & Balk, R. A. (2012). Systemic steroids in severe sepsis and septic shock. American journal of respiratory and critical care medicine, 185(2), 133–139. [203] Pearl, R., & Reed, L. J. (1920). On the rate of growth of the population of the United States since 1790 and its mathematical representation. Proceedings of the National Academy of Sciences of the United States of America, 6(6), 275–288. [204] Phillips, B. R., Quinn, J. A., & Goldfine, H. (1994). Random motility of swimming bacteria: single cells compared to cell populations. AIChE journal, 40(2), 334–348. [205] Pillow, J. W., Paninski, L., Uzzell, V. J., Simoncelli, E. P., Chichilnisky EJ. (2005). Prediction and decodingofretinalganglioncellresponseswithaprobabilisticspikingmodel.TheJounralofNeuroscience, 25(47), 11003–13. [206] Pillow, J. W., Shlens, J., Paninski, L., Sher, A., Litke, A. M., Chichilnisky, E. J., Simoncelli, E. P. (2008). Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature, 454(7207), 995–9. [207] Pinsky,M.R.,Vincent,J.L.,Devire,J.,Alegre,M.,Kahn,R.J.,&Dupont,E.(1993).Serumcytokine levels in human septic shock. Relation to multiple-system organ failure and mortality. CHEST Journal, 103(2), 565–575. [208] Poggio, T., and Torre, V. (1977). A Volterra representation for some neuron models. Biological Cy- bernetics, 27(2), 113–24. [209] Poole, D., Bertolini, G., & Garattini, S. (2009). Errors in the approval process and post-marketing evaluationofdrotrecoginalfa(activated)forthetreatmentofseveresepsis.TheLancetinfectiousdiseases, 9(1), 67–72. [210] Price,T.H.,Chatta,G.S.,&Dale,D.C.(1996).Effectofrecombinantgranulocytecolony-stimulating factor on neutrophil kinetics in normal young and elderly humans. Blood, 88(1), 335–340. [211] ProCESS Investigators. (2014). A Randomized Trial of Protocol-Based Care for Early Septic Shock. The New England journal of medicine, 370(18), 1683–1693. [212] Bernard, G. R., Vincent, J. L., Laterre, P. F., LaRosa, S. P., Dhainaut, J. F., Lopez-Rodriguez, A., ... & Fisher Jr, C. J. (2001). Efficacy and safety of recombinant human activated protein C for severe sepsis. New England Journal of Medicine, 344(10), 699–709. [213] Pugliese, A., & Gandolfi, A. (2008). A simple model of pathogenimmune dynamics including specific and non-specific immunity. Mathematical biosciences, 214(1), 73–80. [214] De Ravin, S. S., Naumann, N., Cowen, E. W., Friend, J., Hilligoss, D., Marquesen, M., ... & Malech, H. L. (2008). Chronic granulomatous disease as a risk factor for autoimmune disease. Journal of Allergy and Clinical Immunology, 122(6), 1097–1103. [215] Real, L. A. (1977). The kinetics of functional response. American Naturalist, 111(978), 289–300. 330 [216] Remick, D. G. (2007). Pathophysiology of sepsis. The American journal of pathology, 170(5), 1435– 1444. [217] Reynolds, M. G. (2000). Compensatory evolution in rifampin-resistant Escherichia coli. Genetics, 156(4), 1471–1481. [218] Reynolds, A., Rubin, J., Clermont, G., Day, J., Vodovotz, Y., & Bard Ermentrout, G. (2006). A reduced mathematical model of the acute inflammatory response: I. Derivation of model and analysis of anti-inflammation. Journal of theoretical biology, 242(1), 220–236. [219] Rook, A. H., Kehrl, J. H., Wakefield, L. M., Roberts, A. B., Sporn, M. B., Burlington, D. B., ... & Fauci, A. S. (1986). Effects of transforming growth factor beta on the functions of natural killer cells: depressedcytolyticactivityandbluntingofinterferonresponsiveness.TheJournalofImmunology,136(10), 3916–3920. [220] Rybak, M., Lomaestro, B., Rotschafer, J. C., Moellering, R., Craig, W., Billeter, M., ... & Levine, D. P. (2009). Therapeutic monitoring of vancomycin in adult patients: a consensus review of the American Society of Health-System Pharmacists, the Infectious Diseases Society of America, and the Society of Infectious Diseases Pharmacists. American Journal of Health-System Pharmacy, 66(1), 82–98. [221] Schiff, N. D., Victor, J. D., Canel, A., Labar, D. R. (1995). Characteristic nonlinearities of the 3/s ictal electroencephalogram identified by nonlinear autoregressive analysis. Biological Cybernetics, 72(6), 519–26. [222] Schulte, W., Bernhagen, J., & Bucala, R. (2013). Cytokines in Sepsis: Potent Immunoregulators and Potential Therapeutic TargetsAn Updated View. Mediators of Inflammation, ID 165974. [223] Seeley, E. J., Matthay, M. A., & Wolters, P. J. (2012). Inflection points in sepsis biology: from local defensetosystemicorganinjury.AmericanJournalofPhysiology-LungCellularandMolecularPhysiology, 303(5), L355–L363. [224] Smith, A. M., McCullers, J. A., & Adler, F. R. (2011). Mathematical model of a three-stage innate immune response to a pneumococcal lung infection. Journal of theoretical biology, 276(1), 106–116. [225] Song, D., Chan, R. H., Marmarelis, V. Z., Hampson, R. E., Deadwyler, S. A., Berger, T. W. (2007). Nonlinear dynamic modeling of spike train transformations for hippocampal-cortical prostheses. IEEE Transactions on Biomedical Engineering, 54(6), 1053–1066. [226] Song,D.,Chan,R.H.M.,Marmarelis,V.Z.,Hampson,R.E.,Deadwyler,S.A.,Berger,T.W.(2009). Nonlinear modeling of neural population dynamics for hippocampal prostheses. Neural Networks, 22(9), 1340–51. [227] Stearns-Kurosawa, D. J., Osuchowski, M. F., Valentine, C., Kurosawa, S., & Remick, D. G. (2011). The Pathogenesis of Sepsis. Annual Review of Pathology: Mechanisms of Disease, 6, 19–48. [228] Summers, C., Rankin, S. M., Condliffe, A. M., Singh, N., Peters, A. M., & Chilvers, E. R. (2010). Neutrophil kinetics in health and disease. Trends in immunology, 31(8), 318–324. [229] Swabb, E. A., Wei, J., & Gullino, P. M. (1974). Diffusion and convection in normal and neoplastic tissues. Cancer research, 34(10), 2814–2822. [230] Swirski, F. K., Nahrendorf, M., Etzrodt, M., Wildgruber, M., Cortez-Retamozo, V., Panizzi, P., ... & Pittet, M. J. (2009). Identification of splenic reservoir monocytes and their deployment to inflammatory sites. Science, 325(5940), 612–616. 331 [231] Takahata, T., Tanabe, S., and Pakdaman, K. (2002). White-noise stimulation of the HodgkinHuxley model. Biological Cybernetics, 86(5), 403-17. [232] Takata, M., Moore, J. W., Kao, C. Y., & Fuhrman, F. A. (1966). Blockage of sodium conductance increase in lobster giant axon by tarichatoxin (tetrodotoxin). The Journal of general physiology, 49(5), 977–988. [233] Tan, J. C., Indelicato, S. R., Narula, S. K., Zavodny, P. J., & Chou, C. C. (1993). Characterization of interleukin-10receptorsonhumanandmousecells.JournalofBiologicalChemistry,268(28),21053–21059. [234] Tanaka, H., Ishikawa, K., Nishino, M., Shimazu, T., & Yoshioka, T. (1996). Changes in granulocyte colony-stimulating factor concentration in patients with trauma and sepsis. The Journal of Trauma and Acute Care Surgery, 40(5), 718–726. [235] Taniguchi, T., Koido, Y., Aiboshi, J., Yamashita, T., Suzaki, S., & Kurokawa, A. (1999). Change in theratioofinterleukin-6tointerleukin-10predictsapooroutcomeinpatientswithsystemicinflammatory response syndrome. Critical care medicine, 27(7), 1262–1264. [236] Terashima, T., English, D., & Hogg, J. C. (1998). Release of polymorphonuclear leukocytes from the bone marrow by interleukin-8. Blood, 92(3), 1062–1069. [237] Thomson, A. H., Staatz, C. E., Tobin, C. M., Gall, M., & Lovering, A. M. (2009). Development and evaluation of vancomycin dosage guidelines designed to achieve new target concentrations. Journal of antimicrobial chemotherapy, 63(5), 1050–1057. [238] Vazquez-Torres, A., Jones-Carson, J., Mastroeni, P., Ischiropoulos, H., & Fang, F. C. (2000). An- timicrobial actions of the NADPH phagocyte oxidase and inducible nitric oxide synthase in experimental salmonellosis. I. Effects on microbial killing by activated peritoneal macrophages in vitro. The Journal of experimental medicine, 192(2), 227–236. [239] Tracey, K. J., Beutler, B., Lowry, S. F., Merryweather, J., Wolpe, S., Milsark, I. W., ... & Cerami, A. (1986).ShockandTissueInjuryInducedbyRecombinantHumanCachectin.Science,234(4775), 470–474. [240] Tsujimoto,M.,Yip,Y.K.,&Vilcek,J.(1985).Tumornecrosisfactor: specificbindingandinternaliza- tion in sensitive and resistant cells. Proceedings of the National Academy of Sciences, 82(22), 7626–7630. [241] Tsunawaki,S.,Sporn,M.,Ding,A.,&Nathan,C.(1988).Deactivationofmacrophagesbytransforming growth factor-beta. Nature, 334(6179), 260–262. [242] Ulich, T. R., del Castillo, J., Ni, R. X., & Bikhazi, N. (1989). Hematologic interactions of endotoxin, tumornecrosisfactoralpha(TNFalpha), interleukin1, andadrenalhormonesandthehematologiceffects of TNF alpha in Corynebacterium parvum-primed rats. Journal of leukocyte biology, 45(6), 546–557. [243] Ulich, T.R., delCastillo, J., Ni, R.X., Bikhazi, N., &Calvin, L.(1989).Mechanismsoftumornecrosis factor alpha-induced lymphopenia, neutropenia, and biphasic neutrophilia: a study of lymphocyte recir- culation and hematologic interactions of TNF alpha with endogenous mediators of leukocyte trafficking. Journal of leukocyte biology, 45(2), 155–167. [244] vandenBerg,J.M.,Weyer,S.,Weening,J.J.,Roos,D.,&Kuijpers,T.W.(2001).Divergenteffectsof tumor necrosis factor α on apoptosis of human neutrophils. Journal of leukocyte biology, 69(3), 467–473. [245] van Hinsbergh, V. W. (2012). Endotheliumrole in regulation of coagulation and inflammation. In Seminars in immunopathology, 34(1) 93–106. 332 [246] Vogels, M., Zoeckler, R., Stasiw, D. M., & Cerny, L. C. (1975). PF Verhulst’s “notice sur la loi que la populations suit dans son accroissement” from correspondence mathematique et physique. Ghent, vol. X, 1838. Journal of Biological Physics, 3(4), 183–192. [247] Victor,J.D.,andCanel,A.(1992).ArelationbetweentheAkaikecriterionandreliabilityofparameter estimates, with application to nonlinear autoregressive modelling of ictal EEG. Annals of Biomedical Engineering, 20(2), 167–80. [248] Vodovotz, Y., Constantine, G., Rubin, J., Csete, M., Voit, E. O., & An, G. (2009). Mechanistic simulations of inflammation: current state and future prospects. Mathematical biosciences, 217(1), 1–10. [249] Volterra, V. (1930). Theory of functionals and of integro-differential equations. New York: Dover Publications. [250] Volterra, V. Variations and fluctuations of the number of individuals in animal species living together, translated by M. E. Wells in R.N. Chapman, Animal Ecology, New York, 1931. [251] von Bertalanffy, L. (1938). A quantitative theory of organic growth (inquiries on growth laws. II). Human biology, 10(2), 181–213. [252] von Bertalanffy, L. (1949). Problems of organic growth. Nature, 163(4135), 156–158. [253] Waage, A. (1987). Production and clearance of tumor necrosis factor in rats exposed to endotoxin and dexamethasone. Clinical immunology and immunopathology, 45(3), 348–355. [254] Wahl, S. M., Hunt, D. A., Wakefield, L. M., McCartney-Francis, N., Wahl, L. M., Roberts, A. B., & Sporn, M. B. (1987). Transforming growth factor type beta induces monocyte chemotaxis and growth factor production. Proceedings of the National Academy of Sciences, 84(16), 5788–5792. [255] Wajant, H., Pfizenmaier, K., & Scheurich, P. (2003). Tumor necrosis factor signaling. Cell Death & Differentiation, 10(1), 45–65. [256] Wang, H., Bloom, O., Zhang, M., Vishnubhakat, J. M., Ombrellino, M., Che, J., ... & Tracey, K. J. (1999). HMG-1 as a late mediator of endotoxin lethality in mice. Science, 285(5425), 248–251. [257] Watanabe, N., Kuriyama, H., Sone, H., Neda, H., Yamauchi, N., Maeda, M., & Niitsu, Y. (1988). Continuous internalization of tumor necrosis factor receptors in a human myosarcoma cell line. Journal of Biological Chemistry, 263(21), 10262–10266. [258] Watanabe, A., & Stark, L. (1975). Kernel method for nonlinear analysis: identification of a biological control system. Mathematical Biosciences, 27(1), 99-108. [259] Werner, S. L., Kearns, J. D., Zadorozhnaya, V., Lynch, C., ODea, E., Boldin, M. P., ... & Hoffmann, A. (2008). Encoding NF-B temporal control in response to TNF: distinct roles for the negative regulators IB and A20. Genes & development, 22(15), 2093–2101. [260] Wiener, N. (1958). Nonlinear problems in random theory. Cambridge, MA: MIT Press. [261] Wiener, N. (1956). The theory of prediction. In: Beckenbach, E. (Ed.), Modern Mathematics for Engineers. McGraw-Hill, New York. [262] Wise, R., Lister, D., McNulty, C. A. M., Griggs, D., & Andrews, J. M. (1986). The comparative pharmacokinetics of five quinolones. Journal of Antimicrobial Chemotherapy, 18(Supplement D), 71–81. [263] Wise, R., Lockley, R. M., Webberly, M., & Dent, J. (1984). Pharmacokinetics of intravenously admin- istered ciprofloxacin. Antimicrobial agents and chemotherapy, 26(2), 208-210. 333 [264] Yang, S. K., Wang, Y. C., Chao, C. C., Chuang, Y. J., Lan, C. Y., & Chen, B. S. (2010). Dynamic cross-talkanalysisamongTNF-R,TLR-4andIL-1RsignalingsinTNFα-inducedinflammatoryresponses. BMC medical genomics, 3(1), 19. [265] Young, T. R., Buckalew, R., May, A. K., & Boczko, E. M. (2012). A low dimensional dynamical model of the initial pulmonary innate response to infection. Mathematical biosciences, 235(2), 189–200. [266] Zanos, T. P., Courellis, S. H., Berger, T. W., Hampson, R. E., Deadwyler, S. A., Marmarelis, V. Z. (2008). Nonlinear modeling of causal interrelationships in neuronal ensembles. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 16(4), 336–52. [267] Zeni, F., Freeman, B., & Natanson, C. (1997). Anti-inflammatory therapies to treat sepsis and septic shock: a reassessment. Critical care medicine, 25(7), 1095–1100. [268] Zhang, H., Niesel, D. W., Peterson, J. W., & Klimpel, G. R. (1998). Lipoprotein release by bacteria: potential factor in bacterial pathogenesis. Infection and immunity, 66(11), 5196–5201. [269] Zhao, X., and Marmarelis, V. Z. (1997) On the relation between continuous and discrete nonlinear parametric models. Automatica, 33(1), 81–4. [270] Zhao, X., and Marmarelis, V. Z. (1998) Nonlinear parametric models from Volterra kernels measure- ments. Mathematical Computational Modeling, 27(5), 37–43. [271] Zhi, J., Nightingale, C. H., & Quintiliani, R. (1988). Microbial pharmacodynamics of piperacillin in neutropenic mice of systematic infection due to Pseudomonas aeruginosa. Journal of pharmacokinetics and biopharmaceutics, 16(4), 355–375. [272] Zhuge, C., Lei, J., & Mackey, M. C. (2012). Neutrophil dynamics in response to chemotherapy and G-CSF. Journal of theoretical biology, 293, 111–120. 334

Abstract (if available)

Abstract Fundamental to physiology is the concept of homeostasis, the dynamic maintenance of an equilibrium that is, in general, in disequilibrium with the larger environment. Physiologic systems must, then, be able to respond to exogenous perturbations in the environment and to their internal state. That is, physiologic systems can be understood as a set of input‐to‐output transformations between system variables, both exogenous and endogenous. Such systems are quite complex, and knowledge must be summarized in the form of models, either informal or formal. Formal mathematical modeling approaches can be either parametric or non‐parametric. Non‐parametric modeling represents an inductive approach, wherein the model form is estimated directly from observed time‐series data. The fundamental advantage of the non‐parametric approach is that it can effectively summarize very complex systems when little is known about the internal workings. The Volterra series expansion is a canonical non‐parametric methodology for representing the input‐output functional relationship between any two variables. However, as classically formulated, the Volterra series cannot be applied to nonlinear oscillators or chaotic systems, nor can it capture autonomous systems behaviors not directly driven by an exogenous input. Moreover, even when a strict input‐output formulation is theoretically proper, internal system feedbacks or recursion can give rise to inefficiencies in the input‐output functional representation. By relaxing the strict input‐output requirement of the classical formulation and making the outputs recursive, we expand dramatically the class of systems that are amenable to Volterra‐style modeling, including those that operate strictly autonomously. We have demonstrated the method's efficacy on several test systems. In the single‐output setting, auto‐recursive Volterra models can capture autonomous behaviors, such as action potential firing in neural systems or population growth in ecological systems. In the multi‐output case, in which outputs of interest interact dynamically, the concept of recursive outputs allows ""closed"" or ""nested"" loop configurations to be captured in a Volterra‐style framework. ❧ Bacterial sepsis is characterized by a complex inflammatory response, and there has been significant interest in modeling this process as a dynamical system. It is believed that the pathology of sepsis is largely due to the immune response itself, which has motivated the clinical strategy of blocking pro‐inflammatory cytokines. However, this strategy has uniformly failed in clinical trials, motivating a systems approach to the disorder. We have shown that the recursive Volterra methodology can describe both autonomous growth, e.g. that of bacteria, and can capture the dynamics of simulated endotoxemia, a simplified model for actual sepsis. Therefore, we were hopeful that it could be applied to clinical time‐series data for actual sepsis patients. However, the temporal resolution of available data has proven too coarse, and thus the constraints imposed by the data have necessitated a parametric approach to modeling this system. Parametric modeling represents a deductive, hypothesis‐driven methodology, and we build our parametric representation of bacterial infection and the immune response in a hierarchical manner. This allows a careful examination of how the incorporation of different mechanisms, and different formal representations for these mechanisms, affects overall model dynamics and predictions. This approach has generated several predictions salient with respect to both future modeling efforts and to the clinical management of sepsis. We find that the interaction of local and systemic macrophage and neutrophil populations, respectively, is essential both for efficiently mobilizing the immune system and for resolving inflammation. Antibiotic treatment is modeled and found to interact synergistically with the immune response

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Nonlinear dynamical modeling of single neurons and its application to analysis of long-term potentiation (LTP)

PDF

Nonlinear modeling of causal interrelationships in neuronal ensembles: an application to the rat hippocampus

PDF

Principal dynamic mode analysis of cerebral hemodynamics for assisting diagnosis of cerebrovascular and neurodegenerative diseases

PDF

Parametric and nonparametric modeling of cerebral hemodynamics

PDF

Simulating electrical stimulation and recording in a multi-scale model of the hippocampus

PDF

Nonlinear models for hippocampal synapses for multi-scale modeling

PDF

Modeling autonomic peripheral vascular control

PDF

Systems pharmacology for acute myeloid leukemia and feedback control pharmacodynamic models

PDF

Neural spiketrain decoder formulation and performance analysis

PDF

Pattern detection in medical imaging: pathology specific imaging contrast, features, and statistical models

PDF

Experimental and computational models for seizure prediction

PDF

A parallel computation framework for EONS synaptic modeling platform for parameter optimization and drug discovery

PDF

Investigating the role of muscle physiology and spinal circuitry in sensorimotor control

PDF

Animal to human scaling and pharmacokinetic/pharmacodynamic modeling of anticancer drugs

PDF

Computational investigation of glutamatergic synaptic dynamics: role of ionotropic receptor distribution and astrocytic modulation of neuronal spike timing

PDF

Ribavirin systemic and intracellular exposure and its role on viral response and anemia in chronic hepatitis C genotype 1 infection

PDF

Decoding memory from spatio-temporal patterns of neuronal spikes

PDF

A million-plus neuron model of the hippocampal dentate gyrus: role of topography, inhibitory interneurons, and excitatory associational circuitry in determining spatio-temporal dynamics of granul...

PDF

Diffusion MRI of the human brain: signal modeling and quantitative analysis

PDF

Applications of graph theory to brain connectivity analysis

Asset Metadata

Creator Eikenberry, Steffen E. (author)

Core Title Parametric and non‐parametric modeling of autonomous physiologic systems: applications and multi‐scale modeling of sepsis

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Biomedical Engineering

Publication Date 08/08/2014

Defense Date 06/02/2014

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag cytokines,IL-10,Inflammation,Lotka‐Volterra,mathematical model,OAI-PMH Harvest,sepsis,TNF,Volterra model

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Marmarelis, Vasilis Z. (committee chair), D'Argenio, David Z. (committee member), Grzywacz, Norberto M. (committee member), Wong-Beringer, Annie (committee member)

Creator Email seikenbe@usc.edu,steffen.eikenberry@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-456998

Unique identifier UC11286752

Identifier etd-Eikenberry-2791.pdf (filename),usctheses-c3-456998 (legacy record id)

Legacy Identifier etd-Eikenberry-2791.pdf

Dmrecord 456998

Document Type Dissertation

Format application/pdf (imt)

Rights Eikenberry, Steffen E.

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA