Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Data-driven multi-fidelity modeling for physical systems
(USC Thesis Other)
Data-driven multi-fidelity modeling for physical systems
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
Data-Driven Multi-Fidelity Modeling for Physical Systems
by
Orazio Pinti
A Dissertation Presented to the
FACULTY OF THE GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(AEROSPACE ENGINEERING)
May 2024
Copyright 2024 Orazio Pinti
Acknowledgments
First and foremost, I would like to extend my sincerest gratitude to my advisor, Prof. Assad
A. Oberai, for trusting me and granting me a life-changing opportunity. His guidance, driven
by competence, intuition, and most importantly kindness, allowed me to grow immensely,
both on a professional and personal level.
Further, I would like to thank the members of my doctoral committee, Prof. Mitul
Luhar and Prof. Roger Ghanem, and the members of my qualifying exam committee, Prof.
Ivan Bermejo-Moreno and Prof. Paul Newton, for their feedback and support. Special
appreciation goes to the incredible collaborators I have had the honor to work with over the
course of the last years: Prof. Deep Ray, for his continuous support and help, Prof. Farhan
Gandhi and the folks at the Center for Mobility with Vertical Lift (MOVE) at Rensselaer
Polytechnic Institute, Prof. Franca Hoffmann, and Dr. Jeremy Budd.
A special thanks goes to all the caring and great people on this side of the world I am
lucky enough to call friends: Stefano, who has been a real friend for over a decade; Alberto
and Fares, my first PhD friends and roommates whom I shared so much with (especially
pizza); Javier, for his kind nature, solid presence, and limitless help; Maria and Regina, for
their sweet thoughtfulness; and all my lab mates who made this journey memorable.
It is hard to end a journey without looking back to where it started. I would like to
thank all my friends of Barzizza’s for an endless source of joy, laughter, irrational absurdity,
and surrealism, which added to my life much more than I could have ever asked.
Finally, I want to dedicate a few words to some of the most important people in my
life, without whom nothing I have ever done would have been possible. I want to deeply
thank my grandmothers, Maria and Ada, for setting an example of the highest standard of
inner strength and work ethics, and for showing me the utter dignity and grace of selflessly
taking care of others with unconditional love. I want to thank my grandfather, Dario, for
ii
having helped me come to America, for teaching me to “never look back in life with sadness,
but always ahead with excitement and drive, while keeping a fond and grateful look in my
rear-view mirror”, and, especially, for being proud of me.
My mom and dad, Paola and Egidio, worked tirelessly every day to gift me the privilege
of strong roots. Having a place that, no matter what, I can call home is invaluable; I will
never be able to express all my gratitude for that. My brother Davide cured my frenetic
urgency with his stoic, ironic, and grounded perspective on life. I will never not laugh my
heart out with him around. I thank them all from the bottom of my heart.
iii
Table of Contents
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Chapter 1:
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Multi-fidelity modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Literature review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Contributions of this work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Chapter 2:
Multi-fidelity modeling for rotor to rotor aerodynamic interaction . . . . 8
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.1 High-fidelity model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.2 Low-fidelity model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2.3 Multi-fidelity model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3 Numerical results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3.1 Construction of the multi-fidelity model . . . . . . . . . . . . . . . . 21
2.3.2 Testing the performance of the multi-fidelity model . . . . . . . . . . 23
2.3.3 Prediction of integrated rotor loads . . . . . . . . . . . . . . . . . . . 27
2.3.4 Effect of selection of important snapshots . . . . . . . . . . . . . . . . 33
2.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Chapter 3:
Graph Laplacian-based spectral multi-fidelity modeling . . . . . . . . . . . 36
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.2.2 Spectral Multi-Fidelity method (SpecMF) . . . . . . . . . . . . . . . 38
3.2.3 Theoretical Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.3 Numerical results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
iv
3.3.1 Traction on a soft material with a stiff inclusion . . . . . . . . . . . . 53
3.3.2 Aerodynamic coefficients for a family of NACA airfoils . . . . . . . . 60
3.4 Effect of the selection strategy of high-fidelity data . . . . . . . . . . . . . . 65
3.5 Definition of the data space . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.6 Numerical results with augmented data space . . . . . . . . . . . . . . . . . 67
3.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Chapter 4:
B-SpecMF: a Bayesian extension . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.2.1 Distribution for the displacement . . . . . . . . . . . . . . . . . . . . 73
4.2.2 Hyperparameter specification . . . . . . . . . . . . . . . . . . . . . . 76
4.3 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.3.1 Synthetic data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.3.2 Traction on a soft material with a stiff inclusion . . . . . . . . . . . . 80
4.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
Chapter 5:
Future directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
v
List of Tables
2.1 Relative l2 norm error in RMAC and multi-fidelity models in lift distribution. 25
2.2 Relative l2 norm error in RMAC and multi-fidelity models in drag distribution. 26
2.3 Error in thrust computed using RMAC and multi-fidelity models. . . . . . . 31
2.4 Error in torque computed using RMAC and multi-fidelity models. . . . . . . 33
2.5 Relative l2 norm error in multi-fidelity models for lift and drag distribution. . 34
3.1 Range spanned by input parameters for the traction and airfoil problems. . . 55
3.2 Error for the low-fidelity, co-kriging, and SpecMF data for each output component. For the SpecMF the improvement factor, defined as the ratio of the
low- and multi-fidelity error, is also reported. . . . . . . . . . . . . . . . . . . 59
3.3 Errors in the multi-fidelity model constructed with the proposed and random
selection strategies. For the random strategy, we report the mean and the
standard deviation (in parenthesis). . . . . . . . . . . . . . . . . . . . . . . . 66
3.4 Error of the low- and multi-fidelity model for each quantity of interest. . . . 68
4.1 Error for the low-fidelity and Bayesian SpecMF (B-SpecMF) data for each
output component. For the B-SpecMF multi-fidelity estimates, we also include
the percentage of points for which the difference with respect to the highfidelity value is within two standard deviation away. . . . . . . . . . . . . . . 84
vi
List of Figures
1.1 Schematic of low-, high-, and multi-fidelity models in a error vs cost graph. 3
2.1 Rotors blade planform, reproduced from [40] . . . . . . . . . . . . . . . . . . 11
2.2 Longitudinal and vertical distance dx and dy . . . . . . . . . . . . . . . . . . 13
2.3 The snapshot is generated by concatenating the lift or drag load distributions
for the front and rear rotors sampled at D discrete grid points. . . . . . . . . 13
2.4 CFD simulation domain and mesh . . . . . . . . . . . . . . . . . . . . . . . . 15
2.5 Mesh visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.6 High- and low-fidelity simulations in the parameter space. The high-fidelity
snapshots used to construct the multi-fidelity are indicated in green, and the
ones used to test the results are colored in red. . . . . . . . . . . . . . . . . . 21
2.7 Distribution of relative error in the parameter space using the training set as
basis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.8 Test Point 2: Low- and Multi-Fidelity lift disk plots (Units N/m) . . . . . . 24
2.9 Test Point 2: Low- and Multi-Fidelity drag disk plots (Units N/m) . . . . . 24
2.10 Test Point 4: Low- and Multi-Fidelity lift disk plots (Units N/m) . . . . . . 25
2.11 Test Point 4: Low- and Multi-Fidelity drag disk plots (Units N/m) . . . . . 26
2.12 Test Point 5: Low- and Multi-Fidelity lift disk plots (Units N/m) . . . . . . 27
2.13 Test Point 5: Low- and Multi-Fidelity drag disk plots (Units N/m) . . . . . 28
2.14 Test Point 6: Low- and Multi-Fidelity lift disk plots (Units N/m) . . . . . . 29
vii
2.15 Test Point 6: Low- and Multi-Fidelity drag disk plots (Units N/m) . . . . . 30
2.16 Test Point 8: Low- and Multi-Fidelity lift disk plots (Units N/m) . . . . . . 31
2.17 Test Point 8: Low- and Multi-Fidelity drag disk plots (Units N/m) . . . . . 32
2.18 Low- and Multi-Fidelity Thrust and Torque distribution over the parameter
space. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.19 High- and low-fidelity simulations in the parameter space. The high-fidelity
snapshots used to construct the multi-fidelity are indicated in green, and the
ones used to test the results are colored in red. . . . . . . . . . . . . . . . . . 34
3.1 Workflow for the spectral multi-fidelity (SpecMF) method applied to an illustrative problem. (a) Generate low-fidelity data. (b) Compute a graph
Laplacian using the low-fidelity data. (c) Compute the eigen-decomposition
of the graph Laplacian. (d) Perform spectral clustering of the low-fidelity
data and find the points closest to the clusters centroids. (e) Acquire highfidelity data only for these points. (f) Solve a convex minimization problem
to find one influence function for each point with a high-fidelity counterpart.
The influence functions are constructed from the low-lying eigenfunctions of
graph Laplacian. (g) Generate the multi-fidelity approximation of the data
set. (h) For this illustrative example, this is compared with the corresponding
high-fidelity data set. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.2 L-curve for the traction problem in Section 3.3.1. The optimal value for the
regularization parameter is the one corresponding to the elbow of the curve,
marked with a triangle in the graph. . . . . . . . . . . . . . . . . . . . . . . 45
3.3 Schematic of a canonical problem with the data set partitioned into M = 3
clusters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.4 (a) Schematic of the soft body (light grey) with the elliptic stiffer inclusion
(dark grey). The square is compressed on top with a uniform displacement
v = v0, while the bottom is fixed. The vertical traction is integrated over the
top side across equal sections to compute the localized forces fi
. (b) Schematic
of the airfoil with the input parameters. . . . . . . . . . . . . . . . . . . . . 54
3.5 Comparison between the two meshes of the low- and high-fidelity finite element
models used to solve the traction on soft material problem. . . . . . . . . . . 55
3.6 Sample of low- and high-fidelity traction fields for different inclusions. . . . . 56
viii
3.7 Results for the traction problem. (a) Low-fidelity, co-kriging, SpecMF and
high-fidelity data. Low-fidelity data points are shown together with the points
closest to the centroids of the clusters (in blue). (b) Eight eigenfunctions
from the low-lying spectrum projected onto the (f2, f3) plane. (c) Influence
functions for eight control points projected onto the (f2, f3) plane. (d) Error
distribution for the low-fidelity and SpecMF model for each output component. 57
3.8 Results for the entire traction field problem. (a) Low-, high-, and multifidelity solutions for four test cases. (b) Error distribution for the low-fidelity
and SpecMF solutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.9 Example of a mesh used for the high-fidelity CFD simulations. . . . . . . . . 62
3.10 Results for the airfoil problem. (a) Left: low-fidelity data with points closest
to cluster centroids (in blue). Center: multi-fidelity data points obtained with
co-kriging and our multi-fidelity approach (SpecMF). Right: high-fidelity data
points used for validation. (b) Error distribution for the low- and multi-fidelity
data. (c) The first three non-trivial eigenfunctions. (d) Three typical influence
functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.11 Comparison of the curves of the lift, drag and moment coefficients versus angle
of attack for the airfoil NACA 0012 at Re = 6 · 106
. The low-fidelity data are
indicated with a dashed orange line, the multi-fidelity data with a dash-dot
blue line, the high-fidelity data with a solid red line, and the experimental
data are marked with a green upside-down triangle. . . . . . . . . . . . . . . 65
3.12 Schematic of a case where adding the parameter µ1 to the data space (q1, q2)
separates the data set into two distinct clusters, C1 and C2. . . . . . . . . . . 68
4.1 Low- and high-fidelity points for the synthetic dataset problem. . . . . . . . 78
4.2 (a) Points used for training and validation. (b) Low-fidelity data used for
training (larger blue dots), connected with a red segment with their highfidelity counterparts (large red dots). The red segments represent the known
displacements stored in the matrix Zˆ. . . . . . . . . . . . . . . . . . . . . . . 79
4.3 Plot of the log-likelihood of the validation dataset ℓ(ω). The maximum is
achieved for ω
∗ = 4.93. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.4 Comparison of the low-, multi- and the underlying high-fidelity datasets. The
third plot shows the circles indicating the standard deviation associated with
each multi-fidelity estimate. . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
ix
4.5 Multi-fidelity points colored with the value of standard deviation. The training data points are indicated by large red dots. . . . . . . . . . . . . . . . . . 81
4.6 (a) Points used for training. (b) Points used for validation. The data is
presented by projection onto two-dimensional planes. . . . . . . . . . . . . . 82
4.7 Plot of the log-likelihood of the validation dataset ℓ(ω). The maximum is
achieved for ω
∗ = 181.48. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.8 Comparison of the low-, multi- and high-fidelity datasets projected over the
4 planes (ui
, ui+1), i = 1, . . . , 4. The third columns shows circles indicating
the standard deviation associated with each multi-fidelity estimate. . . . . . 85
x
Abstract
This work is focused on the development and implementation of data-driven multi-fidelity
models for physical systems.
Often times, when facing problems such as optimization, uncertainty quantification, or
characterization of a parametric physical system, a large number of simulations is required.
If multiple models of different fidelity and cost are available to perform these simulations,
a multi-fidelity approach can be effective to decrease the overall computational cost of the
task, while maintaining the required level of accuracy. The foundational idea is to define a
strategy that combines data and results from the different models to construct a multi-fidelity
dataset that inherits most of the accuracy of the higher-fidelity model, but at a fraction of
the cost.
We present three different strategies. The first approach exploits the low-fidelity model
to span the input space and generate a large set of data (or snapshots). Then, a low-rank
representation of the snapshots is found by computing a suitable basis. Finally, high-fidelity
data is acquired, and a lifting procedure is used to increase the accuracy of the low-rank
representation. This strategy is applied to study the effect of the separation between the
rotors of a two-rotor assembly on its aerodynamics performance. It is shown how this strategy
can substantially improve upon the low-fidelity results.
The second method that is put forth is a multi-fidelity model based on the spectral
properties of the graph Laplacian. A graph is constructed using the low-fidelity data as nodes,
and a graph Laplacian and its eigen-decomposition are evaluated. After the high-fidelity data
acquisition, a transformation for all the low-fidelity points is defined as an expansion in terms
of the graph Laplacian eigen-functions. The coefficients of this expansion are computed by
minimizing a data misfit term and a regularization term, which penalizes the use of higherorder eigen-functions. Numerical experiments in both solid and fluid mechanics shown how
xi
this approach is effective to learn the low-fidelity data distribution and transform it based
on a small number of high-fidelity data points.
The third method is a probabilistic extension of the previous one. Indeed, the problem of
determining a multi-fidelity approximation of the data is cast as a Bayesian inference problem. The transformation from low- to multi-fidelity points is defined through a probability
distribution, determined by a graph Laplacian-regularized prior, and a data-misfit likelihood
term. This approach is more general, as the form of the transformation is not assumed a
priori. Furthermore, it enables to perform uncertainty quantification studies by analysing of
the statistical properties of the posterior distribution.
xii
Chapter 1
Introduction
1.1 Multi-fidelity modeling
In many interesting tasks in computational engineering, such as optimization, uncertainty
quantification, sensitivity analysis, and design, a large number of simulations of a physical
system of interest is often required. Usually, for that, a model of the system has to be
evaluated multiple times. In this context, a model represents a computable approximation
of the real input-output relation of the system. This can be either an explicit mathematical
model of the physics of the system, or a more general black-box type of solver. The input
is any set of parameters or initial conditions needed to be specified for the evaluation, and
the output comprises all the resulting quantities of interest. The evaluation or the query of
a computational model amounts to implementing it numerically, running a simulation, and
computing the output.
Based on how a model is derived and implemented, it can exhibit different levels of fidelity
and cost. The fidelity of a model is a measure of the discrepancy between its prediction and
the true response of the physical system. On the other hand, the cost of a model quantifies
the computational resources needed for its evaluation. In general, a higher fidelity demands
a greater cost.
In practical scenarios, it is very common to have access to more than one model for the
1
same system, or to be able to tune some hyper-parameters of a given model to increase
its accuracy (and cost), such as mesh size, or the integration time step. For example, in
computational fluid dynamics one can choose a direct numerical simulation (DNS) model
or a Reynolds-averaged Navier-Stokes (RANS) model to simulate the same flow, leading to
more or less accurate (and expensive) results. Similarly, in solid mechanics, one can choose
a more or less fine mesh to discretize the domain, thus incurring in different levels of error
in the results. It is then useful to distinguish between high- and low-fidelity models.
A model is said to be high-fidelity if it can capture the true underlying behavior of the
system within a level of accuracy that is equal to or greater than the one required for the
given task. If a model is not high-fidelity, it is said to be low-fidelity.
Lower-fidelity models are designed to trade some of the predictive accuracy in favor
of a more competitive evaluation cost. This can be attained in several ways. The most
common methods include: using simplified physics or making stronger modeling assumptions,
linearizing the dynamics of the system, employing projection-based or data-fitting surrogate
model, or using coarser numerical discretizations. It is worth noting that a high- vs lowfidelity model pair can also be represented by experiments and numerical simulations.
Completing a task that requires a large number of simulations solely relying on a highfidelity model is typically prohibitive or impractical, while employing only lower-fidelity
models might not lead to results that are accurate enough. This is where multi-fidelity
models come into play. The goal of multi-fidelity methods is to retain as much accuracy
as possible from a high-fidelity model, while incurring a fraction of the cost by leveraging
lower-fidelity models. More specifically, consider a set of models of different fidelity and cost,
and a task that requires a large number of evaluations. Then, one can pose the problem of
finding a strategy to combine the predictions from some or all these models so that a given
accuracy metric (e.g., the minimum or average accuracy) is maximized across all evaluations,
while ensuring that the total computational cost remains within a predefined budget. Or,
similarly, combining the predictions from different models so that the total computational
2
Figure 1.1: Schematic of low-, high-, and multi-fidelity models in a error vs cost graph.
cost is minimized, while the minimum required accuracy is still achieved. In other words,
the general objective of multi-fidelity methods is constructing models that exhibit cost-toaccuracy predictions that are as effective as possible (see Figure 1.1).
Based on the application, the multi-fidelity strategy can either combine data from different sources in a single surrogate model, or define a criterion to decide which model at what
accuracy and cost ought to be evaluated at each step of the algorithm. This is also referred
by Peherstorfer et al. [1] as the model management strategy.
In a typical multi-fidelity framework, low-fidelity data is generated to explore the input
space, and obtain an approximation of the response of the system. Then, a limited number of
high-fidelity data points are computed or measured, and techniques that learn the response
from the low-fidelity data, and improve it by using the high-fidelity data are applied.
In this work, we focus on methods where only two models are available. This simplifies
the task of selecting which model has to be evaluated and when. With this in mind, we
consider the following three steps blueprint for constructing a multi-fidelity model:
1. Generate low-fidelity data.
2. Analyze the low-fidelity data and determine a high-fidelity data acquisition policy
(unless also the high-fidelity data are generated beforehand).
3
3. Define a strategy to “inject” the accuracy of the few select high-fidelity data to the
larger lower-fidelity dataset or in the decision-making process.
1.2 Literature review
Multi-fidelity methods have been widely used in optimization [2, 3], uncertainty quantification [4], uncertainty propagation [5, 1], and statistical inference [6] (see Fern´andez-Godino
et al. [7] and Peherstofer et al. [1] for two comprehensive reviews).
In uncertainty quantification, the parameter inputs are usually modeled as random variables drawn from a prescribed distribution, and the general objective is to compute statistics
of the output. If more than one model for the output is available, one can exploit the correlation of the different approximations, which play the role of auxiliary random variables
whose statistics are easier to compute. Along the lines of multilevel Monte Carlo methods
[8, 9], different techniques to construct an unbiased estimator of the mean and variance of
the highest-fidelity model output based on lower-fidelity approximations have been proposed
[10, 11]. In [12] a multi-fidelity multi-level method is proposed, when different models and
different levels of accuracy for each model are considered at the same time. In [11], an
optimal strategy to share the computational resources across different models so that the
variance of a multi-fidelity estimator is minimized is also put forth.
In co-kriging methods [13, 14, 15, 16] the multi-fidelity response is expressed as a weighted
sum of two Gaussian processes, one modeling the low-fidelity data, and the other representing
the discrepancy between the low- and high-fidelity data. The parameters of the mean and
correlation functions of these processes are determined by maximizing the log-likelihood
of the available data. Co-kriging has also been extensively investigated in the context of
multi-fidelity optimization [17, 18, 19, 20].
Other methods make use of radial basis functions (RBFs) to model the low-fidelity response. Specifically, the low-fidelity surrogate is written as an expansion in terms of a
4
set of RBFs, and the coefficients are determined by interpolating the available low-fidelity
data. The multi-fidelity approximation is then obtained in different ways. These include
determining a scaling factor and a discrepancy function, which can be modeled using a
kriging surrogate [16], or another expansion in terms of RBFs [21, 22]. In some cases the
multi-fidelity surrogate is constructed by mapping the low-fidelity response directly to the
high-fidelity response [23].
More recently, deep neural networks have been used to fit low-fidelity data and learn the
complex map between the input and output vectors in the low-fidelity model. Then, the
relatively small amount of high-fidelity data is used in combination with techniques such
as transfer learning [24, 25], embedding the knowledge of a physical law through physicsinformed loss functions [26, 27, 28, 29], or, in the case of multiple levels of fidelity, concatenating multiple neural networks together [30]. An approach that involves training a
physics-constrained generative model, conditioned on the low-fidelity snapshots, to produce
solutions that are higher-fidelity and higher-resolution has also been proposed [31].
Another class of methods, suitable when the response of the system consists of a highdimensional vector, first performs order-reduction using the low-fidelity data, and then inject
accuracy using the high-fidelity data in a reduced-dimensional latent space. This has been
accomplished by computing the low- and high-fidelity Proper Orthogonal Decomposition
(POD) manifolds, aligning them with each other, and replacing the low-fidelity POD modes
with their high-fidelity counterparts [32]. This has also been done by first solving a subset
selection problem to construct a surrogate model of the low-fidelity response in terms of a
few important snapshots, then generating their high-fidelity counterparts, and finally using
these in the multi-fidelity surrogate model [33, 34, 35].
5
1.3 Contributions of this work
In what follows, we present three different multi-fidelity approaches. In Chapter 2, we
present a method introduced by Narayan et al. [33] and apply it to a problem of interactional aerodynamics. Here, after a large amount of low-fidelity data is computed, a low-rank
representation is obtained by finding a suitable basis to express the snapshots. A major
difference with the common Proper Orthogonal Decomposition (POD) method is that the
elements of such basis are sought to be snapshots themselves, and therefore the basis will
not be, in general, orthonormal. The key parameter values corresponding to the snapshots
forming the basis are chosen to query the high-fidelity model, and the resulting high-fidelity
snapshots are used in a lifting procedure to determine multi-fidelity solutions at other parameter values. We use this method to tackle the problem of predicting the aerodynamics
performance (defined by total thrust and torque) of a two-rotor assembly in forward flight
as a function of the rotor-to-rotor vertical and longitudinal spacing. The two models used
for this study are a LES-based CFD solver, which provides a very accurate representation
of the rotor behavior, and a blade-element theory based solver, which is extremely efficient
to query, but also less accurate.
In Chapter 3 we present the second strategy, a multi-fidelity model based on the spectral
properties of the graph Laplacian. The starting point is similar to the previous method,
that is to compute a large number of low-fidelity data to approximate their distribution.
Thereafter, each data point is treated as the node of a weighted graph, where the weights of
the edges are defined based on the distance in the data space. A normalized graph Laplacian
and its eigen-decomposition are evaluated, and the graph is embedded in the eigen-functions
space. Here, data are clustered to identify coherent structures in the distribution, and
the low-fidelity points closer to the centroids of the clusters are identified. The proposed
high-fidelity data acquisition policy entails querying the high-fidelity model to compute the
counterpart of the centroids only. The assumption is that data points within the same
6
cluster behave similarly. Thereafter, a non-linear transformation is applied to the eigenfunctions, to enforce them to be non-negative and a partition of unity, and a mapping for
all the low-fidelity points is defined as an expansion in terms of these transformed graph
Laplacian eigen-functions. The coefficients of this expansion are computed by minimizing a
high-fidelity data misfit term, together with a regularization term which penalizes the use
of higher-order eigen-functions. This method is applied to problems of linear elasticity and
fluid dynamics, and it is shown how the low-fidelity data can gain considerable accuracy
through the use of only a few key accurate data points.
The third method is a generalization of the previous one. Indeed, the multi-fidelity
approximation of the low-fidelity data is framed as a maximum a posteriori (MAP) estimation, conditioned on the high-fidelity data, acquired with the above-mentioned strategy.
The probability distribution for the multi-fidelity points is defined in a Bayesian context.
The prior is given by a zero-mean normal distribution, with covariance defined based on
the inverse of the graph Laplacian. This plays the role of regularization term, as the graph
Laplacian penalizes transformations of the graph that are less smooth and regular. The
likelihood term is determined by the square of the multi-fidelity misfit with respect to the
high-fidelity data. This approach has several advantages. First, the few hyper-parameters at
play can be found in a pure Bayesian manner, by maximizing the likelihood of an auxiliary
high-fidelity dataset, used for the hyper-parameters specification only. Second, the form of
the multi-fidelity transformation is not prescribed a priori, allowing for more expressivity.
Finally, the covariance of the posterior distribution provides insight on the confidence that
the multi-fidelity model has on each estimate. This is especially helpful for applications in
uncertainty quantification.
In the last chapter, we conclude with ideas on where to invest future efforts.
7
Chapter 2
Multi-fidelity modeling for rotor to rotor aerodynamic interaction
2.1 Introduction
In recent years there has been a huge interest in large electric multicopters (eVTOL aircraft)
for Urban Air Mobility (for example, in the Uber Elevate vision and the NASA UAM Grand
Challenge), commercial package delivery, and cargo, among other applications. The current
batteries that would be needed to power these eVTOL aircraft exhibit very low energy
density relative to hydrocarbon fuels used by larger convectional VTOL aircraft. With this
limitation, it is especially important to maximize eVTOL aircraft aerodynamic performance
in order to realize practical payload capacity, endurance and range. Therefore, a time efficient
aerodynamic characterization of various multicopter designs becomes very appealing. One
area that requires particular attention is the understanding of the interactional aerodynamic
effects of rotors operating in close proximity and their impact on performance.
A number of recent studies have used Computational Fluid Dynamics (CFD) to simulate and understand the complex flows associated with interactional aerodynamics of rotors
operating in close proximity. Researchers at the NASA Advanced Supercomputing Division
have used CFD to simulate large and small-scale quadcopters in hover and forward flight.
8
Yoon et al. [36] investigated the effects of rotor separation for an XV-15 derivative quadcopter in hover, and observed up to a 4% decrease in rotor efficiency for rotors in close
proximity. At the smaller scale, Yoon et al. simulated the Straight Up Imaging (SUI) Endurance quad-copter at 10 m/s cruise [37] and reported a 28% thrust deficit on the aft rotors
when compared to those rotors operating in isolation. Other work by Diaz and Yoon [38]
found that vertical rotor separation via over/under mounting influenced rotor interaction on
a quadcopter in cruise. Misiorowski, Gandhi and Oberai also used CFD to simulate quadcopters operating in cruise in the plus and cross configurations [39], and provided physical
insights into the difference in interactional aerodynamics for the two configurations. Healy,
Misiorowski and Gandhi conducted CFD simulations of in-line large UAM scale rotors in
cruise to systematically examine the effects of vertical and longitudinal rotor spacing [40]
and the effects of rotor canting [41] on interactional aerodynamic effects.
While the high-fidelity CFD studies described above provide detailed physics of multirotor interaction, in most cases they are too computationally expensive for performing optimization or uncertainty quantification (UQ) studies over a range of parameters. This is
because these studies require a large number (often in the order of 100-10,000) of simulations of the system at different configurations. In these cases, lower-fidelity models, with
simplified physics or other approximations, are used instead. However, these lower fidelity
models approximate the underlying physics, and that leads to inaccuracies in the predicted
quantities of interest. It is therefore useful to develop multi-fidelity methods that combine
the desirable characteristics of both high- and low-fidelity approaches. That is, they inherit
the computational efficiency of the low-fidelity model, while retaining the accuracy of the
high-fidelity model.
The method used in this study was developed by Narayan et al. in [33], further analyzed
by Zhu et al. and Hampton et al. in [42, 43], and applied to topology optimization under
uncertainty by Keshavarzzadeh in [44]. It assumes access to an accurate, but computationally
expensive, high-fidelity model and a less-accurate, but computationally inexpensive, low9
fidelity model. Both models can generate solution fields, termed snapshots, for a given set
of parameter values. The goal of the multi-fidelity method is to use these two models to
generate accurate snapshots for a wide range of parameters while incurring low computational
costs. The starting point is to perform a large number of low-fidelity simulations to span
the domain of interest in the parametric space. It then comprises the following steps:
1. Use the low-fidelity snapshots to determine the parameter values corresponding to a
small number of ”important” snapshots. We refer to this step as the subset selection
step. Several numerical methods can be employed to determine the important snapshots in this step [45]. These include leverage sampling, [46, 47], a pivoted Cholesky
decomposition [33, 42], or a pivoted QR decomposition [48, 49, 50]. In this work we
utilize the QR decomposition because of its efficiency and robustness [51, 52].
2. Generate a low-rank surrogate model using the important low-fidelity snapshots as
basis vectors, and coefficients that are determined from the entire set of low-fidelity
snapshots. We refer to this step as generating the low-rank surrogate model.
3. Perform high-fidelity simulations at the parameter values corresponding to the important snapshots, and use these as basis in the surrogate model derived in Step 2. This
leads to a more accurate model for all parameter values. We refer to this step as the
lifting procedure.
The desirable features of the proposed multi-fidelity approach are:
(i) It is model agnostic; that is, it can be applied to any system and any combination of
low- and high-fidelity models without regard to the specific application and physics.
(ii) By relying on the subset selection strategy, it uses the computationally expensive highfidelity model only at configurations that make the most impact.
We apply the multi-fidelity approach described above to predict the lift and drag distribution on the rotors in an in-line 2-rotor assembly in forward flight. Our parameters of
10
interest are the longitudinal and vertical distance between the two rotors. Using the lift
and drag distributions one may compute the total thrust generated and torque required for
each rotor. For the high-fidelity model we rely on CFD solutions of the system, and for the
low-fidelity model we use the Rensselaer Multicopter Analysis Code (RMAC) [53], which is
based on a blade element approximation of the original model.
The format of the remainder of this manuscript is as follows. In Section III, we formulate
the problem of interest by describing the high-fidelity and the low-fidelity models and the
strategy for combining them. We present numerical results in Section IV, and conclusions
in Section V. Finally, in the Appendix we present additional multi-fidelity results.
2.2 Problem formulation
We analyze the aerodynamic behavior of two rotors of a multicopter in forward flight. The
rotors used are modified two-bladed Whirlwind propellers [54] with radius R = 0.8382 m
(33 in), whose blade chord distribution is shown in Fig. 2.1. While the original Whirlwind
propeller is untwisted, fixed pitch and intended for axial flow applications, most multicopter
blades employ some degree of twist. In our problem −12°/R twist is applied to the blade.
The blades are set to a fixed root pitch of 24◦ and spun at 2,170 RPM in nose-level 40 kts
edgewise flight. Additional details on the selection of root pitch, twist rate and rotor RPM
can be found in [40].
Figure 2.1: Rotors blade planform, reproduced from [40]
11
The rotors are located in the longitudinal plane, aligned with the forward velocity. The
relative position of the rotors can be described using is determined by the longitudinal and
the vertical distance between the rotor centers. These two quantities, denoted by dx and dy
(Fig. 2.2), define a two-dimensional parameter space µ = (dx, dy).
For each value of these parameters we wish to evaluate certain fields of interest denoted
by u, namely the disk plots, obtained from the solution of the physical problem. For each
point in the parameter space we wish to evaluate the lift and drag distributions for the
front and rear rotors. The lift distribution represents the vertical force per unit radial
length generated by a blade as a function of its radial coordinate and azimuthal location.
Similarly, the drag distribution represents the drag force per unit radial length on the blade
as a function of its radial coordinate and azimuthal location. The vector u(µ) is used to
denote point-wise values of either of these fields for the front and rear rotors, and is referred
to as a snapshot (see Figure 2.3). We note that the total thrust generated by the rotor
can be computed by integrating the lift distribution along the radial direction and then
averaging over all azimuthal angles. Similarly, the total torque delivered to the rotor can be
computed by integrating the moment arm of the drag distribution along the radial direction
and then averaging over all azimuthal angles. The lift and drag distributions, and therefore
the snapshot u(µ), can be computed using the high-fidelity CFD model or the lower-fidelity
blade-element theory model. Both these models are described next.
2.2.1 High-fidelity model
CFD simulations are conducted using the commercial Navier-Stokes solver AcuSolve which
uses a stabilized 2nd order upwind finite element method with 2nd order time integration,
capable of modeling weakly compressible flows, and has been validated for external aerodynamic problems [55, 56]. In particular, the streamwise upwind Petrov-Galerkin (SUPG)
method is used to solve the flow equations [57, 58]. The SUPG method is a stabilized finite
element method that has been used extensively to solve advective-diffusive system of equa12
Figure 2.2: Longitudinal and vertical distance dx and dy
Figure 2.3: The snapshot is generated by concatenating the lift or drag load distributions
for the front and rear rotors sampled at D discrete grid points.
tions. In this method, terms that are proportional to residual of the original PDEs are added
to the standard Galerkin method. These terms add stability to the method while retaining
its consistency. For some special cases, it can be shown that this method is equivalent to
a second order upwind finite difference scheme. The system of equations thus obtained are
integrated in time using the Generalized α implicit time integration method. This method
supresses high frequency disturbances in the flow and allows solution stability with CourantFriedrichs-Lewy (CFL) number greater than 1 [59]. AcuSolve predictions of a NACA 0012
lift, drag and pressure coefficients at three angles of attack (0◦
, 10◦ and 15◦
) have been
shown to compare to NASA experiment with lift and drag coefficients lying within 0.98%
and 1.22% of experiment respectively [55]. AcuSolve validations have also been presented
13
for a variety of external flow conditions: massively separated flow around a square cylinder,
unsteady flow past an off-shore riser, and unsteady flow past a wind turbine with linearly
elastic blades [56]. Simulation results for an SUI Endurance rotor in hover obtained using
AcuSolve at two different rotor speeds (3000 RPM and 3900 RPM) were previously shown
to compare well against experiment in [39] where thrust matched the experiment within 3%
for both cases.
For a two-rotor unit, the computational domain is shown in Figure 2.4a, comprising of
a rectangular prism with far-field boundary conditions on the front and top surfaces set
to the freestream velocity and zero relative pressure. The sides, bottom and rear of the
computational domain are set to outflow with backflow conditions enabled, which allows for
flow in either direction across the boundary with zero pressure offset. In particular, kinetic
energy, eddy frequency and eddy viscosity variables are specified only on nodes where flow is
re-entering the domain. Flow is determined to be re-entering the domain if the component
of local velocity in the boundary-normal direction is into the domain. An “area average”
approach is used in which solution variables are averaged over the entire outflow surface
and the result is prescribed as the boundary condition for all the nodes with inflow. For the
forward-flight cases presented in this study, flow into the volume due to back flow conditions is
small, less than 0.5% of the freestream velocity. All boundaries of the computational domain
are at least 25 rotor radii away from the center of the 2-rotor assembly in all directions.
As indicated in Figure 2.4a, the computational domain consists of two cylindrical rotating
volumes (one for each rotor) where the mesh inside the volume rotates along with the rotor
geometry. Equations within the rotating volumes are solved in the inertial frame using an
Arbitrary Lagrangian Eularian formulation. At each timestep, the position of each node
within the rotating volume is updated based on its initial position, the rotation rate and
the timestep. Each rotating volume is a cylinder with radius 1.06 rotor radii. The height of
the cylinder extends two tip chord lengths above and below the rotor plane. Each rotating
volume is bounded by a sliding mesh interface which passes information into and out of
14
(a) Diagram of the computational domain (b) Cross-section of wake mesh refinement
Figure 2.4: CFD simulation domain and mesh
the non-rotating volume that comprises the remainder of the computational domain. The
continuity in the field variables across the sliding interface is enforced by requiring these to
be equal to each other in a weak sense. The precise form of the terms used to achieve this
is motivated by terms that are commonly used in the discontinuous Galerkin finite element
method.
The domain was discretized using a mesh comprised entirely of unstructured tetrahedral
elements. Within both rotating volumes, the blade surface mesh was set to ensure 200
elements around the airfoil contour. The elements on the blade were refined by a factor
of 10x near the leading (0-10% chord) and trailing edge (90-100% chord), compared to the
elements along the remainder of the chord. The boundary layer in the wall normal direction
is highly resolved, with the first element height set to ensure y
+ < 1. A portion of the blade
surface mesh is presented in Fig. 2.5a and a slice cutting through the boundary layer mesh
is shown in Fig. 2.5b. Around the rotors, a refinement region with element size prescribed as
1/2 tip chord is established, and extends 0.6R above the rotor plane, and 3R below (Figure
2.4b), with the mesh refinement below the rotor plane skewed towards the rear to better
capture the rotor wakes as they convect downstream. The entire computational domain is
comprised of 122 million elements, with 33 million in each rotating volume, and 56 million in
15
the nonrotating volume. A mesh refinement study was performed in which the surface mesh
size, edge refinement, boundary layer, and wake refinement were doubled independently.
The results of the refinement study indicated that the thrust and torque changed by less
than 1.5% and 2.5% respectively when compared to the original mesh (which is used for
simulations in this study) [40].
(a) Blade surface mesh near blade tip (b) Chordwise slice through the mesh
Figure 2.5: Mesh visualization
A detached eddy simulation (DES) is used with the Spalart-Allmaras (SA) turbulence
model. Based on the grid sizes, the region in which RANS is used only extends 0.54%
tip-chord lengths off of the blade. All simulations are run initially using time steps corresponding to 10° of rotation for at least 40 revolutions to reduce computational cost of the
rotor wake development. Each simulation is then restarted for additional revolutions at 1°
time steps until integrated thrust and torque over one revolution has converged (typically
3 additional revolutions). The initial 10° time steps are possible without causing numerical
divergence due to the stability afforded by the Streamline Upwind Petrov-Galerkin (SUPG)
stabilized finite element method and Generalized α implicit time integration method. All
runs were performed on 512 2.6 GHz Intel Xeon E5 -2650 processors, part of the Center for
Computational Innovations (CCI) at Rensselaer Polytechnic Institute.
16
2.2.2 Low-fidelity model
The low-fidelity model used in this study is the Rensselaer Multicopter Analysis Code
(RMAC, described and validated in [53]). Blade element theory is used to calculate the
lift and drag at a differential blade element. In the RMAC simulations presented, the blade
is divided into 48 radial segments. Repeating the lift and drag calculations at each element yields a radial load distribution, which can be integrated to determine the individual
blade loads. This can be performed for multiple azimuthal angles to produce a lift and drag
distribution over the entire rotor disk.
A helical vortex-wake model is applied [60] to calculate the flow induced by the rotor.
In this model, the rotor wake is represented by a vortex trailed from the tip of each rotor
blade. As the rotor turns, the trailed vortex filaments are convected downward by the mean
rotor induced flow and backward by the free-stream velocity. By using the Biot-Savart law,
the vortex-induced flow can be calculated at any point in space, which makes it capable of
predicting rotor-rotor interference effects. For the simulations presented, the maximum wake
age is 4 rotor revolutions (1440°), with a 2° resolution, resulting in 180 distinct azimuthal
locations for the rotor lift and drag distributions.
2.2.3 Multi-fidelity model
In this section we describe the method used to combine the two different fidelity models
and construct a multi-fidelity approximation of the fields of interest, namely the rotors’ disk
plots of lift and drag, at N¯ points in the parameter space µi = (d
i
x
, di
y
), with i = 1, . . . , N¯,
where a low-fidelity solution is known. The disk plots are stored as vectors, or snapshots,
containing the values of lift and drag per unit radial length at D discrete radial and azimuthal
locations over both front and rear rotors’ disk. In this section we describe the method used
to combine the low- and high-fidelity snapshots to construct a multi-fidelity approximation
of the lift and drag distributions on the two rotors, denoted by the vector u (referred to
17
as the snapshot). We wish to evaluate this field for both lift and drag at N¯ points in the
parameter space denoted by µi = (d
i
x
, di
y
), i = 1, . . . , N¯. Here dx and dy represent the
longitudinal and vertical separation between the rotors. The low-, high- and multi-fidelity
snapshots are denoted by the symbols u¯(µ), uˆ(µ), and u˜(µ), respectively. The goal is to
construct N¯ multi-fidelity snapshots u˜i = u˜(µi) by performing N¯ low-fidelity simulations
and N ≪ N¯ high-fidelity simulations. As described below, this is accomplished in three
distinct steps.
2.2.3.1 Subset selection problem
We consider a large set of points in the parameter space S = {µi}
N¯
i=1, and compute the
corresponding low-fidelity snapshots u¯i = u¯(µi). Out of these, we aim to select the N ≪ N¯
most ”important” snapshots. This is done by identifying the N-dimensional subspace of
snapshots that minimizes the average l2-projection error of all low-fidelity snapshots onto it.
The solution of this problem is found through a truncated rank revealing QR decomposition
[33, 48] of the matrix U¯ = [u¯1, . . . , u¯N¯ ],
UP¯ = QR. (2.1)
where Q is an orthogonal matrix, R is a an upper-triangular matrix, and P is the permutation matrix that rearranges the snapshots in a descending order, such that the n most
important snapshots are contained in the n first columns of the matrix UP¯ . This approach is
similar to a Proper Orthogonal Decomposition (POD), which is also often used to identify an
optimal basis ([61, 62, 63, 64, 65]). However, the critical difference is that, in the method described above, the optimal basis is constrained to comprise of individual snapshots, whereas
with POD each basis vector is linear combination of all snapshots.
In some instances we may be required to select the n most important snapshots not
from the entire set of snapshots, but from a subset of M ≪ N¯ snapshots within it. This
18
would be the scenario when the parameter values at which the high-fidelity simulations are
performed are determined by considerations other than the need to generate an accurate
multi-fidelity model. For example, scenarios where the choice of the high-fidelity simulations
is motivated by the desire to compute accurate response for configurations that are most
likely, or configurations that illustrate an effect most strongly. In this case we may still select
a smaller set of N < M using the principle described above. However, the QR decomposition
will no longer be applicable and this set would be selected in a combinatorial manner.
Once the permutation matrix P is determined, we re-index the set S and the snapshots
{u¯i}
N¯
i=1 to reflect the new-found ranking, such that the n most important snapshots occupy the N first indices. We also construct a reduced matrix that contains the important
snapshots,
U¯N = [u¯1, . . . , u¯N ]. (2.2)
2.2.3.2 Low-rank surrogate model
Next, a low-fidelity surrogate model u¯
s
(µ) is constructed using the important snapshots as
basis vectors,
u¯
s
(µ) = Xn
i=1
u¯igi(µ). (2.3)
The values of the functions gi at a given parameter point µj are found by minimizing the
residual
Rj = |u¯
s
(µj ) − u¯(µj )|
2
, (2.4)
which yields
g(µj ) = G−1fj
, (2.5)
where G ≡ U¯ T
N U¯N ∈ R
N×N is the so-called Gramian matrix and fj ≡ U¯ T
N u¯j ∈ R
N . The
resulting surrogate model is the l2-orthogonal projection of u¯j onto the subspace formed by
the n most important snapshots.
19
2.2.3.3 Lifting procedure
In this step we perform the high-fidelity simulations at the important parameter values to
obtain the snapshots {uˆi}
N
i=1. We use these in lieu of the low-fidelity basis in the surrogate
model (Eq. 2.3) thereby ”lifting” the accuracy of the surrogate model itself. This leads to
the final form of the multi-fidelity model,
u˜(µj ) = X
N
i=1
uˆigi(µj ). (2.6)
The overall computational cost of this algorithm is the cost of N¯ low-fidelity simulations
plus the the cost of n high-fidelity simulations. Once an estimate of the computational cost
of a typical low- or high-fidelity simulation is available, the user may expend their computational budget toward any one of these costs. Performing more low-fidelity simulations better
covers the parameter space and helps identifying the important snapshots with greater accuracy. On the other hand, performing more high-fidelity simulations allows for a larger basis
in the surrogate model, thereby injecting more accuracy into the model.
In the following section we apply the method described above to construct multi-fidelity
lift and drag distributions for two rotors in a multi-rotor assembly. As mentioned earlier, in
this application each snapshot comprises of either the lift or drag distributions for the two
rotors. Hence the method described above is applied twice: once for the lift distribution and
once of the drag distribution.
2.3 Numerical results
The multi-rotor assembly considered in this study has two identical counter-rotating rotors
aligned with the longitudinal axis, where the front rotor rotates clockwise and the rear
rotor rotates counter-clockwise. As mentioned above, the parameters of the problem are
the longitudinal and vertical distance between the rotors, denoted respectively as dx and dy.
20
The region of investigation in the parameter space, which defines the different configurations
under analysis, is given by dx ∈ [2.25R, 4R] and dy ∈ [0, 0.75R].
2.3.1 Construction of the multi-fidelity model
In order to apply the multi-fidelity algorithm, N¯ = 105 low-fidelity snapshots are obtained
using RMAC by uniformly sampling in the parameter domain. In addition to this, M =
9 high-fidelity CFD simulations at the parameter values (dx, dy) ∈ {2.5R, 3R, 3.5R} ×
{0, 0.25R, 0.5R} are available. Of these, we will use N = 4 for constructing the multifidelity model and the remainder as benchmark for testing the performance of the multifidelity model.
In the subset selection step of the multi-fidelity approach, we select the four (out of
nine) important snapshots using only the low-fidelity snapshots and the strategy described
in second paragraph of Section III C.1. We apply this strategy to both the lift and drag
snapshots, and for both cases it yields the same set of important snapshots. The location of
these snapshots in the parameter space, as well as of those used for testing the algorithm, is
shown in Figure 2.6.
2.25
2.375
2.5
2.625
2.75
2.875
3
3.125
3.25
3.375
3.5
3.625
3.75
3.875
4
dx=R
0
0.125
0.25
0.375
0.5
0.625
0.75
dy=R
1
2
3
4
5
6
7
8
9
Low-Fidelity High-Fidelity Test
Figure 2.6: High- and low-fidelity simulations in the parameter space. The high-fidelity
snapshots used to construct the multi-fidelity are indicated in green, and the ones used to
test the results are colored in red.
21
Relative Error Distribution
Lift Snapshots
1
3
7
9
2.25
2.375
2.5
2.625
2.75
2.875
3
3.125
3.25
3.375
3.5
3.625
3.75
3.875
4
dx=R
0
0.125
0.25
0.375
0.5
0.625
0.75
dy=R
Drag Snapshots
1
3
7
9
2.25
2.375
2.5
2.625
2.75
2.875
3
3.125
3.25
3.375
3.5
3.625
3.75
3.875
4
dx=R
0
0.125
0.25
0.375
0.5
0.625
0.75
dy=R
0
0.5
1
1.5
Figure 2.7: Distribution of relative error in the parameter space using the training set as
basis.
Using the 4 important low-fidelity snapshots and the strategy described in Section III
C.2, we construct the low-rank surrogate model u¯
s
(µ) given in Eq. 2.3 for lift and drag
distributions. We determine whether this model can accurately represent the low-fidelity
snapshots by evaluating its error, defined by
ei = 100 ×
∥u¯
s
i − u¯i∥
∥u¯i∥
, i = 1, . . . , N. ¯ (2.7)
where ∥· ∥ represents the l2 norm of a vector. This error is plotted in Figure 2.7. We observe
that it is overall small (less than 1.5% everywhere), and goes to zero at the 4 locations
corresponding to the basis snapshots. This gives us confidence that the four snapshots
accurately capture the behavior of all the other low-fidelity snapshots.
Finally, we apply the lifting procedure. As described in Section III C.3 this is done
by replacing the low-fidelity basis vectors in the surrogate model with their high-fidelity
counterparts. This yields a multi-fidelity model for the snapshots given by Equation (2.6)
in the entire parameter space.
22
2.3.2 Testing the performance of the multi-fidelity model
We test the performance of the method by comparing the multi-fidelity predictions with the
high-fidelity snapshots from the test set. The lift and drag results for all test points (as
defined in Fig. 2.6) are presented in Figs. 2.9-2.17. We discuss the results for test point
2, but similar conclusions and observations hold for the other tests points as well. Figure
2.8a shows the rotor lift distribution for CFD (top), RMAC (middle), and the multi-fidelity
(bottom). As expected, all three models predict high lift on the advancing side of the rotors,
with some bias toward the front of the rotors. When compared to the CFD solutions, the
RMAC clearly exaggerates the tip losses, as well as the negative lift in the reverse-flow region
for both rotors. Comparing the front and rear rotors, aside from a reflection about the 0−180°
line (due to the different spin directions), there is a large loss of lift on the front of the rear
disk clearly visible in the CFD and multi-fidelity solutions due to the impingement of the
front rotor wake on the rear rotor [39, 40]. This effect is present, though less pronounced,
in the RMAC simulations as well. Fig. 2.8b shows the difference between the RMAC and
multi-fidelity predictions relative to the CFD. The prediction of lift distribution of the multifidelity model is significantly better than the RMAC prediction, which has significant error,
especially in the tip region, where the tip-vortex effects are strongest.
Figure 2.9a shows similar results for the drag distribution on the rotor disk. Once again,
the accuracy of the multi-fidelity model (as shown in Fig. 2.9b) is substantially better than
that of the low-fidelity model. As was the case in lift, the error in the RMAC predictions is
primarily on the tip of the blade, where the tip-vortex is most influential.
The accuracy of the distributions is quantified by the score si
,
si = 100 ×
∥ui − uˆi∥
∥uˆi∥
(2.8)
which is the ratio of the l2 norm of the difference between the snapshot predicted by a model
ui (either RMAC or the multi-fidelity model) and the CFD snapshot uˆi (taken to be the
23
Front (CW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
Rear (CCW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
0 500 1000 1500 2000 2500 3000
(a) Lift disk plots for the three methods
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
-500 0 500
(b) Error in the Low- and Multi-Fidelity lift
Figure 2.8: Test Point 2: Low- and Multi-Fidelity lift disk plots (Units N/m)
Front (CW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
Rear (CCW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
0 100 200 300 400
(a) Drag disk plots for the three methods
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
-150 -100 -50 0 50 100 150
(b) Error in the Low- and Multi-Fidelity drag
Figure 2.9: Test Point 2: Low- and Multi-Fidelity drag disk plots (Units N/m)
24
Front (CW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
Rear (CCW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
0 500 1000 1500 2000 2500 3000
(a) Lift disk plots for the three methods
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
-500 0 500
(b) Error in the Low- and Multi-Fidelity lift
Figure 2.10: Test Point 4: Low- and Multi-Fidelity lift disk plots (Units N/m)
truth), normalized by the l2 norm of the CFD snapshot. This results in scores for both
models, with lower scores representing a distribution that better matches the CFD. The
scores for lift and drag are tabulated in Tables 2.1 and 2.2, respectively. As expected, the
multi-fidelity model has much better scores than the low-fidelity model. The error in RMAC
is largely driven by the outboard 5% of the rotor, where tip vortex causes large negative lift.
If this segment is ignored, the scores reduce significantly, though they are still higher than
the multi-fidelity model.
Table 2.1: Relative l2 norm error in RMAC and multi-fidelity models in lift distribution.
Lift Test Point 2 Test Point 4 Test Point 5 Test Point 6 Test Point 8
Front Rear Front Rear Front Rear Front Rear Front Rear
RMAC 48.77 50.79 47.89 52.87 50.79 51.14 49.88 50.25 49.97 51.14
RMAC* 24.53 26.21 22.95 29.35 26.97 26.14 25.49 24.85 25.69 26.34
MF 4.32 3.65 7.14 4.37 2.38 3.03 4.07 4.12 2.09 1.99
*
Ignoring the outboard 5%
25
Front (CW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
Rear (CCW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
0 100 200 300 400
(a) Drag disk plots for the three methods
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
-150 -100 -50 0 50 100 150
(b) Error in the Low- and Multi-Fidelity drag
Figure 2.11: Test Point 4: Low- and Multi-Fidelity drag disk plots (Units N/m)
Table 2.2: Relative l2 norm error in RMAC and multi-fidelity models in drag distribution.
Drag Test Point 2 Test Point 4 Test Point 5 Test Point 6 Test Point 8
Front Rear Front Rear Front Rear Front Rear Front Rear
RMAC 78.7 71.97 78.8 71.07 78.15 73.6 76.96 74.43 76.2 73.41
RMAC* 35.51 32.44 35.27 28.65 36.62 31.57 35.72 32.81 34.91 30.77
MF 12.69 3.02 15.04 4.19 4.14 2.7 5.54 3.15 5.89 1.92
*
Ignoring the outboard 5%
26
2.3.3 Prediction of integrated rotor loads
Also of interest is the ability of the multi-fidelity model to predict the integrated rotor loads,
such as thrust or torque, as a function of the longitudinal and vertical hub-to-hub separation.
These predictions are plotted in Fig. 2.18 for the low- and multi-fidelity models, using the
blue and black netted surfaces, respectively. As seen in the figures in the left column, the
front rotor surfaces appear nearly flat, implying that its thrust and torque are not influenced
by the relative spacing between the rotors. In other words, the front rotor behaves much
like an isolated rotor. For the front rotor (Fig. 2.18, left column) the low-fidelity solutions
are offset from the multi-fidelity solutions, predicting a higher overall thrust and torque.
But the multi-fidelity solutions are more accurate, by virtue of the lifting using high-fidelity
snapshots. At the parameter points corresponding to the basis snapshots, the multi-fidelity
predictions are observed to coincide with the high-fidelity solutions (denoted by the black
Front (CW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
Rear (CCW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
0 500 1000 1500 2000 2500 3000
(a) Lift disk plots for the three methods
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
-500 0 500
(b) Error in the Low- and Multi-Fidelity lift
Figure 2.12: Test Point 5: Low- and Multi-Fidelity lift disk plots (Units N/m)
27
Front (CW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
Rear (CCW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
0 100 200 300 400
(a) Drag disk plots for the three methods
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
-150 -100 -50 0 50 100 150
(b) Error in the Low- and Multi-Fidelity drag
Figure 2.13: Test Point 5: Low- and Multi-Fidelity drag disk plots (Units N/m)
28
Front (CW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
Rear (CCW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
0 500 1000 1500 2000 2500 3000
(a) Lift disk plots for the three methods
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
-500 0 500
(b) Error in the Low- and Multi-Fidelity lift
Figure 2.14: Test Point 6: Low- and Multi-Fidelity lift disk plots (Units N/m)
dots on Fig. 2.18).
More interesting than the front rotor results are the aft rotor thrust and torque predictions
seen in the right column of Fig. 2.18. Consistent with previous studies [39, 41, 40], the aft
rotor thrust shows an increasing deficit (relative to the front rotor thrust) as the longitudinal
separation decreases. At low values of longitudinal separation, the thrust deficit increases as
the vertical offset is reduced. But this sensitivity to vertical offset is observed to diminish for
higher hub-to-hub longitudinal separation. The low-fidelity aft rotor thrust predictions (blue
netted surface) show a greater deficit at low hub-to-hub separations and a smaller deficit at
higher hub-to-hub separations, than the multi-fidelity predictions (black netted surface). In
contrast, the multi-fidelity predictions show a lower sensitivity to relative position, corrected
as they are by the lifting of the high-fidelity CFD snapshots. In the bottom right sub-figure in
Fig. 2.18, the aft rotor torque does not show as a high a degree of sensitivity to relative rotor
position as the aft rotor thrust. This is discussed in [40], and attributed to compensatory
29
Front (CW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
Rear (CCW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
0 100 200 300 400
(a) Drag disk plots for the three methods
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
-150 -100 -50 0 50 100 150
(b) Error in the Low- and Multi-Fidelity drag
Figure 2.15: Test Point 6: Low- and Multi-Fidelity drag disk plots (Units N/m)
30
Front (CW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
Rear (CCW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
0 500 1000 1500 2000 2500 3000
(a) Lift disk plots for the three methods
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
-500 0 500
(b) Error in the Low- and Multi-Fidelity lift
Figure 2.16: Test Point 8: Low- and Multi-Fidelity lift disk plots (Units N/m)
effects of changes in induced and profile drag on the aft rotor, for low hub-to-hub separation.
Regardless, the multi-fidelity aft rotor torque predictions are more accurate than the lowfidelity predictions and are anchored by the high-fidelity snapshots where these snapshots
are available.
The relative errors in thrust and torque at the five test points where high-fidelity solutions
are available are tabulated in Tables 2.3 and 2.4, respectively. Overall, the multi-fidelity predictions are seen to have significantly lower error than the low-fidelity (RMAC) predictions,
as compared to the CFD results.
Table 2.3: Error in thrust computed using RMAC and multi-fidelity models.
Thrust Error [%] Test Point 2 Test Point 4 Test Point 5 Test Point 6 Test Point 8
Front Rear Front Rear Front Rear Front Rear Front Rear
RMAC 2.97 -4.33 2.08 2.68 3.94 0.02 2.93 -0.14 2.74 2.58
MF -0.68 -1.37 -0.58 1.89 0.50 -0.90 -0.60 -1.48 0.07 -0.62
31
Front (CW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
Rear (CCW)
CFD
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
V1
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
0 100 200 300 400
(a) Drag disk plots for the three methods
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
RMAC
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
MF
0.2
0.4
0.6
0.8
1
45
225
90
270
135
315
180 0
-150 -100 -50 0 50 100 150
(b) Error in the Low- and Multi-Fidelity drag
Figure 2.17: Test Point 8: Low- and Multi-Fidelity drag disk plots (Units N/m)
32
0.7
0
5
.5
dy=R
(a) Thrust of Front Rotor [N]
0.25
1100
1200
dx=R
1300
2.5
0
1400
3 3.5
1500
4
0.7
0
5
.5
dy=R
(b) Thrust of Rear Rotor [N]
0.25
1100
1200
dx=R
1300
2.5
0
1400
3 3.5
1500
4
RMAC Multi-Fidelity CFD
0.7
0
5
.5
dy=R
(c) Torque of Front Rotor [Nm]
0.25
110
dx=R
120
2.5
0
3 3.5
130
4
0.7
0
5
.5
dy=R
(d) Torque of Rear Rotor [Nm]
0.25
110
dx=R
120
2.5
0
3 3.5
130
4
Figure 2.18: Low- and Multi-Fidelity Thrust and Torque distribution over the parameter
space.
Table 2.4: Error in torque computed using RMAC and multi-fidelity models.
Torque Error [%] Test Point 2 Test Point 4 Test Point 5 Test Point 6 Test Point 8
Front Rear Front Rear Front Rear Front Rear Front Rear
RMAC 6.32 2.47 7.10 0.18 5.69 3.12 4.05 4.50 3.83 2.54
MF 0.88 -1.34 0.49 -0.83 0.10 0.80 -0.30 1.14 0.41 0.25
2.3.4 Effect of selection of important snapshots
We now examine the effect of selecting a different set of snapshots as basis vectors for the
multi-fidelity model. In particular, instead of selecting snapshots corresponding to the points
shown in Figure 2.6 as the basis vectors, we select the snapshots corresponding to the points
shown in Figure 2.19. We then construct a new surrogate model and perform the lifting
procedure to obtain a new multi-fidelity model. For the testing points that are common
between the two sets shown in Figures 2.6 and 2.19, we compute the relative error of this
new multi-fidelity model (denoted by MF*
) and the multi-fidelity model constructed in
33
Table 2.5: Relative l2 norm error in multi-fidelity models for lift and drag distribution.
Test Point 2 Test Point 4 Test Point 5
Front Rear Front Rear Front Rear
Lift MF 4.32 3.65 7.14 4.37 2.38 3.03
MF* 13.9 16.32 9.8 10.64 7.23 4.79
Drag MF 12.69 3.02 15.04 4.19 4.14 2.7
MF* 20.23 9.59 15.8 4.75 6.56 2.92
Section IV A (denoted by MF). This is reported in Table 2.5. As can be seen, the error for
the multi-fidelity model where the basis vectors are optimally selected is consistently smaller.
This result highlights the importance of the subset selection step in the overall multi-fidelity
approach.
2.25
2.375
2.5
2.625
2.75
2.875
3
3.125
3.25
3.375
3.5
3.625
3.75
3.875
4
dx=R
0
0.125
0.25
0.375
0.5
0.625
0.75
dy=R
1
2
3
4
5
6
7
8
9
Low-Fidelity High-Fidelity Test
Figure 2.19: High- and low-fidelity simulations in the parameter space. The high-fidelity
snapshots used to construct the multi-fidelity are indicated in green, and the ones used to
test the results are colored in red.
2.4 Conclusions
In this paper we have described a multi-fidelity approach for predicting lift and drag distributions in a multi-rotor assembly, when the rotors separation is varied. This approach
combines the results from a low-fidelity, low-cost model, which is used to span the entire
parameter space, with that of a high-fidelity, high-cost model, which is used to perform a
34
few select simulations. In the example presented in this work, the low-fidelity model is a
blade-element solver and the high-fidelity model is a CFD solver. We have demonstrated
how the results from these very different models can be combined to produce multi-fidelity
results that are much more accurate than the low-fidelity model, and much cheaper than
the high-fidelity model by themselves. The load distributions predicted by the multi-fidelity
are significantly closer to the CFD results at the test points, when compared with those of
the lower-fidelity model. Further, the total thrust and torque predicted by the multi-fidelity
model have relative errors that are less than 2% for all test points.
35
Chapter 3
Graph Laplacian-based spectral multifidelity modeling
3.1 Introduction
The approach developed in this chapter relies on the spectral properties of the graph Laplacian constructed from the low-fidelity data. It uses these properties to determine the points
at which high-fidelity data ought to be acquired, and to embed the structure of the lowfidelity data into the multi-fidelity model. Thereafter, a transformation that maps every
low-fidelity data point to a multi-fidelity counterpart is determined by minimizing the discrepancy between the multi- and high-fidelity data, while preserving the underlying structure
of the low-fidelity data distribution.
This approach also differs from most co-kriging and RBFs-based methods in how it treats
outputs with multiple quantities of interest. While most methods tend to ignore the joint
distribution of these quantities, the proposed method explicitly utilizes it in constructing
the multi-fidelity response. The proposed approach also has strong connections with semisupervised classification algorithms on graphs [66, 67, 68, 69, 70], and relies on theoretical
results on consistency of graph-based methods in the limit of infinite data [71, 72]. The
method is tested with problems in solid and fluid mechanics. By utilizing only a small
36
fraction of high-fidelity data, the accuracy of a large set of low-fidelity data is significantly
improved.
3.2 Methodology
The proposed multi-fidelity approach relies heavily on the spectral properties of the graph
Laplacian. Hence, we first present a brief introduction of what the graph Laplacian is and
why it is useful. Afterwards, we dive into the specifics of the methodology.
3.2.1 Background
A complete, weighted graph is a pair G = (V, W), where V = {u
1
, . . . , u
N } is a set of
vertices (or nodes) embedded in R
D, and W = [Wij ] is an adjacency matrix. We consider
adjacency matrices of the type
Wij = d(||u
i − u
j
||2), (3.1)
where d(·) is a monotonically decreasing function and || · ||2 is the l2 norm. We define the
degree matrix D to be a diagonal matrix with
Dii =
X
N
j=1
Wij , (3.2)
and a family of graph Laplacians,
L = D−p
(D − W)D−q
. (3.3)
Different choices of p and q result in different normalizations of the graph Laplacian [73,
74, 75]. The graph Laplacian can be used to perform spectral clustering, which amounts to
finding an optimal partition of the graph using the spectral properties of L [76, 77]. The
37
eigenfunctions of L form the set of orthonormal functions from the nodes of the graph to
the real numbers ϕ
(m)
: V → R that solve the eigenvalue problem
Lϕ(m) = λmϕ
(m)
, (3.4)
with ϕ
(m) = [ϕ
(m)
1
, . . . , ϕ(m)
N ]
T and ϕ
(m)
i = ϕ
(m)
(u
i
). For the un-normalized graph Laplacian
(p = q = 0), the eigenfunctions satisfy the following property,
ϕ
(m)
· Lϕ(m) = λm (3.5)
=
1
2
X
i, j
Wij
ϕ
(m)
i − ϕ
(m)
j
2
. (3.6)
This implies that eigenfunctions with small eigenvalues provide a mapping of the graph
to a line that promotes the clustering of vertices that are strongly connected. Note that
the eigenvalue problem (3.4) admits the trivial solution that maps all vertices to a point,
e.g. ϕ
(1) = √
1
N
[1, . . . , 1]T
, and has a zero eigenvalue. The eigenfunction corresponding
to the smallest non-zero eigenvalue, also called Fiedler vector, represents the non-trivial
solution to the problem of embedding the graph onto a line so that connected vertices stay
as close as possible [78]. Similarly, the eigenfunctions corresponding to the k lowest non-zero
eigenvalues, [ϕ
(2)
, . . . , ϕ
(k+1)], represent the optimal embedding of the graph into R
k
, where
the coordinates of a vertex u
i are given by ξi = [ϕ
(2)
i
, . . . , ϕ(k+1)
i
].
3.2.2 Spectral Multi-Fidelity method (SpecMF)
Given a parametric physical problem, we denote by µ ∈ R
P
the vector of input parameters,
and by q¯(µ) ∈ R
Q and qˆ(µ) ∈ R
Q the low- and high-fidelity output vectors, respectively.
These vectors represent a set of Q output quantities of interest of the problem. Low- and
high-fidelity data points, denoted by u¯ ∈ R
D and uˆ ∈ R
D, respectively, with Q ≤ D ≤
Q + P, are constructed from components of input parameters and output quantities. That
38
is, u¯ = R(µ, q¯) and uˆ = R(µ, qˆ), where R is a restriction operator that extracts the
appropriate components of the input parameters and output quantities of interest. The
choice of R is problem dependent and is described in Section 3.5. Two obvious choices are
when the data comprises all input and output components, that is, R(µ, q) = [µ, q], and
when it comprises only of the output components, that is, R(µ, q) = q.
Our goal is to combine a dense set of low-fidelity data points with a few, select, highfidelity points to generate a dense set of multi-fidelity points. The steps to accomplish
this are: (1) generate a dense collection of low-fidelity data points, (2) identify key input
parameter values at which to acquire the more expensive high-fidelity data, and finally (3)
combine the low- and high-fidelity data to construct a multi-fidelity model. These steps are
also outlined in Figure 3.1 in the context of a two-dimensional illustrative example. This
overall approach is referred to as Spectral Multi-Fidelity (SpecMF) method, since the spectral
properties of the graph Laplacian are utilized to embed the structure of the low-fidelity data
into the multi-fidelity model. Below, we describe each step in detail.
Step 1 – Construction of the low-fidelity graph. We begin by sampling N¯ ≫ 1
points in the parameter space from a simple prescribed probability density to generate the
set S¯ = {µ
i}
N¯
i=1. Then, we generate the low-fidelity data points u¯
i = u¯(µ
i
), i = 1, . . . , N¯, and
collect them in the set D¯ = {u¯
i}
N¯
i=1. For uniformity, we scale them so that each component
lies within [−1, 1]. We apply the same scaling factors to the high-fidelity data collected in
Step 2.
We treat each data point as the vertex of an undirected, complete, weighted graph, and
exploit the useful properties of the associated matrices. Hence, we construct a graph with
u¯
i as vertices, and with weights given by the entries of the adjacency matrix (3.1) where d(·)
is chosen to be a Gaussian kernel,
d(r) ≡ exp(−r
2
/σ2
). (3.7)
39
Figure 3.1: Workflow for the spectral multi-fidelity (SpecMF) method applied to an illustrative problem. (a) Generate low-fidelity data. (b) Compute a graph Laplacian using the
low-fidelity data. (c) Compute the eigen-decomposition of the graph Laplacian. (d) Perform
spectral clustering of the low-fidelity data and find the points closest to the clusters centroids. (e) Acquire high-fidelity data only for these points. (f) Solve a convex minimization
problem to find one influence function for each point with a high-fidelity counterpart. The
influence functions are constructed from the low-lying eigenfunctions of graph Laplacian. (g)
Generate the multi-fidelity approximation of the data set. (h) For this illustrative example,
this is compared with the corresponding high-fidelity data set.
40
In the equation above, σ is a characteristic scale. It can be treated as a hyperparameter, or its
value can also be determined for each vertex by analyzing the statistics of its neighborhood
[79]. From the adjacency matrix, we construct the diagonal degree matrix D and a graph
Laplacian L, using (3.2) and (3.3), respectively. For the applications in this paper, we have
employed a normalized graph Laplacian with p, q = 0.5 and determined σ using the approach
described in [79].
Step 2 – Selection strategy. Next, we describe the strategy for selecting N ≪ N¯ nodes
for which high-fidelity data is acquired. These nodes are chosen to be close to the centroids
of the clusters associated with the low-fidelity data D¯. To find these, we compute the eigendecomposition of the graph Laplacian and leverage the properties of its low-lying spectrum.
We embed each data point into the eigenfunctions space and apply a standard clustering
algorithm (e.g. K-means) to determine the clusters and their centroids. This is accomplished
by,
1. Compute the low-lying eigenfunctions of the graph Laplacian, ϕ
(m)
, m = 1, . . . , K,
with K = 3N. During the selection strategy step (Step 2) we use only the first N of
these eigenfunctions. However, in the multi-fidelity transformation step (Step 3) we
utilize all of the K eigenfunctions.
2. For every low-fidelity data point, u¯
i
, compute the coordinates in the eigenfunction
space ξ
i ∈ R
N . These are given by ξ
i = [ϕ
(1)
i
, . . . , ϕ(N)
i
], i = 1, . . . , N¯.
3. Use K-means clustering on the points {ξ
i}
N¯
i=1 to find N clusters.
4. For each cluster, determine the centroid and the low-fidelity data point closest to it.
5. Re-index the low-fidelity data set D¯ and the parameters set S¯ so that the points
identified above correspond to the first N points.
6. Acquire high-fidelity data at the parameter values corresponding to these points, and
41
assemble the data set D = {uˆ
i}
N
i=1, with uˆ
i = uˆ(µ
i
). Note that the elements of D are
the high-fidelity counterparts of the first N elements of D¯.
7. Scale the high-fidelity data with the same scaling factors used in Step 1 for the lowfidelity data.
In Section 3.4 we compare the performance of this strategy against a random selection
of the input parameters at which the high-fidelity data is acquired for one of the numerical
problems.
Step 3 – Multi-fidelity transformation. In this step we generate a multi-fidelity approximation {u˜
i}
N¯
i=1 that learns the data distribution from the low-fidelity data set and uses
the select high-fidelity data to transform this distribution. The proposed multi-fidelity approach seeks a transformation that maps every low-fidelity data point to a new location in
the data space, where the displacements are weighted sums of the N known displacements
of the select points at which the high-fidelity counterpart is known. That is,
u˜
i = u¯
i +
X
N
j=1
(uˆ
j − u¯
j
)ψ
(j)
i
, i = 1, . . . , N. ¯ (3.8)
Here u˜
i are the multi-fidelity data points, uˆ
j − u¯
j
is the displacement vector that maps the
j-th low-fidelity point to its high-fidelity location, and ψ
(j)
i
, j = 1, . . . , N, are the influence
functions that determine the effect of the j-th displacement vector on the i-th point. We
require the influence functions to encode the structure of the low-fidelity data distribution,
and therefore a natural choice is to write them in terms of the eigenfunctions of the graph
Laplacian. For consistency, we also require the influence functions to be a partition of
unity. These requirements are satisfied by applying a softmax activation to a set of auxiliary
functions v
(j)
i
that are constructed from a linear combination of the low-lying eigenfunctions
42
of the graph Laplacian. In particular, the influence functions are given by
ψ
(j)
i =
exp (v
(j)
i
)
PN
k=1 exp (v
(k)
i
)
, (3.9)
and the auxiliary functions v
(j)
i
are,
v
(j)
i =
X
K
m=1
αjmϕ
(m)
i
. (3.10)
The parameter αjm determines the contribution of the m-th eigenfunction to the j-th auxiliary function, and K denotes the cutoff in the spectrum of the graph Laplacian. This cutoff
should be proportional to the number of high-fidelity data points. A suggested value, which
is used in this study, is K = 3N.
The parameters α = {αjm} are determined by solving the minimization problem
α
∗ = arg min
α
J(α), J(α) = Jdata(α) + ωJreg(α). (3.11)
with
Jdata(α) = 1
N
X
N
i=1
||u˜
i
(α) − uˆ
i
||2
2
, and (3.12)
Jreg(α) = 1
τ
2KN
X
N
j=1
v
(j)
(α) · (L + τI)
2v
(j)
(α). (3.13)
The first term in (3.11) is a data misfit term, which forces the multi-fidelity points to
be close to the corresponding high-fidelity points. The second term is a structure-preserving
term that promotes contributions from eigenfunctions with small eigenvalues. Its form is
motivated by a similar term used in semi-supervised learning applications that utilize the
graph Laplacian [71]. To examine the effect of this term, we substitute (3.10) in (3.13) to
43
find
Jreg(α) = 1
KN
X
N
j=1
X
K
m=1
α
2
jm
1 +
λm
τ
2
. (3.14)
Thus, the values of αjm corresponding to larger values of λm are penalized more. Since
the structure of the low-fidelity data is encoded in the eigenfunctions corresponding to the
smaller eigenvalues, this term helps in carrying this structure over to the multi-fidelity model.
It also makes the proposed algorithm relatively insensitive to the selection of the spectrum
cutoff K, as the contribution from the higher-order eigenfunctions is weighed less. The
presence of τ > 0 makes the minimization problem convex and easy to solve [71], and a good
candidate for its value is the smallest non-zero eigenvalue, i.e. τ = λ2. This amounts to
solving a problem with a scaled spectrum of the Laplacian.
The regularization constant ω in (3.11) balances the interplay between the data misfit
term and the structure-preserving regularization term. If its value is too small, the multifidelity model is likely to over-fit the high-fidelity data and ignore the structure learned from
the low-fidelity data. On the other hand, if its value is too large, the method will yield
multi-fidelity points that are significantly different from their high-fidelity counterpart. The
optimal value of this parameter may be determined using the L-curve method [80].
The L-curve is a plot of the data misfit loss Jdata(α∗
(ω); ω) versus the regularization loss
Jreg(α∗
(ω); ω) obtained after solving the minimization problem (3.11) for different values of
regularization parameter ω. It can be used to visualize the balance between the two terms
and provides a way to tune the regularization parameter. The optimal value ω
∗
is indeed
chosen to be the one corresponding to the elbow of the L-curve, i.e. the point of maximum
curvature. In fact, the elbow of the curve separates the regions where the final solution is
either dominated by a large data misfit error or by a large regularization loss. Therefore,
any perturbation of the parameter ω
∗ would lead to a significant increase in one loss term or
the other. Because the minimization problem (3.11) is inexpensive to solve, we can use the
L-curve criterion to determine the optimal value of ω. In Figure 3.2 we show the L-curve
used to determine the regularization parameter for the traction on soft material problem in
44
Figure 3.2: L-curve for the traction problem in Section 3.3.1. The optimal value for the
regularization parameter is the one corresponding to the elbow of the curve, marked with a
triangle in the graph.
Section 3.3.1.
Finally, we note that if we do not include the parameters µ in the definition of uˆ, i.e.
uˆ = qˆ(µ), they not appear explicitly in any of the equations. This means that the method
is insensitive to the dimension of the input space, and it can be applied to a generic point
cloud {u¯
i}
N¯
i=1 embedded in R
D that has to be transformed based on a few more accurate,
or updated, control points {uˆ
i}
N
i=1, with N ≪ N¯. This could represent a set of point-wise
measurements that are dense in space but not very accurate, for which a smaller number of
more precise measurements are available.
3.2.3 Theoretical Analysis
The proposed method is endowed with some desirable theoretical properties, which are described below.
Property 1 An explicit expression for the gradient of the loss function with respect to the
optimization parameters can be computed, lowering the computational cost of the algorithm.
Proof. We want to evaluate the gradient of the loss function J = Jdata + ωJreg in Eq. (3.11)
45
with respect to the parameters α. The gradient of the data misfit term is given by
∂Jdata
∂αuv
=
2
N
X
N
i=1
u˜
i − uˆ
i
·
∂u˜
i
∂αuv
=
2
N
X
N
i=1
u˜
i − uˆ
i
·
X
N
j=1
uˆ
j − u¯
j
∂ψ(j)
i
∂αuv
=
2
N
X
N
i=1
X
N
j=1
u˜
i − uˆ
i
·
uˆ
j − u¯
j
ϕ
(v)
i ψ
(j)
i
δju − ψ
(u)
i
. (3.15)
In deriving this expression we have made use of
∂ψ(j)
i
∂αuv
=
∂
∂αuv
exp (v
(j)
i
)
P
k
exp (v
(k)
i
)
=
P
k
exp (v
(k)
i
) · exp (v
(j)
i
)ϕ
(v)
i
δju − exp (v
(j)
i
) exp (v
(u)
i
)ϕ
(v)
i
hP
k
exp (v
(k)
i
)
i2
= ϕ
(v)
i ψ
(j)
i
δju − ψ
(u)
i
, (3.16)
and
∂v(j)
i
∂αuv
=
∂
∂αuv
X
K
m=1
αjmϕ
(m)
i = ϕ
(v)
i
δju. (3.17)
Here δij is the Kronecker delta. The gradient of the regularization term is given by
∂Jreg
∂αuv
=
1
KN
X
N
j=1
X
K
m=1
∂
∂αuv
α
2
jm
1 +
λm
τ
2
=
2αuv
KN
1 +
λv
τ
2
. (3.18)
Combining (3.15) and (3.17) we have the expression for the total gradient,
∂J
∂αuv
=
2
N
X
N
i=1
X
N
j=1
u˜
i − uˆ
i
·
uˆ
j − u¯
j
ϕ
(v)
i ψ
(j)
i
δju − ψ
(u)
i
+
2ωαuv
KN
1 +
λv
τ
2
. (3.
Property 2 In the limit of a small data misfit term (which happens as the optimization
iterations converge), the Hessian of the loss function is positive definite. This proof is based
on recognizing that (i) the data misfit term is in the form of a least-squares residual and (ii)
the regularization term is a positive-definite quadratic form. This ensures that the resulting
optimization problem is solved easily.
Proof. The Hessian can be obtained by differentiating the gradient in (3.15) and (3.17) with
respect to the optimization parameters. This yields,
Hpquv ≡
∂
2
Jdata
∂αpq∂αuv
=
2
N
hX
N
i=1
||u˜
i − uˆ
i
|| ∂
2
z
i
∂αpq∂αuv
+
X
N
i=1
∂u˜
i
∂αpq
·
∂u˜
i
∂αuv
+
ω
K
1 +
λv
τ
2
δpuδqvi
, (3.20)
where z
i ≡
u˜
i−uˆ
i
||u˜
i−uˆ
i
|| · u˜
i
. In the equation above, the first two terms emerge from the
data-misfit term, while the third term is the contribution of the regularization term. The
inner product of the Hessian with an arbitrary realization of parameters denoted by ˜α writes
X
pquv
α˜pqHpquvα˜uv =
2
N
hX
N
i=1
||u˜
i − uˆ
i
||X
pquv
α˜pq
∂
2
z
i
∂αpq∂αuv
α˜uv
+
X
N
i=1
X
pq
∂u˜
i
∂αpq
α˜pq2
+
ω
K
X
pq
1 +
λq
τ
2
α˜
2
pqi
.
(3.21)
In the equation above, the term ∂
2z
i
∂αpq∂αuv
is a symmetric fourth order tensor and therefore
has real eigenvalues. We let ζ = λmin(
∂
2z
i
∂αpq∂αuv
), i = 1, . . . , N be the smallest eigenvalue.
Therefore, we have
X
pquv
α˜pqHpquvα˜uv ≥
2
N
h
ζ
X
N
i=1
||u˜
i − uˆ
i
||X
pq
α˜
2
pq +
X
N
i=1
X
pq
∂u˜
i
∂αpq
α˜pq2
+
ω
K
X
pq
α˜
2
pqi
, (3.22)
With ζ ≥ 0, without any restriction on the data misfit, we have
X
pquv
α˜pqHpquvα˜uv ≥
2ω
NK
X
pq
α˜
2
pq, (3.23)
which implies that the Hessain is positive definite. With ζ < 0, we require PN
i=1 ||u˜
i −uˆ
i
|| ≤
ω
2|ζ|K
, and use this in (3.22) to arrive at
X
pquv
α˜pqHpquvα˜uv ≥
ω
NK
X
pq
α˜
2
pq, (3.24)
Thus, assuming PN
i=1 ||u˜
i −uˆ
i
|| ≤ ω
2|ζ|K we conclude that the Hessian is positive definite.
Property 3 Under the assumptions (a) the low-fidelity data is partitioned into M clusters,
(b) the high-fidelity data differs from the low-fidelity data by distinct rigid translations
applied to each cluster, and (c) the high-fidelity version of one point per cluster is known,
the proposed approach permits a transformation that maps each low-fidelity point to the
true high-fidelity point in the limit of infinite low-fidelity data and as the regularization
parameter tends to zero. That is, the multi-fidelity data set converges to its high-fidelity
counterpart. This is a consistency result that demonstrates the proposed method can solve
this special problem exactly.
Proof. We make the following assumptions on the data:
1. The low-fidelity data u¯
i are distributed among M distinct clusters.
2. C(i) is the cluster number for the i−th data point. Without loss of generality, we may
4
Figure 3.3: Schematic of a canonical problem with the data set partitioned into M = 3
clusters.
assume that points are numbered such that C(i) = i for i = 1, . . . , M.
3. For any point i, the high-fidelity data is related to the low-fidelity data via the transformation
uˆ
i = u¯
i + δ
C(i)
. (3.25)
where δ
j
, j = 1, . . . , M, are the rigid translations for each cluster.
4. For i = 1, . . . , M, the high-fidelity data uˆ
i
is available and is used to construct the
data matching term. Using assumptions 2 and 3, we note that these points,
uˆ
i = u¯
i + δ
i
, i = 1, . . . , M. (3.26)
Under the assumptions above, the proposed method admits a multi-fidelity solution whose
error vanishes as the regularization parameter approaches zero.
In the limit of infinite data, the graph Laplacian of the low-fidelity graph yields M
eigenfunctions with null eigenvalues [71] such that,
ϕ
(j)
i =
1, C(i) = j
−1, C(i) ̸= j
. (3.27)
49
That is, the j-th eigenfunction attains a value of +1 for all points that belong to the j-th
cluster and a value of −1 for all other points.
For the proposed method we consider the choice K = Nˆ = M. Further, we consider the
special case where the unknown parameters are given by, αjm = αδjm. With this choice,
from (3.27) and (3.10) we conclude that
v
(j)
i =
α, C(i) = j
−α, C(i) ̸= j
. (3.28)
From (3.9) this yields,
ψ
(j)
i =
1
1+(M−1) exp(−2α)
, C(i) = j
exp(−2α)
1+(M−1) exp(−2α)
, C(i) ̸= j
. (3.29)
For i = 1, . . . , M, the difference between the multi-fidelity expression (3.8) and the highfidelity data (3.26) is given by
u˜
i − uˆ
i = u¯
i +
X
M
j=1
δjψ
(j)
i −
u¯
i + δ
i
= δ
i
(ψ
(i)
i − 1) + X
M
j=1,j̸=i
δ
jψ
(j)
i
=
exp(−2α)
1 + (M − 1) exp(−2α)
≡b
i
z }| {
− (M − 1)δ
i +
X
M
j=1,j̸=i
δ
j
, from (3.29)
= γ(α)b
i
.
Where γ(α) ≡
exp(−2α)
1+(M−1) exp(−2α)
. Using this expression in the definition of the data matching
term (3.12), we arrive at,
Jdata(α) = γ(α)
2X
M
i=1
||b
i
||2
M
. (3.30)
Using (3.28) in (3.13) we conclude that the regularization term is given by,
Jreg(α) = α
2
M
, (3.31)
where we have used the fact the eigenvalues for the eigenfunctions considered in this expansion are zero.
Therefore the total objective function is equal to,
J(α) = γ(α)
2X
M
i=1
||b
i
||2
M
+ ω
α
2
M
, (3.32)
Further, it is easily verified that
dγ
dα = 2γ
(M − 1)γ − 1
. (3.33)
Setting dJ
dα = 0 to find the stationary point of the objective function, we arrive at,
2γ
(M − 1)γ − 1
X
M
i=1
||bi
||2
M
+ ω
α
M
= 0. (3.34)
In the limit ω → 0, to leading order this equation yields the solution
α = −
ln ω
4
. (3.35)
Using this solution in the expression of the data matching term we note that Jdata(α) =
O(ω), which tends to zero as ω → 0. Also Jreg(α) = O(ω
2
ln2 ω) which also tends to zero
as ω → 0. Thus both the data matching and regularization terms tend to zero as the
regularization parameter tends to zero.
We now show that with choice of α, for any point indexed by i, the coordinates predicted
by the multi-fidelity approximation tend to the high-fidelity coordinates as the regularization
is reduced. To accomplish this we compute
51
u˜
i − uˆ
i = u¯
i +
X
M
j=1
δ
jψ
(j)
i −
u¯
i + δ
C(i)
= δ
C(i)
(ψ
(C(i))
i − 1) + X
M
j=1,j̸=C(i)
δ
jψ
(j)
i
=
exp(−2α)
1 + (M − 1) exp(−2α)
− (M − 1)δ
C(i) +
X
M
j=1,j̸=C(i)
δ
j
, from (3.29)
= γ(α)b
C(i)
.
Therefore,
||u˜
i − uˆ
i
|| = γ(α)||b
C(i)
|| = ||b
C(i)
||ω
1/2
(1 + O(ω)). (3.36)
Clearly this difference tends to zero as ω → 0.
3.3 Numerical results
In what follows, we apply the proposed method to two numerical problems. The first is an
application to linear elasticity, where the dimension of the input parameters space and the
data space are P = 5 and D = 5, respectively. The second is a fluid dynamics problem,
with P = 5 and D = 3. Furthermore, for the elasticity problem, we also apply the SpecMF
method to an entire field discretized on a grid with 100 points, so that D = 100. These
problems were solved on a Apple M1 Pro processor with 8 cores, and the computational
time for a fixed value of regularization parameter is around 1-2 minutes.
We compare our results with a co-kriging method [13, 14] applied to each output quantity
of interest individually, i.e. qk = qk(µ), k = 1, . . . , Q. In co-kriging, the multi-fidelity
approximation Zk(µ) is constructed as a weighted sum of two Gaussian processes, Zk(µ) =
γZ¯
k(µ) + Z
d
k
(µ), where Z¯
k models the low-fidelity data {(µ
i
, q¯
i
k
)}
N¯
i=1, γ is a scaling factor,
5
and Z
d
k models the discrepancy between the high-fidelity data {(µ
i
, qˆ
i
k
)}
N
i=1 and γZ¯
k. For
both these Gaussian processes the covariance function is an anisotropic Gaussian kernel,
where different length scales are used for each coordinate of the input parameter space:
cov(Z¯
k(µ
i
), Z¯
k(µ
j
)) = ¯σ
2
k
exp(−
X
P
p=1
(µ
i
p − µ
j
p
)
2
¯l
2
k, p
), (3.37)
cov(Z
d
k
(µ
i
), Zd
k
(µ
j
)) = σ
2
k
exp(−
X
P
p=1
(µ
i
p − µ
j
p
)
2
l
2
k, p
). (3.38)
The hyper-parameters of these kernels, such as ¯σk, σk,
¯lk, p, lk, p, the scaling factor γ and the
mean values of Z¯
k and Z
d
k
, are computed by maximizing the log-likelihood of the data [14].
In our experience, this optimization problem could be very sensitive to the initial guess and
the prescribed bounds in the search space. We addressed this issue by utilizing a grid search
and multiple random initial guesses to arrive at the best results. All computations were
performed using the open-source computing platform OpenMDAO [81].
3.3.1 Traction on a soft material with a stiff inclusion
Problem description. We examine a problem of linear elasticity which involves a soft
square sheet in plane stress with an internal stiffer elliptic inclusion. The length of the edge
of the square is L = 10 cm, its Young’s modulus is E = 1kP a, and the Young’s modulus of
the inclusion is Ei = 4E. Both the body and the inclusion are incompressible. The bottom
edge of the square is fixed, and a uniform downward displacement v0 = −5 mm is applied
to the top edge. The top edge is traction free in the horizontal direction while the vertical
edges are traction-free in both directions (Fig. 3.4a). We wish to predict attributes of the
vertical traction field on the upper edge as a function of the inclusion shape, orientation
and location. This problem is motivated by the need to identify stiff tumors within a soft
background tissue, which is particularly relevant to detecting and diagnosing breast cancer
tumors [82, 83].
53
(a) (b)
1/1
Figure 3.4: (a) Schematic of the soft body (light grey) with the elliptic stiffer inclusion
(dark grey). The square is compressed on top with a uniform displacement v = v0, while the
bottom is fixed. The vertical traction is integrated over the top side across equal sections to
compute the localized forces fi
. (b) Schematic of the airfoil with the input parameters.
Parameters and quantities of interest. The input parameters of the problem are the
coordinates of the center of the elliptical inclusion (xc, yc), its orientation θ, and its major
and minor semi-axes a and b (see Fig. 3.4a). The minimum and maximum values for these
parameters are reported in Table 3.1. The output quantities of interest include the values
of the localized vertical forces on the top edge. These are determined by dividing the top
edge into M = 4 sections of equal length and integrating the vertical traction σyy over each
section. This results in M values of localized forces fi
, i = 1, . . . , M (see Fig. 3.4a),
fi =
Z i
L
M
(i−1) L
M
σyy(x, L)dx, i = 1, . . . , M. (3.39)
In addition to these forces, the maximum value of traction on the top edge is included as a
quantity of interest. Therefore, the M + 1 quantities of interest are qi = fi
, i = 1, . . . , M,
and qM+1 = maxx |σyy(x, L)|. As the location, orientation and size of the inclusion is varied,
the traction field on the top surface changes, which in turn changes the M components of
the localized forces, and the maximum value of traction.
We consider the case where the data space is constructed only from the output vector,
54
1/1
Figure 3.5: Comparison between the two meshes of the low- and high-fidelity finite element
models used to solve the traction on soft material problem.
that is uˆ(µ) = qˆ(µ). The case of including the input vector in the data space, that is
uˆ(µ) = [µ, qˆ(µ)], yields comparable results and is described in Section 3.6.
Soft body with inclusion
Parameter Min Max Units
xc 2.5 7.5 cm
yc 5 7.5 cm
θ 0 180 degree
a 1 2 cm
b 1 2 cm
Airfoil
Parameter Min Max Units
η 1 6 %c
xη 4 7 0.1 × c
t 10 20 %c
α -5 12 degree
Re 103 107 1
Table 3.1: Range spanned by input parameters for the traction and airfoil problems.
Low- and high-fidelity models. We employ two finite element-based models that differ
in the number of elements of the mesh. The low-fidelity model uses a coarse mesh with
around 400 elements, whereas the high-fidelity model has a fine mesh with around 25, 000
elements. In Figure 3.5 we show a comparison between the two meshes. It is verified that
the high-fidelity model produces a solution that is mesh-converged.
Numerical results. We generate N¯ = 1, 120 samples of the input parameters by treating
each parameter as an independent random variable that is uniformly distributed within its
range. For each instance, we generate the low- and high-fidelity solutions. We decide to use
N = 30 high-fidelity data points to construct the multi-fidelity results, and the remainder
55
40
60
80
100
σyy [Pa]
40
60
80
100
40
60
80
100
40
60
80
100 High-fidelity
Low-fidelity
0 5 10
x [cm]
0
5
10
y [cm]
0 5 10
x [cm]
0
5
10
0 5 10
x [cm]
0
5
10
0 5 10
x [cm]
0
5
10
Figure 3.6: Sample of low- and high-fidelity traction fields for different inclusions.
for testing the performance of method. We notice that the low- and high-fidelity solutions
show similar trends, but the low-fidelity versions tend to underestimate the magnitude of
traction field. In Figure 3.6 we show a sample of low- and high-fidelity solutions.
To visualize the five-dimensional data set, we project the data points in the (f1, f2)
and (f2, f3) planes. In the first and fourth columns of Figure 3.7a, we show a comparison
between the scaled low- and high-fidelity data sets. We notice that low-fidelity data captures
the structure of the high-fidelity data, however has a smaller spread.
The graph Laplacian is constructed from the low-fidelity data points; its low-lying eigenfunctions are shown in Fig. 3.7b in the (f2, f3) plane. We observe that the eigenfunctions
localize different regions of the low-fidelity data. Thereafter, we determine the coordinates of
each low-fidelity data point in the eigenfunctions space and then perform K-means clustering
to find the points closest to the centroids of N = 30 clusters. These points are shown in
blue in leftmost column of Figure 3.7a, and correspond to the points where we utilize the
high-fidelity data. We observe that these points appear to be evenly distributed over the
span of the low-fidelity data.
Next, we determine the influence functions for the multi-fidelity approximation by solving
the minimization problem described in (3.11). For that, we select a value of τ = 3 · 10−10
,
56
f1
−1
0
1
f2
Low-Fidelity
Centroids
f1
Co-Kriging
f1
SpecMF
f1
High-Fidelity
−1 0 1
f2
−1
0
1
f3
−1 0 1
f2
−1 0 1
f2
−1 0 1
f2
(a)
−1
0
1
f3
φ
2 φ
3 φ
4 φ
5
−1 0 1
f2
−1
0
1
f3
φ
6
−1 0 1
f2
φ
7
−1 0 1
f2
φ
8
−1 0 1
f2
φ
9
(b)
−1
0
1
f3
ψ
1 ψ
2 ψ
3 ψ
4
−1 0 1
f2
−1
0
1
f3
ψ
5
−1 0 1
f2
ψ
6
−1 0 1
f2
ψ
7
−1 0 1
f2
ψ
8
(c)
0
100
200
300
f1
SpecMF
Low-Fidelity
0 2 4 6 8 10 12
Error [%]
0
100
200
f2
SpecMF
Low-Fidelity
0 5 10 15 20
Error [%]
0
100
200
300
f3
SpecMF
Low-Fidelity
0 5 10 15 20
Error [%]
0
200
400
f4
SpecMF
Low-Fidelity
0 2 4 6 8 10 12 14
Error [%]
0
50
100
150
σ
max
yy
SpecMF
Low-Fidelity
0 5 10 15 20
Error [%]
(d)
1/1
Figure 3.7: Results for the traction problem. (a) Low-fidelity, co-kriging, SpecMF and
high-fidelity data. Low-fidelity data points are shown together with the points closest to
the centroids of the clusters (in blue). (b) Eight eigenfunctions from the low-lying spectrum
projected onto the (f2, f3) plane. (c) Influence functions for eight control points projected
onto the (f2, f3) plane. (d) Error distribution for the low-fidelity and SpecMF model for
each output component.
57
which corresponds to the smallest non-zero eigenvalue, and a spectrum cutoff of K = 90.
To determine the value of the regularization parameter in (3.11) we make use of the L-curve
method to study the data misfit loss versus the regularization loss for different values of
ω ∈ [10−9
, 10−6
] (see Figure 3.2). The optimal value corresponding to the elbow of the curve
is found to be ω
∗ = 8.53 · 10−9
. A subset of the influence functions and the corresponding
data points are shown in Fig. 3.7c. We observe that all the influence functions peak at their
respective data point and vanish away from it.
The final multi-fidelity SpecMF approximation, which is generated via (3.8), is shown
in third column of Figure 3.7a. The results of the co-kriging method are also shown in the
second column of the same figure. We observe that the SpecMF approximation appears to
stretch the low-fidelity data distribution to make it closer to the high-fidelity distribution.
In contrast, the co-kriging method appears to have distorted the underlying structure of the
data.
To quantify the error in the SpecMF data u˜
i
, we compute the relative absolute difference
with respect to the high-fidelity data uˆ
i at each point i and for every component k,
e
i
k =
|u˜
i
k − uˆ
i
k
|
E(|uˆk|)
× 100%, i = 1, . . . , Nval, k = 1, . . . , D. (3.40)
Here E(·) denotes the average over all validation points i, and Nval is the number of validation
points (in this case, Nval = N¯ − N). Similar errors are computed for the low-fidelity data
and the co-kriging based multi-fidelity approximation. The histograms of the errors for the
SpecMF and low-fidelity data are shown Figure 3.7d for each output component. In each case
we observe that the error distribution for the SpecMF data is more closely centered around
zero, and presents a much smaller spread with respect to the distribution of the low-fidelity
errors. The mean value of these errors is reported in Table 3.2 for each data component. We
observe that the error for the low-fidelity data ranges between 5-10%. Co-kriging method
reduces this error, but the SpecMF method is 1.5-2 times more accurate, improving the
58
accuracy of the low-fidelity data by factor of 5-9 times.
Error [%] Soft body with inclusion Airfoil
Quantity of interest f1 f2 f3 f4 σ
max
yy σyy CL CD CM
Low-fidelity 4.48 7.15 7.21 4.65 10.19 6.95 40.40 28.84 216.96
Co-kriging 1.04 1.95 1.91 0.90 2.94 - 30.21 34.33 57.44
SpecMF 0.52 1.2 1.06 0.51 2.0 1.52 18.56 10.77 45.95
Improvement Factor (SpecMF) 8.60 5.95 6.8 9.16 5.09 4.58 2.18 2.68 4.73
Table 3.2: Error for the low-fidelity, co-kriging, and SpecMF data for each output component. For the SpecMF the improvement factor, defined as the ratio of the low- and
multi-fidelity error, is also reported.
Multi-fidelity model for the entire traction field. To test the method with a higher
dimensional data space, we consider a modified version of the traction problem. Instead
of computing the four localized forces and the maximum value of the traction on the top
surface, that is Q = D = 5, we use the entire traction field discretized over 100 points. Thus
the dimension of the output and data vectors is Q = D = 100, and uj = σyy(x
j
99 , L), j =
0, . . . , 99. To generate the multi-fidelity approximations for this field, we use the same
number of high-fidelity simulations as the previous case, that is N = 30. The results are
shown in Figure 3.8. In Figure 3.8a we show the low-, high-, and multi-fidelity traction fields
for four test cases, where it is clearly seen that the multi-fidelity solution is significantly more
accurate that its low-fidelity counterpart. In Figure 3.8b we have plotted the histograms of
the errors e
i
, defined in an analogous way for both the SpecMF and low-fidelity data as,
e
i =
||u˜
i − uˆ
i
||2
E(||uˆ
i
||2|)
× 100%, i = 1, . . . , Nval, (3.41)
where the index i denotes the i-th validation sample, || · ||2 is the l2 norm, and E(·) denotes
the mean over all test samples. We observe that the error distribution for the multi-fidelity
data is closer to zero, and that there is almost no overlap between the low- and multi-fidelity
error distributions. The mean error for the low-fidelity data is around 7% whereas for the
59
50
75
100
σyy [Pa]
High-fidelity
Low-fidelity
SpecMF
0 5 10
x [cm]
50
75
100
σyy [Pa]
0 5 10
x [cm]
(a)
0
50
100
150
σyy(x, L)
SpecMF
Low-Fidelity
0 2 4 6 8 10
Error [%]
(b)
1/1
Figure 3.8: Results for the entire traction field problem. (a) Low-, high-, and multi-fidelity
solutions for four test cases. (b) Error distribution for the low-fidelity and SpecMF solutions.
multi-fidelity data it is around 1.5%. These values are also reported in Table 3.2. We note
that this example demonstrates that the proposed method can be employed for constructing
multi-fidelity approximations of fields, with large values of D.
3.3.2 Aerodynamic coefficients for a family of NACA airfoils
Problem description. The multi-fidelity approach is used to tackle a problem in aerodynamics where the goal is to predict the lift, drag and pitching moment coefficients for a
family of airfoils operating at different conditions. We consider the 4-digit NACA airfoils,
whose shape is defined by 3 geometric parameters, and investigate how the aerodynamic
performance of these airfoils changes at different Reynolds numbers and angles of attack.
Parameters and quantities of interest. The parameters of the problem comprise both
design and operating condition variables. They are the maximum camber of the airfoil η,
the distance of the maximum camber from the leading edge xη, the thickness of the airfoil t,
the angle of attack α, and the Reynolds number Re =
Uc
ν
(see Figure 3.4b). Here, U is the
flow speed, c is the chord of the airfoil, and ν is the kinematic viscosity of the fluid. In our
analysis, the Reynolds number is varied by changing the flow speed U. The range of each
parameter is reported in Table 3.1.
60
The quantities of interest are the aerodynamic coefficients CL, CD, and CM, defined as,
CL =
L
1
2
ρU2c
, CD =
D
1
2
ρU2c
, CM =
M
1
2
ρU2c
2
, (3.42)
where L, D and M are the lift, drag and the pitching moment about a point located at quarter
chord from the leading edge, respectively, and ρ is the density of the fluid. Hence, the vector
of quantities of interest is q(µ) = [CL, CD, CM], with input parameters µ = [η, xη, t, α, Re].
The data space is formed by the output quantities of interest only, i.e. uˆ(µ) = q(µ). A
different case, where the Reynolds number is included in the data space, is analyzed in
Section 3.5.
Low- and high-fidelity models. The low-fidelity data is generated using XFOIL [84],
a code based on the vortex panel method for the analysis of subsonic airfoils. In this case,
the lift and moment coefficients are calculated by direct integration of surface pressure,
whereas the drag is recovered by applying the Squire-Young formula [85]. To generate the
low-fidelity results, the surface of each airfoil is discretized with 40 panels, and the Reynolds
and Mach numbers are set by using a kinematic viscosity of ν = 10−5 m2
s
−1 and speed of
sound cs = 340 m s−1
.
High-fidelity results are generated via 2D Reynolds-averaged Navier–Stokes (RANS) simulations with a SST k − ω turbulence model [86] using OpenFOAM. The computational
domain is a cuboid of dimension 1000c × 1000c × c. A hybrid mesh is employed, comprising
a C-grid structured mesh in the proximity of the airfoil of size 4c × 6c, and an unstructured mesh in the rest of the domain (see Fig. 3.9). The number of finite volumes in the
mesh varies between 100,000 and 800,000, depending on the Reynolds number. At the outer
boundary, Dirichlet boundary conditions for both velocity and pressure are prescribed, while
on the airfoil surface a no-slip condition for velocity and zero-gradient condition for pressure
are applied. The turbulence intensity of the flow at the outer boundary is set to 2%.
61
Figure 3.9: Example of a mesh used for the high-fidelity CFD simulations.
Numerical results. We sample N¯ = 5, 400 instances of the input parameter vector from
a multivariate uniform distribution and employ XFOIL to generate the set of low-fidelity
data (shown in the first column of 3.10a in three independent planes). Then, we construct
the graph Laplacian and compute its eigendecomposition. The first three non-trivial eigenfunctions are shown in Figure 3.10c in the normalized 3-dimensional data space. We embed
the low-fidelity data points in the eigenfunction space, and use K-means clustering to find
N = 70 clusters and locate the points closest to their centroids (shown as blue dots in the first
column of Figure 3.10a). Thereafter, we run CFD simulations to acquire the high-fidelity
data at these points. We run Nval = 400 additional high-fidelity simulations, corresponding
to randomly selected low-fidelity data points, to be used as a validation set to quantify the
performance of the multi-fidelity models. These data points are shown in the fourth column
of Figure 3.10a.
We solve the minimization problem (3.11) to determine the transformation (3.8) which
yields the SpecMF data points. For this, we used a spectrum cutoff of K = 210, a value of
τ = 9.07 · 10−4
, and a regularization parameter ω
∗ = 8.87 · 10−7
.
62
Three typical influence functions plotted over the low-fidelity data set are shown in Figure
3.10d. The resulting multi-fidelity data points are plotted in the third column of Figure 3.10a.
The results obtained using co-kriging are plotted in the second column of this figure. When
compared to both the low-fidelity and co-kriging data sets, the SpecMF points appear to
better represent the structure observed in the high-fidelity data points. This is accomplished
by a significant upward shift in the low-fidelity values for the moment coefficient, and a
compression in the lift versus drag plane.
The average error for the the low- and multi-fidelity models, as defined in (3.40), is
reported in Table 3.2. For the SpecMF method, the average error for lift and drag coefficients
drops by a factor greater than two, while for the pitching moment it drops more than four
times. The co-kriging method reduces the error for the lift and pitching moment only, and
not as effectively. The distribution of errors for the low-fidelity and SpecMF data is plotted
in 3.10d. Once again we observe that the distribution of the error for the SpecMF data is
centered closer to zero and displays a narrower spread.
To better visualize the results, in Figure 3.11 we select the case of the NACA 0012
airfoil profile at Reynolds Re = 6 · 106 and plot the curves of the aerodynamic coefficients
versus angle of attack for the three models, including experimental data available for lift
and drag coefficients [87]. We observe that the high-fidelity CFD simulations match the
experimental results within 10% error for lift and 15% error for drag. In Figure 3.9 we show
the computational mesh used to perform these simulations.
By analyzing the lift and drag coefficients curves, we note that the multi-fidelity model
could learn and retain the low-fidelity trend, and adjust the magnitudes in light of the
sparse high-fidelity data. For the moment coefficient, the low-fidelity data shows a significant
disagreement with the high-fidelity results. Nonetheless, the predictions of the multi-fidelity
model more closely match the high-fidelity data in magnitude. The prediction of the trend
is also correct for angles of attack smaller than 8 degrees. It is verified that none of the highfidelity data points related to the NACA 0012 airfoil were used in constructing the multi63
−1 0 1
CL
0.0
0.1
0.2
CD
Low-fidelity
−1 0 1
CL
Co-Kriging
−1 0 1
CL
SpecMF
−1 0 1
CL
High-fidelity
−1 0 1
CL
−0.2
0.0
0.2
CM
−1 0 1
CL
−1 0 1
CL
−1 0 1
CL
0.0 0.1 0.2
CD
−0.2
0.0
0.2
CM
Centroids
0.0 0.1 0.2
CD
0.0 0.1 0.2
CD
0.0 0.1 0.2
CD
(a)
0
20
40
60
CL
SpecMF
Low-Fidelity
0 20 40 60 80 100 120
Error [%]
0
50
100
CD
SpecMF
Low-Fidelity
0 20 40 60 80 100 120 140
Error [%]
0
20
40
CM
SpecMF
Low-Fidelity
0 100 200 300 400 500 600 700 800
Error [%]
(b)
CL
−1
0
1 −1 CD
0
1
CM
−1
0
1
φ
2
CL
−1
0
1 −1 CD
0
1
CM
−1
0
1
φ
3
CL
−1
0
1 −1 CD
0
1
CM
−1
0
1
φ
4
(c)
CL
−1
0
1 −1 CD
0
1
CM
−1
0
1
ψ
1
CL
−1
0
1 −1 CD
0
1
CM
−1
0
1
ψ
2
CL
−1
0
1 −1 CD
0
1
CM
−1
0
1
ψ
3
(d)
1/1
Figure 3.10: Results for the airfoil problem. (a) Left: low-fidelity data with points closest to
cluster centroids (in blue). Center: multi-fidelity data points obtained with co-kriging and
our multi-fidelity approach (SpecMF). Right: high-fidelity data points used for validation.
(b) Error distribution for the low- and multi-fidelity data. (c) The first three non-trivial
eigenfunctions. (d) Three typical influence functions.
64
Figure 3.11: Comparison of the curves of the lift, drag and moment coefficients versus angle
of attack for the airfoil NACA 0012 at Re = 6 · 106
. The low-fidelity data are indicated
with a dashed orange line, the multi-fidelity data with a dash-dot blue line, the high-fidelity
data with a solid red line, and the experimental data are marked with a green upside-down
triangle.
fidelity model. We conclude that the multi-fidelity model could correct the data structures
in the 3-dimensional data space using the few select high-fidelity data points available, and
has increased the accuracy for all low-fidelity points.
3.4 Effect of the selection strategy of high-fidelity data
In Step 2 of the method, we outline a strategy to select the parameters values at which to
acquire high-fidelity data. The strategy consists in determining the low-fidelity data points
that are the closest to the centroids of the clusters arising within the low-fidelity data, and
then employ the high-fidelity model to compute their counterpart.
We want to compare the performance of this strategy against a random selection of the
low-fidelity points for which we acquire a high-fidelity version. To do that, we consider the
problem of predicting the traction on a soft body and the results obtained in Section 3.3.1.
Then, we solve the same problem with a random selection of the parameters at which to
compute high-fidelity data points. The number of high-fidelity data points used is fixed,
N = 30, and the problem is solved 200 times with different random selections. Then, we
compute the mean and standard deviation of the error for each component as defined in
65
(3.40), and display them in Table 3.3.
We notice that the errors committed with a random selection is larger than the errors
attained with the proposed selection approach, but is still considerably smaller than the
low-fidelity errors. This suggests that the proposed selection strategy ought to be preferred
to a random choice when possible, but that the multi-fidelity method can be successfully
applied even when the set of high-fidelity data is predetermined.
Error [%] Soft body with inclusion
Quantity of interest t1 t2 t3 t4 σ
max
yy
Low-fidelity 4.48 7.15 7.21 4.65 10.19
SpecMF + proposed selection strategy 0.52 1.2 1.06 0.51 2.0
SpecMF + random selection 0.63 (0.11) 1.54 (0.16) 1.42 (0.16) 0.81 (0.12) 2.38 (0.18)
Table 3.3: Errors in the multi-fidelity model constructed with the proposed and random
selection strategies. For the random strategy, we report the mean and the standard deviation
(in parenthesis).
3.5 Definition of the data space
The data space uˆ where the low-fidelity graph is constructed is defined as uˆ(µ) = R(µ, qˆ(µ)),
with R being a restriction operator that extracts the appropriate components of µ and qˆ.
The choice of R is problem dependent. It should include all the variables directly related to
the predictions one wants to make, i.e. the subset of the relevant components of q, and the
parameters that would help identifying clusters and structures in the data.
For example, including bifurcation parameters in the data space is important, as small
changes in their value can cause significant changes in the topology or qualitative nature of
the physical solution. Similarly, parameters that identify different and distinct regimes of the
solution, e.g. the Reynolds number in fluid dynamics, provide valuable information about
different data points, and can help separating the point cloud. This is especially important
when the performance of the low-fidelity model strongly depend upon certain parameters. If
66
the error of the low-fidelity data significantly differs based on a parameter, adding it to the
data space will make sure that the transformation (3.8) will act accordingly.
In Figure 3.12 we show a schematic of a simple case where adding the parameter µ1 to the
data space (q1, q2) leads to a clear separation of the data points. In this illustrative example,
including the parameter in the data space allows for a better selection of the high-fidelity
data to acquire and a more appropriate treatment of the two clusters.
It is important to notice that the transformation (3.8) does not lead to any displacements
along the parameters directions, as the high-fidelity data points will have by construction
the same input parameters. Hence, the multi-fidelity transformation will act only on the
space of the quantities of interests, as it should.
3.6 Numerical results with augmented data space
In this Section we analyze the results of the numerical problems when the input parameters
are included in data space.
For the problem of traction on soft body (Section 3.3.1), with input parameters µ =
[xc, yc, θ, a, b] and quantities of interest q = [f1, f2, f3, f4, σmax
yy ], we consider the case
where the data space is formed by concatenating all inputs and all outputs, i.e. uˆ = [µ, qˆ].
On the other hand, for the problem of the aerodynamic coefficients of NACA airfoils (Section 3.3.2), the input parameters and the set of quantities of interest are µ = [η, xη, t, α, Re]
and q = [CL, CD, CM], respectively. In this case, we include only the Reynolds number in
the data space, i.e. uˆ = [Re, CL, CD, CM].
For both problems, we apply once again the proposed multi-fidelity method considering
the graph constructed in the augmented data space. The numerical results are reported in
Table 3.4. We note that in these particular numerical experiments, using only the quantities
of interest in the definition of the data space (that is uˆ = qˆ) leads to better results in almost
all cases.
67
Error [%] Soft body with inclusion Airfoil
Quantity of interest f1 f2 f3 f4 σ
max
yy CL CD CM
Low-fidelity 4.48 7.15 7.21 4.65 10.19 40.40 28.84 216.96
SpecMF (uˆ = q) 0.52 1.24 1.07 0.51 2.02 18.56 10.77 45.95
SpecMF (augmented data space) 0.65 1.52 1.56 0.63 1.87 20.01 12.04 49.62
Table 3.4: Error of the low- and multi-fidelity model for each quantity of interest.
Figure 3.12: Schematic of a case where adding the parameter µ1 to the data space (q1, q2)
separates the data set into two distinct clusters, C1 and C2.
3.7 Discussion
We have proposed a multi-fidelity approach to predict the response of a system when two
mechanisms of generating data of different fidelity and cost are available. The method
includes three steps: (i) acquire a large number of low-fidelity data, (ii) identify and acquire
a small number of key high-fidelity data, and (iii) use the high-fidelity data to improve
the accuracy of all low-fidelity data. This is accomplished by constructing an undirected,
complete graph from the low-fidelity data and computing its graph Laplacian. The lowlying spectrum of the graph Lalpacian is then used to cluster the low-fidelity data and to
determine points closest to the centroids of the clusters. Thereafter, high-fidelity data is
acquired only for these select points. This data, along with the spectral decomposition of
the graph Laplacian, is used to solve a minimization problem which yields a transformation
that maps each low-fidelity data point to new multi-fidelity coordinates. It is shown that this
68
minimzation problem is convex. In numerical experiments, the approach yields multi-fidelity
data that is significantly more accurate that its low-fidelity counterpart. In particular, in
a problem motivated by biomechanics, this approach improves the accuracy of 1,120 lowfidelity data points by a factor of 5-9 (depending on the quantity of interest) by only relying
on 30 high-fidelity simulations (less than 3% of the low-fidelity simulations). Similarly,
in a problem of aerodynamics, it improves the accuracy of 5,400 low-fidelity data points
by a factor of 2-4 while only using 70 high-fidelity simulations (1.3% of the low-fidelity
simulations). The computational cost of the method scales as O(DN¯ 2 + NN¯), where D is
the dimension of the data space, and N¯ and N are the number of low- and high-fidelity
data points, respectively. The process of constructing constructing the adjacency matrix
involves computing O(N¯ 2
) dot products of the differences between D-dimensional vectors,
and therefore scales as O(DN¯ 2
). Further, the cost of computing the low-lying spectrum of
the graph Laplacian using iterative methods scales as O(N¯ 2 + NN¯)[88, 89]. However, both
these costs can be reduced by setting a cutoff on the number of edges per node on the graph
using, for example, an efficient implementation of the k−nearest neighbors algorithm [90].
In this case, the cost of the specMF algorithm scales as O(knnDN¯ log N¯ + NN¯), where knn
is the cutoff on the maximum number of neighbors for every node.
Some limitations and remarks of the proposed method, which also serve to delineate
future directions for improvement, are discussed next. In its present form, the choice of
the input parameters and output quantities to be included in the data space is somewhat
arbitrary. However, it might be possible to develop certain problem-dependent heuristics to
identify the parameters and quantities that yield better performance.
The SpecMF method learns the data distribution from the low-fidelity model, and then
adjusts it based on a few higher-fidelity data points. Thus, the underlying requirement is
that the structure of the low- and high-fidelity data does not change significantly. If the
structure arising form the low-fidelity model is fundamentally inaccurate, the benefit from
using the method will be limited. We also note that this is a common requirement among
69
most multi-fidelity models. We prove the convergence of the method for the case where
the low- and high-fidelity clusters differ by translations. This suggests that the method will
perform well when the low- and high-fidelity distributions have the same topology but differ
by well-behaved transformations. The graph Laplacian spectrum yields additional insights.
A clear spectral gap signifies that the data are effectively clustered, and also provides a way
of choosing a suitable number of high-fidelity runs.
Finally, theoretical analysis of the performance of the method in the limit of a large
number of high-fidelity data points will lead to a better understanding of its properties.
70
Chapter 4
B-SpecMF: a Bayesian extension
4.1 Introduction
As we already discussed, we are interested in solving the problem of predicting a quantity
u ∈ R
D as a function of some input parameters µ ∈ R
P
. We do not have direct access
to the underlying true value u, but for each input parameter µ
i we can use two models
to approximate it. One model is very accurate but expensive to query, and its estimate is
denoted as uˆ
i
, while the second model is less accurate but cheaper, and is denoted as u¯
i
.
This problem is well suited to be cast in probabilistic terms. Indeed, when viewed through
a Bayesian lens, the low-fidelity data can serve as a prior distribution for the truth, while the
sparse yet accurate high-fidelity data points can be used to construct the likelihood of our
multi-fidelity model. Hence, the problem can be framed as finding the posterior distribution
of the underlying data.
A major advantage of using a probabilistic approach is the possibility to compute statistics of the posterior distribution, and quantify the confidence of the predictions of the multifidelity model. This is especially useful for applications in uncertainty quantification and
sensitivity analysis.
As we will see, another advantage of the proposed Bayesian extension is that the maximum a posteriori (MAP) multi-fidelity estimate can be found in closed form by solving a
71
linear system. This is because both the prior and the likelihood for the unknown displacement field that maps low- to multi-fidelity points are chosen to be normal.
Similar to the standard SpecMF method, also in the Bayesian extension there is a regularization constant to to be determined. However, the problem of hyper-parameters specification can be solved in a pure Bayesian fashion. In fact, we can maximize the likelihood of an
additional high-fidelity dataset with respect to the hyper-parameters in order to determine
their values.
In what follows we provide a brief summary of the mathematical background before
presenting the Bayesian extension of the SpecMF method. We then proceed with numerical
results, and conclude with some final remarks and observations.
4.2 Methodology
The first two steps of the SpecMF method presented in Chapter 3 are the same for the
proposed Bayesian extension, that is: (1) generate a dense set of low-fidelity points and
build a graph in the data space, and (2) perform spectral clustering, select the points closest
to N centroids and compute their high-fidelity counterpart.
More specifically, let us construct an adjacency matrix W ∈ R
N¯×N¯
for the low-fidelity
data D¯ = {u¯
i}
N¯
i=1 via a monotonic decreasing function d(·) that measures similarity:
Wij = d
||u¯
i − u¯
j
||2
2
(4.1)
From this, we can compute a graph Laplacian L ∈ R
N¯×N¯
and its low-laying spectrum
{λk, ϕk}
N¯
k=1, with ϕk ∈ R
N¯
. This is used to perform spectral clustering over the low-fidelity
data D¯ and identify the centroids. Thereafter, the high-fidelity data corresponding to the
points closest to the centroids are acquired (see Chapter 3).
7
Ultimately, we seek a multi-fidelity approximation written as
u
i = u¯
i + z
i
, i = 1, . . . , N¯ (4.2)
where z
i ∈ R
D is the unknown displacement that maps the i-th low-fidelity point to the
true value. Let us define the displacement matrix
Z =
z
1
.
.
.
z
N¯
= [z1, . . . , zD] ∈ R
N¯×D (4.3)
Here, the superscript indicates a row, while the subscript indicates a column. This
convention is retained in the remainder of the chapter.
4.2.1 Distribution for the displacement
In this Bayesian framework, the displacement vectors z
i are treated as random variables
with a probability distribution that ought to be determined. To do that, we will define both
a prior and a likelihood for the matrix Z. In the end, we will combine these distributions to
find the posterior to sample from.
Prior We want each component of the displacement field zm, m = 1, . . . , D, to be able to
fit the data, while being as smooth and regular as possible across the different nodes of the
graph. This can be obtained by choosing a prior distribution for Z that is regularized by
the graph Laplacian:
p(Z) ∝ exp
−
ω
2
⟨Z,(L + τIN¯ ) Z⟩F
(4.4)
= exp
−
X
D
m=1
ω
2
z
T
m · (L + τIN¯ ) zm
!
(4.5)
73
where IN¯ ∈ R
N¯×N¯
is the identity matrix, and ⟨·, ·⟩F denotes the Frobenius inner product,
that is ⟨A, B⟩F = tr(ATB) = AijBij .
To give a clear interpretation of this prior we can diagonalize the Laplacian as L = ΦΛΦT
and express Z as a linear combination of the eigenfunctions via the coefficients A = [αij ] ∈
R
N¯×D, that is Z = ΦA. If we substitute this in Eq. 4.5, we obtain:
p(A) ∝ exp
−
ω
2
tr
AT Φ
T Φ (Λ + τIN¯ ) Φ
T ΦA
(4.6)
= exp
−
ω
2
tr
AT
(Λ + τIN¯ ) A
(4.7)
= exp
−
ω
2
X
N¯
i=1
X
N¯
j=1
α
2
ij (λi + 1)!
(4.8)
This is equivalent to promoting (or deeming as more likely), the coefficients associated to
eigenfunctions with small eigenvalues, which carry information about the structure of the
graph (see Chapter 4).
Likelihood Given the low- and high-fidelity datasets D = {uˆ
i}
N
i=1 and D¯ = {u¯
i}
N¯
i=1, we
can construct the matrix of known displacements zˆ
j = uˆ
j − u¯
j
, j = 1, . . . , N, as:
Zˆ =
zˆ
1
.
.
.
zˆ
N
= [zˆ1, . . . , zˆD] ∈ R
N×D (4.9)
We assume that the error in the high-fidelity data is multivariate normal with zero mean
and variance σ
2
, that is:
u
i = uˆ
i + ε, i = 1, . . . , N, (4.10)
with ε ∼ ND(0, σ2ID). Hence, the likelihood is written as:
p(Zˆ|Z) ∝ exp
−
||Zˆ − PN Z||2
2
2σ
2
!
(4.11)
where PN ∈ R
N×N¯
extracts the first N rows from the matrix on its right, and is given by
PN :=
IN 0N×(N¯−N)
.
Posterior The posterior distribution is proportional to the product of the prior and the
likelihood, and is given by,
p(Z|Zˆ) ∝ p(Zˆ|Z)p(Z) (4.12)
∝ exp
−
||Zˆ − PN Z||2
2
2σ
2
−
ω
2
⟨Z,(L + τ IN¯ ) Z⟩F
!
(4.13)
=
Y
D
m=1
exp
−
||zˆm − zm||2
2
2σ
2
−
ω
2
z
T
m · (L + τIN¯ ) zm
(4.14)
Here, ω plays the role of a regularization constant weighting the prior. Because both the
prior and the likelihood are chosen to be normal distributions, we can write the expression
above as
p(Z|Zˆ) ∝ exp
1
2
⟨Z − Z˜, C
−1
z
(Z − Z˜)⟩F
(4.15)
=
Y
D
m=1
exp
−
1
2
(zm − z˜m)
T
· C
−1
z
(zm − z˜m)
(4.16)
with
C
−1
z =
1
σ
2
P
T
N PN + ω(L + τI) (4.17)
Z˜ =
1
σ
2 CzP
T
N Zˆ. (4.18)
75
That is, the posterior distribution is also normal, where the mean for the m-th component
is given by z˜m and the covariance is given by Cz. Further, since for a normal distribution
the mean and the mode are the same, this means that the maximum a-posteriori (MAP)
estimate Z˜ can be found in closed form by solving the linear system above.
The covariance matrix of the posterior distribution Cz provides a measure of confidence
of the model prediction. In fact, the resulting multi-fidelity posterior distribution for each
data point u
i
, i = 1, . . . , N¯, is a multivariate normal distribution with mean u˜
i = u¯
i + z˜
i
,
with z˜
i being the i-th row of Z˜, and a diagonal covariance matrix given by Cz, iiID.
An important difference with respect to the standard SpecMF method is that now the
displacement field is not explicitly expressed in terms of the first K = O(N) graph Laplacian
eigenfunctions. However, because the Laplacian eigenfunctions form a basis in R
N¯
, the MAP
estimate of the proposed approach is equivalent to what one would get using the standard
SpecMF by setting K = N¯, and without applying a softmax transform. This achieves two
things at the same time: (i) the less restrictive form that the displacement field Z can take
increases the expressivity of the model, and (ii) the absence of non-linearity introduced by
the softmax allows for the MAP estimate Z˜ to be found in closed form.
The computational cost of solving the linear system (4.18) is obviously higher compared
to the standard SpecMF, if no softmax was used in either methods. However, there are
efficient low-rank approximations of the graph Laplacian that can be used to mitigate this
cost [91].
4.2.2 Hyperparameter specification
One of the attractive features of the proposed method is that it only has one hyperparameter,
ω, that needs to be determined. In this section we describe a simple and effective approach
for determining this parameter.
To choose the value of ω, we can use an additional high-fidelity dataset of M points
and determine the value of ω that maximizes their marginal log-likelihood. The marginal
76
distribution for a subset of zm is obtained simply by dropping off the variables that don’t
appear in this set from Z˜ and Cz. This is possible because the posterior distribution is a
multivariate normal distribution.
Let us term Ξ = [ξ1, . . . , ξD] ∈ R
M×D the displacements at the M points used for the
hyper-parameters specification, and let ˜ξm ∈ R
M and Γz ∈ R
M×M be the corresponding
mean and covariance matrix, obtained by the extracting M rows from z˜m, and the M rows
and columns from Cz. Then, we have:
p(Ξ) = Y
D
m=1
1
q
(2π)
M |Γz|
exp
−
1
2
ξm − ˜ξm
T
· Γ
−1
z
ξm − ˜ξm
(4.19)
=
1
q
(2π)
MD |Γz|D
exp
−
X
D
m=1
1
2
ξm − ˜ξm
T
· Γ
−1
z
ξm − ˜ξm
!
(4.20)
Once we acquire the M high-fidelity data Ξ, we can maximize the log-likelihood ℓ(ω; Ξ) with
respect to ω, which is given up to a constant by
ℓ(ω; Ξ) = −
D
2
log |Γz| − X
D
m=1
1
2
ξm − ¯ξm
T
· Γ
−1
z
ξm − ¯ξm
. (4.21)
We note that the log-likelihood above depends on ω through the vectors ¯ξm and the
matrix Γz.
4.3 Numerical Results
We present numerical results for two problems. In the first one we use a synthetic 2-
dimensional dataset to analyze the performance of the method in a controlled setting. The
second problem is the one also considered in the previous chapter, that is the prediction
of the traction over the top side of a compressible square sheet in plane stress with a solid
inclusion of variable size and location.
Figure 4.1: Low- and high-fidelity points for the synthetic dataset problem.
4.3.1 Synthetic data
For this first problem we use a synthetic dataset of N¯ = 1000 points in a 2-dimensional
space. The data is distributed according to three clear structures: a circle, a wave, and a
strip. The low-fidelity data differ from their high-fidelity counterpart by three independent
affine transformations applied to these three structures separately, and iid Gaussian noise
(see Figure 4.1). In what follows, we will use N = 15 high-fidelity data point to construct
the likelihood term (4.11), and M = 15 points to maximize the log-likelihood (4.21) and
determine the value of ω. The remaining points of the dataset are used only for visual
comparison with the resulting multi-fidelity data.
High-fidelity dataset Following the first two steps of the standard SpecMF method, we
compute the graph Laplacian L, perform spectral clustering over the low-fidelity data, and
identify the points closest to the centroids of N = 15 clusters. Furthermore, we also sample
M = 15 extra high-fidelity to determine ω. To select these points we can exploit the same
clustering; in fact, we can sample one random point from each cluster, to make sure that this
additional dataset covers the whole dataset uniformly. The N points used for computing
the likelihood distribution, are referred to as training points, and the M points used for the
78
(a) (b)
Figure 4.2: (a) Points used for training and validation. (b) Low-fidelity data used for training
(larger blue dots), connected with a red segment with their high-fidelity counterparts (large
red dots). The red segments represent the known displacements stored in the matrix Zˆ.
hyper-parameter specification, are termed as validation points, are shown in Figure 4.2a.
From the training dataset, we can compute the matrix of known displacements Zˆ. These
displamcement vectors are shown in Figure 4.2b. On the other hand, with the validation
dataset we can maximize the log-likelihood from Eq. (4.21) with respect to ω. In Figure 4.3
we show the trend of ℓ(ω), with the optimal value found to be ω
∗ = 4.93.
Multi-fidelity dataset Once the value of ω is determined, we can compute the covariance
matrix Cz from Eq. (4.17), and solve the system (4.18) for the mean displacement Z˜. From
that, we obtain the multi-fidelity approximations,
u˜
i = u¯
i + z˜
i
, i = 1, . . . , N, ¯ (4.22)
with z˜
i being the i-th row of Z˜. Moreover, the diagonal values of Cz provide the variance
associated to each estimate u˜
i
.
In Figure 4.4, we show the initial low-fidelity dataset in the first plot, the resulting multifidelity dataset in the second plot, and the underlying true high-fidelity data in the fourth
79
Figure 4.3: Plot of the log-likelihood of the validation dataset ℓ(ω). The maximum is achieved
for ω
∗ = 4.93.
plot. Further, the third plot shows the uncertainty of the multi-fidelity model through circles
centered at each multi-fidelity point estimate, with radius equal to the associated standard
deviation. The uncertainty of the model predictions is also shown in Fig 4.5, where the
multi-fidelity data points are colored based on the value of standard deviation attained. We
notice that the multi-fidelity model was able to learn the structures from the low-fidelity
data, and adjust them based on the few high-fidelity points used for training, and the final
multi-fidelity dataset closely resembles the underlying truth. By looking at Figure 4.5, we
see that the uncertainty of the multi-fidelity predictions, measured by the standard deviation
attained at each point, is smaller nearby the points used for training, and increases further
away. We verified that for around 81% of the multi-fidelity points, the distance with respect
to corresponding high-fidelity counterpart is less than two standard deviations away.
4.3.2 Traction on a soft material with a stiff inclusion
We now tackle the elasticity problem considered in the previous chapter. The objective is
to predict localized forces over the upper side by a soft 2-dimensional square sheet in plane
stress with a stiff inclusion of variable size and location. For more details about the problem
80
Figure 4.4: Comparison of the low-, multi- and the underlying high-fidelity datasets. The
third plot shows the circles indicating the standard deviation associated with each multifidelity estimate.
Figure 4.5: Multi-fidelity points colored with the value of standard deviation. The training
data points are indicated by large red dots.
81
(a)
(b)
Figure 4.6: (a) Points used for training. (b) Points used for validation. The data is presented
by projection onto two-dimensional planes.
setup, the low- and high-fidelity models, and the definition of the input parameters and
quantities of interest, the reader is referred to Chapter 3.
High-fidelity dataset The total number of low-fidelity points is N¯ = 1120, and we want
to use N = 30 high-fidelity points for training, that is for computing Zˆ and constructing
the likelihood distribution, and M = 30 additional high-fidelity points for validation, that
is determining the value of ω. The training dataset is determined by computing the graph
Laplacian and performing spectral clustering over the low-fidelity data (see Chapter 3), and
the validation set is formed by sampling one random point per cluster. The resulting datasets
are shown in Figure 4.6.
The optimal regularization constant is found by maximizing the log-likelihood of the
validation set ℓ(ω), and the value is found to be ω
∗ = 182.48 (see Figure 4.7).
Multi-fidelity dataset After solving for Z˜ and Cz via (4.18) and (4.17), we obtain the
multi-fidelity MAP estimates and the associated variance. The results are shown in Figure
82
Figure 4.7: Plot of the log-likelihood of the validation dataset ℓ(ω). The maximum is achieved
for ω
∗ = 181.48.
4.8 by projecting the data over the 4 planes (ui
, ui+1), i = 1, . . . , 4, in the normalized data
space (where the low-fidelity data are scaled to take values in between -1 and 1). In the first
column of this figure we show the low-fidelity dataset, in the second the multi-fidelity MAP
estimate, in the third we plot circles indicating the standard deviation associated with each
MAP estimate, and in the fourth the underlying high-fidelity data.
We notice how the MAP estimates of the multi-fidelity model follow a distribution that
closely matches the high-fidelity one. In particular, the low-fidelity data distribution is
stretched to fit the high-fidelity points used for training, while still preserving the structure
learned from the low-fidelity data set.
We can quantify the error of the multi-fidelity data in the same way we did for the
standard SpecMF method, via Eq. (3.40). The numerical results are reported in Table 4.1.
We notice how the error of the MAP estimates of the Bayesian SpecMF (B-SpecMF) is very
close to the standard SpecMF method, leading to an improvement of around 5 to 7 times
over the low-fidelity model. However, with the B-SpecMF method we also have access to
the uncertainty of the predictions, measured by the variance of the MAP estimates given
by the diagonal values of Cz. For each component, we can then compute the percentage of
83
data points for which the difference between the multi-fidelity value and true corresponding
high-fidelity value is within two standard deviations. This is reported in the last row of
Table 4.1. We see how for almost all components, around 95% of the multi-fidelity MAP
estimates are within this 2 standard deviation threshold, expect for the last component (the
maximum stress), which shows a 82% rate. This is likely caused by the fact that the first
four components, defined as integral quantities, tend to experience a higher correlation and
coherent behavior. This helps the multi-fidelity model to increase the accuracy of a larger
portion of data points close to the centroids, when these are mapped to the true high-fidelity
values. On the other hand, the local maximum value of stress can show a lower correlation
for different input parameters, which limits the degree of which accuracy can be injected by
the training data points.
Error [%] Soft body with inclusion
Quantity of interest f1 f2 f3 f4 σ
max
yy
Low-fidelity 4.48 7.15 7.21 4.65 10.19
B-SpecMF 0.63 1.3 1.32 0.62 2.39
< 2-std 95.9% 94.1% 94.6% 98.4% 82.3%
Table 4.1: Error for the low-fidelity and Bayesian SpecMF (B-SpecMF) data for each output
component. For the B-SpecMF multi-fidelity estimates, we also include the percentage of
points for which the difference with respect to the high-fidelity value is within two standard
deviation away.
4.4 Conclusions
We presented and analyzed the performance of a Bayesian extension of the SpecMF method
put forth in Chapter 3. The methodology shares the same premise with the standard
SpecMF, but the multi-fidelity approximation is expressed as a posterior distribution whose
statistics can be found in closed form. This is done by using a prior regularized by the graph
Laplacian. The resulting posterior distribution enjoys several properties:
84
Figure 4.8: Comparison of the low-, multi- and high-fidelity datasets projected over the 4
planes (ui
, ui+1), i = 1, . . . , 4. The third columns shows circles indicating the standard
deviation associated with each multi-fidelity estimate.
85
• The MAP estimate can be found in closed form by solving a linear system.
• The error of the MAP estimate is comparable to the standard SpecMF method.
• Each multi-fidelity estimate is equipped with an uncertainty measure, given by the
covariance matrix of the posterior distribution.
• Within each cluster, the variance of an estimate is larger the farther it is from the
centroid, for which the high-fidelity corresponding value is known.
• The uncertainty level of each multi-fidelity estimate can be used to inform next evaluations of the high-fidelity model.
In summary, the Bayesian SpecMF provides a probabilistic extension of the SpecMF
method with a number of advantages, in terms of accuracy, competitive computational cost,
and the possibility to perform uncertainty quantification analysis.
86
Chapter 5
Future directions
Two graph-based multi-fidelity models have been put forth, and despite providing a comprehensive analysis of the theory and performance of these methods, several promising directions
for future research can be outlined.
The first venue for exploration is trying to use a more sophisticated model for noise to
define the likelihood of the multi-fidelity distribution. While the current version employs the
commonly used iid Gaussian noise, one could think of using correlated noise, non necessity
additive, to model the behavior of the error of the high-fidelity data as a function of the
input parameters. This is a justifiable criticism, as the accuracy of a computational model is
likely affected by the input parameters, e.g. the accuracy of a CFD model for predicting the
behavior of an airfoil can highly depend on Reynolds number and angle of attack. Hence,
the error should not simply be uncorrelated, zero-mean normal noise.
On the theoretical side, it will be beneficial to study the behavior of the method in the
limit for large data. This can lead to important results when it comes to consistency, thus
providing a more rigorous foundation. In particular, one can ask what happens in the limit
of infinite low-fidelity data, so that the clusters become continuous, dense set. Or also, does
the method converge when the number of high-fidelity points increases, and if so, with which
rate?
Moreover, the scalability of the proposed methodology in the limit of large datasets is yet
another area for exploration. Developing efficient ways to handle increasingly larger datasets
87
poses an important challenge, that can be tackled by considering, for example, techniques
to perform order reduction of the graph Laplacian and ease the burden of inverting a large
matrix.
Another theoretical front is the rigorous analysis of which kind of graph-to-graph transformations can be well approximated by the proposed methods with arbitrarily high precision. Understanding the limitations and strengths of the current approach in capturing
specific types of transformations will contribute to the method’s adaptability and guide its
application in various contexts and fields.
The current research primarily focuses on deriving a multi-fidelity approximation for
datasets in a low- to moderately high-dimensional space. But the proposed methodology
can be extended to the study of high-dimensional data, to tackle images or fields, where
the quantity of interest is defined as a set of values attained over the grid points of a
computational mesh. Naturally, investigating the performance and limitation of the method
in the context of a high-dimensional data space will deepen the understanding of it, as well
as broaden its applicability.
Finally, future work could also investigate the possibility of iterative strategies, where
the final multi-fidelity data set is generated in multiple steps. This can be useful, for example, for defining a better high-fidelity data acquisition policy. That is, determining where
additional high-fidelity data should be acquired, based on where the prediction of the model
are expected to be affected by greater error. In particular, the information of the variance
of the estimates provided by the B-SpecMF method can be used in designing adaptive data
acquisition strategies, informed by the evolving uncertainty landscape.
In conclusion, this thesis lays the groundwork for a class of graph-based multi-fidelity
methods for data-driven modeling of physical systems, and the outlined future directions
hope to push the field forward, thereby allowing to address always more challenging realworld applications.
88
Bibliography
[1] Benjamin Peherstorfer, Karen Willcox, and Max Gunzburger. “Optimal model management for multifidelity Monte Carlo estimation”. In: SIAM Journal on Scientific
Computing 38.5 (2016), A3163–A3194. doi: 10.1137/15M1046472.
[2] T. Goel et al. “Ensemble of surrogates”. In: Structural and Multidisciplinary Optimization 33.3 (Mar. 2007), pp. 199–216. issn: 1615-1488. doi: 10.1007/s00158-006-
0051-9. url: https://doi.org/10.1007/s00158-006-0051-9.
[3] Robert B Gramacy and Herbert KH Lee. “Adaptive design and analysis of supercomputer experiments”. In: Technometrics 51.2 (2009), pp. 130–145.
[4] Douglas Allaire and Karen Willcox. “A mathematical and computational framework
for multifidelity design and analysis with computer models”. In: International Journal
for Uncertainty Quantification 4.1 (2014).
[5] L. W. T. Ng and K. E. Willcox. “Multifidelity approaches for optimization under
uncertainty”. In: International Journal for Numerical Methods in Engineering 100.10
(2014), pp. 746–772. doi: https : / / doi . org / 10 . 1002 / nme . 4761. url: https :
//onlinelibrary.wiley.com/doi/abs/10.1002/nme.4761.
[6] Jari Kaipio and Erkki Somersalo. “Statistical inverse problems: Discretization, model
reduction and inverse crimes”. In: Journal of Computational and Applied Mathematics
198.2 (2007). Special Issue: Applied Computational Inverse Problems, pp. 493–504.
issn: 0377-0427. doi: https : / / doi . org / 10 . 1016 / j . cam . 2005 . 09 . 027. url:
https://www.sciencedirect.com/science/article/pii/S0377042705007296.
[7] M Giselle Fern´andez-Godino et al. “Review of multi-fidelity models”. In: arXiv preprint
arXiv:1609.07196 (2016).
[8] Stefan Heinrich. “Multilevel monte carlo methods”. In: Large-Scale Scientific Computing: Third International Conference, LSSC 2001 Sozopol, Bulgaria, June 6–10, 2001
Revised Papers 3. Springer. 2001, pp. 58–67.
[9] Michael B. Giles. “Multilevel Monte Carlo methods”. In: Acta Numerica 24 (2015),
259–328. doi: 10.1017/S096249291500001X.
[10] Leo W. T. Ng and Karen E. Willcox. “Multifidelity approaches for optimization under uncertainty”. In: International Journal for Numerical Methods in Engineering
100.10 (2014), pp. 746–772. doi: https : / / doi . org / 10 . 1002 / nme . 4761. eprint:
https://onlinelibrary.wiley.com/doi/pdf/10.1002/nme.4761. url: https:
//onlinelibrary.wiley.com/doi/abs/10.1002/nme.4761.
89
[11] Benjamin Peherstorfer, Karen Willcox, and Max Gunzburger. “Optimal Model Management for Multifidelity Monte Carlo Estimation”. In: SIAM Journal on Scientific
Computing 38.5 (2016), A3163–A3194. doi: 10 . 1137 / 15M1046472. eprint: https :
//doi.org/10.1137/15M1046472. url: https://doi.org/10.1137/15M1046472.
[12] Gianluca Geraci, Michael S Eldred, and Gianluca Iaccarino. “A multifidelity multilevel
Monte Carlo method for uncertainty propagation in aerospace applications”. In: 19th
AIAA non-deterministic approaches conference. 2017, p. 1951.
[13] Marc C Kennedy and Anthony O’Hagan. “Predicting the Output from a Complex
Computer Code When Fast Approximations Are Available”. In: Biometrika 87.1 (2000),
pp. 1–13. issn: 00063444. doi: 10.1093/biomet/87.1.1.
[14] Alexander IJ Forrester, Andr´as S´obester, and Andy J Keane. “Multi-fidelity optimization via surrogate modelling”. In: Proceedings of the royal society a: mathematical,
physical and engineering sciences 463.2088 (2007), pp. 3251–3269. doi: 10 . 1098 /
rspa.2007.1900.
[15] Paris Perdikaris et al. “Multi-fidelity modelling via recursive co-kriging and Gaussian–
Markov random fields”. In: Proceedings of the Royal Society A: Mathematical, Physical
and Engineering Sciences 471.2179 (2015), p. 20150018.
[16] Chanyoung Park, Raphael T Haftka, and Nam H Kim. “Remarks on multi-fidelity
surrogates”. In: Structural and Multidisciplinary Optimization 55 (2017), pp. 1029–
1050. doi: 10.1007/s00158-016-1550-y.
[17] Douglas Allaire and Karen Willcox. “A mathematical and computational framework
for multifidelity design and analysis with computer models”. In: International Journal
for Uncertainty Quantification 4.1 (2014).
[18] Tushar Goel et al. “Ensemble of surrogates”. In: Structural and Multidisciplinary Optimization 33 (2007), pp. 199–216.
[19] Robert B Gramacy et al. “Modeling an augmented Lagrangian for blackbox constrained
optimization”. In: Technometrics 58.1 (2016), pp. 1–11.
[20] Robert B Gramacy and Herbert K H Lee. “Bayesian treed Gaussian process models
with an application to computer modeling”. In: Journal of the American Statistical
Association 103.483 (2008), pp. 1119–1130.
[21] C´edric Durantin et al. “Multifidelity surrogate modeling based on radial basis functions”. In: Structural and Multidisciplinary Optimization 56 (2017), pp. 1061–1075.
doi: 10.1007/s00158-017-1703-7.
90
[22] Xueguan Song et al. “A radial basis function-based multi-fidelity surrogate model:
exploring correlation between high-fidelity and low-fidelity models”. In: Structural and
Multidisciplinary Optimization 60 (2019), pp. 965–981. doi: 10.1007/s00158-019-
02248-0.
[23] Qi Zhou et al. “A variable fidelity information fusion method based on radial basis
function”. In: Advanced Engineering Informatics 32 (2017), pp. 26–39. issn: 1474-0346.
doi: 10.1016/j.aei.2016.12.005.
[24] Souvik Chakraborty. “Transfer learning based multi-fidelity physics informed deep
neural network”. In: Journal of Computational Physics 426 (2021), p. 109942. issn:
0021-9991. doi: 10.1016/j.jcp.2020.109942.
[25] Subhayan De et al. “On transfer learning of neural networks using bi-fidelity data
for uncertainty propagation”. In: International Journal for Uncertainty Quantification 10.6 (2020), pp. 543–573. doi: 10.1615/Int.J.UncertaintyQuantification.
2020033267.
[26] Michael Penwarden et al. “Multifidelity modeling for Physics-Informed Neural Networks (PINNs)”. In: Journal of Computational Physics 451 (2022), p. 110844. issn:
0021-9991. doi: 10.1016/j.jcp.2021.110844.
[27] Xuhui Meng and George Em Karniadakis. “A composite neural network that learns
from multi-fidelity data: Application to function approximation and inverse PDE problems”. In: Journal of Computational Physics 401 (2020), p. 109020. issn: 0021-9991.
doi: 10.1016/j.jcp.2019.109020.
[28] Xuhui Meng, Hessam Babaee, and George Em Karniadakis. “Multi-fidelity Bayesian
neural networks: Algorithms and applications”. In: Journal of Computational Physics
438 (2021), p. 110361. issn: 0021-9991. doi: 10.1016/j.jcp.2021.110361.
[29] M. Raissi, P. Perdikaris, and G.E. Karniadakis. “Physics-informed neural networks: A
deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations”. In: Journal of Computational Physics 378 (2019),
pp. 686–707. issn: 0021-9991. doi: 10.1016/j.jcp.2018.10.045.
[30] Shibo Li et al. “Multi-fidelity Bayesian optimization via deep neural networks”. In:
Advances in Neural Information Processing Systems 33 (2020). Ed. by H. Larochelle
et al., pp. 8521–8531.
[31] Nicholas Geneva and Nicholas Zabaras. “Multi-fidelity generative deep learning turbulent flows”. In: Foundations of Data Science 2.4 (2020), pp. 391–428. doi: 10.3934/
fods.2020019.
91
[32] Christian Perron, Dushhyanth Rajaram, and Dimitri Mavris. “Development of a multifidelity reduced-order model based on manifold alignment”. In: AIAA Aviation 2020
Forum. 2020, p. 3124. doi: 10.2514/6.2020-3124.
[33] Akil Narayan, Claude Gittelson, and Dongbin Xiu. “A stochastic collocation algorithm
with multifidelity models”. In: SIAM Journal on Scientific Computing 36.2 (2014),
A495–A521. doi: 10.1137/130929461.
[34] Vahid Keshavarzzadeh, Robert M Kirby, and Akil Narayan. “Parametric topology
optimization with multiresolution finite element models”. In: International Journal for
Numerical Methods in Engineering 119.7 (2019), pp. 567–589. doi: 10.1002/nme.6063.
[35] Orazio Pinti et al. “Multi-fidelity approach to predicting multi-rotor aerodynamic interactions”. In: AIAA Journal 60.6 (2022), pp. 3894–3908. doi: 10.2514/1.J060227.
[36] Steven Yoon, Henry C Lee, and Thomas H Pulliam. “Computational analysis of multirotor flows”. In: 54th AIAA aerospace sciences meeting. 2016, p. 0812.
[37] Seokkwan Yoon et al. “Computational aerodynamic modeling of small quadcopter
vehicles”. In: American Helicopter Society (AHS) 73rd Annual Forum Fort Worth,
Texas. 2017.
[38] Patricia Ventura Diaz and Steven Yoon. “High-fidelity computational aerodynamics
of multi-rotor unmanned aerial vehicles”. In: 2018 AIAA Aerospace Sciences Meeting.
2018, p. 1266.
[39] Matthew Misiorowski, Farhan Gandhi, and Assad A. Oberai. “Computational Study
on Rotor Interactional Effects for a Quadcopter in Edgewise Flight”. In: AIAA Journal
57.12 (2019), pp. 5309–5319. doi: 10.2514/1.J058369. eprint: https://doi.org/
10.2514/1.J058369. url: https://doi.org/10.2514/1.J058369.
[40] Richard Healy, Matthew Misiorowski, and Farhan Gandhi. “A CFD-Based Examination of Rotor-Rotor Separation Effects on Interactional Aerodynamics for eVTOL
Aircraft”. In: Journal of the American Helicopter Society 67 (2022). doi: 10.4050/
JAHS.67.012006.
[41] Richard Healy, Farhan Gandhi, and Mihir Mistry. “Computational Investigation of
Multirotor Interactional Aerodynamics with Hub Lateral and Longitudinal Canting”.
In: AIAA Journal (2021), pp. 1–11. doi: 10.2514/1.J060530. url: https://doi.
org/10.2514/1.J060530.
[42] Xueyu Zhu, Akil Narayan, and Dongbin Xiu. “Computational aspects of stochastic
collocation with multifidelity models”. In: SIAM/ASA Journal on Uncertainty Quantification 2.1 (2014), pp. 444–463.
92
[43] Jerrad Hampton et al. “Practical error bounds for a non-intrusive bi-fidelity approach
to parametric/stochastic model reduction”. In: Journal of Computational Physics 368
(2018), pp. 315–332.
[44] Vahid Keshavarzzadeh, Robert M. Kirby, and Akil Narayan. “Parametric topology
optimization with multiresolution finite element models”. In: International Journal
for Numerical Methods in Engineering 119.7 (Apr. 2019), 567–589. issn: 1097-0207.
doi: 10.1002/nme.6063. url: http://dx.doi.org/10.1002/nme.6063.
[45] Daniel J. Perry et al. “Allocation strategies for high fidelity models in the multifidelity
regime”. In: SIAM/ASA Journal on Uncertainty Quantification 7 (2019), pp. 203–231.
[46] Dimitris Papailiopoulos, Anastasios Kyrillidis, and Christos Boutsidis. “Provable deterministic leverage score sampling”. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 2014, pp. 997–1006.
[47] Christos Boutsidis, Petros Drineas, and Malik Magdon-Ismail. “Near-optimal columnbased matrix reconstruction”. In: SIAM Journal on Computing 43.2 (2014), pp. 687–
717.
[48] Tony F. Chan. “Rank revealing QR factorizations”. In: Linear Algebra and its Applications 88-89 (1987), pp. 67–82. issn: 0024-3795. doi: https : / / doi . org / 10 .
1016/0024-3795(87)90103-0. url: https://www.sciencedirect.com/science/
article/pii/0024379587901030.
[49] Tony F. Chan and Per Christian Hansen. “Some Applications of the Rank Revealing
QR Factorization”. In: SIAM Journal on Scientific Computing 13.3 (1992), 727–741.
issn: 0196-5204. doi: 10.1137/0913043. url: https://doi.org/10.1137/0913043.
[50] Ming Gu and Stanley C Eisenstat. “Efficient algorithms for computing a strong rankrevealing QR factorization”. In: SIAM Journal on Scientific Computing 17.4 (1996),
pp. 848–869.
[51] Tony F. Chan. “Alternative To The SVD: Rank Revealing QR-Factorizations”. In: Advanced Algorithms and Architectures for Signal Processing I. Ed. by Jeffrey M. Speiser.
Vol. 0696. International Society for Optics and Photonics. SPIE, 1986, pp. 31 –38. doi:
10.1117/12.936872. url: https://doi.org/10.1117/12.936872.
[52] Yoo Pyo Hong and C-T Pan. “Rank-Revealing QR Factorizations and the Singular
Value Decomposition”. In: Mathematics of Computation 58.197 (1992), pp. 213–232.
[53] Robert Niemiec and Farhan Gandhi. “Development and Validation of the Rensselaer
Multicopter Analysis Code (RMAC): A Physics-Based Comprehensive Modeling Tool”.
In: 75th Annual Forum of the American Helicopter Society (Philadelphia, PA, May 13-
16, 2019).
93
[54] Whirlwind Propellers. https://whirlwindpropellers.com/airboats/shop/razor-x/. 2018.
url: https://whirlwindpropellers.com/airboats/shop/razor-x/.
[55] ACUSIM Software, Inc. AcuSolve Validation - NACA 0012 Airfoil. Tech. rep. 2010.
doi: https : / / www . altair . com / resource / acusolve - validation - naca0012 -
airfoil.
[56] David Corson, Rajeev Jaiman, and Farzin Shakib. “Industrial application of RANS
modelling: capabilities and needs”. In: International Journal of Computational Fluid
Dynamics 23.4 (2009), pp. 337–347. doi: 10.1080/10618560902776810. eprint: https:
/ / doi . org / 10 . 1080 / 10618560902776810. url: https : / / doi . org / 10 . 1080 /
10618560902776810.
[57] Alexander N. Brooks and Thomas J.R. Hughes. “Streamline upwind/Petrov-Galerkin
formulations for convection dominated flows with particular emphasis on the incompressible Navier-Stokes equations”. In: Computer Methods in Applied Mechanics and
Engineering 32.1 (1982), pp. 199–259. issn: 0045-7825. doi: https://doi.org/10.
1016/0045-7825(82)90071-8. url: https://www.sciencedirect.com/science/
article/pii/0045782582900718.
[58] Charles A Taylor, Thomas JR Hughes, and Christopher K Zarins. “Finite element
modeling of blood flow in arteries”. In: Computer methods in applied mechanics and
engineering 158.1-2 (1998), pp. 155–196.
[59] Kenneth E. Jansen, Christian H. Whiting, and Gregory M. Hulbert. “A generalizedalpha method for integrating the filtered Navier–Stokes equations with a stabilized
finite element method”. In: Computer Methods in Applied Mechanics and Engineering
190.3 (2000), pp. 305–319. issn: 0045-7825. doi: https://doi.org/10.1016/S0045-
7825(00)00203-6. url: https://www.sciencedirect.com/science/article/pii/
S0045782500002036.
[60] J. Gordon Leishman. “Introduction: Rotor Wakes”. In: Journal of Aircraft 39.5 (2002),
pp. 721–721. doi: 10.2514/2.2999. eprint: https://doi.org/10.2514/2.2999. url:
https://doi.org/10.2514/2.2999.
[61] J. L. Lumley. “The Structure of Inhomogeneous Turbulent Flows”. In: Proceedings of
the International Colloquium on the Fine Scale Structure of the Atmosphere and Its
Influence on Radio Wave Propagation (1967).
[62] Gal Berkooz, Philip Holmes, and John L Lumley. “The proper orthogonal decomposition in the analysis of turbulent flows”. In: Annual review of fluid mechanics 25.1
(1993), pp. 539–575.
94
[63] Philip Holmes et al. Turbulence, Coherent Structures, Dynamical Systems and Symmetry. 2nd ed. Cambridge Monographs on Mechanics. Cambridge University Press, 2012.
doi: 10.1017/CBO9780511919701.
[64] Kunihiko Taira et al. “Modal Analysis of Fluid Flows: An Overview”. In: AIAA Journal
55.12 (2017), pp. 4013–4041. doi: 10.2514/1.J056060. eprint: https://doi.org/
10.2514/1.J056060. url: https://doi.org/10.2514/1.J056060.
[65] Clarence W. Rowley, Tim Colonius, and Richard M. Murray. “Model reduction for
compressible flows using POD and Galerkin projection”. In: Physica D: Nonlinear
Phenomena 189.1 (2004), pp. 115–129. issn: 0167-2789. doi: https://doi.org/10.
1016/j.physd.2003.03.001. url: https://www.sciencedirect.com/science/
article/pii/S0167278903003841.
[66] Andrea L Bertozzi et al. “Uncertainty quantification in graph-based classification of
high dimensional data”. In: SIAM/ASA Journal on Uncertainty Quantification 6.2
(2018), pp. 568–595.
[67] Dejan Slepcev and Matthew Thorpe. “Analysis of p-Laplacian regularization in semisupervised learning”. In: SIAM Journal on Mathematical Analysis 51.3 (2019), pp. 2085–
2120.
[68] Mikhail Belkin, Irina Matveeva, and Partha Niyogi. “Regularization and semi-supervised
learning on large graphs”. In: International Conference on Computational Learning
Theory. Springer. 2004, pp. 624–638.
[69] Andrea L Bertozzi et al. “Posterior consistency of semi-supervised regression on graphs”.
In: Inverse Problems 37.10 (2021), p. 105011.
[70] Matthew M Dunlop et al. “Large data and zero noise limits of graph-based semisupervised learning algorithms”. In: Applied and Computational Harmonic Analysis
49.2 (2020), pp. 655–697.
[71] Franca Hoffmann et al. “Consistency of semi-supervised learning algorithms on graphs:
Probit and one-hot methods”. In: Journal of Machine Learning Research 21 (2020),
pp. 1–55.
[72] Nicolas Garcia Trillos and Dejan Slepˇcev. “A variational approach to the consistency
of spectral clustering”. In: Applied and Computational Harmonic Analysis 45.2 (2018),
pp. 239–281.
[73] Mikhail Belkin and Partha Niyogi. “Convergence of Laplacian eigenmaps”. In: Advances in neural information processing systems 19 (2006).
95
[74] Ronald R. Coifman and St´ephane Lafon. “Diffusion maps”. In: Applied and Computational Harmonic Analysis 21.1 (2006), pp. 5–30. issn: 1063-5203. doi: https :
//doi.org/10.1016/j.acha.2006.04.006. url: https://www.sciencedirect.
com/science/article/pii/S1063520306000546.
[75] Nicolas Garcia Trillos et al. “Error estimates for spectral convergence of the graph
Laplacian on random geometric graphs towards the Laplace–Beltrami operator”. In:
arXiv preprint arXiv:1801.10108 (2018).
[76] Stephen Guattery and Gary L Miller. “Graph embeddings and laplacian eigenvalues”.
In: SIAM Journal on Matrix Analysis and Applications 21.3 (2000), pp. 703–723.
[77] Ulrike Von Luxburg. “A tutorial on spectral clustering”. In: Statistics and computing
17.4 (2007), pp. 395–416.
[78] Mikhail Belkin and Partha Niyogi. “Laplacian eigenmaps and spectral techniques for
embedding and clustering”. In: Advances in neural information processing systems 14
(2001).
[79] Lihi Zelnik-Manor and Pietro Perona. “Self-tuning spectral clustering”. In: Advances
in Neural Information Processing Systems 17 (2004).
[80] Per Christian Hansen and Dianne Prost O’Leary. “The use of the L-curve in the
regularization of discrete ill-posed problems”. In: SIAM journal on scientific computing
14.6 (1993), pp. 1487–1503. doi: 10.1137/0914086.
[81] Justin S. Gray et al. “OpenMDAO: An Open-Source Framework for Multidisciplinary
Design, Analysis, and Optimization”. In: Structural and Multidisciplinary Optimization
59 (4 2019), pp. 1075–1104. doi: 10.1007/s00158-019-02211-z.
[82] Armen Sarvazyan and Vladimir Egorov. “Mechanical imaging-a technology for 3-d
visualization and characterization of soft tissue abnormalities: A review”. In: Current
Medical Imaging 8.1 (2012), pp. 64–73.
[83] Paul E Barbone and Assad A Oberai. “A review of the mathematical and computational foundations of biomechanical imaging”. In: Computational Modeling in Biomechanics (2010), pp. 375–408.
[84] Mark Drela. “XFOIL: An Analysis and Design System for Low Reynolds Number
Airfoils”. In: Low Reynolds Number Aerodynamics. Ed. by Thomas J. Mueller. Berlin,
Heidelberg: Springer Berlin Heidelberg, 1989, pp. 1–12. isbn: 978-3-642-84010-4.
[85] H. B. Squire and A. D. Young. “The Calculation of the Profile Drag of Aerofoils”. In:
Aeronautical Research Committee, Repts. and Memoranda 1838 (1938).
96
[86] Florianr Menter. “Zonal two equation k-w turbulence models for aerodynamic flows”.
In: 23rd fluid dynamics, plasmadynamics, and lasers conference. 1993, p. 2906.
[87] Charles L. Ladson. “Effects of independent variation of Mach and Reynolds numbers on the low-speed aerodynamic characteristics of the NACA 0012 airfoil section”.
In: Washington, D.C: National Aeronautics and Space Administration, Scientific and
Technical Information Division (1988).
[88] Pin-Yu Chen, Baichuan Zhang, and Mohammad Al Hasan. “Incremental eigenpair
computation for graph Laplacian matrices: theory and applications”. In: Social Network
Analysis and Mining 8.1 (2018), p. 4. doi: https://doi.org/10.1007/s13278-017-
0481-y.
[89] James W. Demmel. “Iterative Methods for Eigenvalue Problems”. In: Applied Numerical Linear Algebra. Society for Industrial and Applied Mathematics, 1997. Chap. 7,
361–387. doi: 10.1137/1.9781611971446.
[90] Hengzhao Ma and Jianzhong Li. “A True O(n log n) Algorithm for the All-k-NearestNeighbors Problem”. In: Combinatorial Optimization and Applications. Ed. by Yingshu
Li, Mihaela Cardei, and Yan Huang. Springer International Publishing, 2019, pp. 362–
374. isbn: 978-3-030-36412-0. doi: https://doi.org/10.1007/978-3-030-36412-
0\_29.
[91] Jeremy Budd, Yves van Gennip, and Jonas Latz. “Classification and image processing
with a semi-discrete scheme for fidelity forced Allen–Cahn on graphs”. In: GAMMMitteilungen 44.1 (2021), e202100004. doi: https : / / doi . org / 10 . 1002 / gamm .
202100004. eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/gamm.
202100004. url: https://onlinelibrary.wiley.com/doi/abs/10.1002/gamm.
202100004.
97
Abstract (if available)
Abstract
This work is focused on the development and implementation of data-driven multi-fidelity models for physical systems. Often times, when facing problems such as optimization, uncertainty quantification, or characterization of a parametric physical system, a large number of simulations is required. If multiple models of different fidelity and cost are available to perform these simulations, a multi-fidelity approach can be effective to decrease the overall computational cost of the task, while maintaining the required level of accuracy. The foundational idea is to define a strategy that combines data and results from the different models to construct a multi-fidelity dataset that inherits most of the accuracy of the higher-fidelity model, but at a fraction of the cost. We present three different strategies. The first approach exploits the low-fidelity model to span the input space and generate a large set of data (or snapshots). Then, a low-rank representation of the snapshots is found by computing a suitable basis. Finally, high-fidelity data is acquired, and a lifting procedure is used to increase the accuracy of the low-rank representation. This strategy is applied to study the effect of the separation between the rotors of a two-rotor assembly on its aerodynamics performance. It is shown how this strategy can substantially improve upon the low-fidelity results. The second method that is put forth is a multi-fidelity model based on the spectral properties of the graph Laplacian. A graph is constructed using the low-fidelity data as nodes, and a graph Laplacian and its eigen-decomposition are evaluated. After the high-fidelity data acquisition, a transformation for all the low-fidelity points is defined as an expansion in terms of the graph Laplacian eigen-functions. The coefficients of this expansion are computed by minimizing a data misfit term and a regularization term, which penalizes the use of higher-order eigen-functions. Numerical experiments in both solid and fluid mechanics shown how this approach is effective to learn the low-fidelity data distribution and transform it based on a small number of high-fidelity data points. The third method is a probabilistic extension of the previous one. Indeed, the problem of determining a multi-fidelity approximation of the data is cast as a Bayesian inference problem. The transformation from low- to multi-fidelity points is defined through a probability distribution, determined by a graph Laplacian-regularized prior, and a data-misfit likelihood term. This approach is more general, as the form of the transformation is not assumed a priori. Furthermore, it enables to perform uncertainty quantification studies by analysing of the statistical properties of the posterior distribution.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Efficient inverse analysis with dynamic and stochastic reductions for large-scale models of multi-component systems
PDF
Physics-based data-driven inference
PDF
Reconstruction and estimation of shear flows using physics-based and data-driven models
PDF
A high-fidelity geometric contact method for thin fabrics using shell finite elements in explicit and implicit time scheme
PDF
Model falsification: new insights, methods and applications
PDF
Scalable exact inference in probabilistic graphical models on multi-core platforms
PDF
Efficient learning: exploring computational and data-driven techniques for efficient training of deep learning models
PDF
Model-driven situational awareness in large-scale, complex systems
PDF
Integration of multi-physics data into dynamic reservoir model with ensemble Kalman filter
PDF
Development and applications of a body-force propulsor model for high-fidelity CFD
PDF
Trustworthy spatiotemporal prediction models
PDF
Designing data-effective machine learning pipeline in application to physics and material science
PDF
Hybrid physics-based and data-driven computational approaches for multi-scale hydrologic systems under heterogeneity
PDF
Learning the semantics of structured data sources
PDF
Probabilistic data-driven predictive models for energy applications
PDF
Cyberinfrastructure management for dynamic data driven applications
PDF
Learning controllable data generation for scalable model training
PDF
Exploiting web tables and knowledge graphs for creating semantic descriptions of data sources
PDF
Impurity gas detection for spent nuclear fuel (SNF) canisters using ultrasonic sensing and deep learning
PDF
Machine learning in interacting multi-agent systems
Asset Metadata
Creator
Pinti, Orazio
(author)
Core Title
Data-driven multi-fidelity modeling for physical systems
School
Viterbi School of Engineering
Degree
Doctor of Philosophy
Degree Program
Aerospace Engineering
Degree Conferral Date
2024-05
Publication Date
01/30/2024
Defense Date
01/26/2024
Publisher
Los Angeles, California
(original),
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
computational engineering,data-driven modeling,multi-fidelity modeling,OAI-PMH Harvest
Format
theses
(aat)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Oberai, Assad (
committee chair
), Ghanem, Roger (
committee member
), Luhar, Mitul (
committee member
)
Creator Email
pinti@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-oUC113817105
Unique identifier
UC113817105
Identifier
etd-PintiOrazi-12643.pdf (filename)
Legacy Identifier
etd-PintiOrazi-12643
Document Type
Dissertation
Format
theses (aat)
Rights
Pinti, Orazio
Internet Media Type
application/pdf
Type
texts
Source
20240131-usctheses-batch-1124
(batch),
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Repository Email
cisadmin@lib.usc.edu
Tags
computational engineering
data-driven modeling
multi-fidelity modeling