University of Southern California Dissertations and Theses

Experimental and analytical studies of infrastructure systems condition assessment using different sensing modality

(USC Thesis Other)

Experimental and analytical studies of infrastructure systems condition assessment using different sensing modality

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Experimental and Analytical Studies of Infrastructure Systems
Condition Assessment Using Dierent Sensing Modality
by
Mohamed H. Abdelbarr
A Dissertation Presented to the
FACULTY OF THE GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulllment of the
Requirements of the Degree
DOCTOR OF PHILOSOPHY
(Civil Engineering)
May 2018
Copyright 2018 Mohamed H. Abdelbarr
Dedicated to my parents.
For their endless love, support and encouragement.
ii
Acknowledgments
First, I would like to thank my parents Hala and Hassan for their personal support and
patience always; my sister Sarah and her husband Abdullah, my brothers Ali and Ahmed
who have given me their unequivocal support throughout, as always.
I would like to thank my dear advisor, Prof Sami F Masri, for his invaluable guidance
throughout my PhD studies. His consistent positive impact on my academic, scientic,
and professional development has been tremendous. I am extremely grateful for his strong
support, valuable advice, and continuous encouragement throughout my studies.
I am also thankful to Dr. John Carey for his assistance and collaboration during
the experimental phase of this work. I also appreciate the advice and support of Prof
Mohammad Jahanshahi, Prof Nizar Lajnef, Dr. Patrick Brewick, and Dr. Anthony Massari.
I would like to thank my qualication and oral exam committee members Professors L.
Carter Wellford, Wei-Men Shen, Ketan Savla, and Qiming Wang. I would like to thank
the team members of the structural health monitoring research group at USC and my
ocemates, Dr Miguel Ricardo Hernandez-Garcia and Dr. Yulu Luke Chen.
Finally, I acknowledge the partial nancial support by Qatar Foundation (QF), Qatar
University (QU), Federal Highway Administration (FHWA), and the King Abdulaziz
City for Science and Technology (KACST). I also acknowledge the USC Viterbi School of
Engineering Fellowship for their generous support and the USC Sonny Astani Department
iii
of Civil and Environmental Engineering for the support through teaching and research
assistantships.
iv
Table of Contents
Dedication ii
Acknowledgments iii
List of Figures ix
List of Tables xviii
Abstract xx
1 Introduction 1
1.1 Background and motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Health Monitoring and Condition Assessment Using Inexpensive Color
and Depth (RGB-D) Fusion for 2D Displacement-Field Measurement 7
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1.1 Background and motivation . . . . . . . . . . . . . . . . . . . . . . 7
2.1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1.3 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2 Consumer-Grade RGB-D camera . . . . . . . . . . . . . . . . . . . . . . . 16
2.3 Microsoft Kinect sensor (rst version) . . . . . . . . . . . . . . . . . . . . 19
2.4 3D dynamic displacement eld measurement using Kinect sensor . . . . . 21
2.4.1 Kinect sensor data acquisition . . . . . . . . . . . . . . . . . . . . . 21
2.4.2 Calibration of the RGB and IR cameras . . . . . . . . . . . . . . . 21
2.4.3 3D scene reconstruction from depth image . . . . . . . . . . . . . . 22
2.4.4 Aligning RGB and depth images . . . . . . . . . . . . . . . . . . . 23
2.4.5 Point clouds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.4.6 Target detection and tracking . . . . . . . . . . . . . . . . . . . . . 25
2.5 Experimental design to measure dynamic displacement eld . . . . . . . . 27
2.5.1 Experiment setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.5.2 RGB-D data acquisition system . . . . . . . . . . . . . . . . . . . . 28
2.6 Data processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
v
2.6.1 Data ltering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.6.2 Data alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.7 Experimental results and data analysis . . . . . . . . . . . . . . . . . . . . 32
2.7.1 Initial test results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.7.2 Harmonic excitation . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.7.3 Random excitation . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.8 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.9 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3 3D Dynamic Displacement-Field Measurement for Structural Health
Monitoring and Condition Assessment Using Inexpensive RGB-D Based
Sensor 49
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.1.1 Background and motivation . . . . . . . . . . . . . . . . . . . . . . 49
3.1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.1.3 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.2 Kinect data extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.2.1 RGB-D sensor calibration and data registration . . . . . . . . . . . 57
3.2.2 Target detection and tracking . . . . . . . . . . . . . . . . . . . . . 58
3.3 Theory and methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.3.1 Displacement calculation . . . . . . . . . . . . . . . . . . . . . . . 59
3.3.2 Rotation calculation . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.4 Experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.4.1 Testbed structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.4.2 Consumer-Grade RGB-D camera . . . . . . . . . . . . . . . . . . . 68
3.4.3 Contact-type sensors . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.4.4 Data acquisition systems . . . . . . . . . . . . . . . . . . . . . . . 69
3.5 Data preprocessing and verication . . . . . . . . . . . . . . . . . . . . . . 71
3.5.1 Kinect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.5.2 Accelerometers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.5.3 Inertial measurement unit (IMU) . . . . . . . . . . . . . . . . . . . 72
3.5.4 Data synchronization and alignment . . . . . . . . . . . . . . . . . 77
3.6 Experimental validation for translational motion . . . . . . . . . . . . . . 77
3.6.1 Harmonic test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.6.2 Random test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
3.7 Experimental validation for rotational motion . . . . . . . . . . . . . . . . 86
3.7.1 Static test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3.7.2 Harmonic test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.7.3 Random test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
3.8 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.9 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
vi
4 Health Monitoring and Condition Assessment Using Self-Powered
Piezo-Floating-Gate (PFG) Sensors 97
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.1.1 Background and motivation . . . . . . . . . . . . . . . . . . . . . . 97
4.1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.1.3 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
4.2 Self-powered PFG sensor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
4.2.1 Design principles and working mechanism . . . . . . . . . . . . . . 100
4.2.2 Sensor output calibration . . . . . . . . . . . . . . . . . . . . . . . 104
4.3 Experimental study for health monitoring of beams . . . . . . . . . . . . . 107
4.3.1 Description of the test apparatus . . . . . . . . . . . . . . . . . . . 107
4.3.2 Finite element model for the testbed . . . . . . . . . . . . . . . . . 108
4.3.3 Eect of damage on strain eld . . . . . . . . . . . . . . . . . . . . 111
4.4 Damage detection using the self-powered PFG sensor . . . . . . . . . . . . 115
4.4.1 Overview of damage scenarios . . . . . . . . . . . . . . . . . . . . . 115
4.4.2 Damage classication . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.5 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
4.6 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5 Vibration-Based Condition Assessment and Damage Detection Local-
ization and Quantication of Downtown Los Angeles 52-Story High-
Rise Building 123
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
5.1.1 Background and motivation . . . . . . . . . . . . . . . . . . . . . . 123
5.1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
5.1.3 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
5.2 Chain-like structure system identication approach (ChainID) . . . . . . . 126
5.3 52-Story high-rise building description, instrumentation and computational
model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
5.4 Identication of data based reduced-order models . . . . . . . . . . . . . . 133
5.4.1 Sample data processing results . . . . . . . . . . . . . . . . . . . . 133
5.4.2 Decomposition approach (ChainID) implementation . . . . . . . . 134
5.5 Change detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5.5.1 Change detection using reduced-order models of structural systems 140
5.5.2 Change detection using modal parameters . . . . . . . . . . . . . . 144
5.6 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
5.7 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
6 Development and Validation of Nonlinear Data-Driven Computational
Models for Condition Assessment and Change Detection Based on Vi-
bration Signature 152
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
6.1.1 Background and motivation . . . . . . . . . . . . . . . . . . . . . . 152
vii
6.1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
6.1.3 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
6.2 Identication approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
6.3 Experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
6.3.1 Nonlinear gap element . . . . . . . . . . . . . . . . . . . . . . . . . 159
6.3.2 Instrumentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
6.4 Data processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
6.5 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
6.5.1 Identication of linear testbed structure . . . . . . . . . . . . . . . 164
6.5.2 Identication of nonlinear testbed structure . . . . . . . . . . . . . 166
6.6 Analysis and discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
6.6.1 Statistical analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
6.6.2 Change detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
6.7 Change detection using global identication . . . . . . . . . . . . . . . . . 182
6.8 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
7 Summary and Conclusions 186
Bibliography 189
viii
List of Figures
1.1 Some subset of condition assessment challenges discussed in this thesis. . 2
2.1 Camera calibration for the RGB and depth camera: (a) calibration images
for the RGB camera, (b) calibration images for the IR camera, and (c)
the 3D plot of the spatial conguration for stereo calibration. . . . . . . 22
2.2 An example of (a) color-depth alignment, and (b) a color point cloud. . 25
2.3 Extracting points of interest from (a) color image and mapping to (b)
depth image. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.4 Test apparatus: (a) the experimental setup for the Kinect dynamical
displacement measurement, (b) the aluminum plate, and (c) the Microsoft
Kinect. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.5 Kinect data post-processing: (a) raw depth data, (b) depth data Fourier
transform, (c) ltered depth, and LVDT data, and (d) aligned Kinect and
LVDT data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.6 Displacement measurements of Kinect and LVDT under dierent frequen-
cies: (a) 0.5 Hz, (b) 1.0 Hz, and (c) 1.5 Hz. . . . . . . . . . . . . . . . . 34
2.7 Displacement measurements of Kinect and LVDT under dierent displace-
ment levels: (a) 5 mm, (b) 10 mm, and (c) 20 mm. . . . . . . . . . . . . 34
2.8 Target orientation and angle with respect to Kinect: (a) target displace-
ment perpendicular to Kinect (depth-based), (b) target displacement
parallel to Kinect (pixel-based), (c) target displacement angled to Kinect
(depth-and-pixel-based). . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.9 Sample depth-based measurements (1.0 Hz, 20 mm): (a) x-direction, (b)
y-direction, and (c) z-direction. . . . . . . . . . . . . . . . . . . . . . . 37
2.10 Sample pixel-based measurements (1.0 Hz, 20 mm): (a) x-direction, (b)
y-direction, and (c) z-direction. . . . . . . . . . . . . . . . . . . . . . . . 37
2.11 Sample depth-and-pixel-based measurements (1.0 Hz, 20 mm): (a) x-
direction, (b) y-direction, and (c) z-direction. . . . . . . . . . . . . . . . 38
2.12 Depth-based measurements errors: (a) normalized errors, and (b) peak
errors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.13 Pixel-based measurements errors: (a) normalized errors, and (b) peak
errors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
ix
2.14 Depth-and-pixel-based measurements errors: (a) normalized errors, and
(b) peak errors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.15 Depth-based measurements for three RMS levels: (a) RMS = 1.94 mm,
(b) RMS = 8.44 mm, and (c) RMS = 13.81 mm. . . . . . . . . . . . . . 42
2.16 Pixel-based measurements for three RMS levels: (a) RMS = 1.94 mm, (b)
RMS = 8.44 mm, and (c) RMS = 13.81 mm. . . . . . . . . . . . . . . . 42
2.17 Sketch showing PDF peaks mean and standard deviation errors. . . . . . 43
2.18 PDF of the estimated Kinect displacement measurements based on depth-
based data: (a) RMS = 1.94 mm, (b) RMS = 8.44 mm, and (c) RMS =
13.81 mm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.19 PDF of the estimated Kinect displacement measurements based on pixel-
based data: (a) RMS = 1.94 mm, (b) RMS = 8.44 mm, and (c) RMS =
13.81 mm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.20 PDF of the estimated Kinect peak displacement measurements based on
depth-based data: ( (a) RMS = 1.94 mm, (b) RMS = 8.44 mm, and (c)
RMS = 13.81 mm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.21 PDF of the estimated Kinect peak displacement measurements based on
pixel-based data: (a) RMS = 1.94 mm, (b) RMS = 8.44 mm, and (c)
RMS = 13.81 mm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.1 Extracting points of interest from ArUco markers: (a) in a color image
and mapping to (b) a depth image where the axis of motion parallel is to
X-direction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.2 Overview of measuring dynamic displacements using the Kinect sensor.
The process of displacement calculation consists of stereo calibration for
color and depth images, target (ArUco markers) detection, tracking and
displacement calculation using Equation (3.1). . . . . . . . . . . . . . . . 61
3.3 Computed angle using Kinect data (a) Pixel-based data; refer to the
computed rotation angle based on Kinect data when the axis of rotation
is parallel to the Kinect's Z -axis (i.e., the depth axis) , and (b) depth-and-
pixel-based data; refer to the computed rotation angle based on Kinect's
data when the axis of rotation is parallel to the Kinect Y -axis. . . . . . 62
3.4 Sensors locations: (a) accelerometers Acc 1, Acc 2, Acc 5, Acc 6, Acc
7 are used to measure acceleration in the X-direction; accelerometers
Acc 3, Acc 4 are used to measure acceleration in the Y -direction; and an
IMU is used to measure rotation (torsion) in module 3 around the IMU's
y-axis that coincide with the structure's axis of symmetry, and (b) ArUco
markers deployed on test-bed structure and the Microsoft Kinects used;
Kinect
side
, Kinect
front
, Kinect
top
(note that the depth axis for Kinect
side
is perpendicular to the shaker direction of motion, the depth axis for
Kinect
front
is parallel to the shaker direction motion, and the depth axis
for Kinect
top
is perpendicular to the plane of motion [i.e., XY plane]. . . 64
x
3.5 Overview of the experimental setups: (a) the experimental setup used for
the evaluation of harmonic translation, harmonic rotation, static rotation
and random translation, and (b) the experimental setup used for the
evaluation of random rotation, including slider-crank assembly (uniaxial
shaker connected to round rotating table by rigid arm to convert linear
motion to circular motion). . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.6 Testbed frequency response based on acceleration data acquired from
accelerometer sensors (a) Acc 1 located at module 3, and (b) Acc 3
located at module 3 (note that the amplitude scales are signicantly
dierent). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.7 Mode shapes from FEA model: (a) the rst mode of response at 1.3 Hz,
(b) the second mode of response at 4.3 Hz, and (c) the third mode of
response at 7.3 Hz. Note that the Abaqus FEA software was used to
generate the model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.8 Test apparatus and components: (a) the Kinect
front
xture, (b) the
Kinect
side
xture, (c) the Kinect
top
xture, (d) the 4 accelerometers and
IMU deployed on module 3, (e) the 2 Kg mass attached to module 2, and
(f) the Microsoft Kinect sensor. Note that the size of markers (100 mm
100 mm) are relatively large for calibration process, while smaller markers
can be used. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.9 Sample of (a) recorded data from an accelerometer (Acc 7), and (b) corre-
sponding computed displacement time history and LVDT measurements. 72
3.10 Sketch for (a) slider-crank geometry, and (b) IMU axis. . . . . . . . . . . 74
3.11 Sample aligned IMU-Gyroscope and ground truth (GT) data under har-
monic excitation (2.0 Hz, 16.0
o
): (a) x-direction, and (b) y-direction. The
GT is the computed angle using the slider-crank mechanism. . . . . . . 74
3.12 Sample aligned IMU-Gyroscope and GT data under random excitation:
(a) x-direction (RMS of the GT signal = 2.35
o
), and (b) y-direction (RMS
of the GT signal = 2.00
o
). The GT is the computed angle using the
slider-crank mechanism. . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.13 PDFs of the IMU-Gyroscope measurements for the three directions: (a)
IMU-x-direction (RMS of the GT signal = 2.35
o
), (b) IMU-y-direction
(RMS of the GT signal = 2.00
o
), and (c) IMU-z-direction (RMS of the
GT signal = 2.83
o
). The GT is the computed angle using the slider-crank
mechanism. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.14 Kinect data post-processing: (a) raw depth data, and (b) aligned Kinect
and GT data after applying the Kinect post-processing approach that
consists of Kinect data ltering, smoothing and alignment. . . . . . . . 78
3.15 Sample measurements for the structure
oors (1.3 Hz): (a) pixel-based
measurements (Kinect
side
) based on marker MS1, and (b) depth-based
measurements (Kinect
front
) based on marker MF1. Note that x, y, z
directions refer to the Kinects' axes. . . . . . . . . . . . . . . . . . . . . 79
xi
3.16 Sample measurements for the structure
oors (1.3 Hz): (a) pixel-based
measurements (Kinect
side
) based on markers MS1, MS2, MS3, MS4 at-
tached at 4
th

oor, 3
rd

oor, 2
nd

oor, and 1
st

oor, respectively; and
(b) depth-based measurements (Kinect
front
) based on markers MF1, MF2,
MF3, MF4 attached at 4
th

oor, 3
rd

oor, 2
nd

oor, and 1
st

oor, respec-
tively. Note that the response for the 3
rd
and 4
th

oors are almost similar
due to the mass attached at the 3
rd

oor. . . . . . . . . . . . . . . . . . 79
3.17 Superimposed plots for the Kinects data versus GT measurements (1.3 Hz):
(a) rst
oor, and (b) fourth
oor. The GT is the computed displacement
from acceleration records. . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.18 Depth- and pixel-based measurements errors (1.3 Hz). . . . . . . . . . . 81
3.19 Depth- and pixel-based measurements for three RMS levels: (a) pixel-
based 4
th

oor (marker MS1; RMS = 16.42 mm), (b) depth-based 4
th

oor (marker MF1; RMS = 16.42 mm), (c) pixel-based 2
nd

oor (marker
MS3; RMS = 10.01 mm), (d) depth-based 2
nd

oor (marker MF3; RMS
= 10.01 mm), (e) pixel-based 1
st

oor (marker MS4; RMS = 6.81 mm),
and (f) depth-based 1
st

oor (marker MF1; RMS = 6.81 mm). The GT is
the computed displacement from acceleration records. . . . . . . . . . . 83
3.20 PDFs of the estimated Kinect displacements based on pixel-based mea-
surement versus GT: (a) RMS = 6.81 mm (marker MS4), (b) RMS =
10.01 mm (marker MS3), and (c) RMS = 16.42 mm (marker MS1). The
GT is the computed displacement from acceleration records. . . . . . . . 85
3.21 PDFs of the estimated Kinect displacements based on depth-based mea-
surement versus GT: (a) RMS = 6.81 mm (marker MF4), (b) RMS =
10.01 mm (marker MF3), and (c) RMS = 16.42 mm (marker MF1). The
GT is the computed displacement from acceleration records. . . . . . . . 85
3.22 Normalized estimation errors: (a) depth-and-pixel-based rotational mea-
surements (based on the data acquired from the front markers set (MF1,
MF2, MF3, MF4) and the side markers set (MS1, MS2, MS3, MS4),
and (b) pixel-based rotational measurements (marker MT1), depth-and-
pixel-based rotational measurements (mean of the side and front markers
sets). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.23 Testbed mode shapes: (a) the initial condition, (b) the rst mode of
response at 1.3 Hz, and (c) the second mode of response at 4.3 Hz. It
can be seen that the rst mode of response is pure translational and the
second mode is a pure torsional. . . . . . . . . . . . . . . . . . . . . . . . 89
3.24 Tracking of top marker (MT1) during torsional test: (a) the initial position,
(b) the maximum torsion. . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.25 Markers tracking comparison for translational and rotational motion: (a)
transnational motion (marker MS1, 1.3 Hz test), and (b) rotational motion
(marker MT1, 4.3 Hz test). . . . . . . . . . . . . . . . . . . . . . . . . . 90
xii
3.26 Sample torsional measurements for the structure fourth
oor (4.3 Hz):
(a) marker MT1 (Kinect
top
), and (b) marker MT1 (Kinect
top
), MS1
(Kinect
side
), MF1 (Kinect
front
); GT is the computed angle of rotation
from IMU data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
3.27 Sample torsional measurements for the structure's fourth
oor based on
marker MF1 (Kinect
front
) for two RMS levels: (a) RMS = 1.85
o
, and (b)
RMS = 5.46
o
; GT is the computed angle of rotation from IMU data. . . 92
3.28 Sample torsional measurements for the structure's fourth
oor based on
marker MT1 (Kinect
top
) for two RMS levels: (a) RMS = 1.85
o
, and (b)
RMS = 5.46
o
; GT is the computed angle of rotation from IMU data. . . 93
3.29 PDFs of the estimated Kinect torsional measurements for the structure's
fourth
oor based on marker MF1 (Kinect
front
) for two RMS levels: (a)
RMS = 1.85
o
, and (b) RMS = 5.46
o
; GT is the computed angle of rotation
from IMU data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.30 PDFs of the estimated Kinect torsional measurements for the structure's
fourth
oor based on marker MT1 (Kinect
top
) for two RMS levels: (a)
RMS = 1.85
o
, and (b) RMS = 5.46
o
; GT is the computed angle of rotation
from IMU data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
4.1 PFG sensor basic concepts. . . . . . . . . . . . . . . . . . . . . . . . . . 101
4.2 Prototype and working principles of the self-powered PFG sensor. . . . . 103
4.3 Experimental setup deployed for sensor output calibration. . . . . . . . . 104
4.4 Quantication of uncertainty in the PFG sensor data. . . . . . . . . . . 105
4.5 Calibration of the PFG sensor gates injection thresholds. . . . . . . . . . 106
4.6 Experimental setup overview. . . . . . . . . . . . . . . . . . . . . . . . . 109
4.7 Sketch for the beam dimensions, sensors locations and damage locations. 109
4.8 Finite element model for the cantilever beam. . . . . . . . . . . . . . . . 110
4.9 Deformed shape of the cantilever beam. . . . . . . . . . . . . . . . . . . 111
4.10 Sample strain output (E11) for cases 1, 2, 6, and 7. . . . . . . . . . . . . 113
4.11 Strain eld (E11) for case#2. . . . . . . . . . . . . . . . . . . . . . . . . 113
4.12 Strain eld (E11) for case#4. . . . . . . . . . . . . . . . . . . . . . . . . 114
4.13 PDF of strain output (E11) for cases 1, 2, 6, and 7. . . . . . . . . . . . . 115
4.14 PDF of strain output (E11) at location 1 for cases 2, 3, 4, and 5. . . . . 115
4.15 Eect of notch (hole) size on the variation of strain eld distribution. . . 116
4.16 Experimental setup for damaged beam (cases 2 and 4). . . . . . . . . . . 116
4.17 The experimental strain and PZT output for cases 2 and 4 at location 1. 118
4.18 PDF of the experimental strain and PZT output for cases 2 and 4 at
location 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
4.19 Comparison between strain, PZT, and simulation output variation between
case 2 and 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.20 PFG sensor output after 1000 cycles (case#2 and case#4) . . . . . . . 120
4.21 PFG sensor voltage variation after 1000 cycles (case#2 and case#4). . . 121
xiii
4.22 Bridge girder under dierential settlement. . . . . . . . . . . . . . . . . . 122
5.1 Modeling of the 52-story building used in this study (a) reduced order
representation (mathematical model), (b) 2D view for the ETABS model
in x-direction, and (c) 3D view for the ETABS model. . . . . . . . . . . 126
5.2 Finite element 3D-model (ETABS 3D-model) for the 52-story building
with axis orientation, nomenclature and location of sensors deployed in
the building [92]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
5.3 Caltech ETABS property modier automater used to introduce dam-
age/change and generate dierent damage/change states [120]. . . . . . 132
5.4 Sample of relative displacement z
(i)
and relative velocity _ z
(i)
computed
between 10
th
and 9
th

oors are illustrated in the rst two rows. The third
row shows the measured restoring force time-history for element G
(10)
from reference condition (state#1) (a) x-direction, and (b) y-direction.
Note that, dierent amplitude scales are used in LHS and RHS columns of
plots for enhanced viewing. Since the excitation was applied in x-direction
only, the relative displacement computed z
(10)
between 10
th

oor and 9
th

oor in y-direction is relatively small (less than 0.15 in) or almost less
than 5% of the relative displacement computed in x-direction. . . . . . . 136
5.5 The identied mass normalized stiness like coecient (a
(i)
10
) for all
oors
starting from the reference conguration (state#1) in x- and y-directions.
Note that after the 5
th
, the identied mass normalized stiness like
coecient are almost equal for the same
oor in x- and y-directions due
to symmetry in the structure. In Addition, an abrupt change can be seen
at the 50
th

oor in in both directions due to the large change in mass
after the 50
th

oor. For enhanced viewing the values of the identied mass
normalized stiness like coecient for the basement and 1
st

oors are not
plotted. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
5.6 Comparison of measured and estimated restoring forces G
(i)
for the 10
th

oor in the reference conguration (state#1) in (a) x-direction, and (b)
y-direction. The blue solid lines correspond to the measured restoring force
time-history; however, the red dotted lines correspond to the reconstructed
time-history for the restoring force computed using the reduced-order
model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
5.7 Sample of phase plot of restoring forceG
(i)
versus relative displacementz
(i)
for the 10
th
floor in the reference conguration (state#1) and correspond-
ing estimated restoring force surface for the 10
th

oor in (a) x-direction,
and (b) y-direction. In the rst row the blue solid lines correspond to the
phase plot for the measured restoring force and the red dotted lines corre-
spond to the reconstructed restoring force. In the second row, the actual
restoring force measurements were plotted as point cloud. In addition,
the restoring force surface is almost planar, as expected for linear elements. 139
xiv
5.8 Comparison of the computed root mean square (RMS) for relative dis-
placement z
(i)
and mass normalized restoring force G
(i)
for all
oors in x-
and y-directions for reference and damage conditions. Note the sudden
change in relative displacement at damaged locations (5
th
- 7
th
)
oors for
damaged state#2 in x- and y-directions, and 5
th
and 10
th

oors in x- and
y-directions consecutively for damaged state#3). . . . . . . . . . . . . . 141
5.9 Identied change in the mass normalized restoring force surfaces for
damage state#1 (a) for elements G
(5)
,G
(7)
, andG
(10)
in x-direction after
introducing damage at 5
th
to 7
th

oors in x-direction, and (b) for elements
G
(5)
,G
(7)
, andG
(10)
in y-direction after introducing damage at 5
th
to 7
th

oors in y-direction simultaneously. Note that, dierent amplitude scales
are used in LHS and RHS columns of plots for enhanced viewing. . . . . 142
5.10 Identied change in the mass normalized restoring force surfaces for
damage state#2 (a) for elements G
(5)
,G
(8)
, andG
(10)
in x-direction after
introducing damage at 5
th

oor in x-direction, and (b) for elements G
(5)
,
G
(8)
, and G
(10)
in y-direction after introducing damage at 10
th

oor in
y-direction simultaneously. Note that, dierent amplitude scales are used
in LHS and RHS columns of plots for enhanced viewing. . . . . . . . . . 143
5.11 Magnitude of relative changes in the mass normalized stiness-like coe-
cients (a
(i)
10
) of the reduced-order models in the x- and y-directions of the
test structure computed using ChainID approach for (a) structural state#2
and (b) structural state#3. The horizontal dotted line corresponds to
the thresholdj
(i)
10
j = 5% used in this study to determine if signicant
changes in the stiness-likea
(i)
10
coecients have occurred with respect the
reference condition (state#1) parameters. . . . . . . . . . . . . . . . . . 145
5.12 Mode shapes of the rst ve lateral modes in (a) x-direction, and (b)
and y-direction extracted from the ETABS 3D-model in the reference
condition (state#1). Note that the modes shapes are shown in 2D view
(XZ and YZ planes). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
5.13 Magnitude of relative change in the natural frequencies !
i
obtained using
(a) NExT/ERA approach and (b) ChainID approach estimated for the
building structure in the x- and y-directions. The horizontal dotted line
corresponds to the thresholdj!j = 5% used in this study to determine
if signicant changes in the modal parameters have occurred with respect
the reference condition parameters. . . . . . . . . . . . . . . . . . . . . 149
6.1 Model of non-linear multidegree-of-freedom chain-like system. . . . . . . 155
6.2 Photograph of testbed components details. . . . . . . . . . . . . . . . . . 161
6.3 Location of the accelerometers deployed on testbed structure. . . . . . . 163
xv
6.4 Sample of (a) acceleration, velocity, and displacement time-histories for
module 2, and (b) relative displacement and relative velocity computed
between modules 2 and 3 are illustrated in the rst two rows. The third
row shows a comparison between the measured restoring force time-history
for element G
(2)
and the reconstructed time-history using the identied
restoring force coecients from the linear system in the reference structural
conguration test#1 (the 2 curves almost match). . . . . . . . . . . . . 164
6.5 Identied restoring force surfaces (a) for element G
(2)
for the reference
linear condition (test#1) and after removal of one element from module 5
(test#4), and (b) for elementG
(5)
for the reference condition (test#1) and
after removal of one element from module 5 (test#4). Identied restoring
force surfaces for test#1 are transparent. Note that, for enhanced viewing,
dierent amplitude scales are used in the LHS and RHS columns of plots
and the surface plots for LHS are almost the same. . . . . . . . . . . . . 167
6.6 Phase plots of (a) restoring force G
(5)
versus relative displacement z
5
and restoring force G
(5)
versus relative velocity _ z
5
for module 5 in the
reference conguration (test#1), and (b) restoring forceG
(5)
versus relative
displacement z
5
and restoring force G
(5)
versus relative velocity _ z
5
after
removal of one element from module 5 (test#4). The blue solid lines
correspond to the phase plot for the measured restoring force, however
the red dotted lines correspond to the reconstructed time history for the
restoring force. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
6.7 Sample of (a) acceleration, velocity, and displacement time-histories for
module 2 (test#7), and (b) relative displacement and relative velocity
computed between modules 2 and 3 (test#7) are illustrated in the rst two
rows. The third row shows a comparison between the measured restoring
force time-history for element G
(2)
and the reconstructed time-history
using the identied restoring force coecients from the nonlinear system
in the test#7 (the 2 curves almost match). . . . . . . . . . . . . . . . . . 169
6.8 Identied restoring force surfaces (a) for element G
(2)
for module 2 in the
reference linear condition (test#1), and (b) for element G
(2)
in module 2
after the introduction of nonlinear gap element in module 2 (nonlinear
conguration [test#7]). The black dots correspond to the measured
restoring force. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
6.9 Phase plots of (a) restoring force G
(2)
versus relative displacement z
2
and restoring force G
(2)
versus relative velocity _ z
2
for module 2 in the
reference conguration (test#1), and (b) restoring forceG
(2)
versus relative
displacementz
2
and restoring forceG
(2)
versus relative velocity _ z
2
after the
introduction of nonlinear gap element in module 2 (nonlinear conguration
[test#7]). The blue solid lines correspond to the phase plot for the
measured restoring force, however the red dotted lines correspond to the
reconstructed time history for the restoring force. . . . . . . . . . . . . . 172
xvi
6.10 Comparison of probability density functions of the identied mass-normalized
restoring force coecients a
(5)
10
, a
(5)
01
, a
(5)
20
, a
(5)
30
from the reduced order rep-
resentation of structural module 5 for the testbed structure in the linear
dynamic response congurations (test#1, test#3, test#4, test#5). The
solid blue lines correspond to the pdf of the coecients in the baseline
condition (test#1). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
6.11 Comparison of probability density functions of the identied mass-normalized
restoring force coecients a
(2)
10
, a
(2)
01
, a
(2)
20
, a
(2)
30
from the reduced order rep-
resentation of structural module 2 for the testbed structure in the linear
(test#1) and nonlinear (test#7, test#8) dynamic response congurations.
The solid blue lines correspond to the pdf of the coecients in the baseline
condition (test#1). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
6.12 Comparison of probability density functions of the identied mass-normalized
restoring force coecients a
(2)
10
, a
(2)
01
, a
(2)
20
, a
(2)
30
from the reduced order rep-
resentation of structural module 2 for the testbed structure in the linear
(test#1) and combined linear/nonlinear (test#6) dynamic response con-
gurations. The solid blue lines correspond to the pdf of the coecients
in the baseline condition (test#1). . . . . . . . . . . . . . . . . . . . . . 175
6.13 Comparison of probability density functions of the identied mass-normalized
restoring force coecients a
(5)
10
, a
(5)
01
, a
(5)
20
, a
(5)
30
from the reduced order rep-
resentation of structural module 5 for the testbed structure in the linear
(test#1) and combined linear/nonlinear (test#6) dynamic response con-
gurations. The solid blue lines correspond to the pdf of the coecients
in the baseline condition (test#1). . . . . . . . . . . . . . . . . . . . . . 176
xvii
List of Tables
2.1 Comparison of the technical specications for some inexpensive RGB-D
sensors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2 Time history for world coordinates of target points. . . . . . . . . . . . . . 26
2.3 Comparison of normalized error, mean error, and standard deviation error
under depth-based and pixel-based measurements. . . . . . . . . . . . . . 46
3.1 Storage arrangement for time history for world coordinates of target points
of ArUco marker. Note that for each ArUco marker, the coordinates for its
four corner points are stored. . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.2 Comparison of natural frequencies from FEA model and experimental
measurements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.3 Comparison of normalized error, mean error, and standard deviation error
under harmonic and random measurements of angle of rotation obtained
from IMU. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.4 Comparison of normalized error, mean error, and standard deviation error
under depth-based and pixel-based measurements. . . . . . . . . . . . . . 85
3.5 Comparison of normalized error, mean error, and standard deviation error
under pixel-based and depth-pixel-based torsion measurements . . . . . . 95
4.1 Properties of PZT-5A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4.2 Material properties used in the nite element model. . . . . . . . . . . . . 110
4.3 Summary of simulated damage cases. . . . . . . . . . . . . . . . . . . . . . 112
5.1 Summary of structural state conditions (damage/change scenarios). . . . . 133
5.2 Identied mass normalized restoring force coecients a
(i)
qr
for the 10
th

oor
in the reference conguration (state#1) in x- and y-directions. The bold
numbers are mass normalized stiness like coecient (a
(10)
10
). Note that the
mass normalized stiness like coecient has the signicant contribution in
the non-parametric representation of the restoring force in both directions
and the higher order terms have negligible values compared to the mass
normalized stiness like coecient. . . . . . . . . . . . . . . . . . . . . . . 137
xviii
5.3 Summary of natural frequencies and damping ratios for the rst ve lateral
modes in the x- and y-directions of the 52-story building structure identied
from the reduced-order model developed using ChianID and NExT/ERA
approaches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
6.1 Summary of structural tests. . . . . . . . . . . . . . . . . . . . . . . . . . 162
6.2 Identied Chebyshev restoring force coecients C
(i)
qr
for module 2 from the
linear system in the reference structural conguration (test#1). . . . . . . 166
6.3 Identied Chebyshev restoring force coecients C
(i)
qr
for module 2 after the
introduction of nonlinear gap element in module 2 (nonlinear conguration
[test#7]). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
6.4 Summary of mean and coecient of variation c
v
of the identied mass
normalized restoring force coecients a
10
, a
01
for modules 2, 4, and 5 for
the testbed structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
6.5 Summary of mean and coecient of variation c
v
of the identied mass
normalized restoring force coecients a
20
, a
30
for modules 2, 4, and 5 for
the testbed structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
6.6 Summary of relative mean change (=
r
) and signal-to-noise ratio (=
r
)
in the identied mass normalized restoring force coecients a
10
, a
01
for
modules 2, 4, and 5 for the testbed structure. . . . . . . . . . . . . . . . . 178
6.7 Summary of relative mean change (=
r
) and signal-to-noise ratio (=
r
)
in the identied mass normalized restoring force coecient a
20
for modules
2, 4, and 5 for the testbed structure. . . . . . . . . . . . . . . . . . . . . . 179
6.8 Comparison of numerical and experimental modal parameters. . . . . . . 183
6.9 Global system identication and relative changes in experimentally identied
natural frequencies using ChainID. . . . . . . . . . . . . . . . . . . . . . . 183
xix
Abstract
The recent developments in sensor-based technologies and computational tools provide a
potential opportunity to overcome challenges that are widely encountered in the eld of
condition assessment and health monitoring of structural systems. The main challenges can
be categorized based on sensors types, measurements types and data analysis methodologies.
The rst part of this dissertation focuses on the calibration and application of newly
developed class of sensors capable of resolving some problems related to sensor and
measurement problems.
The quantication of evolving 2D/3D displacement elds associated with a continuous
structural system is an important challenging task in health monitoring. The inexpensive
RGB-D cameras are very promising technology to quantify 3D displacement elds as
they are non-contact displacement sensors, with full-eld measurements, and 3D data
acquisition. Microsoft Kinect is a representative of this class of sensors but originally
designed for gaming purposes. This study investigated the accuracy and performance
of the Microsoft Kinect sensor as a representative RGB-D sensor in measuring dynamic
multi-component deformations by performing experimental and analytical studies. The
calibration process was divided into two phases: (1) sensor-calibration and (2) system
application. In the rst phase, the laboratory experiments were performed to determine the
performance envelope for dynamic measurements such as amplitude and frequency bounds,
linearity of measurements, distortion, eect of noise. In the second phase, the Kinect was
deployed to acquire dynamic displacement eld measurements, including 3D transnational
motion with simultaneous rotational/torsional motion components from
exible structure.
The calibration results showed that if the observed displacement eld generated discrete
xx
(pixel) sensor measurements with sucient resolution (observed displacements more than
10 mm) beyond the sensor noise
oor, then the subject sensors could provide reasonable
accuracy for transnational motion (about 5%) when the frequency range of the evolving
eld is within about 10 Hz. However, the expected error for torsional measurements was
around 6% for static motion and 10% for dynamic rotation for measurements greater than
5

.
Another major challenge for long-term monitoring and condition assessment is eciently
storing the acquired data and providing the health monitoring system including sensors
and data acquisition with external power requirements. A newly developed self-powered
wireless sensor uses piezoelectric transducers for the self-powering of wireless sensors
by harvesting energy from the mechanical loading experienced by the structure. An
experimental study was performed on a cantilever aluminum beam subjected to bending
to calibrate the proposed technology the output of the piezo-
oating-gate sensor in a
fatigue damage environment. Dierent piezoelectric transducers were mounted on the
beam for both powering the sensor and monitoring the damage progression. Furthermore,
conventional strain gages were mounted in the vicinity of the piezoelectric transducers
to calibrate their output. The change of charge on the
oating-gates of the sensor due
to electron injection were considered as damage indicator parameters. The experimental
results show that the performance of the proposed sensor is satisfactory for detecting
damage progression in beams.
The second part of this dissertation focused on implementing promising and robust data-
driven methodologies to build high-delity, reduced-order models for dierent structural
systems. The developed reduced-order models were used for condition assessment and
xxi
modeling of critical components. Dierent data sets were obtained from a high-delity 3D
nite element model developed based on experiential data acquired from a 52-story high-
rise building and re-congurable test structure designed, built, and tested at University
of Southern California (USC) were employed to evaluate the robustness of the developed
data-based reduced-order models for detecting, locating, quantifying, and classifying
structural changes in probabilistic way.
xxii
Chapter 1
Introduction
1.1 Background and motivation
T
HE American Society of Civil Engineers' (ASCE) 2017 report card for America's
infrastructure reveals that the current national infrastructure grade is 'D', which is
close to failing [11]. According to the report, 2.0 trillion dollars investment are required to
uplift the nation's infrastructure and earn a grade of 'B'. The failure to provide these invest-
ments to close this infrastructure investment gap will lead to serious economic consequences
such as 3.9 trillion dollars in losses to the Unites States (US) gross domestic product
(GDP) by 2025 [11]. Therefore, there is an urgent need to develop eective approaches and
technologies for the condition assessment and evaluation of these infrastructure systems.
However, there are still some major problems and technical issues that need investigation
and resolution to enhance the structure health monitoring (SHM) methodologies and apply
the SHM eld to full-scale structures, under realistic eld conditions.
With the growing opportunities in the development of new generation of sensor-based
technologies such vision-based sensors, inexpensive sensors, energy harvesting sensors, etc.,
1
coupled with powerful computational tools and algorithms which are being developed and
enhanced in the areas of articial intelligence, machine learning, there is a tremendous
opportunity for capitalizing on all these dierent components to develop useful applications
to solve these problems. Furthermore, the availability of large amounts of data (Big Data)
provides an opportunity for more accurate and extensive analysis to extract meaningful
and feasible features for condition assessment (Data Analytics). With that in mind, this
thesis focuses on some subsets of challenges and situations summarized in Figure 1.1, that
are widely encountered in the condition assessment of civil and mechanical infrastructure
systems related to sensor types, measurement types, and data analysis methodologies.
Figure 1.1: Some subset of condition assessment challenges discussed in this thesis.
Quantitative and accurate measurements concerning the time history of a multi-component
deformation eld of a distributed system undergoing dynamic response is an important and
2
challenging problem in the broad eld of condition assessment. There are only very limited,
and relatively quite expensive, methodologies for obtaining multi-component deformations
of a dynamically deformation eld. The development of inexpensive RGB-D cameras can
oer the potential of conveniently tracking evolving multi-component deformation elds in
monitored structures, with an accuracy that compares favorably to similar measurements
achieved by much more elaborate and sophisticated sensors which are either orders-of-
magnitude more expensive, and/or suer from fundamental limitations regarding their
theory of operation and ability to track the motion of discrete points on the monitored
structure. However, most inexpensive RGB-D cameras are originally not designed for
scientic purposes. Therefore, it is important to evaluate their feasibility and performance
before their deployment in acquiring dynamic measurements of the displacement eld and
condition assessment of target structures.
One of the most serious challenges that have hampered the practical application of the
eld of SHM for damage detection and condition assessment in extended structures, is the
infeasibility of using a sucient number of sensors (such as strain gages, accelerometers,
etc.) to provide a high enough spatial resolution, so as to capture small changes that
may be precursors to serious structural damage. Keep in mind that the collection of
measurements from the target structures needs not only the sensors themselves, but
also an instrumentation network and enough power supply to collect and transmit the
measurements to a computer server for subsequent analysis. Furthermore, Continuous
acquisition of data (i.e., continuous time history) versus situations where data statistics are
only available are major challenges that arise in choosing the most feasible approach for
acquiring data, especially for long term-monitoring. This major impediment to practical
3
application of SHM to complex systems has been partially resolved by a recently developed
Michigan State University (MSU) self-powered strain sensor [145].
Furthermore, after collecting data, there is a need to investigate and resolve some high
priority research challenges that include the detection, localization, quantication, and
classication of structural changes in monitored structures. Specially, when the structure is
subjected to non-linear phenomena as well as signicant uncertainties. Therefore, one of the
main motivations for the proposed work is the development of reduced-order mathematical
models for data processing and monitoring the health of representative structural systems
encountered in civil structures subjected to dynamic loads. Consequently, in order to
develop robust and reliable models capable of detecting, locating, quantifying changes
for structures incorporating complex nonlinear phenomena, experimental and analytical
investigations are needed to study and resolve some of the problems that hinder the
development of such data analysis approaches.
1.2 Scope
With the previous section in mind, Chapter 2 is focused on assessing the accuracy and
performance characteristics of a newly-developed class of inexpensive non-contact vision-
based sensors (RGB-D cameras) to monitor 1-D and 2-D displacements of rigid structures
under dynamic loads. Kinect v1 is a representative of this class of sensors that are designed
for video games, where the requirements for gaming is quite dierent than using it as an
accurate scientic sensor. This study presents the results of a comprehensive experimental
investigation to provide precise quantitative measurements that evaluate some key metrics
4
used in the structural dynamics eld, such as noise eects, amplitude bounds, amplitude
accuracy relative to frequencies, direction of motion of target with respect to sensor, etc. It
should be noted that for typical civil infrastructure, the low-frequency modes can capture
most of the system's response, hence this study concentrates on dynamic testing of lower
frequency ranges (currently, 0 to 2 Hz) below the Nyquist frequency (15 Hz) of the Kinect
sensor.
Chapter 3, deals with using Kinect v1 equipped with an RGB camera and an active depth
sensor to monitor evolving multi-component displacement elds of
exible structures under
dynamic loads. Chapter 2 focused on the Kinect calibration and performance evaluation
for 2-D dynamic transnational motion. The goal of this part of the study is to complete
the evaluation of the Kinect v1 performance envelope for dynamical displacement eld
measurements, including 3D transnational motion with simultaneous rotational/torsional
motion components. Performance characteristics that are evaluated include linearity of
measurements, distortion, noise eects, displacement and amplitude bounds, displacement
accuracy relative to motion frequencies, displacement accuracy relative to direction of
motion of target with respect to sensor, in
uence of lighting conditions, and the eects of
distance between sensor and target.
In Chapter 4, the performance of the self-powered piezoelectric
oating-gates (PFG) sensors
for the detection of damage progression in metallic beams under bending is investigated
for the purposes of long-term condition assessment. The objective of this study was to
validate the proposed sensing technology through small-scale laboratory testing as a rst
step before applying it on full-scale structures, as well as evaluating data summarization
concepts that the sensor used to extract feasible features that can be correlated to damage.
5
Structural health monitoring of large high-rise buildings is particularly important because
of their important role in civil infrastructure. In Chapter 5, data from a high-delity
3D nite element model for a 52-story high-rise oce building located in downtown
Los Angeles developped based on data acquired from state-of-the-art strong-motion
accelerometers located at each
oor and operated by CSN, was used to develop data-driven
computational models for response prediction and change detection. Several simulated
damage congurations scenarios in the building lateral load resisting system were studied
based on the simulated data from white-noise base excitation dynamic tests performed on
a 3D-model.
Chapter 6 describes a re-congurable test apparatus that was designed and built for
investigating complex dynamic systems with nonlinear structural components. Physical
experiments were performed to collect a statistically signicant ensemble of measurements
that were then used to investigate the development of reduced-order, low-complexity
mathematical models for detecting and locating, in a probabilistic framework involving
hypothesis testing, linear and nonlinear structural changes at various locations within the
testbed structure. The general methodology used in this study provides a robust nonlinear
model that is reliable for computational studies, as well a robust tool for capturing the
correct physics of the system.
The conclusions of the presented studies are summarized in Chapter 7.
6
Chapter 2
Health Monitoring and Condition Assessment
Using Inexpensive Color and Depth (RGB-D)
Fusion for 2D Displacement-Field Measurement
2.1 Introduction
2.1.1 Background and motivation
T
HE Quantitative measurements concerning the time history for the multi-component
deformation eld of a distributed system undergoing dynamic response is an
important and challenging problem in the broad eld of structural dynamics. For example,
ultra-lightweight,
exible, gossamer space structures are composed of ultra-thin membranes
and in
atable tubes to be packed tightly for launch, and then expand to large-scale (tens
or hundreds of meters) structures in space [77]. It is important to monitor full-eld
deformation behaviors for multiple components of gossamer space structure during testing,
deployment, and in-service conditions; however, attaching sensors on thin-lm membranes
7
can add mass and stiness for the structures, increase power consumption, and raise cost
[101]. The other example is measuring dynamical behaviors of large structures such as high-
rise buildings, long-span bridges, or wind power turbines for structural health monitoring
(SHM) purpose. Although accelerometers are very common sensors for structural health
monitoring applications, in order to determine displacement, it requires to perform double
integration of acceleration data, which needs sucient knowledge and careful judgment to
select suitable digital lters for noise and DC bias removal to obtain accurate estimation
[153]. Moreover, it is impractical to install large number of accelerometers on a structure
to measure full-eld deformation. As technology of digital imaging advances, image sensors
gradually have higher resolution, fast sampling rate, more features (2D, 3D, multispectral,
etc.), and cost less expensive. To monitor structural dynamical behaviors, vision-based
techniques become more promise to oer non-contact, full-eld, multi-component, static
and dynamic deformation measurement.
This study focused on a comprehensive experimental study to assess the performance
characteristics of a newly-developed class of vision-based sensors | the RGB-D cameras,
which include the color camera to take RGB images and the depth sensor to acquire pixel-
wise range data at moderate sampling rate. The RGB-D cameras can oer the potential of
conveniently tracking evolving multi-component deformation elds in monitored structures
with an accuracy that compares favorably to similar measurements achieved by much more
elaborate and sophisticated sensors which are either orders-of-magnitude more expensive,
and/or suer from fundamental limitations regarding their theory of operation and ability
to track the motion of discrete points on the monitored structure.
8
With the above in mind, the state-of-the-art techniques for the dierent classes of contact-
type and contactless displacement sensors will be reviewed, and then discuss the theory
of operation and performance envelope of a new breed of vision-based sensors, which is
subsequently followed by a comprehensive quantitative assessment of a representative
sensor | Microsoft Kinect (rst version).
Contact-type sensors for displacement measurement can be classied into direct-measurement
devices and indirect-measurement devices. In this review, direct-measurement sensors
include linear variable dierential transformers (LVDT) and global positioning systems
(GPS), and indirect-measurement sensors contain accelerometers.
The LVDTs are robust position-to-electrical transducers providing highly accurate one-
dimensional displacement measurements at a direction point. The LVDT operation requires
a xed platform as a reference point to measure the relative displacement between the
reference point and a point on the structure; however, this is a major disadvantage of
LVDTs, science it is not easy to nd a rigid platform close to the structure in many cases.
GPS has been applied for continuous monitoring of the dynamic displacement directly for
large-scale structures since 1990s [105, 27, 23, 28, 75, 178]. Conventional GPS technology
has accuracy limited to1 cm horizontally and2 cm vertically with up to 20 Hz sampling
rates. The highly precise real time kinematic (RTK) GPS consists of a base station on a
known ground point and multiple receivers called "rovers" installed on a structure. The
base station sends correction information to the rovers via wireless transmission to enhance
accuracy [164]. The RTK GPS has a resolution of5 mm horizontally and10 mm
vertically with 10 Hz sample rate. The drawback of GPS technology is that the GPS
signals cannot be received indoor. The other issue is multipath interference which is caused
9
by re
ective GPS signals o surrounding surfaces such as water, metal, and glass adding
error on true GPS signals [88].
Accelerometers are very common sensors widely used to monitor structural dynamics.
Accelerometers use a seismic mass spring-mass-damper system to measure the force (in
g's) acting on the mass, caused by gravity or motion. Nowadays, the technology of
accelerometers is very mature. One-axis, two-axis, and tri-axis miniature micro-electro-
mechanical systems (MEMS) accelerometers with dierent performance levels and dierent
price-levels can be found in various applications. Numerous low-power cost-eective tiny
wireless sensor nodes equipped with tri-axis MEMS accelerometers can be congured as a
wireless sensor network (WSN) and distributed over a structure for monitoring structural
dynamics with dense spatial resolution in three dimensions [106, 135]. However, it is very
challenging to estimate displacements using accelerometers. To calculate displacements
from recorded acceleration data, numerical integration should be implemented twice to
obtain velocity rst, and then displacement. During the numerical integration process, the
drift error and DC bias will be amplied. A thorough technical discussion of the serious
challenges and potential pitfalls in the use of digital signal processing and resulting large
errors inherent in the derivation of displacement time histories from directly-measured
acceleration records is available in the work of Smyth et al [153, 154].
Most of the contact-type sensors are well-developed and can acquire precise data with very
high sampling rate. A contact-type sensor can only obtain displacement for one single
spot on a structure. If full-eld deformation needs to be measured, this requires one or
several intelligent sensor networks consisting of large amount of sensor nodes. However,
it will raise the complexity of related cabling, power consumption, data communication,
10
synchronization, and computation needs. The sensor networks also add mass on a tested
structure, and probably change the structure's property. If the tested structure undergoes
damage during its monitoring phase, it could damage contact-type sensors and degrade
the performance of the monitoring system. To overcome these problems, the technology
for deformation measurement is trending toward developing cost-eective non-contact
sensors that can quickly deploy on remote sites to measure full-eld deformation dynamics
in three dimensions.
Optical methods are very applicable to develop non-contact techniques for deformation
measurement. Optical methods based on laser technology can provide very precise data.
Currently, laser-based optical devices, including the Doppler vibrometers (LDV) and
terrestrial laser scanners (TLS), are used to survey the de
ection of structures. The
operation of laser Doppler vibrometers is based on optical interferometry to measure
velocity and one-dimensional displacement at a xed point on a vibrating surface from
a remote site. The continuous scanning laser Doppler vibrometer (CSLDV) sweeps the
laser beam across the surface continuously to obtain full-eld vibration data; however, this
instrument is quite expensive. Representative publications that deal with the applications
of laser Doppler vibrometers can be found in the work of Rezaei et al. [144].
Terrestrial laser scanners (TLS) can generate a 3D model for the scanned object with
two working principles: phase-shift and pulsed time-of-
ight (ToF) technologies. The
time-of-
ight scanners measure the travel time of a laser pulse emitted onto an object and
re
ected back. Distance is calculated by taking half of the multiplication of the travel
time and the speed of light. The phased-shift scanners continuously emit a modulated
laser beam with amplitude in a sinusoidal form onto an object, and receive the re
ected
11
signal. The phase shift is measured by comparing the phase of incoming signal with the
outgoing light. The travel time of the laser beam can be determined by dividing the phase
shift (in radians) by the product of 2 and the modulation frequency. Phase-shift laser
scanners can acquire dense data at higher speed but have shorter measurement range;
conversely, the time-of-
ight laser scanners have longer range but not as fast as acquisition
range. Terrestrial laser scanner can scan still objects and generate precise 3D points (called
point clouds), but due to hardware limitation [158, 140], terrestrial laser scanners are not
applicable for dynamic measurement. Recently, Kim et al. [91] used a specic model
(RIEGL VZ-400) of ToF terrestrial laser scanner to measure 2D dynamic displacement of
a cantilever beam with line scan mode (repetitively moving the laser beam along a line);
however, only a few commercial models have this function. Overall, the laser scanners are
infeasible due to their exorbitant cost.
An alternative optical method is to use digital cameras with CCD (charge couple device)
or CMOS (complementary metal oxide semiconductor) image sensors. In recent years, the
technology of digital cameras has been improved and advanced signicantly to provide
high-quality images or videos, with light-weight and compact-size bodies. Many creative
approaches utilizing digital cameras to measure dynamic displacement had been proposed
[127, 170, 100, 57, 37, 99, 42, 79, 175, 146, 35, 138, 17]. The in-plane (2D) motion can be
determined using a single xed camera to track predened targets, such as high-contrast
patterns or LEDs on a planar surface of an object. For images taken by a digital camera,
digital image processing plays an important role to perform o-line or on-line analysis.
For example, the two-dimensional digital image correlation (2D DIC) approach measures
in-plane displacement by analyzing a series of images of random texture painted on a
12
planer surface of an object to track shifts of the random texture before and after applying
loading on the target structure [132]. Three-dimensional digital image correlation (3D
DIC) can determine full-eld out-of-plane deformation by processing two sets of images
taken by two digital cameras (stereo vision) simultaneously [68]. However, the drawback
of the digital image correlation approach is the requirement for preparing random texture
on the tested object, especially if it is to be applied on a large-scale structure.
Close-range photogrammetry is a non-contact technique to establish three-dimensional
geometric measurement from photographs taken by hand-held digital cameras. Two
photogrammetric approaches can be used to reconstruct 3D models from 2D images. One
is by processing a series of images captured by a single camera from dierent orientation.
The other one utilizes the stereo vision system composed of two (or multiple) cameras to
take images simultaneously; therefore, for full-eld deformation dynamic measurement, the
stereo vision system is a better choice [30]. Close-range photogrammetry using the stereo
vision system is an economical approach to obtain dense 3D deformation measurement
rapidly. Although digital image sensors and digital image processing can provide diverse
techniques for non-contact deformation measurement, target illumination is a big issue,
especially when a shadow on the object can confuse the image processing software.
Recently, the quick development of low-cost, o-the-shelf, depth sensors has attracted
researchers' attention to the potential applications of this type of 3D sensing devices
in various science and engineering elds, even though some of the depth sensors were
originally designed to be a natural user interface (NUI) for entertainment purposes.
There are two major depth sensing technologies: (1) time of
ight, and (2) structured light.
The time-of-
ight (ToF) depth sensors determine the point-by-point distance between
13
the device and the test object by measuring the
ight time of a quick scanning infrared
laser. The structured-light depth sensors project a known pattern on an object with the
infrared laser, and acquire the 3D shape of the object by observing the distortion of the
projected pattern, using an infrared camera. In general, the time-of-
ight sensors have a
higher frame rate but lower spatial resolution, and the structured-light sensors have higher
spatial resolution but a very low frame rate. However, in 2010, Microsoft released Kinect
v1 using a patented technology (similar to structured-light) which can acquire 640480
depth images at 30 frames per second (fps). In 2013. Kinect v2 was launched which
uses time-of-
ight technology but the depth resolution is 512424, at 30 fps. Although
both sensors have a relatively low price (under $200 dollars), they provide reasonable
performance and depth accuracy. The 3D measurement features make the cost-eective
Kinect v1 sensors have wide applications, including gesture recognition, augmented reality,
human pose estimation, 3D scanning and printing, and robotic navigation.
The Kinect v1 sensor was used to monitor respiratory motion for medical applications
[8, 162, 176]. For earth science, geologists used Kinect v1 to quantify surface deformation
in analogue modeling of volcanoes [166]. For civil engineering, researchers used Kinect
v1 to measure the dynamic de
ection of concrete beams [141]. Currently, most low-cost
o-the-shelf depth sensors are subject to easy interference by sunlight; consequently,
this restricts their outdoor applications. However, the depth sensing technology is still
advancing to manufacture aordable 3D hand-held sensors to provide high quality data
for indoor and outdoor applications.
For full-eld dynamic deformation measurement, the depth sensors have several advantages
over other sensors including: (1) non-contact method; (2) quick 2D/3D measurement;
14
(3) high spatial resolution; (4) aordable price (currently, hundreds to thousands of US
dollars), (5) light weight and compact size, and (6) easy operation. These features make
the depth sensors a very promising technology to track and quantify evolving deformation
eld.
2.1.2 Objectives
This study focused on using Microsoft's rst-generation Kinect sensors to monitor the
multi-component, full-eld 1D and 2D displacements of rigid structures under dynamic
loads. Before employing Kinect sensors to measure dynamical responses of structures, it is
necessary to verify the performance of the Kinect sensor to quantify dynamic displacement.
Many studies had been conducted to analyze the accuracy of Kinect's depth data captured
from stationary objects; however, there is a lack of in-depth quantitative assessment for
the performance of the Kinect sensor.
This study presents the results of a comprehensive experimental investigation to provide
precise quantitative measurements that evaluate some key metrics used in the structural
dynamics eld, and to assess the accuracy of the displacement time history of deformation
elds typically encountered in the vibration measurement of structures undergoing complex
dynamic response.
The assessment reported herein includes the investigation of: noise eects, amplitude
bounds, amplitude accuracy relative to frequencies, direction of motion of target with
respect to sensor, etc. It should note that for typical civil infrastructure, the low-frequency
modes can explain most of the system response, hence this study concentrates dynamic
15
testing of lower frequency ranges (currently, 0 to 2 Hz) below to Nyquist frequency (15
Hz) of the Kinect sensor.
2.1.3 Scope
The remainder of this study is organized as follows. Section 2.2 brie
y introduces newly-
developed consumer-grade RGB-D cameras. In Section 2.3, the description of the rst
version of Kinect sensor is illustrated. Section 2.4 presents the approach to quantify 2D
dynamic displacement eld, which includes: the Kinect data acquisition, the calibration
procedure for the RGB and IR camera, the 3D scene reconstruction approach, and target
tracking. Section 2.5 illustrates experimental setup for the comprehensive quantitative
assessment of the performance of the Microsoft Kinect. Section 2.6 explains data process-
ing procedures, which contain data ltering and data alignment. Section 2.7 discusses
experimental results and data analysis. The conclusions are summarized in Section 2.8.
2.2 Consumer-Grade RGB-D camera
RGB-D cameras can capture RGB (color) images and depth maps in several frames per
second using various techniques including structured-light, time-of-
ight, or stereo vision.
It has a great benet to obtain RGB images and depth maps simultaneously using a
single imaging device. The RGB images can provide dierent types of "features" (edges,
corners, color, texture, intensity gradients, etc.) to recognize contents on the images. The
depth maps containing pixel-wise range measurements can easily reconstruct 3D real world
coordinates from 2D pixel coordinates. Hence, RGB-D cameras have wide applications
16
including robotic navigation, human movement monitoring, 3D scanning, natural interface,
etc.
Although 3D imaging and depth sensing are not new technologies; formerly, to acquire
RGB and depth images simultaneously required integration of multiple sensors, bulky data
acquisition devices, and costly price. In recent year, consumer-grade RGB-D cameras,
which allow capturing RGB images and depth maps at same time with small form factor,
inexpensive price, low power consumption, and plug-and-play feature, have become very
handy for computer vision research. This research evaluated several commercially available
structured-light and time-of-
ight RGB-D cameras including Microsoft Kinect v1 and v2,
Asus Xtion Pro Live, SoftKinetic DS325, and Mesa SwissRanger SR4000. The comparison
of the specications for some of the listed RGB-D cameras is listed in Table 2.1.
Basically, the technologies behind Microsoft Kinect v1 and Asus Xtion Pro Live are
identical; the dierences are Microsoft Kinect v1 having tri-axis accelerometer, microphone
array, tilt motor, and requiring extra power supply. The Microsoft Kinect v1, v2, and
Asus Xtion Pro Live have higher RGB and depth resolutions than SoftKinetic DS325;
however, SoftKinetic cameras have better frame rates.
The Microsoft Kinect v2 has higher hardware and software demands. It requires Windows 8
(and above) computers with 64-bit faster processors, dedicate USB 3.0 host controllers, and
some compatible graphic adapters. The Microsoft Kinect v1 can run on dierent platforms
(Windows, Linux, Mac, etc.), have various choices of programming tools (Microsoft SDK,
OpenNI, OpenKinect, etc.), and requires common hardware congurations; therefore, this
study chosen Kinect v1 on this research phase. Microsoft Kinect v1 composes a color
CMOS sensor, an infrared CMOS sensor, and an infrared projector to capture 640480
17
RGB images and depth maps at 30 fps using PrimeSense's light coding technology. The
Kinect v1 sensor has two versions: one is Kinect for Xbox 360 and the other is Kinect for
Windows. The major dierence between two version is that the Kinect for Windows has
the "near mode" to obtain valid depth values as close as 40 cm; however, the minimum
range of Kinect for Xbox 360 is 80 cm. The Microsoft Kinect sensor used in this study
was Kinect for Xbox 360.
Table 2.1: Comparison of the technical specications for some inexpensive RGB-D sensors.
Specications Microsoft Kinect v1 SoftKinetic DS325 SwissRanger SR4000
Depth Camera:
Technology Structured Light Time of Flight Time of Flight
Range 0.8 - 4.0 m 0.15 - 1.0 m 0.8 - 5.0 m
Field-of-view (H V) 58

45

74

58

43

x 34

Resolution 640 480 pixels 320 240 pixels 176 144 pixels
Frame rate 30 fps 25 - 60 fps 50 fps
RGB Camera:
Field-of-view (H x V) 57

43

63.2

49.3

N/A
Resolution 640 x 480 pixels 720p HD N/A
Frame rate 30 fps N/A
Communication USB USB USB
Size (W x H x D) 27.94 x 7.62 x 7.62 cm 10.5 x 3.1 x 2.7 cm 6.5 x 6.5 x 6.8 cm
Price $99 $249 $4295
18
2.3 Microsoft Kinect sensor (rst version)
Many studies had evaluated the performance and accuracy of the Microsoft Kinect for
dierent types of applications. Stowers et al. [161] used Kinect as a visual control system
to estimate a quad-copter UAV's altitude dynamically. In this research, depth calibration
and photometric calibration was performed rst to measure accurate altitude. The result
showed the UAV hovered around5 cm of a selectable altitude. Fabian et al. [48] installed
a Kinect sensor on a wheeled mobile robot to operate as a visual odometer. This research
established an accurate error propagation model for a calibrated Kinect sensor to improve
the performance of visual odometry. Stone et al. [160] developed a marker-less vision-based
system using calibrated Kinects to assess the fall risk in a home by measuring a person's
gait in detail. In this research, the gait parameters, including number of steps, walking
speed, and left/right stride length, were estimated from accurate 3D point clouds obtained
by Kinects. Dutta et al. [45] used calibrated Kinects to monitor the risk of injury in
the workplace. These studies focused on 3D measurement for a longer distance (3 m
and farther). Dutta et al. [45] mentioned that the largest errors were measured at the
furthest target. It also found relatively larger errors close to the edge of the depth image,
which could be caused by the camera calibration errors. Ballester et al. [14] evaluated a
Kinect sensor to capture 3D motion data in a time series for use in physics experiments.
In this study, the depth data was provided by the OpenNI software suite which used
factory-default calibration parameters and the 3D coordinates were computed by the
optical geometric relationship of the Kinect sensor. Generally, the motion trajectories
captured from the Kinect without further calibrations are visually following the predicted
routes.
19
The depth accuracy related to operation, illumination, and thermal conditions was in-
vestigated by several researches. Chow et al. [38] studied possible in
uences on depth
accuracy, including warm-up time, ambient light, incidence angle, and surface color for a
non-calibrated Kinect sensor. The authors suggested waiting at least 60 minutes to warm
up a Kinect for stable depth data acquisition. Fiedler et al. [53] tested a Kinect sensor at
various thermal condition and found that temperature variations have strong impact on
depth accuracy. Andersen et al. [10] and Smisek et al. [152] compared accuracy of depth
measurements captured by dierent software suites (Microsoft SDK, OpenNI, etc.), and
discovered that there is a signicant dierence of depth error between these software tools.
Mallick et al. [108] attempted to characterize noise in the depth images of a Kinect sensor.
In the study, the authors summarized the source of noise and possible control strategies to
eliminate measurement error.
Two major calibration techniques could be implemented to enhance 3D measurement
accuracy. One is the disparity-depth calibration and the other one is camera calibration.
The disparity-depth calibration creates a relationship between an 11-bit raw disparity
value and a measured depth value. The dierent proposed disparity-depth relations can be
found in ROS.org (http://wiki.ros.org/kinect_calibration/technical), OpenK-
inect.org (http://openkinect.org/wiki/Imaging_Information), and the study pub-
lished by Khoshelham et al. [87].
The camera calibration is the process to determine intrinsic parameters (focal length,
principal point, and lens distortions) and extrinsic parameters (rotation and translation) of
a camera by solving the relationship between a set of known points in a 3D space and their
projections on a 2D image. The intrinsic parameters contain the camera's characteristics,
20
and the extrinsic parameters are related to the camera's position. Zhang [184] indicated
that the calibration target can be a precise stationary 3D object, a moving 2D planar
plate, a moving 1D line, or self-calibration without any calibration object. Currently, the
2D-plane-based calibration procedure according to Zhang [183] is the popular technique
for easy implementation. Bouguet [21] provides a camera calibration toolbox for MATLAB
to process calibration patterns and estimate the metric information for a camera.
2.4 3D dynamic displacement eld measurement using Kinect
sensor
2.4.1 Kinect sensor data acquisition
A Kinect sensor is composed of an RGB camera to acquire color images and a depth sensor
to generate depth maps. The depth sensor consists of two components: an IR emitter
and an IR camera. The IR emitter reads a specic sparkle pattern from the rmware and
projects the sparkle pattern on objects. The IR camera captures the projecting sparkle
pattern on objects and coverts into range measurements. A Kinect sensor can record RGB
and depth video simultaneously, with resolution of 640480 pixels at frame rate of 30 Hz.
2.4.2 Calibration of the RGB and IR cameras
Camera calibration processes were performed to compute intrinsic and extrinsic parameters
for the RGB camera and the IR camera. The intrinsic parameters included focal length,
principal point, and distortion coecients for each camera. The extrinsic parameters
21
contained a rotation matrix and a translation matrix for a stereo system. The intrinsic
and extrinsic parameters can be used for 3D reconstruction and RGB-depth registration.
To estimate these parameters, images of a chessboard were captured by the RGB and IR
cameras simultaneously, from dierent orientations (Figure 2.1a and Figure 2.1b). The
spatial relationships between the two cameras and the various positions of the chessboard
are shown in Figure 2.1c. The intrinsic and extrinsic parameters estimation was performed
using Camera Calibration Toolbox for MATLAB [22].
Calibration images
(a)
Calibration images
(b) (c)
Figure 2.1: Camera calibration for the RGB and depth camera: (a) calibration images for
the RGB camera, (b) calibration images for the IR camera, and (c) the 3D plot of the
spatial conguration for stereo calibration.
2.4.3 3D scene reconstruction from depth image
Kinect's depth image stores distance measurements along the Z-axis to the lens center of
the IR camera. It is convenient to reconstruct a 3D scene from a depth image using the
following formula, which was derived from a pinhole camera model [22];
22
X
IR
=
(u
D
c
IR
x
)Z
IR
f
IR
x
; (2.1a)
Y
IR
=
(v
D
c
IR
y
)Z
IR
f
IR
y
; (2.1b)
Z
IR
=Z
D
; (2.1c)
where (X
IR
;Y
IR
;Z
IR
) are the world coordinates of a 3D point relative to the IR camera;
(u
D
;v
D
) are the pixel coordinates of a depth image; (c
IR
x
;c
IR
y
) are the principal point on
the image plane, and f
IR
x
,f
IR
y
are the focal lengths of the IR camera. The depth value Z
D
is measured by a Kinect sensor. The intrinsic parameters f
IR
x
;f
IR
y
;c
IR
x
;c
IR
y
of a Kinect's
IR camera can be estimated through a camera calibration procedure.
2.4.4 Aligning RGB and depth images
Since the RGB camera and the IR camera are located onto a Kinect side by side, and the
eld-of-views are dierent between the two cameras, the pixel coordinates on a color image
are not at the same location on a depth image, which is taken simultaneously. The Kinect's
development software such as (OpenNI and Windows SDK) provides programming routines
using predened calibration parameters in the rmware, to easily align a pair of depth
and color images. If a computer vision application requires more accurate results, one can
use a stereo camera calibration approach to obtain intrinsic and extrinsic parameters for
the depth and color image registration.
23
The depth to color image registration based on the stereo camera calibration could follow
a simple three-step procedure: First, reconstruct a 3D scene from the depth image which
is already mentioned in the previous section. Second, transform the 3D coordinates of the
depth camera to the 3D coordinates of the RGB camera. Third, project the 3D coordinates
to the pixel coordinates on the RGB image. The second step can be represented by the
following transformation:
2
6
6
6
6
6
4
X
RGB
Y
RGB
Z
RGB
3
7
7
7
7
7
5
= R
2
6
6
6
6
6
4
X
IR
Y
IR
Z
IR
3
7
7
7
7
7
5
+ t; (2.2)
where (X
RGB
;Y
RGB
;Z
RGB
) are the world coordinates relative to the RGB camera; (X
IR
;Y
IR
;Z
IR
)
are the world coordinates relative to the IR camera; and the rotation matrix R and the
translation matrix t are the extrinsic parameters estimated from the stereo camera cali-
bration. Then, the world coordinates (X
RGB
;Y
RGB
;Z
RGB
) can be projected onto the pixel
coordinates (u
RGB
;v
RGB
) on the RGB image using the following formula:
u
RGB
=
X
RGB
Z
RGB
f
RGB
x
+c
RGB
x
; (2.3)
v
RGB
=
Y
RGB
Z
RGB
f
RGB
y
+c
RGB
y
; (2.4)
where f
RGB
x
;f
RGB
y
;c
RGB
x
;c
RGB
y
are the intrinsic parameters for a Kinect's RGB camera.
Figure 2.2a shows an example of color-depth alignment and a color point cloud for a rigid
plate.
24
(a) (b)
Figure 2.2: An example of (a) color-depth alignment, and (b) a color point cloud.
2.4.5 Point clouds
As discussed in Section 2.4.3, when the depth value Z
D
of a pixel (u
D
;v
D
) is obtained, the
pixel can be converted back to a 3D point (X
D
;Y
D
;Z
D
) in the world coordinates using
Equation (3.1). If a pair of RGB and depth images are registered, the color information
(R;G;B) of the pixel can also be added to the point. The collection of all color 3D points
is called the point cloud, which looks like a group of unconnected points
oating in a 3D
space (see Figure 2.2b).
2.4.6 Target detection and tracking
To quantify the dynamic displacement eld using a Microsoft Kinect sensor, marking
high-contrast visible targets on the surface of an object can help recognizing and tracking
points of interest on the color images. The corner detection algorithm proposed by Geiger
et al. [59] was used to identify the target points on a chessboard and then locate their
pixel coordinates (u
RGB
;v
RGB
) on the RGB image. For an aligned RGB and depth image
25
pair, the depth image's pixel coordinates (u
D
;v
D
) were identical to the RGB image's pixel
coordinates (u
RGB
;v
RGB
). The locations of identied target points on a RGB image and
the relative depth image are shown in Figure 2.3.
Hence, on a depth image, the depth value Z
i
(u
i
D
;v
i
D
) of a target point i could be found by
using its pixel coordinates (u
i
RGB
;v
i
RGB
) detected on the RGB image; that isZ
i
(u
i
RGB
;v
i
RGB
).
After the depth valueZ
i
of a target pointi was obtained, the world coordinates (X
i
;Y
i
;Z
i
)
of the point with respect to the Kinect sensor were computed using the transformation
formulas described in Section 2.4.3.
For a sequential pair of RGB and depth frames acquired by Kinect sensor, a procedure
was programmed in MATLAB to read the time step, detect target points on the RGB
image, locate their pixel coordinates on the RGB image, obtain their depth values on the
depth image, and compute their world coordinates with respect to the Kinect camera. The
time history of target points' positions was stored in accordance with the format shown in
Table 3.1. The trajectories of displacements for the target points can be plotted according
to the time history record.
Table 2.2: Time history for world coordinates of target points.
time point 1 point 2 . . . point n
t
1
X
11
Y
11
Z
11
X
12
Y
12
Z
12
. . . X
1n
Y
1n
Z
1n
t
2
X
21
Y
21
Z
21
X
22
Y
22
Z
22
. . . X
2n
Y
2n
Z
2n
.
.
.
t
m
X
m1
Y
m1
Z
m1
X
m2
Y
m2
Z
m2
. . . X
mn
Y
mn
Z
mn
26
(a) (b)
Figure 2.3: Extracting points of interest from (a) color image and mapping to (b) depth
image.
2.5 Experimental design to measure dynamic displacement
eld
This section presents the approach to evaluate performance of a Kinect sensor for 3D
dynamic displacement measurement. The Kinect sensor captured videos of the vibration
motion of a rigid plate mounted on a shaking table; subsequently, the videos are processed
to locate target points, and track the time history of the displacement. In order to quantify
measurement errors, the displacement measured by the Kinect sensor was veried by
LVDT measurements.
2.5.1 Experiment setup
To perform the dynamic tests, a rigid aluminum plate, as a measurement object for a
Kinect sensor, was mounted on a linear long-stroke vibration exciter (APS 400 ELECTRO-
SEIS
r
). The displacement of the exciter was recorded by an LVDT transducer to validate
27
the Kinect's data. The Kinect sensor (without a plastic case) fastened on a rigid plate
was mounted on a stable structure. The distance between the target plate and the Kinect
sensor is about 1 m, which is close to the minimum range 0.8 m (since the depth error
increases when the range becomes farther). The target plate was covered by a white paper
to reduce infrared re
ection from the shiny aluminum surface, which could cause false
depth values. A black-and-white chessboard pattern was attached on the target plate.
The inner corners of the black squares represented the points of interest to be detected.
The photographs of the experimental setup are shown in Figure 2.4.
(a) (b) (c)
Figure 2.4: Test apparatus: (a) the experimental setup for the Kinect dynamical displace-
ment measurement, (b) the aluminum plate, and (c) the Microsoft Kinect.
2.5.2 RGB-D data acquisition system
The Kinect sensor streams RGB and depth videos simultaneously to computer via USB
communication. One important issue for data acquisition using a Kinect sensor is sampling
rate. According to specications of the Microsoft Kinect v1, it can record RGB and depth
videos at frame rate up to 30 fps; however, unlike real-time embedded system, which
has dedicated microprocessor to acquire data with desired sampling rate; the typical
personal computer and operating system are not designed for real-time data acquisition.
28
Even though the Kinect is set to record data at 30 fps, the actual frame rate sometimes
could be lower because the CPU has excessive load when acquiring and compressing
RGB and depth images simultaneously. One approach to mitigate this problem is using a
high-end multiple-cores CPU and run the data acquisition program alone to improve the
performance.
The other important hardware requirement is that Kinect sensor has high demand of USB
bandwidth; therefore, it should reserve dedicated USB controller for a Kinect sensor to
maintain stable sampling rate. Since a single USB bus is usually expanded into several
ports, it needs to make sure no other devices share USB bandwidth with the Kinect sensor.
One solution for desktop PC is installing an external USB card for Kinect.
The Kinect data acquisition program used in this study was developed using C/C++
computer language, with OpenNI 1.0 (Open Natural Interaction) library, on Microsoft
Windows platform. The advantage of C/C++ is its high performance to overcome the
frame rate issue. The OpenNI library is composed of open source APIs (Application
Programming Interfaces) for programmers to access RGB and depth data from RGB-D
sensors, which use the PrimeSense PS1080 processor. OpenNI also provides a function to
align RGB and depth images with reasonably accuracy, using manufacture's calibration
parameters. In the current phase of this study, this method was adopted to register RGB
and depth data. By using OpenNI library, the data acquisition program saves the RGB
and depth streams together into a video le with the OpenNI le format (.oni). After
data acquisition procedure, a series of RGB and depth images can be extracted from the
ONI video le for further processing. Since there is no timing information stored in the
29
ONI le, the data acquisition program also saves frame numbers and timestamps into a
separate text le (.txt).
2.6 Data processing
Currently, most CPU resource was dedicated for Kinect data acquisition, hence the
recorded ONI video le, which contains an RGB video and a depth video, was processed
after the data collection phase. For the post processing, the procedure involved: extracting
each pair of the RGB image and the depth map in sequence from the ONI video le,
identifying target points on the RGB image using the corner detection algorithm, obtaining
corresponding depth values on the depth map, and computing 3D world coordinates for
the target points. The raw data generated from this procedure contained DC bias and
noise, which would be removed before comparing with LVDT data. Moreover, this study
used another computer to control the vibration exciter and collect LVDT measurement
without synchronizing with the Kinect's data acquisition system; hence, it required to
perform data alignment between Kinect and LVDT recordings.
2.6.1 Data ltering
After extracting depth data from the recorded ONI video le, the next step was ltering
and enhancing the signal obtained from Kinect. The ltering process helps to remove
DC bias, reduce noise, and increase smoothness in the acquired depth signal. Two FFT
(Fast Fourier Transform) digital lters were applied to process depth data: the high-pass
lter was used to remove the DC bias, and the low-pass lter was employed to clean
30
high-frequency noise. The concept of the FFT ltering approach is fairly straightforward:
rst, computing the frequency spectrum of a depth signal using the FFT transformation;
second, eliminating unwanted low- or high-frequency components in the spectrum based
on the setting of the cut-o frequency; and third, converting the updated spectrum back
to time domain to restore the ltered signal. A demonstration of noise-polluted zero-mean
raw depth data (harmonic signal with 20 mm amplitude and 1.25 Hz frequency) and its
frequency-domain representation are shown in Figures 2.5a and 2.5b. The ltering results
are presented in Figure 2.5d.
2.6.2 Data alignment
Currently, this study used two separate computers for the experiments: one was for Kinect
data acquisition, and the other was utilized to control the vibration exciter and collect
LVDT measurement. Because the o-the-shelf Kinect sensor lacks control function to be
triggered at exact certain time, it still had diculty to perform hardware synchronization
to coordinate the Kinect and LVDT data acquisition at this stage. The data alignment
procedure was required to synchronize Kinect and LVDT measurements after data was
processed.
To record the full-length time history of dynamical displacements using Kinect sensor,
the strategy of testing was planned as follow: rst step, start the Kinect data acquisition
program; second step, excite the vibration shaker and record the LVDT data; and nal
step, after the vibration test ends, stop the Kinect data collection process.
The Kinect and LVDT signals both had similar waveform and were acquired at the
same sampling rate (30 Hz). The cross-correlation approach was used to perform an
31
alignment procedure by measuring how much the Kinect signal resembled the LVDT
measurement. The main idea was that the amplitude of each sample point in the cross-
correlation estimation, which computed the "sliding dot product" of Kinect and LVDT
data, represented the degree of similarity at the sample point; therefore, the goal was to
nd the location of delay in the cross-correlation signal at which the amplitude reached its
maximum value (i.e. Kinect and LVDT signals were maximally correlated). Subsequently,
the two signals were aligned by shifting one of the signals according to the delay. The
procedure used MATLAB function xcorr to compute cross-correlation of Kinect and
LVDT data.
To enhance accuracy of the alignment procedure using cross-correlation approach, it is
important to mention that, before aligning the Kinect and LVDT measurements, the
two signals were resampled at higher rate (1000 Hz) with spline interpolation (MATLAB
function spline), which also smoothed the peaks to obtain better alignment results. An
example of data alignment is illustrated in Figures 2.5c and 2.5d. In Figure 2.5c, the plot
shows the time shift between Kinect and LVDT data. After performing cross-correlation
alignment, the result is displayed in Figure 2.5d.
2.7 Experimental results and data analysis
2.7.1 Initial test results
The initial test was conducted to evaluate the basic performance of Kinect hardware
(depth sensor) and software (OpenNI) to measure dynamical displacements at a stable
sampling rate (30 Hz). Figure 2.6 demonstrates the examples of aligned Kinect depth
32
22 22.5 23 23.5 24 24.5 25
−20
−10
0
10
20
Time (s)
Displacement (mm)
(a)
0.5 1 1.5 2 2.5
0
5
10
15
20
Frequency (Hz)
|PSD|
(b)
0 10 20 30 40
−30
−20
−10
0
10
20
30
Time (s)
Displacement (mm)

Kinect
LVDT
(c)
22 22.5 23 23.5 24 24.5 25
−20
−10
0
10
20
Time (s)
Displacement (mm)

Kinect
LVDT
(d)
Figure 2.5: Kinect data post-processing: (a) raw depth data, (b) depth data Fourier
transform, (c) ltered depth, and LVDT data, and (d) aligned Kinect and LVDT data.
33
measurement and LVDT data at dierent harmonic frequencies of 0.5 Hz, 1 Hz, and 1.5
Hz while maintaining a constant amplitude of 20 mm. Figure 2.7 illustrates aligned data
for dierent harmonic amplitudes of 5 mm, 10 mm, and 20 mm, which were recorded at
a constant frequency of 1 Hz. These plots show that there are 30 data point counts in
1 second interval, which implies data acquired at 30 fps by the Kinect sensor without
information lost. The test results indicate that the Kinect sensor has basic capability to
capture low-frequency vibration; while, more comprehensive experiments were performed
to verify the accuracy of the Kinect measurements as presented in following sections.
20 22 24
−20
−10
0
10
20
Time (s)
Displacement (mm)

Kinect
LVDT
(a)
20 22 24
−20
−10
0
10
20
Time (s)
Displacement (mm)

Kinect
LVDT
(b)
20 22 24
−20
−10
0
10
20
Time (s)
Displacement (mm)

Kinect
LVDT
(c)
Figure 2.6: Displacement measurements of Kinect and LVDT under dierent frequencies:
(a) 0.5 Hz, (b) 1.0 Hz, and (c) 1.5 Hz.
22 23 24 25
−20
−10
0
10
20
Time (s)
Displacement (mm)

Kinect
LVDT
(a)
22 23 24 25
−20
−10
0
10
20
Time (s)
Displacement (mm)

Kinect
LVDT
(b)
22 23 24 25
−20
−10
0
10
20
Time (s)
Displacement (mm)

Kinect
LVDT
(c)
Figure 2.7: Displacement measurements of Kinect and LVDT under dierent displacement
levels: (a) 5 mm, (b) 10 mm, and (c) 20 mm.
34
2.7.2 Harmonic excitation
Data collection
To evaluate the performance of the Kinect sensor for dynamical displacement eld measure-
ments, series of experiments were performed to measure harmonic displacements of a rigid
plate with a chessboard pattern under dierent combinations of vibration frequencies (0
Hz, 0.25 Hz, 0.5 Hz, 0.75 Hz, 1 Hz, 1.25 Hz, 1.5 Hz, 1.75 Hz, 2 Hz) and peak displacements
(5 mm, 10 mm, 15 mm, 20 mm). The harmonic motions were excited by sinusoidal signals
except the case of 0 Hz. To perform the 0 Hz test, a low-frequency square wave was
generated to create two constant range measurements.
(a) (b) (c)
Figure 2.8: Target orientation and angle with respect to Kinect: (a) target displacement
perpendicular to Kinect (depth-based), (b) target displacement parallel to Kinect (pixel-
based), (c) target displacement angled to Kinect (depth-and-pixel-based).
The videos of harmonic motions of the rigid plate were recorded by Kinect v1 in three
dierent orientations | Case I: motion of the rigid plate is parallel to Kinect's depth axis
(see Figure 2.8a); case II: motion of the rigid plate is perpendicular to Kinect's depth axis
35
(see Figure 2.8b); and case III: motion of the rigid plate has an angle of 45

to Kinect's
depth axis (see Figure 2.8c). For case I, when referring to Equations (3.1a), (3.1b), and
(3.1c), the z-direction depth values of a chessboard corner (target point) are variable, but
the x-direction values remain constant (see Figure 2.9). For case II, the z-direction depth
values of a chessboard corner remain constant, but the x-direction values are variable (see
Figure 2.10). For case III, both z and x of a chessboard corner are variable. Because the
rigid plate has no vertical motion for all the three cases, the y-direction values of target
points are constants for all cases (see Figure 2.11). To identify the three scenarios, case I
is referred to "depth-based measurement" because it is dominated by the depth value z,
case II is called "pixel-based measurement" for it is governed by the pixel coordinate u,
and case III is named "depth-and-pixel-based measurement" in this study.
Data analysis
The harmonic displacements measured by the Kinect RGB-D camera were compared to
corresponding LVDT records. For each test, the normalized error and the normalized peak
error between Kinect and LVDT data were calculated using the following equations:
Normalized error =
kX
LVDT
X
Kinect
k
2
kX
LVDT
k
2
; (2.5)
Normalized peak error =
kP
LVDT
P
Kinect
k
2
kP
LVDT
k
2
; (2.6)
36
Where X
LVDT
is the array corresponding to the sampled LVDT displacement time history,
X
Kinect
is the processed displacement time history for the Kinect sensor, P
LVDT
is the
LVDT displacement peaks, and P
Kinect
is the Kinect displacement peaks.
22 23 24 25
−20
−10
0
10
20
Time (s)
Displacement (mm)
(a)
22 23 24 25
−20
−10
0
10
20
Time (s)
Displacement (mm)
(b)
22 23 24 25
−20
−10
0
10
20
Time (s)
Displacement (mm)
(c)
Figure 2.9: Sample depth-based measurements (1.0 Hz, 20 mm): (a) x-direction, (b)
y-direction, and (c) z-direction.
22 23 24 25
−20
−10
0
10
20
Time (s)
Displacement (mm)
(a)
22 23 24 25
−20
−10
0
10
20
Time (s)
Displacement (mm)
(b)
22 23 24 25
−20
−10
0
10
20
Time (s)
Displacement (mm)
(c)
Figure 2.10: Sample pixel-based measurements (1.0 Hz, 20 mm): (a) x-direction, (b)
y-direction, and (c) z-direction.
Figure 2.12 to Figure 2.14 show six sets of tting curves representing the relationship
between accuracy of the displacement measurements and the frequencies of harmonic
excitation, which were estimated using Equation (2.5) (normalized error) and Equation
(2.6) (normalized peak error), under three testing scenarios discussed below.
37
22 23 24 25
−20
−10
0
10
20
Time (s)
Displacement (mm)
(a)
22 23 24 25
−20
−10
0
10
20
Time (s)
Displacement (mm)
(b)
22 23 24 25
−20
−10
0
10
20
Time (s)
Displacement (mm)
(c)
Figure 2.11: Sample depth-and-pixel-based measurements (1.0 Hz, 20 mm): (a) x-direction,
(b) y-direction, and (c) z-direction.
Depth-based measurements: Figures 2.12a and 2.12b illustrate error analysis for the
depth-based measurements. For the cases of the peak displacements of 10 mm, 15 mm,
and 20 mm, the normalized errors remain at the same level of about 5%. However, for
the peak displacement measurement of 5 mm, the percentage of normalized error raises,
when the frequency level increases. Basically, the tting curves of normalized error and
normalized peak error for the depth-based measurement have similar trend.
0 0.5 1 1.5 2
0
5
10
15
Frequency (Hz)
Normalized error (%)

5 mm
10 mm
15 mm
20 mm
(a)
0 0.5 1 1.5 2
0
5
10
15
Frequency (Hz)
Normalized peak Error (%)

5 mm
10 mm
15 mm
20 mm
(b)
Figure 2.12: Depth-based measurements errors: (a) normalized errors, and (b) peak errors.
38
Pixel-based measurements: Figures 2.13a and 2.13b plot the tting curves of nor-
malized errors and peak errors for pixel-based measurements. In Figure 2.13a, for all
displacement measurements, the normalized errors increase when the frequencies increase;
however, the normalized peak errors stay at the same level below 5%, which is shown
in Figure 2.13b. This dierence could be explained by the rolling shutter distortion of
CMOS (Complementary Metal-Oxide Semiconductor) image sensors [3]. Both RGB and
IR cameras of the Kinect sensor consist of rolling shutter CMOS image sensors. When
each frame of video stream is recorded by rolling shutter CMOS sensor, images are not
captured by the entire pixel array simultaneously, rather by scanning row-by-row, in pixels.
The rolling shutter CMOS sensors provide low-noise, low-power, fast data processing, and
an inexpensive solution for most commercial cameras; however, this image acquisition
method creates distortions when shooting moving objects. Therefore, when frequency
increases, faster motion speed causes more distortion for the picture of the target plate,
which diminished accuracy of horizontal (x-axis) displacement measurement. But for the
normalized peak errors, only the upper peaks and the lower peaks of amplitude were
considered, where the velocity of the target plate is zero (i.e. no movement), hence there
was no rolling shutter distortion in the image.
Depth-and-pixel-based measurements: In this case, there was a 45-degree angle
between the direction of motion and the Kinect depth axis; hence, the displacement was
divided into x-component and z-component. In Figure 2.11, the example shows that the
peak amplitude of the x-component (pixel-based) and the z-component (depth-based) are
10 mm for a harmonic excitation with a peak amplitude of 20 mm. Therefore, for the same
excitation, this setup will reduce the displacements of x-component, which can decrease
39
0 0.5 1 1.5 2
0
5
10
15
Frequency (Hz)
Normalized error (%)

5 mm
10 mm
15 mm
20 mm
(a)
0 0.5 1 1.5 2
0
5
10
15
Frequency (Hz)
Normalized peak Error (%)

5 mm
10 mm
15 mm
20 mm
(b)
Figure 2.13: Pixel-based measurements errors: (a) normalized errors, and (b) peak errors.
error from the rolling shutter distortion. The results are illustrated in Figure 2.14a and
Figure 2.14b.
0 0.5 1 1.5 2
0
5
10
15
Frequency (Hz)
Normalized error (%)

5 mm
10 mm
15 mm
20 mm
(a)
0 0.5 1 1.5 2
0
5
10
15
Frequency (Hz)
Normalized peak Error (%)

5 mm
10 mm
15 mm
20 mm
(b)
Figure 2.14: Depth-and-pixel-based measurements errors: (a) normalized errors, and (b)
peak errors.
40
2.7.3 Random excitation
Data collection
Since in practical applications, the motion of interest is not simply harmonic; hence,
additional tests were performed to evaluate the performance of the Kinect sensor through
measuring dynamical displacements under low-frequency random excitation. Three distinct
random signals with dierent levels of maximum amplitude were generated with MATLAB
rand function, and applied a low-pass lter to remove high frequency components above
5 Hz, which were used to excite the shaking table at frequency range of 0 to 5 Hz. The
random displacements measured by the LVDT transducer and the Kinect sensor were
sampled from Gaussian distributions. The experiments were classied into three groups
according to root mean square (RMS) levels: 1.94 mm, 8.44 mm, and 13.81 mm, which
were estimated based on LVDT displacement measurements.
The videos for random motions of the target plate were captured by Kinect sensor in
two dierent scenarios: case I, motion of the target plate is parallel to Kinect's z-axis
(i.e. depth-based measurement); and case II, motion of the target plate is parallel to
Kinect's x-axis (i.e. pixel-based measurement). Figure 2.15 illustrates aligned LVDT and
depth-based Kinect measurements under dierent RMS levels. However, Figure 2.16 shows
aligned LVDT and pixel-based Kinect measurements under dierent RMS levels.
Data analysis
In order to evaluate the performance of the Kinect sensor for displacement measurement
under random excitation, this study compares the probability density functions (PDFs) of
41
sampled Kinect and LVDT data for three dierent RMS amplitude levels (1.94 mm, 8.44
mm, and 13.81 mm). The probability density functions were smoothed based on normal
kernel density estimation.
25 30 35
−40
−20
0
20
40
Time (s)
Displacement (mm)

Kinect
LVDT
(a)
25 30 35
−40
−20
0
20
40
Time (s)
Displacement (mm)

Kinect
LVDT
(b)
25 30 35
−40
−20
0
20
40
Time (s)
Displacement (mm)

Kinect
LVDT
(c)
Figure 2.15: Depth-based measurements for three RMS levels: (a) RMS = 1.94 mm, (b)
RMS = 8.44 mm, and (c) RMS = 13.81 mm.
25 30 35
−40
−20
0
20
40
Time (s)
Displacement (mm)

Kinect
LVDT
(a)
25 30 35
−40
−20
0
20
40
Time (s)
Displacement (mm)

Kinect
LVDT
(b)
25 30 35
−40
−20
0
20
40
Time (s)
Displacement (mm)

Kinect
LVDT
(c)
Figure 2.16: Pixel-based measurements for three RMS levels: (a) RMS = 1.94 mm, (b)
RMS = 8.44 mm, and (c) RMS = 13.81 mm.
Figure 2.18 shows the comparisons of probability density functions between Kinect and
LVDT displacement measurements under case I | depth-based measurement. Figure 2.19
illustrates the comparisons of probability density functions between Kinect and LVDT
displacement estimations under case II | pixel-based measurement. Figures 2.20 and 2.21
display the comparisons of probability density functions of all sampled peak values (positive
42
and negative) between Kinect and LVDT displacement estimations, under case I (depth-
based measurement) and case II (pixel-based measurement) respectively.
Three metrics were utilized to quantify the Kinect performance of random displacement
measurements with respect to LVDT data: the normalized error, the mean error for the two
PDFs of sampled peaks, and standard deviation error for the two PDFs of sampled peaks
(see Figure 2.17), which are dened below. The results were summarized in Table 2.3.
Normalized error =
kX
LVDT
X
Kinect
k
2
kX
LVDT
k
2
; (2.7)
Mean error (PDFs of peaks) =

LVDT

Kinect

LVDT
; (2.8)
Standard deviation error (PDFs of peaks) =

LVDT

Kinect

LVDT
; (2.9)
Mean error
LVDT
Kinect
σKinect
σLVDT
Displacement
Figure 2.17: Sketch showing PDF peaks mean and standard deviation errors.
43
WhereX
LVDT
is the time history of the LVDT displacement, X
Kinect
is the time history of
Kinect data,
LVDT
and
LVDT
are the mean and standard deviation for the probability
density function of all sampled peaks of LVDT data, and
Kinect
and
Kinect
are the mean
and standard deviation for the probability density function of all sampled peaks of Kinect
data.
Overall, the normalized error, mean error, and standard deviation error decrease when
the RMS amplitude levels (1.94 mm, 8.44 mm, and 13.81 mm) increase. For the entire
displacement measurements of random vibrations, the analysis of normalized error shows
relatively low performance of pixel-based (x-direction) measurements, especially for small
RMS level (error of 124.74% for RMS = 1.94 mm). The error could be mainly aected by
rolling shutter distortion, which had been mention in Section 2.7.2. Comparatively, the
Kinect has better performance for depth-based (z-direction) measurements, particularly
for large RMS level (error of 8.6% for RMS = 13.81 mm).
For sampled peaks (positive and negative), which imply that the velocities equal to 0 at
peak locations, mean errors and standard deviation errors are lower than normalized errors
due to no interference from rolling shutter distortion. For larger RMS levels (13.81 mm),
the mean error (2.8% for depth, 2.02% for pixel) and standard deviation error (8.07% for
depth, 5.50% for pixel) indicate that the Kinect sensor has relatively good performance
to detect peaks of displacement time history under random excitation. Overall, the error
analysis of random test agrees with the error analysis of harmonic test.
44
−50 0 50
0
0.05
0.1
Peak (mm)
RMS = 1.938 mm

kinect
LVDT
(a)
−50 0 50
0
0.05
0.1
Peak (mm)
RMS = 8.442 mm

kinect
LVDT
(b)
−50 0 50
0
0.05
0.1
Peak (mm)
RMS = 13.811 mm

kinect
LVDT
(c)
Figure 2.18: PDF of the estimated Kinect displacement measurements based on depth-
based data: (a) RMS = 1.94 mm, (b) RMS = 8.44 mm, and (c) RMS = 13.81 mm.
−50 0 50
0
0.05
0.1
Displacement (mm)
RMS = 1.938 mm

kinect
LVDT
(a)
−50 0 50
0
0.05
0.1
Displacement (mm)
RMS = 8.442 mm

kinect
LVDT
(b)
−50 0 50
0
0.05
0.1
Displacement (mm)
RMS = 13.811 mm

kinect
LVDT
(c)
Figure 2.19: PDF of the estimated Kinect displacement measurements based on pixel-based
data: (a) RMS = 1.94 mm, (b) RMS = 8.44 mm, and (c) RMS = 13.81 mm.
−50 0 50
0
0.05
0.1
Peak (mm)
RMS = 1.938 mm

kinect
LVDT
(a)
−50 0 50
0
0.05
0.1
Peak (mm)
RMS = 8.442 mm

kinect
LVDT
(b)
−50 0 50
0
0.05
0.1
Peak (mm)
RMS = 13.811 mm

kinect
LVDT
(c)
Figure 2.20: PDF of the estimated Kinect peak displacement measurements based on
depth-based data: ( (a) RMS = 1.94 mm, (b) RMS = 8.44 mm, and (c) RMS = 13.81 mm.
45
−50 0 50
0
0.05
0.1
Displacement (mm)
RMS = 1.938 mm

kinect
LVDT
(a)
−50 0 50
0
0.05
0.1
Displacement (mm)
RMS = 8.442 mm

kinect
LVDT
(b)
−50 0 50
0
0.05
0.1
Displacement (mm)
RMS = 13.811 mm

kinect
LVDT
(c)
Figure 2.21: PDF of the estimated Kinect peak displacement measurements based on
pixel-based data: (a) RMS = 1.94 mm, (b) RMS = 8.44 mm, and (c) RMS = 13.81 mm.
Table 2.3: Comparison of normalized error, mean error, and standard deviation error
under depth-based and pixel-based measurements.
RMS = 1.94 mm RMS = 8.44 mm RMS = 13.81 mm
Depth (z) Pixel (x) Depth (z) Pixel (x) Depth (z) Pixel (x)
Normalized error (%) 18.21 124.74 11.80 47.70 8.60 20.02
"Peaks" Mean error (%) 5.60 65.27 3.66 6.50 2.80 2.02
"Peaks" Std error (%) 14.52 80.46 8.16 10.76 8.07 5.50
46
2.8 Summary and conclusions
This study concerns the importance and challenges regarding quantitative measurements
of the time history for the multi-component deformation eld of a structural system under
dynamic load. The newly-developed class of vision-based sensors | RGB-D cameras,
which oer potential of conveniently to track evolving 1D/2D displacements elds in the
broad eld of structural dynamics. The comprehensive experimental study was performed
to evaluation the performance and feasibility of a representative sensor | Microsoft Kinect
(rst version).
The assessment involved testing basic capability of Kinect hardware and software to
acquire vibration data at stable sampling rate and the quantitative performance evaluation
using the Kinect sensor to measure 1D/2D dynamical displacement of a rigid plate under
harmonic and random excitation. The result of initial test shows that the Kinect sensor
is capable to record vibration data at its maximum sampling rate of 30 Hz with proper
hardware and software setup. According to a series of experiments conducted to determine
the accuracy of the Kinect displacement measurements under dierent combination of
testing scenarios including: dierent amplitudes (0.5 mm to 2 mm), various frequencies
(0 Hz to 2 Hz), and dierent relative motions between the sensor and target, the results
indicate that for displacement over 10 mm, the measurements have average error of about
5% if the vibration motion is almost parallel to Kinect's depth axis (z-axis). However, when
the motion is parallel to Kinect's x-axis, higher frequency could cause higher measurements
errors, which is due to rolling shutter distortion. The phenomenon was shown in harmonic
and random test results.
47
Overall, it was found that the Kinect sensor can be used as feasible tool for certain
applications to measure 1D/2D deformation eld of multi-component structures under
static or dynamic loads. This study also provides guidelines about the in
uence of several
important issues (hardware and software limitation, rolling shutter distortion, etc.) that
would arise in the eld implementation.
2.9 Acknowledgments
This research was supported in part by a grant from Qatar University and Qatar Founda-
tion.
48
Chapter 3
3D Dynamic Displacement-Field Measurement
for Structural Health Monitoring and Condition
Assessment Using Inexpensive RGB-D Based
Sensor
3.1 Introduction
3.1.1 Background and motivation
T
HE quantitative measurement of the multi-component displacement eld of a
distributed system undergoing dynamic response is an important and challenging
problem in multidisciplinary areas, and in dierent applications. For example, in the
aerospace industry, displacement eld measurements are usually conducted to monitor the
change in the shape of aircraft wings under distributed aerodynamic and mass-inertial
forces [94]. The experimental eld measurements are used to correct computational
49
methods and make adjustments to the design parameters. Furthermore, displacement
measurements obtained from the computational model and the full-size wing are required
when creating adequate structural layouts of the computational models [95, 16].
Displacement eld measurements are widely deployed in the eld of civil infrastructures
[85, 149, 126, 80, 15]. Civil infrastructures are continuously deteriorating due to misuse,
material aging, and absence of sucient maintenance. Additionally, the increase of load
demands and environmental changes can cause overstressing that leads to the failure of
infrastructures [170]. Acquiring the structural displacements is an important indicator in
assessing structural safety [134], and identifying changes or damage that might occur in a
structure, and hence improve the maintenance options. Furthermore, acquired response
data are used as the basis for the performance analysis of the structure via several system
identication methods [90], model updating [151, 56], and various structural control
systems [61]. The structural response civil infrastructure systems under dynamic loads is
important to accurately evaluate the load-carrying capacity of a
exible structure, and
estimate its physical parameters in a state-space model identied from the displacement
response data [44, 50, 156, 51, 2]. Dynamic displacement measurements are also important
for strain quantication as they can be directly calculated from displacement measurements,
especially for large strain applications [4].
However, it is challenging to perform quantitative measurements using conventional
contact-type sensors to acquire the time history of the multi-component displacement
and torsional/rotational eld of a distributed system undergoing dynamic response. The
development and availability of consumer grade vision-based systems, at aordable cost,
50
provides the potential for developing monitoring systems that can enhance the capabil-
ities and increase the delity to monitor displacement elds (as opposed to a specic
discrete point) of the distributed systems that are undergoing complex three-dimensional
displacement.
Conventional sensors such as accelerometers, linear variable-dierential transformers
(LVDTs), inclinometers, gyroscopes and global positioning systems (GPS) have various
practical limitations in obtaining accurate displacement eld measurements. However,
vision-based methods have shown promising results in both laboratory and eld experiments.
Vision-based approaches have the advantage of both high spatial and temporal resolution.
Also, for full-eld dynamic displacement measurements, vision-based methods have several
advantages: (1) contactless sensing; (2) full-eld displacement measurements; (3) multi-
component measurements (torsion eects); (4) three-dimensional measurements; (5) cost-
eective data acquisition systems; and (6) easy operation. These features make the
vision-based sensors with depth capabilities a promising technology to track and quantify
evolving displacement elds.
Conventional contact-type sensors for displacement measurement can be classied into
direct and indirect measurement devices. Direct measurement sensors include position sen-
sors and GPS. Position sensors such as LVDTs, linear potentiometers, and linear encoders
are robust position-to-electrical transducers providing highly accurate one-dimensional
displacement measurements at a specic location. Their operation requires a xed platform
as a reference point to measure the relative displacement between the reference point and
a point on the structure; however, this is a major disadvantage of position sensors since
it is not easy to nd a rigid platform close to the structure in many cases. Additionally,
51
they are primarily suited to measure uniaxial motion aligned with their operation axis
(i.e., not suitable for use if orthogonal components of motion are simultaneously present).
GPS devices have been deployed for continuous monitoring of the dynamic displacement
directly for high-rise buildings and long span suspension bridges [24, 178, 75, 28, 23]. The
accuracy of conventional GPS technology is limited to10 mm horizontally and20 mm
vertically, with up to 20 Hz sample rates. The highly precise real time kinematic (RTK)
GPS has a resolution of5 mm horizontally and10 mm vertically but it is relatively
expensive and the sampling rate is 10 Hz rate. The main limitation of GPS technology is
that the GPS signals can not be received indoors. In addition, the multipath interference
caused by re
ective GPS signals of surrounding surfaces such as water, metal, and glass,
adds error on true GPS signals [88].
Accelerometers are common sensors widely used to monitor structural dynamics, and can
be considered as an indirect displacement measurement sensor. In principle, the time
history of the displacements can be computed by double-integrating the corresponding
accelerations [106, 135]. However, it is very challenging to accurately estimate displacement
using accelerometers. The displacements are computed from recorded acceleration data by
implementing numerical integration twice to obtain velocity rst, and then displacement.
During the numerical integration process, the drift error and direct current (DC) bias will be
amplied and the displacement estimate signicantly dier from the actual displacements
[153, 154].
Measuring a small dynamic rotational angle accurately is also a challenging issue. Icli-
nometers are commercially available for measuring relatively small rotational angles in
static mode with resolution between 0.0001 to 0.003 degrees [131]. However, inclinometers
52
are highly sensitive to linear acceleration, environmentally induced magnetic elds and
require a long response time. In addition, inclinometers only measure rotations about axes
perpendicular to gravity [73].
Micro-Electro-Mechanical Systems (MEMS) Gyroscopes sensors are popular for measuring
angular rate - the rate at which rotation angle changes. MEMS sensors have several
advantages: small size, light weight, low power consumption, low costs, and simple design
and use. MEMS gyroscopes have been used in wide range of applications [159], such as in
automobiles for ride stabilization and rollover detection [157], robotics [103], bio-medical
applications [12, 123, 167], virtual and augmented reality applications [72, 150], outdoor
measurements [9], navigation and motion tracking applications [171, 185]. In order to
obtain the values for rotational response, it is required to integrate the angular speed with
respect to time. However, the integration process can produce large drift over time due to
the presence of bias and noise in the angular rate signal [137, 136]. These eects are more
severe for low cost MEMS gyroscopes.
Although the contact-type sensors are already well-developed and can acquire precise data
with high sampling rate, they can only obtain displacement/rotation for one single spot
on a structure. If full-eld displacements need to be measured, this requires one or several
sensor networks consisting of large number of sensor nodes. This will raise the complexity
of the corresponding cabling, power consumption, data communication, synchronization,
and computation needs. In some applications, the typical sensor networks add mass
on the target structure, and may change the structure's dynamic characteristics. If the
tested structure undergoes damage during its monitoring phase, it could damage contact-
type sensors and degrade the performance of the monitoring system. To overcome these
53
problems, the technology for displacement measurement is trending toward developing
cost-eective contactless sensors that can be quickly deployed on remote sites to measure
the full-eld displacement dynamic, in three dimensions.
The terrestrial laser scanner (TLS) is an important contactless technique for displacement
measurement based on optical methods. TLS has been widely used for scanning large and
complicated scenes that consist of complex objects and shapes with greater precision and
resolution in a three-dimensional (3D) space by generating precise 3D point cloud data
[140, 130]. TLS applications include the measurement of static displacement, displacement
and dimensions of structural components [89, 129, 133, 62]. TLS systems are limited for
static displacement and displacement, since they are programmed to measure one point at
a time. Recently, Kim et al. [91] used a specic model time-of-
ight TLS RIEGL VZ-400
to measure the 2D dynamic displacement of a cantilever beam with line scan mode (i.e., by
repetitively moving the laser beam along a line); however, only a few commercial models
have this function. Overall, the laser scanners are infeasible for dynamic eld displacement
measurements due to their hardware limitations and exorbitant cost.
Digital cameras are alternative optical methods. In recent years, the technology of
digital cameras has been improved and advanced signicantly to provide high-quality
images or videos with light-weight and compact-size bodies. The techniques using digital
cameras to measure full-eld 3D dynamic displacements are diverse, including close-range
photogrammetry, 2D/3D digital image correlation (DIC), blind identication and motion
magnication have been proposed [177, 17, 32, 52, 142, 79, 175, 42]. Although digital
image sensors and digital image processing can provide diverse techniques for contactless
54
displacement measurements, target illumination is a big issue, especially when a shadow
on the object confuses the image processing software.
Vision-based systems have been developed to measure rotational angle of large civil
structures. Jeon et al. [78] proposed 6-DOF displacement static measurement system
using a paired structured light system that included lasers, cameras, and screens. Lee et
al. [99] introduced multiple camcorders and image processing techniques to measure the
dynamic rotational angle of a structure. Park et al. [138] utilized a system consisting of
a laser source, a frame grabber, and a commercially available home video camcorder to
measure rotational angles for bridge supports. However, vision-based systems are easily
aected by external changes such as weather and illumination, and only work properly if
installed in the correct line of sight. In addition, measurement accuracy in these systems
is aected by the distance between the camera and the target.
Recently, the rapid development of low-cost, o-the-shelf, RGB-D sensors using time-of-

ight or structured light technologies has attracted researchers' attention in various science
and engineering elds although some of the RGB-D sensors were originally designed to
be a natural user interface (NUI) for entertainment purposes. RGB-D sensor consists
of RGB color sensor that capture pixel color information, alongside with depth sensor
that acquire depth data. In 2010, Microsoft released the rst-generation Kinect using a
patented technology based on structured light that could acquire 640480 depth images at
30 frames per second (fps). In 2014, the second-generation Kinect was launched that used
time-of-
ight technology to obtain 512424 depth images at 30 fps. Although both sensors
have a relatively low price (currently under $100), they provide reasonable performance
and depth accuracy. The 3D measurement features make the inexpensive Kinect sensors
55
have wide applications, including gesture recognition [143, 139, 18], augmented reality
[124, 169], robotic navigation [39, 46], medical applications [162, 8], earth science [166]
and structural engineering [141].
Most inexpensive o-the-shelf RGB-D sensors are based on infrared technology and they
are subject to interference by direct sun light. This restricts their outdoor applications.
Several in-depth evaluations of noise sources and system limitations, including ambient
background light, semitransparent and scattering media, dynamic scenery, multi-sensor
interference, etc., for RGB-D sensors are reported by [108], [83], and [148]. The noise
sources investigated in these studies could create invalid depth values or "holes" in the
depth image. The RGB-D sensing technology is still advancing to overcome the system
limitations [125, 122, 84] to manufacture aordable 3D hand-held sensors to provide high
quality data for indoor and outdoor applications.
3.1.2 Objectives
This study focused on using a class of inexpensive RGB-D sensors that are equipped with an
RGB camera and an active depth sensor to monitor evolving multi-component displacement
elds of
exible structures under dynamic loads. Kinect v1 is a representative of this class
of sensors that are designed for video games, where the requirements for gaming is quite
dierent than using it as an accurate scientic sensor. It is important to mention that
the second-generation Kinect (v2), which was released when the tests of this study were
towards the end. Our previous work [33] focused on the Kinect calibration and performance
evaluation for 1D dynamic transnational motion. The goal of this study is to complete
the evaluation of the Kinect v1 performance envelope for dynamical displacement eld
56
measurements, including 3D transnational motion with simultaneous rotational/torsional
motion components. Performance characteristics that are evaluated include linearity of
measurements, distortion, noise eects, displacement and amplitude bounds, displacement
accuracy relative to motion frequencies, displacement accuracy relative to direction of
motion of target with respect to sensor, in
uence of lighting conditions, and the eects of
distance between sensor and target.
3.1.3 Scope
The remainder of this study is organized as follows. Section 3.2 describes the data extraction
approach to obtain the Kinect data. Section 3.3 proposes the methodology used to compute
the time history records of dynamic translation and rotation. Section 3.4 presents the
experimental setup for RGB-D data acquisition to measure dynamic displacement of a

exible structure and illustrates the dierent type of sensors and data acquisition system
used in this study. Section 3.5 explains the data calibration and processing procedures for
dierent sensors used in this study. Sections 3.6 and 3.7 discuss the experimental results
and data analysis for Kinect v1 sensor performance under dierent types of dynamic
excitation. The conclusions are summarized in Section 3.8.
3.2 Kinect data extraction
3.2.1 RGB-D sensor calibration and data registration
Before computing displacement and rotation time histories, Kinect camera calibration was
performed to compute intrinsic and extrinsic parameters for the Kinect's color camera and
57
infrared (IR) camera. The camera intrinsic parameters are focal length, principal point,
and distortion coecients. The extrinsic parameters include rotation and translation. The
Camera Calibration Toolbox for MATLAB was used to estimate the intrinsic and extrinsic
parameters [21]. Furthermore, the Kinect's RGB camera and IR camera are located side
by side. However, the eld-of-views are dierent between the two cameras, which leads to
a shift in the location of the pixel coordinates on the color image compared to the depth
image that are taken simultaneously. The Kinect's development software kits (such as
OpenNI and Windows SDK) provide programming routines using predened calibration
parameters, that were deployed to align color and depth images. More details about the
Kinect's cameras calibration process, and RGB and Depth data alignment can be found
in [34].
3.2.2 Target detection and tracking
The Kinect recorded les contain the data that can be used to quantify the dynamic
displacement eld of the structure. ArUco markers were used to track the points of interest
in the color image. The robust marker detection algorithm developed by [58] was used to
extract and locate the markers' corners and determine their pixel coordinates (u
C
;v
C
) in
each color image. When the color and depth images are aligned, the pixel coordinates of a
depth map are identical to the pixel coordinates of a color image. Figure 3.1 shows the
ArUco markers detected in a color image (Figure 3.1a) and mapped to the corresponding
depth image (Figure 3.1b). It should be mentioned that there is a small time-shift between
a pair of color and depth frames since Kinect v1 does not support hardware synchronization.
Many programming libraries (e.g., Kinect SDK, OpenNI, etc.) minimize the time-shift to
58
a few milliseconds. The small time shift can be considered as part of the measurement
error.
3.3 Theory and methods
(a) (b)
Figure 3.1: Extracting points of interest from ArUco markers: (a) in a color image and
mapping to (b) a depth image where the axis of motion parallel is to X-direction.
3.3.1 Displacement calculation
For a pair of aligned color and depth images, the depth value Z
i
of the ith target point
P
i
in the color image can be directly obtained from the depth image using its pixel
coordinates: Z
i
(u
i
C
;v
i
C
) where u
i
C
and v
i
C
are the coordinates of P
i
in the color image
plane. Subsequently, the world coordinates (X
i
;Y
i
;Z
i
) of the point P
i
with respect to the
59
color camera of the Kinect sensor can be computed using the following equations that are
derived from a pinhole camera model [22]:
X
i
=
(u
i
C
c
x
)Z
i
f
x
; (3.1a)
Y
i
=
(v
i
C
c
y
)Z
i
f
y
; (3.1b)
Z
i
=Z
i
(u
i
C
;v
i
C
); (3.1c)
where (c
x
;c
y
) are the principal points, and f
x
and f
y
are the focal lengths of the color
camera in x and y directions on the image plane respectively. The intrinsic parameters
f
x
;f
y
;c
x
; andc
y
are estimated through the camera calibration as discussed in Section 3.2.1.
The measurement of dynamic displacement between two points on two sequential frames
is computed using the pinhole camera model shown in Equation (3.1). ArUco marker
detections, frame timestamp readings, and target points detection and localization in the
color image were performed in C++. MATLAB was used to obtain the depth values,
and compute the world coordinates with respect to the color camera of the Kinect sensor.
The time histories of the positions for the four corners points of the ArUco markers were
stored in accordance with the format shown in Table 3.1. The displacements in X, Y, or Z
direction of a target point can be plotted according to the time history record. Figure 3.2
summarizes the process of displacements calculation using the Kinect sensor.
60
Table 3.1: Storage arrangement for time history for world coordinates of target points
of ArUco marker. Note that for each ArUco marker, the coordinates for its four corner
points are stored.
time point 1 point 2 . . . point 4
t
1
X
11
Y
11
Z
11
X
12
Y
12
Z
12
. . . X
14
Y
14
Z
14
t
2
X
21
Y
21
Z
21
X
22
Y
22
Z
22
. . . X
24
Y
24
Z
24
.
.
.
t
m
X
m1
Y
m1
Z
m1
X
m2
Y
m2
Z
m2
. . . X
m4
Y
m4
Z
m4
RGB image
u
v
Depth image
u+p
v+q
u
v
Depth image
RGB image
Depth video
Z1
Z3
Z
Y
X
Calibration
(RGBD alignment)
Target Detection Target Tracking
Displacement
Calculation
Z4
Z2
RGB video
(u1,v1)
(u2,v2)
(u4,v4)
(u3,v3)
Figure 3.2: Overview of measuring dynamic displacements using the Kinect sensor. The
process of displacement calculation consists of stereo calibration for color and depth
images, target (ArUco markers) detection, tracking and displacement calculation using
Equation (3.1).
61
3.3.2 Rotation calculation
After computing the time history for world coordinates of each marker, the rotation angle
time history of each marker can be computed based on the geometrical relationships shown
in Figure 3.3, using following equations:

j
=Tan
1
(
Y
j1
Y
j2
X
j1
X
j2
); (3.2a)

j
=Tan
1
(
Z
j1
Z
j2
X
j1
X
j2
); (3.2b)
β
( , , ) ( , , )
( , , ) ( , , )
1 2
4 3
Z
X
Y
4
3 1
2
( , , )
( , , )
( , , )
( , , )
(a)
θ ( , , ) ( , , )
( , , ) ( , , )
( , , )
( , , )
( , , )
1 2
4 3
2
4
1
Z
X
Y
(b)
Figure 3.3: Computed angle using Kinect data (a) Pixel-based data; refer to the computed
rotation angle based on Kinect data when the axis of rotation is parallel to the Kinect's
Z -axis (i.e., the depth axis) , and (b) depth-and-pixel-based data; refer to the computed
rotation angle based on Kinect's data when the axis of rotation is parallel to the Kinect
Y -axis.
where
j
is the estimated rotation angle when the axis of rotation is parallel to the Kinect's
Z -axis (i.e., depth axis) at time step t
j
based on world coordinates computed from Kinect
data,
j
is the estimated rotation angle when the axis of rotation is parallel to the Y -axis
62
(i.e., pixel axis) at time step t
j
based on world coordinates computed from Kinect data.
X
j1
,Y
j1
,Z
j1
,X
j2
,Y
j2
, andZ
j2
are the computed world coordinates based on Kinect data
at time stept
j
for marker corner points 1 and 2, respectively. As discussed in Section 3.7.1,
when the computed rotation angle is based on world coordinates X and Y, it is tagged as
"pixel-based" data. However, when the computed angle is based on world coordinates X
and Z, it is tagged as "depth-and-pixel-based" data.
3.4 Experimental setup
3.4.1 Testbed structure
To perform the dynamic tests, a
exible testbed shown in Figure 3.4 was designed and built.
The testbed was equipped with two sensor congurations: (1) the contact-type sensors
setup that was used for calibration purposes (see Figure 3.4a), and (2) the non-contact-type
sensor setup (i.e., Kinects and target markers, see Figure 3.4b). In designing the testbed
structure, some desirable objectives were incorporated in the design plan. The designed
structure can simulate a frame, or a wing-like structure, and to have the natural frequency
of the structure in the range of 1 to 5 Hz. This requirement was due to the fact that
a high-amplitude response is needed to sense and track the response. Furthermore, the
above frequency range is typical bandwidth for civil structures. In addition, the testbed
was designed to keep the second and third modes of the testbed response below 10 Hz, so
as to result in modal interaction (i.e., couple translational and torsional modes for the
random input studies).
63
Acc 1
Acc 2
Acc 3
Acc 4
Acc 5
Acc 6
Acc 7
IMU

X
Y
Z
IMU
Shaker
Module 1 Module 2 Module 3
(a)
X
(front)
Z
(front)
Kinect
(front)

Kinect
(side)

ArUco
Marker
MF1
MF2
MF3
MF4
MS1
MT1
MS2
MS3
MS4

X
(side)
Y
(side)
Z
(side)
Y
(front)
Shaker
Y
(top)
X
(top)
Kinect
(top)

Z
(top)
(b)
Figure 3.4: Sensors locations: (a) accelerometers Acc 1, Acc 2, Acc 5, Acc 6, Acc 7 are
used to measure acceleration in the X-direction; accelerometers Acc 3, Acc 4 are used to
measure acceleration in the Y -direction; and an IMU is used to measure rotation (torsion)
in module 3 around the IMU's y-axis that coincide with the structure's axis of symmetry,
and (b) ArUco markers deployed on test-bed structure and the Microsoft Kinects used;
Kinect
side
, Kinect
front
, Kinect
top
(note that the depth axis for Kinect
side
is perpendicular
to the shaker direction of motion, the depth axis for Kinect
front
is parallel to the shaker
direction motion, and the depth axis for Kinect
top
is perpendicular to the plane of motion
[i.e., XY plane].
64
The nal design consisted of 3 modular sections each with dimensions 205 140 300
mm as shown in Figure 3.5. As can be seen, each module has a wooden slab with 6.4 mm
thickness, and four steel columns with 3.2 mm diameter. A mass of 2 Kg was attached to
the second module to adjust the mass distribution in order to keep the torsional mode
of the testbed response below 10 Hz. The shaker was connected to a rotating round
wooden table with a diameter of 134 mm via a 407-mm aluminum arm. The shaker and
round table form a simple slider-crank mechanism to convert the linear motion (shaker) to
circular motion (round table). The structure was congured in two dierent ways: (1) the
testbed was mounted on an electromagnetic long-stroke shaker (see Figure 3.5a), and (2)
the testbed was mounted on a round rotating wooden table (see Figure 3.5b). The exciter
could be modied to generate arbitrary dynamic scenarios such as harmonic, swept-sine
and random excitation forces.
The frequency of the rst three lateral mode shapes were estimated through the Fast
Fourier transform (FFT) analysis of the recorded acceleration from sensor Acc 1 and Acc
3, as shown in Figure 3.6. The corresponding mode shapes from nite element analysis
(FEA) model are displayed in Figure 3.7. The rst mode is primarily a bending mode
as expected. The second mode is a pure torsional mode as shown in Figure 3.7b. The
frequency of the structure shown in Figure 3.7c is 7.3 Hz, which is at the lower end of the
target range, since the model includes a concentrated mass at the second module. The
frequency values for the rst three mode shapes from the experimental data and nite
element analysis are listed in Table 3.2.
65
(a) (b)
Figure 3.5: Overview of the experimental setups: (a) the experimental setup used for
the evaluation of harmonic translation, harmonic rotation, static rotation and random
translation, and (b) the experimental setup used for the evaluation of random rotation,
including slider-crank assembly (uniaxial shaker connected to round rotating table by rigid
arm to convert linear motion to circular motion).
0 2 4 6 8 10
Frequency (Hz)
0
0.25
0.5
0.75
1
1.25
1.5
|Y(f)|
(a)
0 2 4 6 8 10
Frequency (Hz)
0
0.05
0.1
0.15
0.2
0.25
|Y(f)|
(b)
Figure 3.6: Testbed frequency response based on acceleration data acquired from ac-
celerometer sensors (a) Acc 1 located at module 3, and (b) Acc 3 located at module 3
(note that the amplitude scales are signicantly dierent).
66
(a) (b) (c)
Figure 3.7: Mode shapes from FEA model: (a) the rst mode of response at 1.3 Hz, (b)
the second mode of response at 4.3 Hz, and (c) the third mode of response at 7.3 Hz. Note
that the Abaqus FEA software was used to generate the model.
Table 3.2: Comparison of natural frequencies from FEA model and experimental measure-
ments.
! [Hz]
Mode Experimental FEM ! [% Dierence]
1 1.54 1.46 5.48
2 4.25 4.10 3.66
3 7.30 7.68 4.95
67
3.4.2 Consumer-Grade RGB-D camera
The Kinect v1 shown in Figure 3.8f was used in this study as the non-contact type sensor.
The Kinect main components are: a color complementary metal-oxide-semiconductor
(CMOS) sensor, an infrared CMOS sensor, and an infrared projector, to capture 640480-
pixel RGB images and generate 640480 pixel depth maps at 30 fps using PrimeSense's
light coding technology. The Kinect v1 can run on dierent platforms (i.e., Windows,
Linux, Mac, etc.), and has various choices of programming tools (i.e., Microsoft SDK,
OpenNI, OpenKinect, etc.).
Three Kinects were mounted on adjoining structures. According to [87], the error in depth
measurements increases quadratically as the sensor-object distance increases. So, the
Kinects were placed about 1000 mm from the testbed structure to minimize the depth
error that close to the Kinect's minimum range of 800 mm. To reduce the rolling shutter
distortion eect [34] in recorded Kinect data under dynamic motion, two lighting panels
were mounted in front and on the side of the testbed structure.
The ArUco markers of size (100 mm 100 mm) were attached on each
oor of the
testbed structure. The size of markers was chosen relatively large compare to the structure
dimensions for calibration process. The ArUco module is based on the ArUco library,
a popular library for detection of square ducial markers developed by [58]. The outer
corners of the marker represented the points of interest to be detected. One of the main
advantages of the chosen markers is that each ArUco marker has a unique pattern, that
facilitates the marker detection process.
68
The approximate location of the Kinect sensors, ArUco markers as well as the markers
nomenclature used throughout this study are shown in Figures 3.4b, 3.8a, 3.8b, and 3.8c.
The Kinect
front
was used to detect markers MF1, MF2, MF3, and MF4, the Kinect
side
was used to detect markers MS1, MS2, MS3, and MS4, and the Kinect
top
for marker MT1.
3.4.3 Contact-type sensors
Three contact-type sensors were deployed to serve as ground truth to validate the accuracy
of the Kinects' data. An LVDT transducer, Schaevitz
TM
Sensors HR 4000 was used to
record the displacement of the shaker stroke. Seven Endevco 7290E variable capacitance
accelerometers were used to measure acceleration at dierent locations of the testbed
structure. The sensors had a full-scale range of +/-10 g with a frequency bandwidth
of 0-500 Hz. In addition, an inertial measurement unit (IMU), Yost Labs 3-Space
TM
Sensor Micro USB, was deployed at the third module. The approximate location of the
accelerometers, IMU, as well as the sensor numbering used throughout this study are
displayed in Figures 3.4a, 3.8d, and 3.8e.
3.4.4 Data acquisition systems
This study used three data acquisition systems. The rst one was for controlling the shaker,
collecting accelerometers and LVDT measurements using NI LabVIEW. The second one
was for acquiring IMU data (accelerometer and gyroscope data) using the YEI 3-Space
Sensor Software Suite. The third one was for Kinect data acquisition. The Kinect data
acquisition software was developed in C/C++ language using OpenNI (ONI), OpenCV
and ArUco libraries, and operated on a Microsoft Windows 7 platform. The color and
69
(a) (b) (c)
(d) (e) (f)
Figure 3.8: Test apparatus and components: (a) the Kinect
front
xture, (b) the Kinect
side
xture, (c) the Kinect
top
xture, (d) the 4 accelerometers and IMU deployed on module 3,
(e) the 2 Kg mass attached to module 2, and (f) the Microsoft Kinect sensor. Note that
the size of markers (100 mm 100 mm) are relatively large for calibration process, while
smaller markers can be used.
70
depth images were aligned with reasonable accuracy using the OpenNI library. The Kinect
data acquisition software saved aligned color and depth frames together into a video le
using the OpenNI le format. The frame numbers and timestamps were also saved into a
text le.
3.5 Data preprocessing and verication
3.5.1 Kinect
After extracting Kinect raw data from the recorded ONI video les, the data was processed
through two steps. The Kinect data was ltered to remove DC bias, reduce noise, and
smooth the signal. Two FFT digital lters were applied to the raw data: a high-pass
FFT lter was used to remove the DC bias, and a low-pass FFT lter was used to clean
high-frequency noise.
3.5.2 Accelerometers
The computed displacements from acceleration data were used as ground truth for the
evaluation of Kinect displacement data at each
oor. In order to obtain the displacement
and velocity time histories, the acceleration records were windowed, detrended, band-pass
ltered and numerically integrated for each accelerometer. Figure 3.9 shows a typical time
history of measured accelerations and computed displacements for an accelerometer. The
computed displacements were calibrated with direct displacement measurements from
the LVDT attached at the base to evaluate the eects of bias and drift due to double
integration [153, 154]. Figure 3.9b illustrates the aligned computed displacement and
71
LVDT measurements. It can be seen that the two signals match well and the computed
root mean square (RMS) error was less than 10% for general random signal. However, for
harmonic signal, the RMS error was less than 2% as it is discussed in Section 3.6.1.
35 36 37 38 39 40
Time [s]
-6
-4.5
-3
-1.5
0
1.5
3
4.5
6
Acceleration [m/s
2
]
(a)
35 36 37 38 39 40
Time [s]
-20
-15
-10
-5
0
5
10
15
20
Displacement [mm]
LVDT
Computed Disp.
(b)
Figure 3.9: Sample of (a) recorded data from an accelerometer (Acc 7), and (b) corre-
sponding computed displacement time history and LVDT measurements.
3.5.3 Inertial measurement unit (IMU)
MEMS IMU consists of tri-axial accelerometers, gyroscopes, and magnetometers. MEMS
gyroscopes are typically angular rate gyroscopes that are designed to measure the angular
rate. To get the rotation angle using a MEMS rate gyroscope, it is required to integrate
the measured angular rate with respect to time. This integration procedure was calibrated
to evaluate the error in computed angular time-histories due to bias and noise in the
acquired angular rate signal [137, 128]. The angular rate records were windowed, detrended,
band-pass ltered and numerically integrated in order to obtain the angular time-histories
and to reduce the drift and signal noise.
72
The experimental slider-crank setup shown in Figure 3.5a was used to calibrate the
gyroscope. Figure 3.10a illustrates the slider-crank mechanism. The angle of rotation is
dened as follows:

slider crank
=Cos
1
(
(Lx)
2
+ 2R
2
L
2
)
2
p
(Lx)
2
+R
2
)R
Tan
1
(
Lx
R
); (3.3)
where
slide crank
is the angle of rotation of the crank, L is the length of the rod, and x
is the shaker displacement measured by the LVDT. The IMU was xed on wooden cube
attached to the round table. Two sets of tests were performed based on the excitation
type. For the rst set, the shaker was excited with 2 Hz harmonic excitation, repeated
exactly three times to calibrate the gyroscope rotation time history around the IMU local
axes (see Figure 3.10b). Figure 3.11 displays a comparison between the computed angle of
rotation and the ground truth (slider-crank). The error analysis based on the harmonic
excitation is summarized in Table 3.3. For the x and z directions, the normalized errors
are approximately 5%. However, for the y direction measurements, the error is less than
3%.
The second set of tests were conducted to measure low-frequency random vibrations using
the IMU gyroscope. Three distinct input random signals with relatively low root mean
square (RMS) levels were generated to excite the shaker (approximately 2.3
o
). The three
input signals were sampled from a Gaussian distribution and ltered by a low-pass lter to
remove high-frequency components above 10 Hz. The random rotation for the slider-crank
and the gyroscope were computed. Figure 3.12 illustrates the computed random rotation
for the slider-crank and the gyroscope for the IMU's x and y directions. Figure 3.13
shows the comparisons of the probability density functions (PDFs) estimated based on
73
R
m
B
t=0
LVDT
q
Round table
Arm of the
slider-crank
(a)
X
Z
Y
(b)
Figure 3.10: Sketch for (a) slider-crank geometry, and (b) IMU axis.
15 15.5 16 16.5 17 17.5 18
Time [s]
-15
-10
-5
0
5
10
15
Angle [
o
]
GT
Gyro
X
(a)
15 15.5 16 16.5 17 17.5 18
Time [s]
-15
-10
-5
0
5
10
15
Angle [
o
]
GT
Gyro
Y
(b)
Figure 3.11: Sample aligned IMU-Gyroscope and ground truth (GT) data under harmonic
excitation (2.0 Hz, 16.0
o
): (a) x-direction, and (b) y-direction. The GT is the computed
angle using the slider-crank mechanism.
74
the computed angles from the slide-crank (ground truth) and gyroscope by integration.
For instance, there is a good correlation between the two computed quantities. Three
metrics were deployed to quantify the accuracy of random rotation measurements for the
IMU-gyroscope sensor with respect to the ground truth data: the normalized error, the
mean error for the two PDFs, and standard deviation error for the two PDFs, which are
dened as follows:
Normalized error =
kX
GT
X
IMU
k
2
kX
GT
k
2
; (3.4)
Mean error =

GT

IMU

GT
; (3.5)
Standard deviation error =

GT

IMU

GT
; (3.6)
wherek:k
2
is the Euclidean norm, X
GT
is the time history of the ground truth rotation,
X
IMU
is the time history of the computed gyroscope rotation measurements,
GT
and
GT
are the mean and standard deviation for the PDFs of the ground truth data, and
IMU
and
IMU
are the mean and standard deviation for the PDFs of the computed gyroscope
rotation measurements, respectively. The results of the quantitative error analyses are
summarized in Table 3.3. On the whole, the normalized error, mean error, and standard
deviation error decrease when the RMS amplitude levels (2.00
o
, 2.35
o
, 2.86
o
) increase.
For the random vibration tests, the normalized error analyses show relatively acceptable
performance as the rotation angle increases. This can be explained by noting that as the
signal RMS increases, the bias and noise eects decrease. Overall, the error analyses of
the random tests are acceptable for the problem of interest in this study.
75
15 15.5 16 16.5 17 17.5 18
Time [s]
-5
-2.5
0
2.5
5
Angle [
o
]
GT
Gyro
X
(a)
15 15.5 16 16.5 17 17.5 18
Time [s]
-5
-2.5
0
2.5
5
Angle [
o
]
GT
Gyro
Y
(b)
Figure 3.12: Sample aligned IMU-Gyroscope and GT data under random excitation: (a)
x-direction (RMS of the GT signal = 2.35
o
), and (b) y-direction (RMS of the GT signal
= 2.00
o
). The GT is the computed angle using the slider-crank mechanism.
-10 -5 0 5 10
Angle [
o
]
0
0.05
0.1
0.15
0.2
0.25
pdf
GT
Gyro
X
(a)
-10 -5 0 5 10
Angle [
o
]
0
0.05
0.1
0.15
0.2
0.25
pdf
GT
Gyro
Y
(b)
-10 -5 0 5 10
Angle [
o
]
0
0.05
0.1
0.15
0.2
0.25
pdf
GT
Gyro
X
(c)
Figure 3.13: PDFs of the IMU-Gyroscope measurements for the three directions: (a)
IMU-x-direction (RMS of the GT signal = 2.35
o
), (b) IMU-y-direction (RMS of the GT
signal = 2.00
o
), and (c) IMU-z-direction (RMS of the GT signal = 2.83
o
). The GT is the
computed angle using the slider-crank mechanism.
Table 3.3: Comparison of normalized error, mean error, and standard deviation error
under harmonic and random measurements of angle of rotation obtained from IMU.
IMU-x-direction IMU-y-direction IMU-z-direction
Harmonic Random Harmonic Random Harmonic Random
(RMS = 2.35
o
) (RMS = 2.00
o
) (RMS = 2.83
o
)
Normalized error (%) 5.81 9.82 1.98 10.28 5.08 6.10
Mean error (%) - 1.52 - 2.2 - 0.90
Std error (%) - 1.24 - 1.63 - 0.91
76
3.5.4 Data synchronization and alignment
Since dierent computers were used to record the Kinect videos and ground truth raw
data (i.e., acceleration or rotation rate), the Kinect video recording and ground truth data
acquisition started at dierent times. In order to compare the two data measurements, a
data alignment procedure was performed. The measurements collected from the Kinect
sensor and the ground truth transducer had similar waveforms; hence, a cross-correlation
technique can be used to estimate the time delay between the two signals by measuring
their similarity. Subsequently, the two signals were synchronized by shifting one of them
according to the time delay. It should be noted that the Kinect, accelerometers and IMU
data were acquired at dierent sampling rates. The Kinect mean sampling rate was 25
Hz, however for the accelerometers and IMU the sampling rate was 200 Hz. To enhance
accuracy of the synchronization procedure using the cross-correlation approach, the two
signals were resampled at a higher sampling rate of 1000 Hz with spline interpolation
before data alignment, which also smoothed the peaks to obtain better alignment results.
Figure 3.14 illustrates an example of data alignment after performing cross-correlation.
3.6 Experimental validation for translational motion
3.6.1 Harmonic test
The RGB-D camera (Kinect v1) was used to acquire harmonic translational motion with
dierent combinations of frequency and peak amplitudes. The excitation frequency was
chosen based on the nite element model results shown in Figure 3.7. In order to get a pure
displacement motion, the testbed was excited by a sinusoidal signal with 1.3 Hz frequency
77
15 16 17 18 19 20
Time [s]
-15
-10
-5
0
5
10
15
Displacement [mm]
(a)
15 16 17 18 19 20
Time [s]
-15
-10
-5
0
5
10
15
Displacement [mm]
GT Kinect
(b)
Figure 3.14: Kinect data post-processing: (a) raw depth data, and (b) aligned Kinect and
GT data after applying the Kinect post-processing approach that consists of Kinect data
ltering, smoothing and alignment.
corresponding to a cantilever mode shape (i.e., rst mode). The harmonic motion of the
testbed structure was recorded by the three Kinect sensors. Therefore, there were two
cases: (1) the direction of motion was perpendicular to the depth axis (i.e., z
side
-axis) of
the Kinect
side
; (2) the direction of motion was parallel to the depth axis (i.e., z
front
-axis)
of the Kinect
front
(see Figure 3.4b).
For the rst case, the z-direction (z
side
) displacements remained steady while the x-
direction (x
side
) displacements were variable (see Figure 3.15a). For the second case, the
z-direction (z
front
) displacements were variable while the x-direction (x
front
) displacements
were constant (see Figure 3.15b). Since the structure had no vertical displacement, all
of the y-direction measurements for Kinect
side
and Kinect
front
were nearly constant. To
dierentiate between the two scenarios, the rst case was called "depth-based" measurement
since it was dominated by the depth value. Similarly, the second case was named "pixel-
based" measurement for it was governed by the color pixel coordinates. Figure 3.16a
illustrates the pixel-based measurements for all
oors. Figure 3.16b shows the depth-based
78
measurements for all
oors. It can be seen that the pixel and depth based measurements
level are almost similar.
20 21 22 23 24 25
Time [s]
-60
-40
-20
0
20
40
60
Displacement [mm]
X-dir Y-dir Z-dir
(a)
20 21 22 23 24 25
Time [s]
-60
-40
-20
0
20
40
60
Displacement [mm]
X-dir Y-dir Z-dir
(b)
Figure 3.15: Sample measurements for the structure
oors (1.3 Hz): (a) pixel-based
measurements (Kinect
side
) based on marker MS1, and (b) depth-based measurements
(Kinect
front
) based on marker MF1. Note that x, y, z directions refer to the Kinects'
axes.
20 21 22 23 24 25
Time [s]
-60
-40
-20
0
20
40
60
Displacement [mm]
4
th
floor 3
rd
floor 2
nd
floor 1
st
floor
(a)
20 21 22 23 24 25
Time [s]
-60
-40
-20
0
20
40
60
Displacement [mm]
4
th
floor 3
rd
floor 2
nd
floor 1
st
floor
(b)
Figure 3.16: Sample measurements for the structure
oors (1.3 Hz): (a) pixel-based
measurements (Kinect
side
) based on markers MS1, MS2, MS3, MS4 attached at 4
th

oor, 3
rd

oor, 2
nd

oor, and 1
st

oor, respectively; and (b) depth-based measurements
(Kinect
front
) based on markers MF1, MF2, MF3, MF4 attached at 4
th

oor, 3
rd

oor,
2
nd

oor, and 1
st

oor, respectively. Note that the response for the 3
rd
and 4
th

oors are
almost similar due to the mass attached at the 3
rd

oor.
The ground truth measurement in this test was the computed displacement from recorded
acceleration data. This process was calibrated rst using the LVDT attached to the small
79
shaker. The RMS error between the processed acceleration and the LVDT was less than
2%. Figure 3.17a illustrates the superimposed direct displacement measurements (LVDT)
versus the indirect measurements (processed acceleration), Kinect depth data (the front
marker (MF4)), and Kinect pixel data (side marker (MS1)) at the rst
oor. It can be
seen that the LVDT, double integrated acceleration, and the Kinect data match perfectly.
Figure 3.17b compares the displacement values for the fourth
oor based on the direct
measurement of the displacement of the three markers (MF1, MS1, MT1), and the ground
truth obtained from acceleration. It can be seen that the displacement measurements from
the three Kinects and the ground truth measurement completely match, with an RMS
error less than 4% with respect to the ground truth data.
20 21 22 23 24 25
Time [s]
-60
-40
-20
0
20
40
60
Displacement [mm]
MF4 MS4 LVDT GT
(a)
20 21 22 23 24 25
Time [s]
-60
-40
-20
0
20
40
60
Displacement [mm]
MF1 MS1 MT1 GT
(b)
Figure 3.17: Superimposed plots for the Kinects data versus GT measurements (1.3 Hz):
(a) rst
oor, and (b) fourth
oor. The GT is the computed displacement from acceleration
records.
80
For each test, the normalized error between the Kinect and ground truth data (computed
displacement via double integration) were computed for the four corner points of the target
markers pattern using the following equation:
Normalized error =
kX
GT
X
Kinect
k
2
kX
GT
k
2
; (3.7)
wherek:k
2
is the Euclidean norm, X
GT
is the array corresponding to the time history
of the sampled ground truth displacement, and X
Kinect
is the processed displacement
time history for the Kinect sensor. Since the estimated normalized errors are similar
for the four target points of each ArUco marker, the mean results for each marker are
shown in Figure 3.18, that shows one set of tting curves representing the accuracy of the
displacement measurements for dierent displacement values for dierent
oor, that were
estimated using Equation 3.7 under the scenarios discussed below.
0 10 20 30 40 50 60
Displacement [mm]
2
4
6
8
10
Normalized error [%]
Depth-based (front)
Pixel-based (side)
Figure 3.18: Depth- and pixel-based measurements errors (1.3 Hz).
For the Depth-based measurements, the normalized errors remain at the same level of
about 5% that is within the expected range based on [33]. However, for the Pixel-
based measurements, the image sensors when capturing images of moving objects [3,
162]. When the displacement amplitude increases under constant frequency (i.e., average
81
speed increased), severer image distortion decreases the accuracy of horizontal (x-axis)
displacement measurements [33].
3.6.2 Random test
Since in practical applications the excitation of interest is not simply harmonic, additional
tests were conducted to measure low-frequency random response using the Kinect sensor.
The structure was excited by a random signal having 6.81 mm RMS level. The signal
was sampled from Gaussian distribution, and was ltered by a low-pass lter to remove
high-frequency components above 10 Hz. Based on the ground truth, the RMS of the
structure displacement response was 16.42 mm, 15.96 mm, 10.01 mm, and 6.81 mm for the
fourth, third, second, and rst
oor, respectively. The ground truth measurements in this
experiment were the calculated displacements from the recorded acceleration data. The
experimental data was classied into three groups according to the response RMS levels.
The random motions of the structure were captured by the three Kinect sensors. The data
collected by the Kinect
side
and Kinect
top
were the pixel-based measurement. However,
the data collected by the Kinect
front
was the depth-based measurement. Figures 3.19a,
3.19c, and 3.19e show the pixel-based measurements for the three dierent RMS levels,
and Figures 3.19b, 3.19d, and 3.19f illustrate the depth-based measurements for the three
dierent RMS levels.
The dierences between sampled Kinect and ground truth measurements were compared
using PDFs for three dierent RMS amplitude levels (16.42 mm, 10.01 mm, and 6.81 mm).
The PDFs were computed based on the normal kernel density estimation. Figure 3.20
illustrates the comparisons of the PDFs computed with the Kinect and ground truth data
82
20 21 22 23 24 25
Time [s]
-40
-30
-20
-10
0
10
20
30
40
Displacement [mm]
GT
Kinect
(a)
20 21 22 23 24 25
Time [s]
-40
-30
-20
-10
0
10
20
30
40
Displacement [mm]
GT
Kinect
(b)
20 21 22 23 24 25
Time [s]
-40
-30
-20
-10
0
10
20
30
40
Displacement [mm]
GT
Kinect
(c)
20 21 22 23 24 25
Time [s]
-40
-30
-20
-10
0
10
20
30
40
Displacement [mm]
GT
Kinect
(d)
20 21 22 23 24 25
Time [s]
-40
-30
-20
-10
0
10
20
30
40
Displacement [mm]
GT
Kinect
(e)
20 21 22 23 24 25
Time [s]
-40
-30
-20
-10
0
10
20
30
40
Displacement [mm]
GT
Kinect
(f)
Figure 3.19: Depth- and pixel-based measurements for three RMS levels: (a) pixel-based
4
th

oor (marker MS1; RMS = 16.42 mm), (b) depth-based 4
th

oor (marker MF1; RMS
= 16.42 mm), (c) pixel-based 2
nd

oor (marker MS3; RMS = 10.01 mm), (d) depth-based
2
nd

oor (marker MF3; RMS = 10.01 mm), (e) pixel-based 1
st

oor (marker MS4; RMS
= 6.81 mm), and (f) depth-based 1
st

oor (marker MF1; RMS = 6.81 mm). The GT is
the computed displacement from acceleration records.
83
for pixel-based measurement. Figure 3.21 shows the comparisons of the PDFs estimated
with the Kinect and ground truth data for depth-based measurement. Three indices were
used to quantify the accuracy of random displacement estimations for the Kinect sensor
with respect to the ground truth data: the normalized error, the mean error for the two
PDFs, and standard deviation error for the two PDFs that are dened as follows:
Normalized error =
kX
GT
X
Kinect
k
2
kX
GT
k
2
; (3.8)
Mean error (PDFs of displacement or rotation) =

GT

Kinect

GT
; (3.9)
Standard deviation error (PDFs of displacement or rotation) =

GT

Kinect

GT
;
(3.10)
wherek:k
2
is the Euclidean norm,X
GT
is the time history of the ground truth displacements,
X
Kinect
is the time history of the Kinect measurements,
GT
and
GT
are the mean and
standard deviation for the PDFs of the ground truth data, and
Kinect
and
Kinect
are the
mean and standard deviation for the PDFs of the Kinect measurements, respectively. The
results of the quantitative error analysis are summarized in Table 3.4.
Overall, the normalized error, mean error, and standard deviation errors decrease when
the RMS amplitude levels (6.81 mm, 10.01 mm, and 16.42 mm) increase. For the random
vibration tests, the normalized error analysis shows relatively similar performance for
the "pixel-based" and "depth-based" measurements. However, for mean error shows
relatively low performance for the depth-based measurements, especially for the small
RMS excitation level (6.811 mm). As mentioned earlier, the depth measurement error of
84
-50 0 50
Displacement [mm]
0
0.01
0.02
0.03
0.04
0.05
pdf
GT
Kinect
(a)
-50 0 50
Displacement [mm]
0
0.01
0.02
0.03
0.04
0.05
pdf
GT
Kinect
(b)
-50 0 50
Displacement [mm]
0
0.01
0.02
0.03
0.04
0.05
pdf
GT
Kinect
(c)
Figure 3.20: PDFs of the estimated Kinect displacements based on pixel-based measurement
versus GT: (a) RMS = 6.81 mm (marker MS4), (b) RMS = 10.01 mm (marker MS3),
and (c) RMS = 16.42 mm (marker MS1). The GT is the computed displacement from
acceleration records.
-50 0 50
Displacement [mm]
0
0.01
0.02
0.03
0.04
0.05
pdf
GT
Kinect
(a)
-50 0 50
Displacement [mm]
0
0.01
0.02
0.03
0.04
0.05
pdf
GT
Kinect
(b)
-50 0 50
Displacement [mm]
0
0.01
0.02
0.03
0.04
0.05
pdf
GT
Kinect
(c)
Figure 3.21: PDFs of the estimated Kinect displacements based on depth-based measure-
ment versus GT: (a) RMS = 6.81 mm (marker MF4), (b) RMS = 10.01 mm (marker
MF3), and (c) RMS = 16.42 mm (marker MF1). The GT is the computed displacement
from acceleration records.
Table 3.4: Comparison of normalized error, mean error, and standard deviation error
under depth-based and pixel-based measurements.
RMS = 6.81 mm RMS = 10.01 mm RMS = 16.42 mm
Depth (z) Pixel (x) Depth (z) Pixel (x) Depth (z) Pixel (x)
Normalized error (%) 21.80 18.74 15.10 15.50 14.46 15.02
Mean error (%) 18.00 5.05 10.89 5.50 5.71 5.02
Std error (%) 1.77 2.02 0.5 1.06 0.80 3.50
85
Kinect v1 is approximately 3 mm for the distance of 1 m [87, 152]. Consequently, when a
6.81-mm RMS signal is compared to a larger RMS signal, the signal-to-noise ratio of the
former one is smaller than the latter one, leading to larger error values. Overall, the error
analyses of the random tests are in agreement with the ndings in Section 3.6.1 for the
harmonic test and [33].
3.7 Experimental validation for rotational motion
3.7.1 Static test
The RGB-D camera (Kinect v1) was deployed to quantify the structure rotation un-
der very low frequency, almost (0 Hz) of the testbed structure around the global axis
Z. To perform the test, the structure was rotated through dierent angles of rotation
(5
o
; 10
o
; 15
o
; 20
o
; 25
o
; 30
o
; 35
o
; 40
o
, and 45
o
) around the global axis Z.
The angle of rotation of the testbed structure was measured by the Kinect sensors in
three dierent directions (see Figure 3.4b). Since the structure had no vertical motions,
for the Kinect
top
the z-direction motion is steady, while the x- and y-direction of motion
varied according to the structure's rotation. This case was called "pixel-based" rotational
measurements since it was dominated by color pixel coordinates as illustrated in Figure 3.3a.
For the Kinect
side
and Kinect
front
, the y-direction measurements with respect to Kinect
coordinates are almost constant, and the angle of rotation evaluation was dominated by
the x- and z-direction of motions. This case is referred to as "depth-and-pixel-based"
rotational measurements as shown in Figure 3.3b.
86
The angles of rotation estimated based on the Kinect sensor data were compared to the
corresponding ground truth measurement. The structure was rotated around its axis of
symmetry manually with the use of a protractor. For each test, the normalized error
between Kinect and ground truth data were computed for the nine markers attached to
the structure using the following equation:
Normalized error =
k
GT

Kinect
k
2
k
GT
k
2
; (3.11)
wherek:k
2
is the Euclidean norm,
GT
is the array corresponding to the ground truth
measurements,
Kinect
is the processed structure rotation time history for the Kinect sensor.
Since the estimated normalized errors are similar for the front markers (MF1, MF2, MF3,
MF4) and side markers (MS1, MS2, MS3, MS4), the mean normalized error for each set of
markers are evaluated. Figure 3.22a and 3.22b show two sets of tting curves representing
the relationship between the accuracy of the structure rotation and angle of rotation for
dierent Kinects.
Figure 3.22a illustrates the error analysis results for the angle of rotation based on depth-
and-pixel based rotational measurements. For the front and side markers sets, the error
analysis is showing the same behavior and trend. For angles of rotation greater than 5
o
,
the normalized errors are approximately 5%. However, for small angles (less than 5
o
) the
normalized error increases due to the precision of Kinect measurements. According to
[87], the depth measurement error of Kinect v1 is approximately 3 mm for a distance of 1
m. This corresponds to an approximate error of a 1.5
o
for 100 mm marker. Figure 3.22b
shows the tting curves of the mean normalized errors for depth-and-pixel-based rotational
measurements (mean error for front and side markers sets), and normalized errors for
87
pixel-based rotational measurements (marker MT1). For the angle of rotation estimations
based on pixel measurements, the normalized errors stay below 3%.
0 10 20 30 40 50
Angle [
o
]
0
2
4
6
8
10
Normalized error [%]
Depth-and-pixel-based front
Depth-and pixel-based side
(a)
0 10 20 30 40 50
Angle [
o
]
0
2
4
6
8
10
Normalized error [%]
Depth-and-pixel-based
Pixel-based
(b)
Figure 3.22: Normalized estimation errors: (a) depth-and-pixel-based rotational measure-
ments (based on the data acquired from the front markers set (MF1, MF2, MF3, MF4) and
the side markers set (MS1, MS2, MS3, MS4), and (b) pixel-based rotational measurements
(marker MT1), depth-and-pixel-based rotational measurements (mean of the side and front
markers sets).
3.7.2 Harmonic test
For the second test, the structure was vibrated through a harmonic excitation with 4.3
Hz frequency and amplitude 13 mm. The test frequency and amplitude were chosen in
order to ensure a pure torsional motion for the structure that matched the second mode
shape with a maximum angle between (5
o
and 10
o
) based on the results in Section 3.7.1.
Figure 3.23 illustrates the dierence in motion between the two harmonic tests performed
at the designed frequencies 1.3 and 4.3 Hz for almost the same base excitation amplitude
of 13 mm. Figure 3.24 shows two frames extracted from the ONI le for the top marker
(MT1) during torsional dynamic test showing to the initial and maximum rotations of the
fourth
oor.
88
(a) (b) (c)
Figure 3.23: Testbed mode shapes: (a) the initial condition, (b) the rst mode of response
at 1.3 Hz, and (c) the second mode of response at 4.3 Hz. It can be seen that the rst
mode of response is pure translational and the second mode is a pure torsional.
(a) (b)
Figure 3.24: Tracking of top marker (MT1) during torsional test: (a) the initial position,
(b) the maximum torsion.
89
The torsional deformation eld of the testbed structure was recorded by the three Kinect
sensors. Since one IMU was used and attached to the fourth
oor (location of maximum
torsional angle), the emphasis was on the data collected by the three Kinects for the three
markers (MF1, MS1, MT1) for the fourth
oor (see Figure 3.4). Figure 3.25 illustrates
the dierence between the pure translation corresponding to the rst mode shape and
pure torsional motion re
ecting the second mode shape. Thus, by tracking marker MS1,
based on data collected from Kinect
side
under 1.3 Hz test, and marker MT1, based on
data collected from Kinect
top
under 4.3 Hz test. The black dots represent the center of
the marker.
-300 -250 -200 -150 -100
X
side
[mm]
-340
-320
-300
-280
-260
-240
-220
Y
side
[mm]
(a)
-100 -50 0 50
X
top
[mm]
-40
-20
0
20
40
60
80
100
Y
top
[mm]
(b)
Figure 3.25: Markers tracking comparison for translational and rotational motion: (a)
transnational motion (marker MS1, 1.3 Hz test), and (b) rotational motion (marker MT1,
4.3 Hz test).
The ground truth measurement in this test was the calculated angle of rotation from
the recorded gyroscope data. To this end, numerical integration was used to obtain the
torsional angle time history for the recorded angular velocity. Figure 3.26a illustrates the
torsional measurement based on pixel-based data (marker MT1) versus the gyroscope data.
It can be seen that the two signals almost match where the RMS error is 9.71%. The
90
dynamic torsion compared to static rotation has greater error due to the rolling shutter
eect explained in Section 3.6. However, for depth-and-pixel-based rotational measurements
(markers MS1, MF1), the RMS error was found to be 15.71% for the discussed case. The
depth-and-pixel-based data introduces higher RMS error since the measurements combine
the eect of rolling shutter from pixel-based data and the noise eect from the depth-based
measurement. Overall, the error analyses of the harmonic test agree with what have been
discussed in Section 3.7.1 for the static tests. Figure 3.26b shows the dynamic torsional
measurements obtained from the three Kinects for the structure's fourth
oor.
20 20.5 21 21.5 22
Time [s]
-8
-6
-4
-2
0
2
4
6
8
Angle of rotation [
o
]
GT MT1
(a)
20 20.5 21 21.5 22
Time [s]
-8
-6
-4
-2
0
2
4
6
8
Angle of rotation [
o
]
MT1 MS1 MF1
(b)
Figure 3.26: Sample torsional measurements for the structure fourth
oor (4.3 Hz):
(a) marker MT1 (Kinect
top
), and (b) marker MT1 (Kinect
top
), MS1 (Kinect
side
), MF1
(Kinect
front
); GT is the computed angle of rotation from IMU data
3.7.3 Random test
The experimental setup shown in Figure 3.5b was used to quantify the structure's rotation
around Z axis under random rotational excitation. A separate test was performed for
rotation to ensure the possibility of applying dierent random excitations with dierent
RMS amplitudes. In order to get a pure rotational motion, the structure was attached
91
to a rotating table linked to the linear exciter (shaker). The structure was excited by
two random signals with two RMS levels similar to the displacement test. The RMS of
structure rotation response was 5.46
o
and 1.85
o
, respectively for the fourth
oor based
on the ground truth data. The ground truth measurement in this test was the computed
angle of rotation time history from the recorded angle rate by the IMU. The experimental
data could be classied into two groups according to RMS levels. The random motions
of the structure were captured by the three Kinect sensors in three dierent orientations.
The data collected by Kinect
side
and Kinect
front
were pixel-and-depth-based measurement.
However, the data collected by the Kinect
top
was the pixel-based measurement. Figure 3.27
illustrates the pixel-and-depth-based measurements under the two dierent RMS levels
while Figure 3.28 shows the pixel-based measurements under the two dierent RMS levels
for the testbed's fourth
oor.
20 21 22 23 24 25
Time [s]
-30
-20
-10
0
10
20
30
Angle [
o
]
GT Kinect
(a)
20 21 22 23 24 25
Time [s]
-30
-20
-10
0
10
20
30
Angle [
o
]
GT Kinect
(b)
Figure 3.27: Sample torsional measurements for the structure's fourth
oor based on
marker MF1 (Kinect
front
) for two RMS levels: (a) RMS = 1.85
o
, and (b) RMS = 5.46
o
;
GT is the computed angle of rotation from IMU data.
The dierences between sampled Kinect and ground truth measurements were compared
using PDFs for the two dierent RMS amplitude levels (5.46
o
and 1.85
o
). The PDFs
were computed based on the normal kernel density estimation. Figure 3.29 shows the
92
20 21 22 23 24 25
Time [s]
-30
-20
-10
0
10
20
30
Angle [
o
]
GT Kinect
(a)
20 21 22 23 24 25
Time [s]
-30
-20
-10
0
10
20
30
Angle [
o
]
GT Kinect
(b)
Figure 3.28: Sample torsional measurements for the structure's fourth
oor based on
marker MT1 (Kinect
top
) for two RMS levels: (a) RMS = 1.85
o
, and (b) RMS = 5.46
o
; GT
is the computed angle of rotation from IMU data.
comparisons of the PDFs estimated from the Kinect and ground truth data for pixel-and-
depth-based measurements. Figure 3.30 illustrates the comparisons of the PDFs computed
with the Kinect and ground truth data for pixel-based measurements. Three indices were
used to quantify the accuracy of the random displacement measurements for the Kinect
sensor with respect to the ground truth data: the normalized error, the mean error for the
two PDFs, and standard deviation error for the two PDFs, which are dened according
to Equations 3.8, 3.9, and 3.10. The results of the error analysis are summarized in
Table 3.5. The normalized error, mean error, and standard deviation error decrease when
the RMS amplitude levels (1.85
o
and 5.46
o
) increase. For the random vibration tests,
the normalized error analysis shows relatively poor performance for the pixel-depth-based
rotational measurements, especially for the small RMS level (error of 65.21% for RMS
= 1.85
o
). The error could be mainly caused by the rolling shutter distortion and depth
measurements noise, which is mentioned in 3.7.1. Comparatively, the Kinect sensor has
better performance for the pixel-based rotation measurements, particularly for large RMS
levels (error of 10.45% for RMS = 5.46
o
). Overall, the error analyses of the random test
93
are in agreement with what have been discussed in Sections 3.7.1 and Section 3.7.2 for the
static and harmonic tests.
-20 -10 0 10 20
Angle [
o
]
0
0.1
0.2
0.3
pdf
GT
Kinect
(a)
-20 -10 0 10 20
Angle [
o
]
0
0.1
0.2
0.3
pdf
GT
Kinect
(b)
Figure 3.29: PDFs of the estimated Kinect torsional measurements for the structure's
fourth
oor based on marker MF1 (Kinect
front
) for two RMS levels: (a) RMS = 1.85
o
,
and (b) RMS = 5.46
o
; GT is the computed angle of rotation from IMU data.
-20 -10 0 10 20
Angle [
o
]
0
0.1
0.2
0.3
pdf
GT
Kinect
(a)
-20 -10 0 10 20
Angle [
o
]
0
0.1
0.2
0.3
pdf
GT
Kinect
(b)
Figure 3.30: PDFs of the estimated Kinect torsional measurements for the structure's
fourth
oor based on marker MT1 (Kinect
top
) for two RMS levels: (a) RMS = 1.85
o
, and
(b) RMS = 5.46
o
; GT is the computed angle of rotation from IMU data.
3.8 Summary and conclusions
The comprehensive experimental study reported in this study presented the feasibility of
a vision-based approach for obtaining direct measurements of the absolute displacement
and rotational time history at selectable locations of dispersed testbed structure. The
94
Table 3.5: Comparison of normalized error, mean error, and standard deviation error
under pixel-based and depth-pixel-based torsion measurements
RMS = 1.85
o
RMS = 5.46
o
Pixel (x,y) Pixel-depth (x,z) Pixel (x,y) Pixel-depth (x,z)
Normalized error (%) 30.11 65.21 10.45 15.94
Mean error (%) 10.03 25.60 0.75 4.21
Std error (%) 3.55 8.28 1.66 0.75
measurements were obtained using an inexpensive RGB-D camera (the rst-generation
Kinect as being a representative one). The performance characteristics of the Kinect were
evaluated for a class of structural dynamics problems. The Kinect sensor provides the
potential of quantifying multi-component displacement elds, including displacement and
rotational elds. The eld measurements could be deployed in dierent elds of structural
health monitoring applications in a cost-eective way.
The results of the performed calibration studies show that the Kinect sensor is capable
of measuring displacements and rotations under dierent situation involving dierent
amplitude levels, various frequency ranges, and dierent relative motions between the sensor
and the target structure locations. The analysis results indicate that for displacements
larger than 10 mm, the estimations have an error of about 5% if the structure's vibration
is parallel to Kinect's depth axis (z-axis). However, when the motion is parallel to Kinect's
x-axis, larger measurement errors were observed due to the rolling shutter distortion of the
CMOS sensors that are used by the Kinect's RGB and IR cameras. For rotation angles
larger than 5
o
, if the structure's rotation axis is parallel to Kinect's depth axis (z-axis),
95
the measurements have an error of about 5% when the rotation axis is parallel to Kinect's
y-axis, larger measurement errors were observed, due the combination of noise in the depth
data and rolling shutter error in pixel data.
Overall, this study showed that the Kinect sensor is a convenient, feasible, and cost-
eective tool to measure the evolving multi-component displacement and rotational elds
for structural dynamics problems. However, there are still additional research studies
that are needed to extract RGB features without the need to have markers attached to
the surface of the target structure. In addition, there is a need to evaluate data fusion
concepts regarding the acquired data from dierent Kinects as one single sensor unit.
3.9 Acknowledgments
This research was supported in part by a grant from Qatar University and Qatar Founda-
tion.
96
Chapter 4
Health Monitoring and Condition Assessment
Using Self-Powered Piezo-Floating-Gate (PFG)
Sensors
4.1 Introduction
4.1.1 Background and motivation
T
HE eld of Structural Health Monitoring (SHM) is of great interest worldwide
because of its potential capability to provide cost-eective, autonomous, continuous,
and reliable condition assessment and damage detection in civil infrastructure systems.
However, although numerous researchers throughout the world have devoted, and continue
to devote, considerable eort to investigate and resolve the technical issues that underpin
the SHM methodology, there are still major hurdles that need further study and resolution
to achieve and exploit the full potential of the SHM eld when applied to full-scale
structures under realistic eld conditions.
97
Among the most serious challenges that have hampered the practical application of the
eld of SHM for damage detection in extended structures such as bridges, is the infeasibility
of using a sucient number of conventional sensors (such as strain gages, accelerometers,
etc.) to provide a high-enough spatial resolution so as to capture small cracks that
may be precursors to serious structural damage. Keeping in mind that the collection
of measurements from the target structures needs not only the sensors themselves, but
also an instrumentation network to collect and transmit the measurements to a computer
server for subsequent analysis. Additionally, there is a need for robust energy sources to
drive the sensors and associated data acquisition network.
While the development of wireless sensors networks (WSNs) has attracted considerable
attention in recent years [179, 93, 172, 181, 173, 36, 180]; although, WSNs have eliminated
the need of performing the arduous task of stringing lots of connecting cables on extended
structures such as bridges, there is still the challenge of ensuring an adequate energy
source to power the sensor network for long-term, autonomous, and continuous monitoring
[47]. Embedded and long-term operational requirements require the use of batteries,
whereas the small volume of the sensor severely limits the energy storage capacity of energy
harvesting devices. Therefore, currently there exists a wide gap between the energy that
can be scavenged from real-world structures and the energy density required for sensing,
computing and communication.
Energy harvesting devices can be a promising solution for energy problem [49, 147, 63, 13,
102]. The main working concept for energy harvesting devices is converting mechanical
energy into electrical energy that can be used and integrated with the monitoring system.
Piezoelectric transducers (PZT) are the most ecient self-powering energy sources for
98
SHM [97, 74, 60, 98, 96]. For SHM, PZTs can be used for the self-powering of wireless
sensors by harvesting energy from the mechanical loading experienced by the structure
[7, 6, 5, 66].
This major impediment to practical application of SHM to civil infrastructure has been
partially resolved by the recently developed Michigan State University's (MSU) self-
powered wireless sensor (SWS) [97, 74, 60, 98, 96] that uses PZTs to power an array of
ultra-low power
oating-gate computational circuits. The recorded cumulative durations
are stored on-board the sensor since it is installed, and can be periodically read using a
Radio Frequency Identication (RFID) scanner [97, 74, 60]. It is important to emphasis
that the recorded data is proportional to the structure response due to dierent events;
such as variations in load location, load magnitude, trac wander, environmental eects
such as temperature and moisture, and material aging and degradation, which make the
sensor ideal for long-term SHM.
4.1.2 Objectives
The main goal of this study is to evaluate the performance of the self-powered PFG sensors
for the detection of damage progression in metallic beam under bending. This type of
testing is widely used in SHM to assess the beam-like structures for aerospace, civil and
mechanical applications. The objective of this study is to validate the proposed sensing
technology through small-scale laboratory testing as rst step before applying on full-scale
structures. Therefore, a bending test was carried out on a cantilever beam with dierent
damage states. Several PZTs were installed on the beam to measure the changes in charge
99
of the
oating-gates due to damage. The drop in cumulative voltage for each memory gate
was then correlated to damage progression.
4.1.3 Scope
The remainder of this study is organized as follows; Section 4.2 brie
y introduces the basic
principles and calibration of the newly developed Self-powered PFG sensor. Section 4.3
illustrates the experimental setup deployed of the quantitative assessment of the Self-
powered PFG sensor. Section 4.4 explains the data processing procedures and discusses
the experimental results. The conclusions are summarized in Section 4.5.
4.2 Self-powered PFG sensor
4.2.1 Design principles and working mechanism
The piezoelectric materials can convert mechanical applied loading to an electrical charge,
using the direct piezoelectricity eect. The open source voltage (V) generated across the
piezoelectric lead zirconate titanate (PZT) ceramic transducer used in this study is given
by the following equation [98]:
V =
SYd
31
h

; (4.1)
where S, Y, d
31
,h and , are the applied strain, Young's modulus of the piezoelectric
material, piezoelectric constant, thickness, and the electrical permittivity, respectively.
100
The generated energyE
n
from a PZT across a load resistance (R) is given by the following
equation:
E
n
=
Z
t
f
o
V (t)
2
R
; (4.2)
where t
f
is the loading time. In the proposed self-powered PFG sensor, the PZT converts
the mechanical energy into high-energy electrons variation (hot electrons). The kinetic
energy of the electrons varies with the frequency and amplitude of the applied load. The
electrons overpass the barrier and are injected into the
oating-gate if the energy of
electrons exceeds the energy barrier of the silicon (3.2 eV) [74, 19, 67] as illustrated in
Figure 4.1a. Additionally, the injection rate for each
oating-gate is constant with time
and the
oating-gate voltage varies linearly with the number of applied loading cycles as
shown in Figure 4.1b.
(a) Working principle of the PFG technology. (b) Linearity of injection of the PFG sensor gates.
Figure 4.1: PFG sensor basic concepts.
As the piezoelectric element is periodically excited, more electrons are injected onto the

oating gate and the total amount of charge on the
oating gate indicates the duration and
extent of the mechanical disturbance. The gate of the silicon transistor is isolated by high
101
quality insulating oxide. Therefore, the injected electrons remain trapped for a long period
of time (greater than 8 years for 8 bits precision). Furthermore, The PFG self-powering
device has been shown capable to operate at pico-watt (10
12
to10
9
W ) power dissipation
levels, with its response remaining invariant for mechanical strain-levels down to a few
micro-strains () [19].
For data storage, each
oating-gates of the self-powered PFG act as nonvolatile memory.
Thus, data can be stored on-board the sensor and retrieved without a need for an external
power source [19]. Each memory gate of the sensor has two fundamental properties:
activation threshold and injection rate. The total droppage of voltage V
i
at memory
gate i can be expressed as follows:
V
i
=V
0
V
sensor
i
=V
inj
i
t
i
; (4.3)
whereV
0
is the Initial voltage of the sensor gates set to 1.2 V, V
sensor
i
is the voltage across
memory gate i after applying a number of loading cycles, V
inj
i
is the voltage droppage
rate for memory gate, and t
i
is the cumulative injection time. t
i
can be calculated
based on the threshold level of each memory gate and the applied loading. the sensor
output is a function of the gate threshold level, the amplitude of the signal, the V
inj
i
and
V
0
of each gate.
A typical prototype for the developed self-powered sensor under discussion is shown in
Figure 4.2a. An example of mechanical harmonic excitation and corresponding voltage
droppage in a sensor with three memory gates are illustrated in Figures 4.2b to 4.2d. The
voltage droppage rate for the sensor can be adjusted by modifying the impedance of the
102
sensor [19]. For higher injection rates, lower sensor impedance is required. In practice,
the selection of the impedance depends on the application of the sensor. For example, for
fatigue analysis, thousands of cycles need to be recorded; therefore, high impedance is
required to record fatigue loading events.
(a) Self-powered PFG sensor prototype. (b) Sample mechanical excitation.
(c) Constant injection rate. (d) Variable injection rate.
Figure 4.2: Prototype and working principles of the self-powered PFG sensor.
103
4.2.2 Sensor output calibration
During the calibration phase, it was necessary to calibrate the output of the newly
developed self-powered PFG sensor and quantify the uncertainty in measurement. The
calibration process consisted of 2 steps: (1) checking the repeatability and uncertainty in
sensor measurements by subjecting the sensor to same input n times, and (2) checking the
actual injection threshold for the sensor gates compared to the designed ones. Thus, an
analogue signal generator was used to generate a controllable input analog signal feeding
the sensor. The signal generator consists of the National Instruments cDAQ-9188 chassis
and NI 9263 analog output module. Figure 4.3a shows the NI 9263 and cDAQ-9188 chassis
used to generate analog output. Figure 4.3b shows typical voltage output from the signal
generator. The module was controlled through LabView program to generate the required
signal.
(a) NI 9263 analog output module.
5 6 7 8
−10
−5
0
5
10
Time (s)
Voltage (V)
(b) Sample analog output from
signal generator.
Figure 4.3: Experimental setup deployed for sensor output calibration.
For all tests the signal used was sinusoidal with 1 Hz frequency, which is close to that
seen in civil engineering type of structures. The signal peak voltage varied based on the
calibration purpose. In order to quantify the uncertainty in the sensor measurements,
104
4 tests were carried out. For each test, the sensor was attached to the signal generator.
The input signal was sinusoidal with 1 Hz frequency and 10 V voltage peak. The input
signal was applied for 1000 cycles and sensor output was read each 200 cycles. Figure 4.4a
illustrates a sample sensor output for one of the tests. For each test, the variation in
voltage for each gate of the sensor was evaluated as follows:
V
(i)
(%) =
V
(i)
0
V
(i)
1000
V
(i)
0
X 100; i = 1;:::; 7 (4.4)
Where V
(i)
voltage variation for gate i, V
(i)
0
voltage of gate i before starting test, and
V
(i)
1000
voltage of gate i after 1000 cycles (end of test). Figure 4.4b shows the voltage
variation for each gate after 1000 cycles for the 4 tests. The dotted line represents the
mean of the voltage variation over the 4 tests, for each gate. It is important to mention
that after each test, the sensor was tunneled and injected to reset all gates to almost the
same voltage, which was 1.2V.
0 200 400 600 800 1000
0
0.2
0.4
0.6
0.8
1
1.2
Cycles
Voltage (V)

Gate 1
Gate 2
Gate 3
Gate 4
Gate 5
Gate 6
Gate 7
(a) Sample PFG sensor output for 1000 cycles (20
Vpp - 1Hz).
1 2 3 4 5 6 7
0
20
40
60
80
100
Gate Number
Voltage Variation %

Test 1 Test 2 Test 3 Test 4
(b) Voltage variation for each gate of the PFG
sensor after 1000 cycles.
Figure 4.4: Quantication of uncertainty in the PFG sensor data.
105
The next step was to calibrate the injection threshold for sensor gates. The sensor was
subjected to dierent input signals for 1000 cycles. The input signals were sinusoidal, 1
Hz frequency with dierent signal peak voltage. The input signal peak voltage varies from
7V to 10V with 0.5V step (7 tests). Figure 4.5a shows the designed injection threshold
for dierent sensor's gates. Figure 4.5b illustrates the number of recording gates under
dierent signal input level.
1 2 3 4 5 6 7
5
6
7
8
9
10
11
Gate Number
Injection Threshold Voltage (V)
(a) Injection thresholds.
7 7.5 8 8.5 9 9.5 10 10.5
0
1
2
3
4
5
6
7
Input Signal Maximum Voltage (V)
Number of Recording Gates
(b) Number of recording gates versus input signal
peak voltage.
Figure 4.5: Calibration of the PFG sensor gates injection thresholds.
Based on the obtained results, we can conclude that the uncertainty in sensor measurement
is less than 8% for the same input. Furthermore, the designed gates injection thresholds
match with the experimental results.
106
4.3 Experimental study for health monitoring of beams
4.3.1 Description of the test apparatus
The second phase of the calibration process was to simultaneously record the outputs of
the PZT transducers and several conventional strain gages mounted in their vicinity. The
type of the PZTs used throughout this study was PZT-5A from Steiner & Martins, Inc.
The properties of PZTs are summarized in Table 4.1.
Table 4.1: Properties of PZT-5A.
Property Value
PZT type PZT-5A discs
Dimensions: diameter (mm) thickness (mm) 12 0.6
Elastic modulus (GPa) 76
Capacitance (F) 2.9
Electrical permittivity (10
9F
V
) 16.38
Piezoelectric constant (d
31
)(10
12m
V
) -190
In order to calibrate the PZT output under dierent frequencies and strain levels, the
testbed shown in Figure 4.6 was designed and built. The PZTs output voltage were
directly recorded simultaneously with strain gages mounted in the vicinity of the PZTs.
The recording device used was the National Instruments cDAQ-9188 chassis with three
9235 strain gage modules, one 9234 accelerometer module and one 9220 DAQ module.
The 9220 module was used to record PZTs output. The 9235 module was used to collect
strain gages output.
107
The aluminum cantilever beam was connected directly to the shaker at one end (free end).
The shaker used was controlled through separate computer, dierent than the one used for
data collection. The shaker gave the advantage of generating dierent type of excitation
with dierent frequencies and amplitudes. Thus, the shaker and the aluminum beam have
the same displacement amplitude and frequency. Series of tests were generated by varying
the shaker displacement amplitude and frequency. A general overview of the experimental
setup, strain gages' locations and PZTs' locations are shown in Figure 4.6. Additionally,
Figure 4.7 shows in detail the exact locations of strain gages, PZTs, as well as the sensor
locations numbering used throughout this study. It is important to mention that 6 strain
gages were used on one side of the beam and 6 PZTs were attached on the other side at
same locations. Therefore, the PZTs and strain gages output can be compared.
4.3.2 Finite element model for the testbed
Before carrying out the experimental tests, it was important to generate series of numerical
simulations using nite element to study the eect of damage on strain eld. The nite
element model for the cantilever beam was developed using Abaqus program. The model
geometry and dimensions are illustrated in Figure 4.7. The simulated beam had 3.2 mm
thickness (t), modeled using shell element. The material properties used in the analysis
are listed in Table 4.2. The boundary conditions were xed at one edge (cantilever beam).
The 3D undeformed shape of the undamaged model (i.e., cantilever beam), as well as the
deformation shape of the undamaged cantilever beam are shown in Figures 4.8 and 4.9.
108
(a) Experimental setup components.
(b) Strain gages conguration and nomenclature. (c) PZTs conguration.
Figure 4.6: Experimental setup overview.
Figure 4.7: Sketch for the beam dimensions, sensors locations and damage locations.
109
Table 4.2: Material properties used in the nite element model.
Material property Value
Density (Kg/m3) 2800
Young modulus (Gpa) 69
Poison ratio 0.334
Figure 4.8: Finite element model for the cantilever beam.
110
Figure 4.9: Deformed shape of the cantilever beam.
4.3.3 Eect of damage on strain eld
In order to study the eect of damage on strain eld, six simulated damage scenarios
were introduced to the beam model. Damage was simulated by introducing notches with
dierent diameters (function of beam thickness t) and at dierent locations (see Figure 4.7).
The rst damage case (case#1) corresponded to the baseline condition or undamaged case.
For the other cases, dierent changes were made to study their eects on the strain eld.
In case#2, 1-t diameter notch was introduced in middle distance between locations 2 and
4. Similar changes were introduced at the same location for case#6 and case#7 with
larger notch size (2-t and 3-t). In case#3, 1-t diameter notch was introduced in middle
distance between the beam edge and location 1. For case#4 and case#5, similar changes
were introduced in same location with larger notch size (2-t and 3-t) above location 1.
Table 4.3 summarizes the description of all simulated damage cases.
111
For each damage scenario, the beam was subjected to a harmonic excitation with 1 Hz
frequency at the free end for 30 seconds duration. The strain output (E11) was extracted
from the nite element model at six specic locations that correspond to the actual strain
gage locations. The strain measurements were acquired at a sampling frequency of 100
Hz. Typical 5 seconds strain time-history sample output from the nite element model for
locations 2 and 4 for case#1, case#2, case#6, and case#7 are illustrated in Figure 4.10a
and Figure 4.10b.
Table 4.3: Summary of simulated damage cases.
Cases Description
Case#1 Baseline condition (Undamaged beam)
Case#2 Notch with diameter = 1t between location 2 and 4
Case#3
Notch with diameter = 1t between location 2 and 4
and notch with diameter = 1t at location 1
Case#4
Notch with diameter = 1t between location 2 and 4
and notch with diameter = 2t at location 1
Case#5
Notch with diameter = 1t between location 2 and 4
and notch with diameter = 3t at location 1
Case#6 Notch with diameter = 2t between location 2 and 4
Case#7 Notch with diameter = 3t between location 2 and 4
Additionally, Figures 4.11 and 4.12 show typical sample strain eld (E11) for damage
cases 2 and 4. The highlighted points are the locations of strain gages in the physical
experimental setup. The probability density function (PDF) based on kernel density
estimation for the strain output at locations 2 and 4 for damage cases 1, 2, 6, and 7 are
112
5 6 7 8 9 10
−60
−40
−20
0
20
40
60
Time (s)
Strain (Micro Strain)

Case−1
Case−2
Case−6
Case−7
(a) Location 2.
5 6 7 8 9 10
−60
−40
−20
0
20
40
60
Time (s)
Strain (Micro Strain)

Case−1
Case−2
Case−6
Case−7
(b) Location 4.
Figure 4.10: Sample strain output (E11) for cases 1, 2, 6, and 7.
illustrated in Figures 4.13a and 4.13b respectively. Figure 4.14 shows the PDF based on
kernel density estimation for the strain output at location 1 for damage cases 2, 3, 4, and
5.
Figure 4.11: Strain eld (E11) for case#2.
It can be concluded from the strain PDFs distribution that the damage cases 2 and
3 (notch diameter = 1t) had very minor eect (around 2% change) on the strain eld
distribution around the damage location. However, when the damage is equal or greater 2
times the beam thickness, a quantied change in strain eld in neighborhood locations of
113
Figure 4.12: Strain eld (E11) for case#4.
damage was found (around 10%) (cases 4, 5, 6, and 7). It was important to mention that
the change in strain elds were evaluated at strain gage locations referring to Figure 4.7.
For each case listed in Table 4.3 for the six strain gages locations, the variation in strain
eld was evaluated by the variation in the peak of the PDF of the strain using the following
equation:

(i)
jk
(%) =

(i)
j

(i)
k

(i)
j
X 100; i = 1;:::; 6 (4.5)
Where
(i)
jk
strain distribution peak variation between case j and case i at location i,
(i)
j
strain distribution peak variation for case j at location i, and
(i)
k
strain distribution peak
variation for case k at location i. It is important to mention that the variation in strain
eld for cases 2, 6, and 7 due to damage introduced were made to the reference of case#1.
However, the variation in strain eld for case 3, 4, and 5 due to damage introduced were
made to the reference of case#2.
Figure 4.15a summarizes the variation in strain distribution peak for case#2 at locations
2 and 4. Figure 4.15b summarizes the variation in strain distribution peak for case#4 at
locations 6. It can be seen that when notch size (damage) is double the beam thickness,
114
the change in strain (10 %) is almost ve times the case when the damage is equal beam
thickness. However, for higher damage size, the rate of change of strain is almost linear.
4.4 Damage detection using the self-powered PFG sensor
4.4.1 Overview of damage scenarios
−100 −50 0 50 100
0
0.002
0.004
0.006
0.008
0.01
0.012
0.014
Strain (Micro Strain)

Case−1
Case−2
Case−6
Case−7
(a) PDF of strain output (E11) at location 2.
−100 −50 0 50 100
0
0.002
0.004
0.006
0.008
0.01
0.012
0.014
Strain (Micro Strain)

Case−1
Case−2
Case−6
Case−7
(b) PDF of strain output (E11) at location 4.
Figure 4.13: PDF of strain output (E11) for cases 1, 2, 6, and 7.
−100 −50 0 50 100
0
0.002
0.004
0.006
0.008
0.01
0.012
0.014
Strain (Micro Strain)

Case−2
Case−3
Case−4
Case−5
Figure 4.14: PDF of strain output (E11) at location 1 for cases 2, 3, 4, and 5.
115
0 0.5 1 1.5 2 2.5 3
0
5
10
15
20
Hole diameter (t)
Strain Peak Distribution Change (%)

Location−2
Location−4
(a) Case#2.
0 0.5 1 1.5 2 2.5 3
0
5
10
15
20
25
Hole diameter (t)
Strain Peak Distribution Change (%)

Location−6
(b) Case#4.
Figure 4.15: Eect of notch (hole) size on the variation of strain eld distribution.
For the experimental tests, two dierent damage congurations of the test-bed structure
were considered for this study. Based on the simulation results from the nite element
analysis, case#2 and case#4 were chosen to be investigated experimentally. For damage
case#2 a 3.2 mm (beam thickness) notch was drilled in mid distance between location 2
and 4 as shown in Figure 4.16a. After recording the strain, PZT and PFG sensor data,
another notch with 6.2 mm diameter was drilled at location 1 as illustrated in Figure 4.16b.
(a) Case#2. (b) Case#4.
Figure 4.16: Experimental setup for damaged beam (cases 2 and 4).
116
For each experimental test, the beam was subjected to a sequence of ve harmonic
excitation, each of 200-second duration. The frequency of the applied displacement at
the free end of the beam was 1 Hz, with peak displacement of 8 mm. As mentioned in
Section 4.3.1, six uniaxial strain gages were installed to measure the longitudinal strains
in each location. In addition, six PZTs were attached on the other side of the beam at the
same strain gage locations used to feed the PFG sensor. It is important to mention that
the excitation level was chosen based on the input impedance of the PFG sensor.
4.4.2 Damage classication
For the sake of brevity, the results for damage detection for case#4 using PFG sensor will
be discussed in this section. Since the proposed technology of
oating gates PFG sensors is
to detect the change in strain eld due damage progression in the sensor neighborhood, the
change detected for case#4 was evaluated with respect to case#2 at location 1. Typical
time-histories of measured strains and PZTs for location 1 for case#2 and case#4 are
displayed in Figures 4.17a and 4.17b. For the sake of analysis and comparison, the PDF for
the recorded strain and PZT output for case#2 and case#4 at location 1 were estimated
using kernel density estimation. The results are illustrated in Figures 4.18a and 4.18b.
The change in strain eld and PZT output between case#2 and case#4 at location 1 were
computed as follows:

(i)
24
(%) =

(i)
2

(i)
4

(i)
2
X 100; i = 1;:::; 6 (4.6)
117
100 101 102 103 104 105
−80
−60
−40
−20
0
20
40
60
80
Time (s)
Strain (Micro Strain)

Case−2
Case−4
(a) Experimental strain output location 1.
100 101 102 103 104 105
−10
−5
0
5
10
Time (s)
Voltage (V)

Case−2
Case−4
(b) Experimental PZT output location 1.
Figure 4.17: The experimental strain and PZT output for cases 2 and 4 at location 1.
−100 −50 0 50 100
0
0.002
0.004
0.006
0.008
0.01
0.012
0.014
Strain (Micro Strain)

Case−2
Case−4
(a) PDF of experimental strain output location 1.
−10 −5 0 5 10
0
0.02
0.04
0.06
0.08
0.1
0.12
Voltage (V)

Case−2
Case−4
(b) PDF of experimental PZT output location 1.
Figure 4.18: PDF of the experimental strain and PZT output for cases 2 and 4 at location
1.
118
P
(i)
24
(%) =
P
(i)
2
P
(i)
4
P
(i)
2
X 100; i = 1;:::; 6 (4.7)
Where
(i)
24
strain distribution peak variation between case 2 and case 4 at location i,
(i)
2
strain distribution peak variation for case 2 at location i, and
(i)
4
strain distribution peak
variation for case 4 at location i. P
(i)
24
PZT distribution peak variation between case 2
and case 4 at location i, P
(i)
2
PZT distribution peak variation for case 2 at location i, and
P
(i)
4
PZT distribution peak variation for case 4 at location i.
Figure 4.19 summarizes the comparison for the strain PDF distribution peak variation
from the simulation results, strain PDF distribution peak variation from the experimental
tests, and PZT PDF distribution peak variation between case 2 and case 4 for all locations.
0 1 2 3 4 5 6
0
5
10
15
20
25
Location
Variation %

Finite element
Piezo
Strain gage
Figure 4.19: Comparison between strain, PZT, and simulation output variation between
case 2 and 4.
For each test, while the strain gages and PZTs output were recorded, the PFG sensor
voltage output for the sensor 7 gates were read after each 200 cycles for all locations.
119
Figure 4.20a illustrates typical output for the PFG sensor readings for case#2 at location 1.
However, Figure 4.20b shows the PFG sensor readings for case#4 at location 1. Figure 4.21
summarized the variation in sensor gates voltage after 1000 cycles for location 1 for case#2
and case#4. The variation in sensor gates voltage were evaluated according to Equation
4.4.
0 200 400 600 800 1000
0
0.2
0.4
0.6
0.8
1
1.2
Cycles
Voltage (V)

Gate 1
Gate 2
Gate 3
Gate 4
Gate 5
Gate 6
Gate 7
(a) PFG sensor output after 1000 cycles (case 2) at
location 1.
0 200 400 600 800 1000
0
0.2
0.4
0.6
0.8
1
1.2
Cycles
Voltage (V)

Gate 1
Gate 2
Gate 3
Gate 4
Gate 5
Gate 6
Gate 7
(b) PFG sensor output after 1000 cycles (case 4) at
location 1.
Figure 4.20: PFG sensor output after 1000 cycles (case#2 and case#4)
4.5 Summary and conclusions
In the present study, a new approach was presented for damage detection in beams
using self-powered PFG sensors. These sensors continuously operate by harvesting the
strain-energy of the beam under loading. Each sensor has seven memory gates and a
specic voltage/strain threshold level for each gate. Each gate is activated when the
voltage outputted by the associated PZT transducer exceeds the dened threshold. The
main dierence of the deployed sensors with previously used PFG sensors is that their
120
1 2 3 4 5 6 7
0
10
20
30
40
50
60
Gate Number
Voltage Variation %

Case−2
Case−4
Figure 4.21: PFG sensor voltage variation after 1000 cycles (case#2 and case#4).

oating-gates have variable injection rates. This makes interpretation of the measured
data a challenging task. In order to verify the performance of the sensors, an experimental
study was conducted on an aluminum beam.
In this study, the sensor can be calibrated and tuned based on the type of application,
to sense and record critical strain levels by tuning the injection threshold of the sensors
channels. The introduction of damage (a 3.2 mm notch) at location 1 (case#4) caused
a reduction in stiness at that cross section by 6.25%. As shown in Figure 4.19, this leads
to a relative change in PZT output and strain eld by almost 10% at location 1. However,
the other locations were not aected by this change. As the beam is damaged (change in
beam stiness), the induced strain increases. The voltage generated by the PZT increases
at the same rate as strains. Thus, the injection rate as well as the number of recording
channels increases as well as shown in Figure 4.21. When the beam was undamaged, the
recorded memory of the sensor becomes invariant.
121
The results presented in this study will be extended in a future work to monitor full scale
structure, bridge decks, and stress concentration in bridge girder at stiener locations
due dierential settlement. Many older steel girder bridges are aected by the distortion-
induced fatigue cracking. In this context, a major challenge is to predict the secondary
stresses caused by out-of-plane distortion and consequently to analyze the resulting fatigue
damage. A new method is presented based on the data provided by the self-powered
wireless sensors to detect the distortion-induced fatigue cracking of steel bridges as shown
in Figure 4.22.
Figure 4.22: Bridge girder under dierential settlement.
4.6 Acknowledgments
This research was supported in part by a grant from the Federal Highway Administration
(FHWA).
122
Chapter 5
Vibration-Based Condition Assessment and
Damage Detection Localization and
Quantication of Downtown Los Angeles 52-Story
High-Rise Building
5.1 Introduction
5.1.1 Background and motivation
T
HE Structural health monitoring (SHM) and condition assessment of high-rise
buildings based on vibration signature has been the focus of many investigators
for a long period of time. The common approaches for SHM involved in damage detection
and condition assessment are divided into two broad classes local and global methods.
However, nowadays the development of dense sensor networks with the potential of
acquiring vast amounts of data, leads to the ability to apply of more sophisticated
123
data processing algorithms to identify, localize and quantify changes/damage in civil
infrastructure including high-rise buildings which are the focus of this study.
Extensive research on the eld of damage/change detection and system identication for
linear structural system have led to a signicant number of well-developed approaches
based on vibration data in the time-domain and frequency-domain. Excellent and careful
reviews of existing methods and technologies, as well as recent publications summarizing
the state-of-the-art of damage/change detection and system identication for can be found
in Doebling et al. [44, 43], Smyth et al. [153], Soon et al. [64], Van et al. [168], Farrar et
al. [51], Chang et al. [31], Lynch et al. [107], Adams [1], Figueiredo et al. [54], Thomas
[165], Hernandez-Garcia et al. [69], Liu et al. [104], Takewaki et al. [163], Chakraborty et
al. [29], Song et al. [155], and Chang et al. [55]. Most of the proposed approaches rely on
the assumption that the dynamical systems are behaving linearly.
The instrumentation network operated by the Community Seismic Network at the California
Institute of Technology (CSN) has been constantly acquiring and storing extensive amounts
of vibration data for building response not only at ground level, but also at elevated
stories. These responses and measurements over the height of buildings were used to
develop multiple reduced-order models to perform computational studies based on real
structures. The developed models were integrated in various means of damage detection
and structural identication frameworks and potentially other forms of analysis.
5.1.2 Objectives
In this exploratory study, a high-delity 3D nite element model for a 52-story high-
rise oce building located in downtown Los Angeles was built based on data acquired
124
from state-of-the-art strong-motion accelerometers located at each
oor and operated
by CSN. The detailed 3D model was used to develop dierent data-driven input-output
reduced-order models based on identication approaches that have been successfully
applied to analytical and experimental structures [109, 115, 69]. Furthermore, several
damage congurations in the building lateral load resisting system were studied, based
on the simulated data from white-noise base excitation dynamic tests performed on a
3D-model. The process of damage detection, localization, and quantication was performed
by exploring the variability in the main features of the developed reduced-order models.
It is important to emphasis that in this study, the identication approaches deployed were
implemented in a deterministic manner. The eects of variability in the environmental or
operational conditions, as well as the uncertainties in the modeling, measurement, and
data analysis processes weren't take into consideration in the identied change-sensitive
features (i.e., stiness-like parameters, modal parameters) of structures.
5.1.3 Scope
An overview of the chain-like system identication decomposition approach (ChainID) is
provided in Section 5.2. The description of the 52-story building structure, instrumentation
and details of the 3D model are provided in Section 5.3. The implementation details of
the ChainID identication methodology is presented in Section 5.4. The change/damage
detection results are illustrated in Section 5.5. The conclusions are summarized in
Section 5.6
125
5.2 Chain-like structure system identication approach (ChainID)
Consider a multidegree-of-freedom (MDOF) system chain like structure shown in Figure 5.1,
consisted of n lumped masses of magnitude m
i
, and subjected to base excitation and/or
directly applied force F
i
. The lumped masses are connected by linear elements. The linear
elements restoring forces depend on the relative displacement and velocity between the
lumped masses.
S(t)
z
1
= x
1
- S
z
2
= x
2
– x
1
z
3
= x
3
– x
2
z
n
= x
n
– x
n-1
m
2
m
1
F
2
(t)
F
1
(t)
G
(1)
G
(2)
G
(3)
.
.
.
m
n
F
n
(t)
G
(n)
x
i
(a) Typical structural
topology for multidegree-
of-freedom chain-like
system.
(b) 2D view for the ETABS model
for the 52-story building [92].
(c) 3D ETABS model for the
52-story building [92].
Figure 5.1: Modeling of the 52-story building used in this study (a) reduced order
representation (mathematical model), (b) 2D view for the ETABS model in x-direction,
and (c) 3D view for the ETABS model.
126
The equation of motion of the system under discussion can be the following [70]:
x
n
+G
(n)
(z
n
; _ z
n
) =
F
n
m
n
; (5.1a)
G
(n)
(z
n
; _ z
n
) =
F
n
m
n
x
n
; (5.1b)
G
(i)
(z
i
; _ z
i
) =
F
i
m
i
x
i
+
m
i+1
m
i
G
(i+1)
(z
i+1
; _ z
i+1
); (5.1c)
for i = n-1, n-2, ...., 1
where G
(i)
(z
i
; _ z
i
) is the mass-normalized restoring force function of the linear element,
x
i
is the absolute acceleration of the mass m
i
, z
i
the relative displacements between
two consecutive masses, and _ z
i
the relative velocities between two consecutive masses.
Equation (6.1) can be rewritten in more compact form as follows:
G
(i)
(z
i
; _ z
i
) =
n
X
j=i
f
j
m
j

n
X
j=i
m
ij
x
j
; (5.2)
where m
ij
=m
j
=m
i
is the ratio between the lumped masses m
j
and m
i
. It is clear that
this approach assumes that the acceleration time responses x
j
are available from the
experiment, in addition to the the applied forces F
i
and/or the base excitation, and the
the lumped masses m
i
values. According to [110, 109, 115, 116, 112], after obtaining the
restoring forces time histories, a non-parametric representation for each linear element
can be generated using a truncated doubly indexed series expansion in a suitable basis as
follows:
G
(i)
(z
i
; _ z
i
) =
qmax
X
q=0
rmax
X
r=0
C
(i)
qr
T
q
(z
0
i
)T
r
( _ z
0
i
); (5.3)
127
where C
(i)
qr
are Chebyshev series coecients, T
k
() is the Chebyshev polynomial of order k,
and z
0
i
; _ z
0
i
are the normalized relative displacement and velocity calculating as following:
z
0
=
z
1
2
(z
max
+z
min
)
1
2
(z
max
z
min
)
; (5.4a)
_ z
0
=
_ z
1
2
( _ z
max
+ _ z
min
)
1
2
( _ z
max
_ z
min
)
; (5.4b)
Following a change of coordinates
=cos
1
(z
0
); (5.5a)
=cos
1
( _ z
0
); (5.5b)
The coecients C
ij
are given by
C
ij
=
8
>
>
>
>
>
<
>
>
>
>
>
:
(2=)
2
i;j6= 0;
(2=
2
) i or j = 0;
(1=
2
) i =j = 0;
(5.6)
where
=
Z

0
Z

0
cos(i)cos(j ) G
(i)
(cos()cos( ))dd ; (5.7)
The integral can be evaluated numerically by dividing the range (0,) inton

intervals of
length ==n

, and the range (0,) into n

intervals of length ==n

. Equation
5.7 can be written as:
=
n

X
k=1
n

X
l=1
cos(i
k
)cos(j
l
) G
(i)
(cos(
k
)cos(
l
)) (5.8)
128
In addition, each of the estimated restoring force functions can be converted to a power
series of the form
G
(i)
(z
i
; _ z
i
) =
qmax
X
q=0
rmax
X
r=0
a
(i)
qr
z
q
i
_ z
r
i
(5.9)
where a
(i)
qr
are constant coecients of the power series, and z
i
, _ z
i
are the relative displace-
ment and velocity. The linear mass-normalized restoring force can be expressed in the
form of
G
(i)
(z
i
; _ z
i
) =
k
i
m
i
z
i
+
c
i
m
i
_ z
i
=a
(i)
10
z
i
+a
(i)
01
_ z
i
(5.10)
where a
(i)
10
, a
(i)
01
are the mass-normalized stiness-like coecients and mass-normalized
damping-like coecients. The two matrices M
1
K and M
1
C, where M, C, and K are
the mass, damping, and stiness matrices respectively, are obtained from the identied
mass-normalized stiness-like coecients and mass-normalized damping-like coecients
a
(i)
10
;a
(i)
01
as following:
M
1
K =
2
6
6
6
6
6
6
6
6
6
6
4
a
(1)
10
+a
(2)
10
m
1
a
(2)
10
m
1
0 0
a
(2)
10
a
(2)
10
+a
(3)
10
m
2
a
(3)
10
m
2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
a
(n1)
10
+a
(1)
10
m
n1
a
(n)
10
m
n1
0 0 a
(n)
10
a
(n)
10
3
7
7
7
7
7
7
7
7
7
7
5
(5.11)
M
1
C =
2
6
6
6
6
6
6
6
6
6
6
4
a
(1)
01
+a
(2)
01
m
1
a
(2)
10
m
1
0 0
a
(2)
01
a
(2)
01
+a
(3)
01
m
2
a
(3)
01
m
2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
a
(n1)
01
+a
(1)
01
m
n1
a
(n)
01
m
n1
0 0 a
(n)
01
a
(n)
01
3
7
7
7
7
7
7
7
7
7
7
5
(5.12)
129
Consequently, the modal parameters and mode shapes can be quantied through the
solution of the eigenvalue problem.
5.3 52-Story high-rise building description, instrumentation
and computational model
The building under discussion in this study is a 52-story (+5 basement levels) dual
system oce building located in downtown Los Angeles (DTLA) that was constructed
and designed in the late 1980's. Due to condentiality restrictions, building information
with respect to location is omitted when necessary per memorandum of understanding
(M.O.U.) agreements made with respective owners/operators. The building's lateral system
consisted of a braced frame core surrounded by a steel moment frame. The structural
system consisted of three major components: an interior concentrically braced core,
outrigger beams spanning approximately 12 m from the core to the building perimeter,
and eight exterior outrigger columns. The building was instrumented on each
oor by
tri-axial accelerometer, operated by CSN to collect data at each
oor.
Since it is not feasible to introduce damage in the building structure under dierent
dynamic environment, it is common to develop a computational model for testing damage
detection scenarios. Therefore, the building under discussion was modeled using the nite
element program ETABS developed by Computers and Structures Incorporated based on
complete set of structural drawings provided by the building owner. The ETABS software
has many dierent analysis methods combination (static, dynamic, linear, nonlinear,
buckling, etc.) and dierent element types. In this study for simplicity, the model was
130
assumed behaving linearly and the dynamic linear analysis was performed. It is important
to mention that the model was calibrated by comparing recorded data from the building
with simulated building response in the linear case [92]. Figure 5.2 shows a snapshot
for the developed 3D ETABS model and sensor locations at each
oor. Each
oor was
instrumented by a tri-axial accelerometer located at the North-East corner of the core.
X [N-S]
Y [E-W]
The location of the sensors on every
floor is the North-East corner of the
Core.
X [N-S]
Y [E-W]
Sensor
Excitation
direction
Figure 5.2: Finite element 3D-model (ETABS 3D-model) for the 52-story building with
axis orientation, nomenclature and location of sensors deployed in the building [92].
For the ETABS 3D-model the damage was introduced by reducing the stiness of the
bracing elements of the interior concentrically braced core. The damage scenarios were
created using the computational tool shown in Figure 5.3 developed by Computers and
131
Structures (developer Dr. Christopher Janover) [120]. The developed tool was used in
the ETABS program to modify stiness of objects to represent damaged elements in the
structure. Afterwards, the damaged states were run through a series of ground motions for
computational study. In addition, the program was developed such as any properties of
group of objects in the model could be modied independently. The program also oered
the option to choose the type of output. Currently the program provides relative and
absolute displacement, velocity and acceleration as output.
Figure 5.3: Caltech ETABS property modier automater used to introduce damage/change
and generate dierent damage/change states [120].
132
5.4 Identication of data based reduced-order models
5.4.1 Sample data processing results
Three dierent structural states/conguration of the building structure were considered
for this study. The rst state conguration (state#1) corresponded to the baseline, or
reference condition of the building structure. In states#2 and #3, the stiness of the
bracing elements of the core were reduced by 50% between dierent
oors. For state#2,
the axial stiness of the bracing elements of the core between the 4
th
and 7
th

oors were
reduced by 50% in x- and y-directions (i.e., XZ and YZ planes) simultaneously. In state#3,
the axial stiness of the bracing elements of the core were reduced by 50% between the
4
th
and 5
th

oors in x-direction (i.e., XZ planes) only and between the 9
th
and 10
th

oors
in y-directions (i.e., YZ planes) only. Table 5.1 summarizes the dierent state conditions
of the building structure used throughout this study.
Table 5.1: Summary of structural state conditions (damage/change scenarios).
State Description
State#1 Baseline structural condition.
State#2 The axial stiness of the bracing elements of the core were reduced by 50%
between the 4
th
and 7
th

oors in both x- and y-directions simultaneously.
State#3 The axial stiness of the bracing elements of the core were reduced by 50%
between the 4
th
and 5
th

oors in x-direction only; and the 9
th
and 10
th

oors in y-direction only.
133
For each of the structural states, the building structure was subjected to base excitation
by applying 1000-seconds long band-limited white-noise base motions in x-direction only.
For nomenclature, the simulation data and output response from the 3D ETABS model
will be referred as "experimental data". In this study, only the acceleration measurements
in both x- and y-directions were assumed to be available. The acceleration responses
were acquired from the model at a sampling frequency of 50 Hz. The corresponding
velocity and displacement time-histories were obtained by signal processing and numerical
integration. The accuracy of the integrated velocity and displacements time histories were
compared with sample computed response time histories from the ETABS model in x- and
y-directions. The estimated and experimental displacement and velocity time-histories
from ETABS showed a close match. Additionally, it was assumed that the building
slabs were rigid; therefore, the available acceleration measurements corresponded to the
acceleration response of the structure at each slab's geometric center. It is important
to mention that the experimental data available was generated through a nite element
software, which eliminate all sources of uncertainty as oppose to data recorded from a real
physical structure. Therefore, one ensemble of time history records was generated for each
structural state/conguration.
5.4.2 Decomposition approach (ChainID) implementation
The building structure in the reference condition (state#1) was deployed to show the results
of implementing the proposed approach in Section 5.2 to build reduced-order model for a
linear system. During the analysis, the system was not assumed linear; however, the linear
dynamic analysis was performed in the 3D ETABS model. The decomposition approach for
134
the restoring force identication was carried out using a third-order Chebyshev polynomial
in both normalized variables z
0
and _ z
0
(i.e., corresponding power series coecients in
both state variables z and _ z) for all
oors in the building structure. Once the relative
displacements and velocities had been computed, the ChainID identication approach was
applied to build the associated non-parametric representation for each
oor in the 52-story
building structure, by computing the corresponding restoring force coecients of each
oor
in the reference structural conguration. Figure 6.4 illustrates sample time-history plots
for the relative displacement z
(i)
, relative velocity _ z
(i)
, and measured mass normalized
restoring force G
(i)
between 10
th

oor and 9
th

oors in x- and y-directions .
From the analysis of the identied restoring force coecientsC
(i)
qr
and corresponding power
series coecients a
(i)
qr
for all building
oors indicated that the linear term associated with
relative displacements in the non-parametric representation (i.e., C
(i)
10
and a
(i)
10
, had the
most signicant contribution to the restoring force G
(i)
in both x- and y-directions, while
the nonlinear terms were negligible. For the sake of brevity, only the identication results
computed for the 10
th

oor will be presented and discussed in this section. The power series
coecients a
(10)
qr
of the non-parametric representation for the 10
th

oor are summarized in
Table 5.2. It can be seen clearly that the mass normalized stiness like coecient a
(10)
10
was
the signicant term in the non-parametric representation. In addition, since the building
structure is geometrically symmetric after the 5
th

oor, the a
(i)
10
coecient values in both
x- and y-directions for all
oors were similar as illustrated in Figure 5.5.
Figure 5.6 illustrates the time-history of the measured and reconstructed mass-normalized
restoring forces for the 10
th

oor in the reference conguration in x- and y-directions. It
can be seen that the two curves are basically identical curves and the reduced-order model
135
300 320 340 360 380 400
-4
-2
0
2
4
300 320 340 360 380 400
-12
-6
0
6
12
300 320 340 360 380 400
-8000
-4000
0
4000
8000
(a)X -direction.
300 320 340 360 380 400
-0.16
-0.08
0
0.08
0.16
300 320 340 360 380 400
-2
-1
0
1
2
300 320 340 360 380 400
-350
-175
0
175
350
(b)Y -direction.
Figure 5.4: Sample of relative displacementz
(i)
and relative velocity _ z
(i)
computed between
10
th
and 9
th

oors are illustrated in the rst two rows. The third row shows the measured
restoring force time-history for element G
(10)
from reference condition (state#1) (a) x-
direction, and (b) y-direction. Note that, dierent amplitude scales are used in LHS
and RHS columns of plots for enhanced viewing. Since the excitation was applied in
x-direction only, the relative displacement computed z
(10)
between 10
th

oor and 9
th

oor
in y-direction is relatively small (less than 0.15 in) or almost less than 5% of the relative
displacement computed in x-direction.
136
Table 5.2: Identied mass normalized restoring force coecients a
(i)
qr
for the 10
th

oor
in the reference conguration (state#1) in x- and y-directions. The bold numbers are
mass normalized stiness like coecient (a
(10)
10
). Note that the mass normalized stiness
like coecient has the signicant contribution in the non-parametric representation of
the restoring force in both directions and the higher order terms have negligible values
compared to the mass normalized stiness like coecient.
X -direction Y -direction
@
@
@
@
@
q
r
0 1 2 3 0 1 2 3
0 29.42 22.86 -0.32 -0.02 0.1077 3.13 0.7609 1.4
1 1735 1.43 -0.21 -0.06 1704 -8.75 -3.721 23.12
2 -2.45 0.00 0.14 0.02 -21.51 13.48 -51.92 42.45
3 0.20 0.00 0.03 0.00 1.128 1.06 -1.9 -1.13
0 5 10 15 20 25 30 35 40 45 50 55
0
500
1000
1500
2000
2500
3000
3500
Figure 5.5: The identied mass normalized stiness like coecient (a
(i)
10
) for all
oors
starting from the reference conguration (state#1) in x- and y-directions. Note that after
the 5
th
, the identied mass normalized stiness like coecient are almost equal for the
same
oor in x- and y-directions due to symmetry in the structure. In Addition, an abrupt
change can be seen at the 50
th

oor in in both directions due to the large change in mass
after the 50
th

oor. For enhanced viewing the values of the identied mass normalized
stiness like coecient for the basement and 1
st

oors are not plotted.
137
was able to replicate the dominant dynamic features in the time response of 10
th

oor.
To assess the quality of the reduced-order model built using the identied restoring force
coecients, the phase plot of the restoring force versus relative displacement, as well as
the reconstructed restoring force surface for the 10
th

oor in the reference conguration
are shown in Figure 5.7 in both x- and y-directions.
It can be seen from the rst row of Figure 5.7, the phase plot of the measured restoring
forceG
(10)
(solid blue line) and relative displacementz
(10)
is compared to the reconstructed
restoring force (red line), the reconstructed reduced-order model captured the dominant
linear dynamics characteristics of the 10
th

oor. Similarly, the reconstructed restoring
force surface is planar despite a third-order expansion on both state variables (i.e., relative
displacement and relative velocity) was used to characterize the dynamics of the 10
th

oor. Also, the actual restoring measurements were plotted as point cloud as shown in
the second row of Figure 5.7.
300 320 340 360 380 400
-1
-0.5
0
0.5
1
10
4
Measured [FEA Model]
Reconstructed
(a)X -direction.
300 320 340 360 380 400
-500
-250
0
250
500
Measured [FEA Model]
Reconstructed
(b)Y -direction.
Figure 5.6: Comparison of measured and estimated restoring forces G
(i)
for the 10
th

oor
in the reference conguration (state#1) in (a) x-direction, and (b) y-direction. The blue
solid lines correspond to the measured restoring force time-history; however, the red dotted
lines correspond to the reconstructed time-history for the restoring force computed using
the reduced-order model.
138
(a)X -direction. (b)Y -direction.
Figure 5.7: Sample of phase plot of restoring force G
(i)
versus relative displacement z
(i)
for the 10
th
floor in the reference conguration (state#1) and corresponding estimated
restoring force surface for the 10
th

oor in (a) x-direction, and (b) y-direction. In the
rst row the blue solid lines correspond to the phase plot for the measured restoring force
and the red dotted lines correspond to the reconstructed restoring force. In the second
row, the actual restoring force measurements were plotted as point cloud. In addition, the
restoring force surface is almost planar, as expected for linear elements.
139
5.5 Change detection
5.5.1 Change detection using reduced-order models of structural sys-
tems
In this study, the lateral dynamic response at the geometric center of each slab, in both x-
and y-directions, were deployed to estimate the corresponding reduced-order models of the
test structure in each structural state/conguration. In Addition, it was assumed that only
the acceleration measurements were available. Furthermore, the ensemble of acceleration,
velocity and displacement time-histories was used to build 52 degree-of-freedom (DOF)
reduced-order model that describe the dynamical properties of the building structure in
each of the structural congurations (i.e., state#1, state#2, and state#3). Additionally,
the mass values for each
oor was extracted from the ETABS 3D-model for x- and
y-directions.
When the structural changes/damage were introduced in the building structure, the
dynamic characteristics and response time-histories of the structure system were aected.
Therefore, the response time-histories and the estimated restoring force surface of each

oor will exhibit variations with respect to the reference case as shown in Figures 5.8
to 5.10.
For the structure in the baseline condition (i.e., state#1), the normalized stiness like
coecient (a
(i)
10
) was estimated for each DOF (i.e.,
oor) in both x- and y-directions from
the reduced-order models built for the building structure. For structural state#2 and
state#3, the presence of signicant changes in the dierent parameters of reduced-order
model was determined by the observed change falling outside pre-dene bounds. If the
140
0 1 2 3
0
5
10
15
20
25
30
35
40
45
50
55
State#1
State#2
State#3
0 1000 2000 3000
0
5
10
15
20
25
30
35
40
45
50
55
State#1
State#2
State#3
0 0.05 0.1 0.15
0
5
10
15
20
25
30
35
40
45
50
55
State#1
State#2
State#3
0 50 100 150
0
5
10
15
20
25
30
35
40
45
50
55
State#1
State#2
State#3
Figure 5.8: Comparison of the computed root mean square (RMS) for relative displacement
z
(i)
and mass normalized restoring forceG
(i)
for all
oors in x- and y-directions for reference
and damage conditions. Note the sudden change in relative displacement at damaged
locations (5
th
- 7
th
)
oors for damaged state#2 in x- and y-directions, and 5
th
and 10
th

oors in x- and y-directions consecutively for damaged state#3).
changes in the normalized stiness like coecient (a
(i)
10
) computed using Equation 5.13,
exceed the detection threshold ratio of the new observations from a given structural
conguration were outside the parameters' condence bounds, there is enough evidence to
conclude that signicant changes (i.e., damage) were observed in that
oor.
a
(i)
10
[%] = 100
a
(i)
10
a
(i)
10
j
ref
a
(i)
10
j
ref
(5.13)
The change detection was based on a classical hypothesis testing-based approach, from
the assumption of a signicance level of 5% is the bound of dening condence region of
the change in mass normalized stiness-like coecients a
(i)
10
with respect to the reference
structural condition in both directions. It has been shown that the restoring force
141
-3000
-2000
-1000
-10
0
1000
0
2000
4
3000
2
10
0
-2
-4
-3000
-2000
-1000
-10
0
1000
0
2000
4
3000
2
0
10
-2
-4
-3000
-2000
-10
-1000
0
1000
0
2000
4
3000
2
0
10 -2
-4
(a)X -direction.
-200
-150
-2
-100
-50
0
50
100
0
150
0.2
200
0.1
0
-0.1
2
-200
-150
-2
-100
-50
0
50
100
0
150
0.2
200
0.1
0
-0.1
2
-0.2
-200
-150
-100
-50
-1
0
50
100
0
150
200
0.1
1
0
-0.1
-0.2
(b)Y -direction.
Figure 5.9: Identied change in the mass normalized restoring force surfaces for damage
state#1 (a) for elements G
(5)
, G
(7)
, and G
(10)
in x-direction after introducing damage at
5
th
to 7
th

oors in x-direction, and (b) for elements G
(5)
, G
(7)
, and G
(10)
in y-direction
after introducing damage at 5
th
to 7
th

oors in y-direction simultaneously. Note that,
dierent amplitude scales are used in LHS and RHS columns of plots for enhanced viewing.
142
-2000
-1500
-1000
-500
-10
0
500
1000
0
1500
4
2000
2
10
0
-2
-4
-2000
-1500
-1000
-10
-500
0
500
1000
0
1500
4
2000
2
0
-2 10
-4
-2000
-1500
-1000
-10
-500
0
500
1000
0
1500
4
2000
2
0
10 -2
-4
(a)X -direction.
-75
-50
-2
-25
0
25
0
50
0.2
75
0.1
0
-0.1
2
-75
-50
-25
-1
0
25
0
50
0.2
75
0.1
1
0
-0.1
-75
-50
-25
-1
0
25
0
50
75
0.1
1
0
-0.1
-0.2
(b)Y -direction.
Figure 5.10: Identied change in the mass normalized restoring force surfaces for damage
state#2 (a) for elements G
(5)
, G
(8)
, and G
(10)
in x-direction after introducing damage
at 5
th

oor in x-direction, and (b) for elements G
(5)
, G
(8)
, and G
(10)
in y-direction after
introducing damage at 10
th

oor in y-direction simultaneously. Note that, dierent
amplitude scales are used in LHS and RHS columns of plots for enhanced viewing.
143
coecients in the chain-like system identication approach are a suitable set of parameters
for structural change detection applications [69].
Figure 5.11a displays the comparison of relative changes in mass normalized stiness-
like coecients a
(i)
10
in x- and y-directions for state#2. The red dotted line corresponds
to the 5% decision bound. It is clear from Figure 5.11a that signicant reductions in
the stiness-like in the x- and y-directions estimated for the structure in the damaged
congurations. Furthermore, the big change is localized at the 5
th
to the 7
th
in both
directions. Similar conclusions can be concluded from the change in restoring force surface
illustrated Figure 5.9.
Similar analysis was done with the results obtained from state#3 scenario. The detected
changes in the mass normalized stiness-like term a
(i)
10
could be directly correlated to
the damage introduced in the structure. From Figure 5.11b, it can be seen a signicant
reduction in the mass normalized stiness-like term a
(i)
10
at the 5
th

oor in x-direction and
10
th

oor in y-direction. These results were expected since damage was introduced in the
building model at the 5
th

oor in x-direction and 10
th

oor in y-direction. Furthermore, the
maximum change in restoring force surfaces was found at the 5
th

oor in x-direction and
10
th

oor in y-direction as displayed in Figure 5.9. However, the change in the restoring
force surface was negligible for the other
oors.
5.5.2 Change detection using modal parameters
In addition to the local identication of the dynamic of each story in the building, the
modal identication (i.e., modal parameters) of the building were estimated using the
identied restoring force coecients as previously discussed in Section 5.2. The estimated
144
0 5 10 15 20 25 30 35 40 45 50 55
-30
-25
-20
-15
-10
-5
0
5
(a) State#2.
0 5 10 15 20 25 30 35 40 45 50 55
-20
-15
-10
-5
0
5
(b) State#3.
Figure 5.11: Magnitude of relative changes in the mass normalized stiness-like coecients
(a
(i)
10
) of the reduced-order models in the x- and y-directions of the test structure computed
using ChainID approach for (a) structural state#2 and (b) structural state#3. The
horizontal dotted line corresponds to the thresholdj
(i)
10
j = 5% used in this study to
determine if signicant changes in the stiness-like a
(i)
10
coecients have occurred with
respect the reference condition (state#1) parameters.
145
modal parameters were also compared to the modal parameters identied by implementing
natural excitation technique (NExT) [76, 26] in combination with eigensystem realization
algorithm (ERA) [82] which a global identication technique.
The estimated values for the modal parameters (i.e., natural frequencies, and damping
ratios) for the rst ve mode shapes for the reference condition (state#1) in both x- and
y-directions are summarized in Table 5.3. The corresponding mode shapes for the rst
ve lateral modes in the x- and y-direction are shown in Figure 5.12. It should be noted
that the natural frequencies estimated from reduced-order models built in this study did
closely agree with the natural frequencies computed using NExT/ERA approach, as well
as the natural frequencies reported by [121]. However, the test structure had a dominant
bending beam-type dynamic behavior. It can be seen from Table 5.3, the corresponding
estimated damping ratios computed using the two dierent approaches are similar for
the rst two mode shapes; however, it is slightly dierent for the other reported mode
shapes. This dispersion is related to the level of contribution in the characterization of
the restoring forces. Clearly, the mass normalized stiness like coecients a
(i)
10
are much
more robust more than the mass normalized damping like coecients a
(i)
01
. Furthermore,
the damping ratios estimated using NExT/ERA for the rst ve mode shapes are within
2% which is typical for typical braced steel structures.
As previously mentioned in this study, the change detection was based on a classical
hypothesis testing-based approach. From the assumption of a signicance level of 5% is
the bound of dening condence region of the change in modal frequencies and damping
ratios identied with respect the reference structural condition in both directions. The
modal parameters, obtained from vibration data from state#2 and state#3, were then
146
(a)X -direction.
(b)Y -direction.
Figure 5.12: Mode shapes of the rst ve lateral modes in (a) x-direction, and (b) and
y-direction extracted from the ETABS 3D-model in the reference condition (state#1).
Note that the modes shapes are shown in 2D view (XZ and YZ planes).
147
Table 5.3: Summary of natural frequencies and damping ratios for the rst ve lateral
modes in the x- and y-directions of the 52-story building structure identied from the
reduced-order model developed using ChianID and NExT/ERA approaches.
Mode
X-direction Y-direction
ChianID NExT/ERA ChianID NExT/ERA
! [Hz] [%] ! [Hz] [%] ! [Hz] [%] ! [Hz] [%]
1
st
Mode 0.176 2.00 0.168 2.57 0.172 1.80 0.169 2.23
2
nd
Mode 0.505 2.56 0.595 1.75 0.505 1.92 0.578 2.24
3
rd
Mode 0.961 5.07 1.140 1.45 0.775 3.39 0.885 1.50
4
th
Mode 1.210 9.39 1.343 1.51 1.070 4.84 1.142 1.37
5
th
Mode 1.589 13.32 1.662 1.85 1.267 5.42 1.243 1.77
compared against the condence bounds. The magnitudes of observed changes in the
natural frequencies were calculated using Equation 5.14 and summarized in Figure 5.13
based on ChainID and NExT/ERA approaches. From Figure 5.13, it is clear that the
changes in the estimated natural frequencies were proportional to the severity of damage
introduced in the building structure. However, the reductions in natural frequencies for
the rst ve modes in both x- and y-directions are not signicant for state#2 and state#3.
!
i
[%] = 100
!
i
!
ref
!
ref
(5.14)
148
5.6 Summary and conclusions
In this study, a 3D-nite element model was developed for 52-story high-rise building under
The Community Seismic Network at the California Institute of Technology to evaluate the
eectiveness and reliability of employing reduced-order models built from experimental
data generated from the utilizing two dierent system identication techniques, to detect
and locate physical changes/damage in physical building structure.
1 2 3 4 5
-8
-6
-4
-2
0
2
4
State#2
State#3
1 2 3 4 5
-8
-6
-4
-2
0
2
4
State#2
State#3
(a) NExT/ERA
1 2 3 4 5
-8
-6
-4
-2
0
2
4
State#2
State#3
1 2 3 4 5
-8
-6
-4
-2
0
2
4
State#2
State#3
(b) ChainID
Figure 5.13: Magnitude of relative change in the natural frequencies !
i
obtained using
(a) NExT/ERA approach and (b) ChainID approach estimated for the building structure
in the x- and y-directions. The horizontal dotted line corresponds to the thresholdj!j
= 5% used in this study to determine if signicant changes in the modal parameters have
occurred with respect the reference condition parameters.
The results of this study clearly demonstrate that the structure health monitoring method-
ology discussed in this study have proved the capabilities for detecting, locating, and
quantifying structural changes/damage in 52-story high-rise building based on simulated
149
data. The advantage of simulated data was to test dierent damage scenarios without
introducing damages in the physical structure and predict the structure response and
behavior under dierent operation conditions; however, the eects of operational and
environmental conditions, and underlying damage mechanisms are not considered in this
study in the structures' dynamic response.
In this preliminary study, the input-output data from the 52-story building structure
under white noise base excitation, was used to develop reduced-order models for dierent
structural congurations by implemented two dierent approaches: the non-parametric
chain-like system identication approach (ChainID) which is the focus of this study,
and a global identication approach (NExT/ERA). The change in the estimated mass-
normalized stiness-like coecient of the reduced-order models were deployed to detect,
localize the actual structural changes introduced in the building structure. The results of
this study showed that the signicant changes identied in the stiness-like parameters of
the reduced-order models built using the ChainID could be correlated to the presence and
location of the actual physical changes/damage introduced to the building structure, even
in the presence of modeling, measurement, and data processing errors using reduced-order
representations.
Additionally, the results of this study show that for the global identication approach, the
variability in the natural frequencies, was on the average of 2% which was not signicant
and could not be reliably correlated with the actual damage location in the structure.
It is important to mention that even though the ChainID approach basically assumes
that the structure has a shear-type building behavior and doesn't take into consideration
150
the dominant bending beam-type dynamic behavior, the chain-like system identication
approach was able to extract the relative changes in the building structure.
5.7 Acknowledgments
The author would like to acknowledge the assistance of Dr. Anthony Thomas Massari in
lending his expertise in developing the ETABS 3D-model for the building under discussion.
151
Chapter 6
Development and Validation of Nonlinear
Data-Driven Computational Models for
Condition Assessment and Change Detection
Based on Vibration Signature
6.1 Introduction
6.1.1 Background and motivation
T
HE eld of structural control and health monitoring is facing an explosive evolution
and development due to the huge growth in the data available from complex dy-
namical systems in various engineering disciplines such as aerospace, civil, and mechanical
structures. It is important to digest the humongous amount of data acquired and develop
reduced-order, reduced-complexity computational models capable of extracting potential
features that characterize the dominant dynamical systems behavior, especially nonlinear
152
systems typically encountered in the eld of structural dynamics. Furthermore, the devel-
oped reduced-order models can be deployed for active control, response prediction, system
identication, change/damage detection techniques, and structural health monitoring
methodologies based on vibration signature analysis.
Extensive research on the eld of damage/change detection and system identication
for linear system have led to a signicant number of well-developed approaches based
on vibration data in time-domain and frequency-domain. However, in real application,
complex systems can have nonlinear properties during normal operation. Furthermore,
the presence of damage in many cases will additionally introduce nonlinear characteristics
in the dynamic response of the system. Concrete cracks, fatigue cracks that open and
close under loading conditions, impact loading, loose connections, composite material
with nonlinear behavior, geometric nonlinearity related to boundary conditions, material
nonlinearities associated with excessive deformation are considered damage and change
that can cause a structure to behave in a nonlinearly [71]. Excellent and careful reviews
and technical papers of existing nonlinear identication techniques and several illustrative
applications can be found in Brewick et al. [25], Kerschen et al. [86], Farrar et al. [65],
Worden et al. [174], Figueiredo et al. [54], Bornn et al. [20], Derkevorkian et al. [41, 40].
Most of the developed approaches for nonlinear systems, are usually able to obtain results
that are satisfactory for estimating the overall behavior of nonlinear systems. However,
they are inaccurate in representing the ne details in complex system due to the wrong
physics that is incorporated in the simulation model.
Several approaches for building computationally ecient mathematical models for single-
degree-of-freedom (SDOF) and multi-degree-of-freedom (MDOF) have been applied to
153
synthetic and experimental data [70, 114, 117, 113, 119]. Among these approaches, a time-
domain non-parametric decomposition technique for chain-like MDOF systems presented
by Masri et al. [111, 110] which is of special interest.
6.1.2 Objectives
The main objective of this study is to develop computational models based on the approach
presented by Masri et al. [111, 110] for linear/nonlinear systems to perform accurate, simu-
lated "experiments" on realistic complex nonlinear systems, in either existing or conceptual
design stages. Furthermore, the developed models serve as a useful tool for condition
assessment, change/damage detection. Preliminary results are discussed for development
of suitable reduced-order mathematical models to characterize the essential features of
the dominant structural behavior, as well as detecting, locating, quantifying changes
in uncertain chain-like systems. Examples of chain-like systems can be airplane wings,
buildings, oshore platforms, and transmission towers. A sophisticated, re-congurable
structure was built to investigate the detection of generic types of linear, nonlinear and
combined linear/nonlinear complex changes widely encountered in the civil infrastructure
eld. Furthermore, to reliably detect, quantify and localize complex change/damage in the
system, the uncertainties encountered with the modeling, measurement and data analysis
process was addressed in the development of reduced-order representation.
6.1.3 Scope
This paper is organized as follows: Section 6.2 provides an overview of the chain-like system
identication approach. The description of the testbed structure and instrumentation are
154
presented in Section 6.3. Section 6.4 explains the data processing procedures used and the
experimental tests performed. The results and the statistical analysis of this study are
discussed in Sections 6.5 and 6.6. Section 6.7 illustrates the results of change detection
using global identication methods. A conclusive summary of the study is provided in
Section 6.8.
6.2 Identication approach
Consider a MDOF system chain-like structure shown in Figure 6.1, consisted of n lumped
masses of magnitude m
i
, and subjected to base excitation and/or directly applied force
F
i
. The lumped masses are connected by arbitrary nonlinear elements. The nonlinear
elements restoring forces depend on the relative displacement and velocity between the
lumped masses. In addition, the restoring forces depend on a vector of parameters p
i
characterizing the nonlinear element.
. . .
S

(t)
z
1
=

x
1
- S
m
1
G
(1)
F
1
(t)
m
2
G
(2)
F
2
(t)
z
2
=

x
2
– x
1
G
(3)
z
3
=

x
3
– x
2
m
n
G
(n)
F
n
(t)
z
n
=

x
n
– x
n-1

x
i
Figure 6.1: Model of non-linear multidegree-of-freedom chain-like system.
155
The equation of motion of the system under discussion can be the following [111]:
x
n
+G
(n)
(z
n
; _ z
n
; p
n
) =
F
n
m
n
; (6.1a)
G
(n)
(z
n
; _ z
n
; p
n
) =
F
n
m
n
x
n
; (6.1b)
G
(i)
(z
i
; _ z
i
; p
i
) =
F
i
m
i
x
i
+
m
i+1
m
i
G
(i+1)
(z
i+1
; _ z
i+1
;p
i+1
); (6.1c)
for i =n 1; n 2;::::; 1
where G
(i)
(z
i
; _ z
i
; p
i
) is the mass-normalized restoring force function of the nonlinear
element, x
i
is the absolute acceleration of the mass m
i
, z
i
the relative displacements
between two consecutive masses, and _ z
i
the relative velocities between two consecutive
masses. Equation 6.1 can be rewritten in more compact form
G
(i)
(z
i
; _ z
i
; p
i
) =
n
X
j=i
f
j
m
j

n
X
j=i
m
ij
x
j
; (6.2)
where m
ij
=m
j
=m
i
is the ratio between the lumped masses m
j
and m
i
. It is clear that
this approach assumes that the acceleration time responses x
j
are available from the
experiment, in addition to the the applied forces F
i
and/or the base excitation, and the
the lumped masses m
i
values. According to [110, 109, 115, 116, 112], after obtaining the
restoring force time histories, a non-parametric representation for each linear element can
be generated using a truncated doubly indexed series expansion in a suitable basis that
approximates the real restoring force functions G
(i)
(z
i
; _ z
i
; p
i
) by approximate functions
G
(i)
(z
i
; _ z
i
).
G
(i)
(z
i
; _ z
i
; p
i
)G
(i)
(z
i
; _ z
i
) =
qmax
X
q=0
rmax
X
r=0
C
(i)
qr
T
q
(z
0
i
)T
r
( _ z
0
i
); (6.3)
156
where C
(i)
qr
are Chebyshev series coecients, T
k
() is the Chebyshev polynomial of order k,
and z
0
i
; _ z
0
i
are the normalized relative displacement and velocity calculating as following:
z
0
=
z
1
2
(z
max
+z
min
)
1
2
(z
max
z
min
)
; (6.4a)
_ z
0
=
_ z
1
2
( _ z
max
+ _ z
min
)
1
2
( _ z
max
_ z
min
)
; (6.4b)
In addition, each of the estimated restoring force functions can be converted to a power
series of the form of
G
(i)
(z
i
; _ z
i
) =
qmax
X
q=0
rmax
X
r=0
a
(i)
qr
z
q
i
_ z
r
i
; (6.5)
where a
(i)
qr
are constant coecients of the power series, and z
i
, _ z
i
are the relative displace-
ment and velocity. The linear mass-normalized restoring force can be expressed in the
form of
G
(i)
(z
i
; _ z
i
) =
k
i
m
i
z
i
+
c
i
m
i
_ z
i
=a
(i)
10
z
i
+a
(i)
01
_ z
i
; (6.6)
Where a
(i)
10
, a
(i)
01
are the mass-normalized stiness-like and mass-normalized damping-like
coecients respectively. The modal parameters of the structure can be evaluated through
the eigen decomposition of the matrix A, which is given by;
A =
2
6
4
0 I
M
1
K M
1
C
3
7
5 = ; (6.7)
157
where and are the complex eigenvector and complex eigenvalue matrices. The
matrices M
1
K and M
1
C are obtained from the identied mass-normalized stiness-like
coecients and mass-normalized damping-like coecients a
(i)
10
; a
(i)
01
as following;
M
1
K =
2
6
6
6
6
6
6
6
6
6
4
a
(1)
10
+a
(2)
10
a
(2)
10
0 0
a
(2)
10
a
(2)
10
+a
(3)
10
a
(3)
10
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 a
(6)
10
a
(6)
10
3
7
7
7
7
7
7
7
7
7
5
(6.8)
M
1
C =
2
6
6
6
6
6
6
6
6
6
4
a
(1)
01
+a
(2)
01
a
(2)
01
0 0
a
(2)
01
a
(2)
01
+a
(3)
01
a
(3)
01
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 a
(6)
01
a
(6)
01
3
7
7
7
7
7
7
7
7
7
5
(6.9)
The natural frequencies !
i
and damping rations
i
of the system can be calculated using
the complex eigenvalues
i
in matrix ;
!
i
=
p
Re(
i
)
2
+Im(
i
)
2
;

i
=Re(
i
)=!
i
;
6.3 Experimental setup
A multi-bay re-congurable test structure with removable elements was designed and built
to perform studies on the detection, location, and quantication of structural changes
that physical structural elements can undergo due to a variety of damage mechanisms.
158
The fabricated laboratory structure consists of six (6) modular sections each with the
dimensions of 4 x 6 x 8 in. as shown in Figure 6.2a. Each module has four (4) supporting
beams assemblies with cross section 0.5 in. by 0.125 in. with a welded pad at each end.
Each pad is bolted to the adjacent section with four (4) screws. Depending on the desired
mass distribution, one (1) or two (2) steel plates are connected at the interface between
sections. This design allows for user to easily modify the structure by removing or replacing
any beam element without disassembling the structure. Each beam assembly consists of a
0.5 x 0.125 x 7.75 in. steel rod with two (2) steel attachment pads. Furthermore, each
pad is attached to the adjacent section with the use of four (4) screws.
A light weight exciter was used for generating a random force prole. It was important
that the exciter doesn't aect the dynamics of the structure. The actuator shown in
Figure 6.2b, was used to generate broadband output force that excited the rst three (3)
modes of the structure. The force generation was fully-automatic operation for repetitive
testing, using the same random force prole.
6.3.1 Nonlinear gap element
To introduce nonlinear eects in the dynamics of the re-congurable testbed structure,
an adjustable gap element simulating a bumper (rigid boundary), was added to specic
modular sections in the structure. The adjustable nonlinear gap element consisted of two
hefty aluminum brackets with an adjustment bolt between the two. A close-up of the gap
element is shown in Figure 6.2c. To adjust the gap, simply loosen the nut and adjust the
length of the bolt. Also shown in the close-up is the impact plate upon which the bolt
head hits. This steel plate can easily be replaced should it get worn from the impacts.
159
Alternatively, the steel impact plate can be replaced with rubber of various densities to
vary the impact force prole.
6.3.2 Instrumentation
Six (6) Endevco 7290E variable capacitance accelerometers were used to measure accelera-
tion in the testbed structure. The sensors had a full-scale range of10g with a frequency
bandwidth of 0-500 Hz. Endevco DC amplier Model 136 was used as signal conditioner
and power supply units for the accelerometers. A Honeywell Model 11 subminiature
tension/compression load cell with a load range of 25 lb. was used to measure the dynamic
load applied by the electromagnetic exciter to the testbed structure. A Sensotec UBP-10
universal in-line transducer amplier was used to provide the bridge excitation to the load
cell and to amplify the signal of the transducer.
The main data acquisition system (DAQ) consisted of a National Instruments PXIe-1082
chassis with a 24-bit PXIe-4492 module. This module had eight (8) simultaneously sampled
analog input channels that can sample up to 204.8 kS/s, with a 114 dB dynamic range.
This system was used to digitize the measurements from the accelerometers and the load
cell. Additional LabJack-U6 devices were used for the control system of the electromagnetic
exciter and programmable adaptive nonlinear element.
6.4 Data processing
Eight (8) dierent tests were carried out for this study. The rst test (test#1) corresponded
to the baseline or reference condition where the structure response was within the linear
160
(a) Overview of the testbed.
(b) Close-up of electromagnetic exciter. (c) Close-up of nonlinear gap element.
Figure 6.2: Photograph of testbed components details.
range. For the remaining tests, structural modications were made to the reference
structure to introduce both linear and nonlinear changes in the system. In test#2, one of
the beam element was replaced by an element with a 40% reduction in the cross section
area in the fth structural module (module 5). For test#3, two elements in module 5
were replaced by elements with 40% reduced cross-section. The bottom beam elements of
module 5 was completely removed from the structure in test#4. Another type of structural
change made to the reference testbed structure was the loosening of the bolted connections
between structural modules. In test#5, the bolts in one of the connections between the
161
modules 4 and 5 were loosened. For the test#6, in addition to the change done in test#5,
two adjustable gap elements were used in modules 2 and 4 to introduce nonlinear structural
changes into the testbed structure. By varying the size of the gap within the adjustable
nonlinear element, the level of the nonlinearity changes. In both nonlinear elements used,
the gap was set to 0.1 mm. For test#7, the nonlinear case was only considered. The
introduced structural changes consisted only of the nonlinear gap elements in modules
2 and 4, both with still the same 0.1-mm gap. For test#8 the nonlinear gap was only
introduced in module 2. The gap size was chosen 0.1 mm. Table 6.1 summarizes the
dierent tests carried out.
Table 6.1: Summary of structural tests.
Test Description
Test#1 Reference condition
Test#2 One (1) element with reduced cross-section in module 5
Test#3 Two (2) elements with reduced cross-section in module 5
Test#4 One (1) element removed from module 5
Test#5 One (1) loose connection between modules 4 and 5
Test#6
One (1) loose connection between modules 4 and 5
Two (2) nonlinear elements with a 0.1-mm gap in modules 2 and 4
Test#7 Two (2) nonlinear elements with a 0.1-mm gap in modules 2 and 4
Test#8 One (1) nonlinear element with a 0.1-mm gap in module 2
For each structural state (test), the structure was subjected to a sequence of forty (40)
forced vibration under 20-seconds long band-limited white noise excitation. Each forced
vibration test includes more than 100 fundamental periods of the system. The applied
load had a frequency bandwidth of 3-35 Hz, and a root-mean square value (RMS) of
162
1 lb. Six (6) uniaxial accelerometers, with a frequency bandwidth of DC-500 Hz, were
used to measure the lateral dynamic response in each of the six structural modules
composing the testbed structure. The acceleration measurements were acquired at a
sampling frequency of 500 Hz. The approximate location of the accelerometers, as well as
the sensor and module numbering used are shown in Figure 6.3. The acceleration records
were windowed, detrended, band-pass ltered and numerically integrated in order to obtain
the displacement and velocity time-histories at each of the measurement locations.
Figure 6.3: Location of the accelerometers deployed on testbed structure.
Since the proposed approach is data-driven, the number of degrees-of-freedom (DOF) for
the reduced-order model was determined based on the number of deployed sensors. A six
(6) DOF system was considered for the testbed structure. In addition, it was assumed
that the structure's mass is uniform throughout the structure, therefore the mass ratios
m
ij
are equal to unity. Sample of measured acceleration, and corresponding computed
velocity, and displacement time-histories for module 2 are shown in Figure 6.4a. Note that
the lateral displacements values are within one millimeters.
163
10 11 12 13 14
-2
0
2
10
-3
10 11 12 13 14
-0.1
0
0.1
10 11 12 13 14
-10
0
10
(a)
10 11 12 13 14
-1
0
1
10
-3
10 11 12 13 14
-0.05
0
0.05
10 10.5 11 11.5 12
-20
0
20
Experimental Reconstructed
(b)
Figure 6.4: Sample of (a) acceleration, velocity, and displacement time-histories for module
2, and (b) relative displacement and relative velocity computed between modules 2 and
3 are illustrated in the rst two rows. The third row shows a comparison between the
measured restoring force time-history for element G
(2)
and the reconstructed time-history
using the identied restoring force coecients from the linear system in the reference
structural conguration test#1 (the 2 curves almost match).
6.5 Experimental results
6.5.1 Identication of linear testbed structure
The relative displacements and velocities were computed before applying the proposed
time-domain identication technique explained in Section 6.2. Then, the non-parametric
reduced-order models for each element in the 6DOF chain-like system were built by
determining the corresponding restoring force coecients (C
(i)
qr
and a
(i)
qr
). The rst two
plots in Figure 6.4b show sample time-histories of relative displacements and velocities
164
computed between the second and rst
oor of the testbed structure in its reference
structural conguration (test#1). The third plot illustrates the time-history records of the
measured and reconstructed mass-normalized restoring forces estimated using Chebychev
series in solid and dashed lines, respectively, for the element G
(2)
connecting the rst and
second modules. The two curves almost match.
It is important to mention that for this study, the system was not assumed linear, and a
fth-order Chebyshev polynomials in both normalized variables z
0
and _ z
0
was chosen to
identify the restoring force for all structural modules in the testbed structure. The analysis
of Chebyshev coecients C
(i)
qr
indicated that the linear terms had the most signicant
contributions to the restoring forces while the eect of the nonlinear terms were negligible.
For the sake of brevity, the identied Chebyshev coecients C
(2)
qr
of the non-parametric
representation for module 2, computed from one of the forty tests are illustrated in
Table 6.2. It can be seen that the linear term associated with the relative displacements
in the non-parametric representation (C
(2)
10
), had the most signicant contribution to the
restoring force G
(2)
, however the nonlinear terms had negligible eect.
Due to structural changes, the estimating restoring force surface for each module will be
aected with respect to the reference condition. This is due to the changes encountered in
the dynamic characteristics and response time-histories of the chain-like system. Figure 6.5
shows a comparison for the reconstructed mass normalized restoring force surface using
the non-parametric representation for the restoring surface forces for module 2 (i.e., G
(2)
)
and module 5 (i.e., G
(5)
). It can be seen, for test#1 (i.e., the reference condition) and
test#4 (i.e. one element removed from module 5) the restoring force surface is almost
165
Table 6.2: Identied Chebyshev restoring force coecients C
(i)
qr
for module 2 from the
linear system in the reference structural conguration (test#1).
T
0
( _ z
0
) T
1
( _ z
0
) T
2
( _ z
0
) T
3
( _ z
0
) T
4
( _ z
0
) T
5
( _ z
0
)
T
0
(z
0
) 0.244 0.405 0.0734 -0.202 0.0380 0.0671
T
1
(z
0
) 20.236 0.097 1.391 -0.278 -0.029 0.036
T
2
(z
0
) 0.183 0.683 -0.091 -0.115 0.007 0.048
T
3
(z
0
) 1.266 0.032 0.015 -0.017 -0.007 -0.006
T
4
(z
0
) 0.181 0.247 0.054 0.001 -0.116 -0.140
T
5
(z
0
) 0.131 -0.043 -0.031 0.000 -0.165 0.024
planar regardless the use of a fth-order expansion on both state variables to characterize
the dynamics of each structural module.
In order to evaluate the assumed reduced order model in capturing the main dynamic
features of the system, the phase plots of the restoring force versus the relative displacement
and relative velocity of module 5 for the reference condition and test#4 are compared. In
addition, the experimental phase plots (blue line) and the reduced order representation
(dotted red) are compared as shown in Figure 6.6. The reduced order representation for
the restoring force could replicate the dominant dynamic features in the time response
exhibit by the structural module.
6.5.2 Identication of nonlinear testbed structure
In this section, the identication results for test#7 (the introduction of two nonlinear
gaps in modules 2 and 4) are discussed to illustrate the capability of the decomposition
166
-20
-0.02
0
5
20
0
10
-4 0
0.02
-5
(a)
-0.05
-10
0
5
10
0
10
-4
0
-5
0.05
(b)
Figure 6.5: Identied restoring force surfaces (a) for element G
(2)
for the reference linear
condition (test#1) and after removal of one element from module 5 (test#4), and (b)
for element G
(5)
for the reference condition (test#1) and after removal of one element
from module 5 (test#4). Identied restoring force surfaces for test#1 are transparent.
Note that, for enhanced viewing, dierent amplitude scales are used in the LHS and RHS
columns of plots and the surface plots for LHS are almost the same.
ChainID approach for extracting the nonlinear characteristics for the nonlinear structure
modules. As shown before for the linear system, samples of measured acceleration, and
corresponding computed velocity and displacement time-histories for module 2 are shown
in Figure 6.7a. The rst two plots in Figure 6.7b show sample time-histories of relative
displacements and velocities computed between the second and rst
oor of the testbed
structure after the introduction of nonlinear gap in modules 2 and 4. The third plot
illustrates the time-history records of the measured and reconstructed mass-normalized
restoring forces in solid and dashed lines, respectively, for the element G
(2)
connecting the
rst and second modules. It can be seen that the 2 curves almost match for the nonlinear
case.
167
-1 -0.5 0 0.5 1
10
-3
-15
-10
-5
0
5
10
15
-0.05 0 0.05
-15
-10
-5
0
5
10
15
(a)
-1 -0.5 0 0.5 1
10
-3
-15
-10
-5
0
5
10
15
-0.05 0 0.05
-15
-10
-5
0
5
10
15
(b)
Figure 6.6: Phase plots of (a) restoring force G
(5)
versus relative displacement z
5
and
restoring force G
(5)
versus relative velocity _ z
5
for module 5 in the reference conguration
(test#1), and (b) restoring force G
(5)
versus relative displacement z
5
and restoring force
G
(5)
versus relative velocity _ z
5
after removal of one element from module 5 (test#4). The
blue solid lines correspond to the phase plot for the measured restoring force, however the
red dotted lines correspond to the reconstructed time history for the restoring force.
Similarly to the identication of the reference condition case, a fth order Chebyshev
polynomial was assumed in both normalized variables z
0
and _ z
0
. Table 6.3 shows the
identied Chebyshev coecients C
(2)
qr
for one of the forty tests for the module 2. It is clear
that the linear coecient C
(2)
10
still had the largest contribution. In addition, a signicant
contributions of higher-order (C
(2)
20
and C
(2)
30
) and cross-product terms (C
(2)
11
, C
(2)
22
, C
(2)
21
,
C
(2)
22
) were observed in modules 2 and 4, however, the contribution of these terms were
168
10 11 12 13 14
-1
0
1
10
-3
10 11 12 13 14
-0.1
0
0.1
10 11 12 13 14
-10
0
10
(a)
10 11 12 13 14
-5
0
5
10
-4
10 11 12 13 14
-0.05
0
0.05
10 10.5 11 11.5 12
-20
0
20
Experimental Reconstructed
(b)
Figure 6.7: Sample of (a) acceleration, velocity, and displacement time-histories for module
2 (test#7), and (b) relative displacement and relative velocity computed between modules
2 and 3 (test#7) are illustrated in the rst two rows. The third row shows a comparison
between the measured restoring force time-history for element G
(2)
and the reconstructed
time-history using the identied restoring force coecients from the nonlinear system in
the test#7 (the 2 curves almost match).
negligible for the other modules. The presence of the combination of the higher order and
cross product terms is due to the introduction of nonlinear gap and the eect of impacts
against a rigid barrier.
Furthermore, the presence of nonlinearity can be visually inspected from the reconstructed
restoring force surface for G
(2)
shown in Figure 6.8b, compared to the restoring surface
plot for the same channel in linear case shown in Figure 6.8a. Additionally, the eects
of the nonlinear element in the response of the structural module 2 can be seen in both
phase plots of the restoring force compared to the reference condition shown in Figure 6.9a
169
Table 6.3: Identied Chebyshev restoring force coecients C
(i)
qr
for module 2 after the
introduction of nonlinear gap element in module 2 (nonlinear conguration [test#7]).
T
0
( _ z
0
) T
1
( _ z
0
) T
2
( _ z
0
) T
3
( _ z
0
) T
4
( _ z
0
) T
5
( _ z
0
)
T
0
(z
0
) -2.7629 3.9958 0.9135 -0.5643 0.1023 -0.0173
T
1
(z
0
) 19.0428 -3.4901 1.5322 -2.5155 0.0595 0.0034
T
2
(z
0
) 1.8364 4.2551 3.2261 -0.0799 -0.3286 0.0583
T
3
(z
0
) 2.4623 0.0868 0.1476 -0.0955 -0.1364 0.1367
T
4
(z
0
) -0.4789 0.1798 0.0985 -0.1503 0.0094 -0.0614
T
5
(z
0
) -0.0345 0.1938 0.0791 -0.0856 -0.1438 -0.1258
for module 2. In the phase plot of restoring force and relative displacement shown in
Figure 6.9b, the change in the restoring force slope, around a relative displacement of z
2
= 0.1 mm, shows a hardening eect in the introduced nonlinearity and the hardening
eect is only observed in one side of the restoring force as the introduced nonlinearity
is not symmetric. A pinching in the restoring force can be seen in the plot of restoring
force versus relative velocity which is typical signature of impacts against a rigid barrier
where abrupt and rapid changes in the velocity take place. It is important to mention
that the nonparametric reduced-order representation captured the dominant features of
the dynamics in the modules where the nonlinear-gap element is attached.
170
-20
-0.02
0
5
0
20
10
-4
0
0.02
-5
(a)
-20
-0.04
0
5
-0.02
20
0
10
-4
0
0.02
-5
(b)
Figure 6.8: Identied restoring force surfaces (a) for element G
(2)
for module 2 in the
reference linear condition (test#1), and (b) for element G
(2)
in module 2 after the
introduction of nonlinear gap element in module 2 (nonlinear conguration [test#7]). The
black dots correspond to the measured restoring force.
6.6 Analysis and discussion
6.6.1 Statistical analysis
The Chebyshev coecients C
(i)
qr
are inconvenient for the use for change detection in linear
and nonlinear system, since they rely on normalized variables z
0
and _ z
0
. Therefore, the
identied mass normalized restoring force coecientsa
(i)
qr
are selected as features for change
detection in uncertain chain-like systems [118]. As a consequence of the variation in the
restoring force due to changes in the testbed, the identied mass normalized restoring
force coecients a
(i)
qr
(i = 1, . . . , 6) are aected [182]. Therefore, the identied restoring
force coecients are a suitable set of parameters for change detection applications. It is
important to mention that thea
(i)
10
coecients are the dominant terms for the identication
171
-1 -0.5 0 0.5 1
10
-3
-30
-20
-10
0
10
20
30
-0.05 0 0.05
-30
-20
-10
0
10
20
30
(a)
-1 -0.5 0 0.5 1
10
-3
-30
-20
-10
0
10
20
30
-0.05 0 0.05
-30
-20
-10
0
10
20
30
(b)
Figure 6.9: Phase plots of (a) restoring force G
(2)
versus relative displacement z
2
and
restoring force G
(2)
versus relative velocity _ z
2
for module 2 in the reference conguration
(test#1), and (b) restoring force G
(2)
versus relative displacement z
2
and restoring force
G
(2)
versus relative velocity _ z
2
after the introduction of nonlinear gap element in module
2 (nonlinear conguration [test#7]). The blue solid lines correspond to the phase plot for
the measured restoring force, however the red dotted lines correspond to the reconstructed
time history for the restoring force.
of the inter-story restoring forces for all tests considered, which is typically encountered
in shear building-type structures. The linear stiness-like coecients a
(i)
10
, the quadratic
stiness-like coecients a
(i)
20
and cubic stiness-like coecients a
(i)
30
are deployed for the
detection and localization of structural changes. The a
(i)
01
coecients are also reported for
the sake of completeness of the identication approach.
172
For each test, forty (40) reduced order models were built using the non-parametric
representation of the restoring force for each of the six modules in the testbed structure.
For each model, the mass normalized restoring force coecients a
(i)
qr
(i = 1, . . . , 6) were
computed. In order to describe and summarize the variability observed in the identied
a
(i)
qr
coecients, the probability density function (pdf) for each of the coecients was
estimated from the coecients realizations using kernel density estimation. Figures 6.10,
6.11, and 6.12 illustrate the pdfs of mass normalized coecients for a
(i)
10
, a
(i)
01
, a
(i)
20
, and a
(i)
30
for the baseline condition in solid lines, while the probability functions from the modied
structural congurations are plotted with dash-lines, dots-lines, and dash-dot lines.
1 1.1 1.2 1.3 1.4 1.5 1.6
10
4
0
0.5
1
1.5
2
10
-3
Test 1
Test 3
Test 4
Test 5
-20 -15 -10 -5 0 5
0
0.1
0.2
0.3
0.4
Test 1
Test 3
Test 4
Test 5
-100 -50 0 50 100
0
0.005
0.01
0.015
0.02
Test 1
Test 3
Test 4
Test 5
-0.1 -0.05 0 0.05 0.1
0
20
40
60
Test 1
Test 3
Test 4
Test 5
Figure 6.10: Comparison of probability density functions of the identied mass-normalized
restoring force coecients a
(5)
10
, a
(5)
01
, a
(5)
20
, a
(5)
30
from the reduced order representation of
structural module 5 for the testbed structure in the linear dynamic response congurations
(test#1, test#3, test#4, test#5). The solid blue lines correspond to the pdf of the
coecients in the baseline condition (test#1).
173
2 2.5 3 3.5
10
4
0
1
2
3
10
-4
Test 1
Test 7
Test 8
0 10 20 30 40
0
0.1
0.2
0.3
Test 1
Test 7
Test 8
-150 -100 -50 0 50 100 150 200
0
0.005
0.01
0.015
Test 1
Test 7
Test 8
-0.4 -0.2 0 0.2 0.4
0
10
20
30
Test 1
Test 7
Test 8
Figure 6.11: Comparison of probability density functions of the identied mass-normalized
restoring force coecients a
(2)
10
, a
(2)
01
, a
(2)
20
, a
(2)
30
from the reduced order representation of
structural module 2 for the testbed structure in the linear (test#1) and nonlinear (test#7,
test#8) dynamic response congurations. The solid blue lines correspond to the pdf of
the coecients in the baseline condition (test#1).
For better visualization of the analysis of data for change detection, the mean and
coecient of variation c
v
for the pdfs of the mass normalized coecients a
(i)
10
, a
(i)
01
, a
(i)
20
,
and a
(i)
30
computed for modules 2, 4, and 5 for all tests are summarized in Tables 6.4 and
6.5. It was observed from the mean and coecient of variation reported in Tables 6.4
and 6.5 that the mass-normalized stiness-like coecients a
(i)
10
have a low variability, with
coecients of variation ranging from 4% to 6%, compared to the more scattered a
(i)
01
, a
(i)
20
,
and a
(i)
30
coecients. However, when the nonlinear the gap is introduced in the testbed
and its eect started to become signicant in the dynamics of the structure, noticeable
reductions in the dispersion of the quadratic a
(i)
20
and cubic stiness-like coecients a
(i)
30
.
As can be seen in Tables 6.4 and 6.5, the variability in each coecient is due to the noise
174
2 2.5 3 3.5
10
4
0
1
2
3
4
10
-4
Test 1
Test 6
0 10 20 30 40 50
0
0.1
0.2
0.3
Test 1
Test 6
-100 0 100 200
0
0.005
0.01
0.015
Test 1
Test 6
-0.4 -0.2 0 0.2 0.4
0
10
20
30
Test 1
Test 6
Figure 6.12: Comparison of probability density functions of the identied mass-normalized
restoring force coecients a
(2)
10
, a
(2)
01
, a
(2)
20
, a
(2)
30
from the reduced order representation
of structural module 2 for the testbed structure in the linear (test#1) and combined
linear/nonlinear (test#6) dynamic response congurations. The solid blue lines correspond
to the pdf of the coecients in the baseline condition (test#1).
in measurement, data processing errors, as well as the model-order reduction performed
in the identication procedure. (for example, the coecient of variation for the second
module for the quadratic stiness-like coecient a
(2)
20
decreased from an initial value of
2.802 for test#1 (reference condition) to 0.56 for test#7 (when 0.1 mm-gap was introduced
to the system). Similarly, for the variation in the cubic stiness-like term drop from 12.463
to 0.858. Similar behavior was observed in module 2 for tests#6 and #8.
6.6.2 Change detection
In order to detect and localize structural changes, two normalized indices =
r
and =
r
were deployed to assess the eectiveness and robustness of the identication approach,
175
1.1 1.2 1.3 1.4 1.5 1.6
10
4
0
0.5
1
10
-3
Test 1
Test 6
-30 -25 -20 -15 -10 -5 0 5
0
0.1
0.2
0.3
Test 1
Test 6
-100 -50 0 50 100
0
0.005
0.01
0.015
Test 1
Test 6
-0.04 -0.02 0 0.02 0.04 0.06
0
20
40
60
Test 1
Test 6
Figure 6.13: Comparison of probability density functions of the identied mass-normalized
restoring force coecients a
(5)
10
, a
(5)
01
, a
(5)
20
, a
(5)
30
from the reduced order representation
of structural module 5 for the testbed structure in the linear (test#1) and combined
linear/nonlinear (test#6) dynamic response congurations. The solid blue lines correspond
to the pdf of the coecients in the baseline condition (test#1).
where
r
and
r
are the mean and standard deviation of the coecients computed from
the reference condition (test#1). is the dierence in the mean values of the estimated
restoring force coecients ( = -
r
). The dimensionless index =
r
, which re
ects
the relative change in the coecients, can be used to assess the level of change in the
testbed, while =
r
, which is the signal-to-noise ratio (SNR), is a measure of the
statistical signicance of the detected changes. Relatively large values of the SNR index
indicate that the mean dierences are signicant, and therefore they can be associated
with changes in the system. However, smaller values of the SNR mean the changes in
the coecients are likely related to the variability within the identied coecients [114].
In this study, the detection threshold was assumedj=
r
j 2. For the sake of brevity,
176
Table 6.4: Summary of mean and coecient of variation c
v
of the identied mass
normalized restoring force coecients a
10
, a
01
for modules 2, 4, and 5 for the testbed
structure.
Test
Module 2 Module 4 Module 5
a
(2)
10
= k
2
a
(2)
01
= c
2
a
(4)
10
= k
4
a
(4)
01
= c
4
a
(5)
10
= k
5
a
(5)
01
= c
5
(10
4
) c
v
c
v
(10
4
) c
v
c
v
(10
4
) c
v
c
v
Test#1 2.486 0.061 7.083 0.296 1.594 0.040 -3.581 0.912 1.368 0.031 -4.849 0.313
Test#2 2.474 0.056 6.394 0.263 1.586 0.033 -3.471 0.706 1.369 0.023 -4.608 0.283
Test#3 2.461 0.061 7.065 0.249 1.599 0.038 -3.919 0.550 1.358 0.027 -4.914 0.276
Test#4 2.436 0.040 10.344 0.192 1.678 0.023 -6.957 0.258 1.091 0.019 -7.309 0.207
Test#5 2.415 0.054 14.285 0.159 1.308 0.025 -6.360 0.351 1.274 0.035 -14.383 0.079
Test#6 2.872 0.038 30.424 0.224 1.592 0.038 -5.871 0.442 1.286 0.032 -19.523 0.074
Test#7 2.889 0.041 26.907 0.187 1.770 0.035 -6.786 0.397 1.364 0.024 -10.762 0.102
Test#8 2.896 0.048 23.430 0.251 1.615 0.034 -8.399 0.328 1.357 0.029 -8.086 0.167
Table 6.5: Summary of mean and coecient of variation c
v
of the identied mass
normalized restoring force coecients a
20
, a
30
for modules 2, 4, and 5 for the testbed
structure.
Test
Module 2 Module 4 Module 5
a
(2)
20
a
(2)
30
a
(4)
20
a
(4)
30
a
(5)
20
a
(5)
30
c
v
(10
3
) c
v
c
v
(10
3
) c
v
c
v
(10
3
) c
v
Test#1 4.857 2.802 8.515 12.463 4.187 9.123 11.693 3.213 -3.044 8.958 4.728 2.748
Test#2 4.384 11.766 38.125 2.131 11.686 2.744 3.760 7.403 2.155 12.729 2.275 7.603
Test#3 7.363 1.935 28.016 3.786 9.125 3.086 4.084 8.763 -0.373 60.935 3.191 4.090
Test#4 5.291 6.804 12.794 6.537 12.034 2.142 1.734 16.236 9.832 2.798 0.337 64.519
Test#5 27.187 0.974 25.272 1.580 20.039 2.473 21.615 1.483 -4.395 5.853 2.304 3.390
Test#6 94.604 0.666 17.568 0.839 104.603 0.603 11.425 1.647 -10.552 2.127 -0.068 107.775
Test#7 63.242 0.560 23.927 0.858 63.205 0.523 4.038 3.224 -11.184 1.708 -6.025 1.448
Test#8 65.566 0.455 22.396 0.737 10.542 2.171 3.425 6.374 0.598 27.289 0.288 28.788
the values of =
r
and =
r
for all tests are summarized and presented in Tables 6.6
177
and 6.7 for modules 2, 4, and 5 for the mass normalized coecients a
(i)
10
, a
(i)
01
, a
(i)
20
. Table
entries in bold correspond to the detected structural changes.
Table 6.6: Summary of relative mean change (=
r
) and signal-to-noise ratio (=
r
)
in the identied mass normalized restoring force coecients a
10
, a
01
for modules 2, 4, and
5 for the testbed structure.
Test
Module 2 Module 4 Module 5
a
(2)
10
= k
2
a
(2)
01
= c
2
a
(4)
10
= k
4
a
(4)
01
= c
4
a
(5)
10
= k
5
a
(5)
01
= c
5
=
r
=
r
=
r
=
r
=
r
=
r
=
r
=
r
=
r
=
r
=
r
=
r
Test#2 -0.005 -0.076 -0.097 -0.329 -0.005 -0.120 -0.031 0.034 0.001 0.024 -0.050 0.159
Test#3 -0.010 -0.160 -0.003 -0.009 0.003 0.072 0.094 0.103 -0.008 -0.244 0.014 0.043
Test#4 -0.020 -0.324 0.460 1.555 0.052 1.307 0.942 1.033 -0.202 -6.460 0.507 1.620
Test#5 -0.028 -0.462 1.017 3.435 -0.180 -4.472 0.776 0.850 -0.069 -2.198 1.966 6.276
Test#6 0.155 2.526 3.295 11.132 -0.002 -0.038 0.639 0.701 -0.060 -2.026 3.027 9.659
Test#7 0.162 2.637 2.799 9.454 0.111 2.754 0.895 0.981 -0.003 -0.100 1.220 3.892
Test#8 0.165 2.685 2.308 7.796 0.013 0.333 1.345 1.475 -0.008 -0.248 0.668 2.131
Detection and localization of linear changes
While the stiness of the fth module was reduced for tests #2 and #3, no presentable
changes were detected. This is due to the relatively small response of the structure (less
than 1 mm), as well as the fact that the structure was very sti compared to its light mass
and the excitation level. It is important to clarify in this case study that for the change
in mass normalized damping like coecients were excluded from the damage detection
procedures. However, the results were reported for sake of completeness. In test#4, the
stiness of the fth module was reduced by removing one column of the four columns of
the module. For each of the previously mentioned scenarios, it can be seen in Tables 6.6
and 6.7 that the detected relative changes in the mean of the identied coecient a
(5)
10
178
Table 6.7: Summary of relative mean change (=
r
) and signal-to-noise ratio (=
r
)
in the identied mass normalized restoring force coecient a
20
for modules 2, 4, and 5 for
the testbed structure.
Test
Module 2 Module 4 Module 5
a
(2)
20
a
(4)
20
a
(5)
20
=
r
=
r
=
r
=
r
=
r
=
r
Test#2 -0.097 0.019 1.791 0.196 -1.708 0.191
Test#3 2.575 0.500 1.179 0.129 -0.877 0.098
Test#4 0.089 0.017 1.874 0.205 -4.230 0.472
Test#5 4.597 0.893 -5.786 0.634 0.444 0.050
Test#6 18.478 3.590 23.980 2.629 2.466 0.275
Test#7 12.021 2.335 14.094 2.545 2.673 0.298
Test#8 12.499 2.428 1.518 0.166 -1.196 0.134
was the most signicant, with corresponding value ofj=
r
j = 6.46 with detected mean
change in the fth module stiness-like coecient =
r
= -0.202. Similarly, for test#5
a reduction in the stiness by loosening one of the four connections between the fourth
and fth module. The detected relative changes in the mean of the identied coecient
a
(5)
10
and a
(5)
10
were the most signicant, with corresponding value ofj=
r
j = 4.472 and
j=
r
j = 2.198 and the level of change in the fourth and fth modules stiness-like
coecient were =
r
= -0.180 and =
r
= -0.069 respectively.
As expected, the estimated stiness-like coecients for the fourth and fth modules had
the largest and most signicant variations among all other coecients, as the structural
changes in these tests consisted of column- stiness reductions (it can be seen that the
179
level of changes correlate with the structural modication introduced in the testbed). The
same conclusion can be extracted from the change in slope (i.e., the stiness) of the phase
plot (G
(5)
z
5
) shown in Figure 6.6, and the change restoring force surface with respect
to the reference condition illustrated in Figure 6.8. Furthermore, from the analysis of the
the probability density functions depicted in Figure 6.10, the mean values of quadratic
a
(5)
20
and cubic stiness-like a
(5)
30
coecients the PDFs are almost zero. As well as large
variation in the identied coecients was observed, which is expected since the system is
almost linear.
Detection and localization of nonlinear changes
The nonlinearity was introduced by adjusting the bumper (nonlinear element) gap size to
0.1 mm in modules 2 and 4 for test#7 and module 2 for test#8. The results in Tables
6.6 and 6.7 indicated that signicant changes in the coecients for module 2 a
(2)
20
(for
tests#7 and #8) and module 4 a
(4)
20
(for test#7) only. In addition, a signicant increase of
16% in linear stiness coecient was noticed in the modules were the nonlinear element
was introduced. This increment was expected because of the stiness hardening eects
produced by the 0.1 mm-gap. Similar conclusion can be seen from the inspection of the
estimated probability density functions for the restoring force coecients a
(2)
10
, and a
(2)
20
.
The nonlinear element caused a clear change in the stochastic characteristics of the restoring
force coecients. For a
(2)
30
and a
(4)
30
the dispersion in the estimated probability density
functions signicantly decreased. However, even though the change in a
(5)
20
coecient is
relatively large but it could not considered signicant since =
r
is around 0.3 which is
less than the predened threshold.
180
Detection and localization of combined linear and nonlinear changes
For the scenario of test#6, the stiness of modules 4 and 5 were decreased by loosening
one of the connections between the two modules. In addition, the nonlinear eect was
introduced by adjusting the bumper (nonlinear element) gap size to 0.1 mm in modules 2
and 4. Since the structural changes in this case consisted of stiness reduction in modules
4 and 5 (linear changes) and 0.1 mm-gap in modules 2 and 4 (nonlinear changes), it was
expected that the estimated mass-normalized stiness-like coecients a
(2)
10
, a
(4)
10
, and a
(5)
10
would have signicant statistically variation; as well as, higher order coecients a
(2)
20
, and
a
(4)
20
for modules 2 and 4 (where the nonlinear changes were introduced).
It can be seen from Tables 6.6 and 6.7 that a signicant positive change for the mass-
normalized stiness-like coecient a
(2)
10
was detected in module 2 with corresponding
=
r
= 2.5, which was expected due to the hardening eect introduced by the nonlinear
gap element. As well as, a signicant change in a
(2)
20
was noticed with corresponding
=
r
= 3.59, and the dispersion for a
(2)
30
in the estimated probability density functions
signicantly decreased as illustrated in Figure 6.12. For module 4, no change was detected
in the stiness-like term a
(4)
10
. This can be explained that the reduction is stiness caused
by the loosening one of the connection was counteracted by the hardening eect of the
nonlinear gap. However, the nonlinear print was re
ected in the change detected in a
(4)
20
coecient with corresponding =
r
= 2.6. Finally, for module 5, only statistically
signicant reduction was detected in the mass-normalized stiness-like coecient a
(5)
10
with =
r
= = -2.026, with no signicant changes detected in the nonlinear coecients.
Analysis of the probability density functions shown in Figures 6.12 and 6.13 will lead to
similar conclusions to what already described ndings.
181
6.7 Change detection using global identication
In addition to the local non-parametric representation for each module using ChainID, a
full-order nite element model of the testbed structure was built. The results from the
global identication (i.e., modal parameters) were estimated using the identied restoring
force coecients. Since, the identied restoring force coecients a
(i)
qr
are random, the
estimated modal parameters were also random quantities. The pdfs of the rst three
natural frequencies were estimated for all tests using the linear stiness-like a
(i)
10
and
damping-like a
(i)
01
restoring force coecients. The estimated modal parameters using the
ChainID were compared to the parameters identied from the nite element model, and by
implementing global identication methods such as natural excitation technique (NExT)
[76, 26] in combination with eigensystem realization algorithm (ERA) [82], and system
realization using information matrix (SRIM) [81]. For the reference condition (test#1), the
mean values of the numerical and experimental frequencies as well as the damping ratios
for the rst three modes are summarized in Table 6.8. It can be seen from Table 6.8 that
the mean natural frequencies for the rst three modes were comparable to the frequencies
obtained from the nite element analysis and global identication methods. Also, as
expected for very lightly damped structure the estimated damping was less than 1%.
For all other tests, the parameters from the rst three modes were computed using the
linear stiness-like a
(i)
10
and damping-like a
(i)
01
restoring force coecients. In addition,
the relative changes (=
r
) with respect the reference condition and SNR for the rst
three natural frequencies are summarized in Table 6.9. But again, these results show
some of the limitations that modal-based detection techniques, especially those related
182
Table 6.8: Comparison of numerical and experimental modal parameters.
Mode
f [Hz] [%]
FEM SRIM NExT/ERA ChainID SRIM NExT/ERA ChainID
1 5.360 5.600 5.472 6.136 0.930 0.621 0.822
2 15.820 17.420 17.620 16.505 0.510 0.539 0.486
3 25.840 29.580 27.977 25.505 0.340 - 0.116
natural frequencies. It is clear the global identication for this problem was not eective
in detecting, locating and estimating the level of changes in real structures.
Table 6.9: Global system identication and relative changes in experimentally identied
natural frequencies using ChainID.
Test
f
1
[Hz] f
2
[Hz] f
3
[Hz]
c
v
=
r
=
r
c
v
=
r
=
r
c
v
=
r
=
r
Test#1 6.136 0.009 - - 16.505 0.006 - - 25.050 0.008 - -
Test#2 6.122 0.008 -0.002 0.255 16.428 0.005 -0.005 0.823 24.952 0.006 -0.004 0.472
Test#3 6.112 0.008 -0.004 0.424 16.380 0.005 -0.008 1.333 25.011 0.008 -0.002 0.190
Test#4 6.039 0.011 -0.016 1.713 15.770 0.005 -0.044 7.843 26.329 0.008 0.051 6.164
Test#5 5.730 0.008 -0.066 7.116 17.322 0.007 0.049 8.726 26.023 0.018 0.039 4.689
Test#6 6.311 0.012 0.028 3.053 16.866 0.011 0.022 3.855 25.687 0.010 0.025 3.072
Test#7 6.491 0.011 0.058 6.209 16.259 0.007 -0.015 2.625 25.248 0.006 0.008 0.955
Test#8 6.268 0.008 0.021 2.304 16.581 0.006 0.005 0.811 25.211 0.008 0.006 0.774
183
6.8 Summary and conclusions
An experimental study was conducted to evaluate the eectiveness, and demonstrate the
robustness and reliability of a data-driven non-parametric identication technique for the
change/damage detection, localization and quantication in MDOF chain-like uncertain
complex system. Chain-like system can be typical structures in dierent engineering elds.
examples are: tall buildings, aircraft wings, transmission towers, and wind blades. A
simulated re-congurable experimental setup was designed, built and tested under dynamic
environment. Dierent scenarios for structural changes were introduced for the testbed.
Structural changes consisted of linear changes (i.e., change in modules stiness), nonlinear
changes (i.e., the gape 0.1 mm gap element), and combined linear/nonlinear changes (i.e.,
change in stiness and the introduction of gap element). The experimental data collected
was used to build reduced-order, reduced-complexity models for the restoring force, which
served for dual purposed; (1) for local and global system identication, and (2) for the
change/damage detection, localization and quantication for uncertain complex system.
The developed models consisted of a set of coecients (i.e. restoring force coecients) of
doubly indexed series expansions of the restoring forces in a stochastic framework.
The results of this study demonstrate the eectiveness and robustness of the proposed
methodology for identifying the linear (i.e., stiness like coecient) and nonlinear (i.e.,
high-order and cross-product coecients) dominant features, which were deployed for
the detection and localization of changes in the testbed structure despite the modeling,
measurement errors, and data processing. It was shown that the restoring force coecients
in linear and nonlinear congurations were proved to be reliable features for change
detection. Moreover, it was shown that the magnitude of change was correlated to the
184
restoring force coecients and the level detectable changes in the testbed structure. It is
important to note that the presented structure health monitoring scheme was performed
in a well-controlled laboratory conguration and therefore, the in
uence of the eect of
operational and environmental conditions were not incorporated into this study. Otherwise,
the method under discussion can provide a useful tool to accurately detect, locate, and
quantify the changes in the structure, as re
ected in the identied restoring force surface.
Acknowledgment
This study was supported in part by a grant from the King Abdulaziz City for Science
and Technology (KACST).
185
Chapter 7
Summary and Conclusions
T
HIS dissertation is a collection of studies focusing on the extension and evaluation
of methodologies and state-of-the-art sensing technologies for short and long-term
condition assessment and health monitoring of target structural systems. This work
identies and analyzes numerous practical challenges and complications that arise in the
application of SHM frameworks in order to take actionable decisions.
This study evaluated the robustness and the range of application of newly-developed
cost-eective 3D sensor, the RGB-D camera (e.g., the Microsoft Kinect) to quantify the
evolving 2D and 3D deformation elds in a continuous structural system, that are required
for numerous applications in the eld of condition assessment of infrastructure systems.
A comprehensive experimental and computational study was performed to evaluate the
performance envelope of a representative RGB-D sensor (the rst generation of Kinect
sensor) with the aim of assessing its suitability for the class of problems encountered in the
structural dynamics eld, where reasonably accurate information of evolving displacement
elds (as opposed to few discrete locations) that have simultaneous dynamic unidirectional
motion, planar translational motion, and translational motion with signicant rotational
186
(torsional) components. This study investigated the in
uence of key system parameters
of concern in selecting an appropriate sensor for such structural dynamic applications,
such as amplitude range, spectral content of the dynamic displacements, location and
orientation of sensors relative to target structure, fusing of measurements from multiple
sensors, sensor noise eects, and rolling-shutter eects. The results of this study showed
that the proposed approach can potentially be used as an economical and robust solution
for obtaining evolving displacements and torsional elds in realistic civil, mechanical, and
aerospace structural systems.
With the constant advancements in the state-of-the-art of sensing technologies for long-
term structural health monitoring, this research evaluated a class of newly developed
self-powered sensors (PFG) based on energy harvesting for damage detection for target
structural systems. The new sensor overcomes the problem of power supply which is of
signicant concern for the application of wireless sensors. An experimental study was
performed on a thin cantilever beam, subjected to dierent loading conditions to verify
the eciency of the proposed technology in detecting and locating damage. Dierent
piezoelectric transducers were mounted on the beam for both powering the sensor and
monitoring the damage progression. The changes of charge on the
oating-gates of the
sensor, due to electron injection, were considered as damage indicator parameters. The
results of this study demonstrate the robustness of the recently developed Michigan
State University (MSU) self-powered strain sensor in monitoring damage progression and
overcoming the power problem widely encountered in WSN.
Finally, for condition assessment and damage/change detection this research proceeded with
the development of reduced-order models using data from a high-delity 3D nite element
187
model developed based on experiential data acquired from a 52-story high-rise building
and a recongurable experiential lab setup. A powerful sub-structuring approach was
used to decompose the structure into its constituent elements and developed mathematical
models capable of accurately detecting changes and dierentiating between structural
changes associated with modications in the linear system properties, as opposed to
changes associated with nonlinear phenomena. Furthermore, as re
ected in the identied
restoring force surface, the sub-structuring approach used can provide a useful tool to
accurately detect, locate, quantify, and classify the changes in the structure. In addition,
this study showed that the presence and location of the actual physical changes introduced
in the testbed structures could be correlated to the changes identied in the stiness-like
parameters of the reduced-order models, even with the presence of errors in modeling,
measurement, and data processing.
188
Bibliography
[1] Adams, D. (2007). Health monitoring of structural materials and components: methods
with applications. John Wiley & Sons.
[2] Adewuyi, A. P., Wu, Z., and Serker, N. K. (2009). Assessment of vibration-based
damage identication methods using displacement and distributed strain measurements.
Structural Health Monitoring, 8(6):443{461.
[3] Ait-Aider, O., Andre, N., Lavest, J. M., and Martinet, P. (2006). Simultaneous
Object Pose and Velocity Computation Using a Single View from a Rolling Shutter
Camera. In Computer Vision ECCV 2006, volume 3952, pages 56{68.
[4] Aktan, A., Catbas, F., Grimmelsman, K., and Tsikos, C. (2000). Issues in infrastructure
health monitoring for management. Journal of Engineering Mechanics, 126(7):711{724.
[5] Alavi, A. H., Hasni, H., Lajnef, N., and Chatti, K. (2016a). Continuous health
monitoring of pavement systems using smart sensing technology. Construction and
Building Materials, 114:719{736.
[6] Alavi, A. H., Hasni, H., Lajnef, N., Chatti, K., and Faridazar, F. (2016b). Damage de-
tection using self-powered wireless sensor data: an evolutionary approach. Measurement,
82:254{283.
[7] Alavi, A. H., Hasni, H., Lajnef, N., Chatti, K., and Faridazar, F. (2016c). An
intelligent structural damage detection approach based on self-powered wireless sensor
data. Automation in Construction, 62:24{44.
[8] Alnowami, M., Alnwaimi, B., Tahavori, F., Copland, M., and Wells, K. (2012). A
quantitative assessment of using the Kinect for Xbox360 for respiratory surface motion
tracking. Proc. SPIE, 8316:83161T{83161T{10.
[9] Aminian, K. and Naja, B. (2004). Capturing human motion using body-xed sensors:
outdoor measurement and clinical applications. Computer animation and virtual worlds,
15(2):79{94.
[10] Andersen, M., Jensen, T., Lisouski, P., Mortensen, A., Hansen, M., Gregersen,
T., and Ahrendt, P. (2012). Kinect Depth Sensor Evaluation for Computer Vision
189
Applications. Technical report, Department of Electrical and Computer Engineering,
Aarhus University.
[11] ASCE (2017). Report Card for America's Infrustructure.
[12] Ayrulu-Erdem, B. and Barshan, B. (2011). Leg motion classication with articial
neural networks using wavelet-based features of gyroscope signals. Sensors, 11(2):1721{
1743.
[13] Badel, A., Sebald, G., Guyomar, D., Lallart, M., Lefeuvre, E., Richard, C., and Qiu,
J. (2006). Piezoelectric vibration control by synchronized switching on adaptive voltage
sources: Towards wideband semi-active damping. The Journal of the Acoustical Society
of America, 119(5):2815{2825.
[14] Ballester, J. and Pheatt, C. (2013). Using the Xbox Kinect sensor for positional data
acquisition. American Journal of Physics, 81(1):71.
[15] Baqersad, J., Niezrecki, C., and Avitabile, P. (2015). Extracting full-eld dynamic
strain on a wind turbine rotor subjected to arbitrary excitations using 3D point tracking
and a modal expansion technique. Journal of Sound and Vibration, 352:16{29.
[16] Barrows, D. (2007). Videogrammetric model deformation measurement technique for
wind tunnel applications. In 45th AIAA Aerospace Sciences Meeting and Exhibit, page
1163.
[17] Bartilson, D. T., Wieghaus, K. T., and Hurlebaus, S. (2015). Target-less computer
vision for trac signal structure vibration studies. Mechanical Systems and Signal
Processing, 60-61:571{582.
[18] Biswas, K. K. and Basu, S. K. (2011). Gesture recognition using microsoft kinect R
.
Proceedings of the 2011 5th International Conference on Automation, Robotics and
Applications (ICARA), pages 100{103.
[19] Borchani, W., Aono, K., Lajnef, N., and Chakrabartty, S. (2016). Monitoring of
postoperative bone healing using smart trauma-xation device with integrated self-
powered piezo-
oating-gate sensors. IEEE Transactions on Biomedical Engineering,
63(7):1463{1472.
[20] Bornn, L., Farrar, C. R., and Park, G. (2010). Damage detection in initially nonlinear
systems. International Journal of Engineering Science, 48(10):909{920.
[21] Bouguet, J. Y. (2008). Camera calibration toolbox for Matlab.
[22] Bradski, G. and Kaehler, A. (2008). Learning OpenCV: Computer Vision with the
OpenCV Library. OReilly Media Inc, Sebastopol, CA.
[23] Breuer, P., Chmielewski, T., G orski, P., and Konopka, E. (2002). Application of GPS
technology to measurements of displacements of high-rise structures due to weak winds.
Journal of Wind Engineering and Industrial Aerodynamics, 90(3):223{230.
190
[24] Breuer, P., Chmielewski, T., Gorski, P., Konopka, E., and Tarczynski, L. (2015).
Monitoring horizontal displacements in a vertical prole of a tall industrial chimney using
global positioning system technology for detecting dynamic characteristics. Structural
Control and Health Monitoring, 22(7):1002{1023. STC-14-0112.R1.
[25] Brewick, P. T. and Masri, S. F. (2016). An evaluation of data-driven identication
strategies for complex nonlinear dynamic systems. Nonlinear dynamics, 85(2):1297{1318.
[26] Caicedo, J. M., Dyke, S. J., and Johnson, E. A. (2004). Natural excitation technique
and eigensystem realization algorithm for phase i of the iasc-asce benchmark problem:
Simulated data. Journal of Engineering Mechanics, 130(1):49{60.
[27] C elebi, M., Prescott, W., Stein, R., Hudnut, K., Behr, J., and Wilson, S. (1999).
GPS Monitoring of Dynamic Behavior of LongPeriod Structures. Earthquake Spectra,
15(1):55{66.
[28] C elebi, M. and Sanli, A. (2002). GPS in pioneering dynamic monitoring of long-period
structures. Earthquake Spectra, 18(1):47{61.
[29] Chakraborty, D., Kovvali, N., Papandreou-Suppappola, A., and Chattopadhyay, A.
(2015). An adaptive learning damage estimation method for structural health monitoring.
Journal of Intelligent Material Systems and Structures, 26(2):125{143.
[30] Chang, C. C. and Ji, Y. F. (2007). Flexible Videogrammetric Technique for Three-
Dimensional Structural Vibration Measurement. Journal of Engineering Mechanics,
133(6):656{664.
[31] Chang, P. C., Flatau, A., and Liu, S. (2003). Health monitoring of civil infrastructure.
Structural health monitoring, 2(3):257{267.
[32] Chen, J. G., Wadhwa, N., Cha, Y.-j., Durand, F., and Freeman, W. T. (2015). Modal
identication of simple structures with high-speed video using motion magnication.
Journal of Sound and Vibration, 345:58{71.
[33] Chen, Y. L., Abdelbarr, M., Jahanshahi, M. R., and Masri, S. F. (2017). Color and
depth data fusion using an RGB-D sensor for inexpensive and contactless dynamic
displacement-eld measurement. Structural Control and Health Monitoring, pages 1{14.
[34] Chen, Y. L., Jahanshahi, M. R., Manjunatha, P., Gan, W., Abdelbarr, M., Masri,
S. F., Becerik-Gerber, B., and Carey, J. P. (2016). Inexpensive multimodal sensor
fusion system for autonomous data acquisition of road surface conditions. IEEE Sensors
Journal, 16(21):7731{7743.
[35] Cheng, C. and Kawaguchi, K. (2015). A preliminary study on the response of steel
structures using surveillance camera image with vision-based method during the Great
East Japan Earthquake. Measurement, 62:142{148.
191
[36] Cho, S., Yun, C.-B., Lynch, J. P., Zimmerman, A. T., Spencer Jr, B. F., and
Nagayama, T. (2008). Smart wireless sensor technology for structural health monitoring
of civil structures. Steel Structures, 8(4):267{275.
[37] Choi, H.-S., Cheung, J.-H., Kim, S.-H., and Ahn, J.-H. (2011). Structural dynamic
displacement vision system using digital image processing. NDT & E International,
44(7):597{608.
[38] Chow, J. C. K., Ang, K. D., Lichti, D. D., and Teskey, W. F. (2012). Performance
Analysis of a Low-Cost Triangulation-Based 3D Camera: Microsoft Kinect System.
ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial
Information Sciences, XXXIX-B5(September):175{180.
[39] Correa, D. S. O., Sciotti, D. F., Prado, M. G., Sales, D. O., Wolf, D. F., and Osorio,
F. S. (2012). Mobile robots navigation in indoor environments using Kinect sensor.
Proceedings of the 2012 Second Brazilian Conference on Critical Embedded Systems
(CBSEC), pages 36{41.
[40] Derkevorkian, A., Hernandez-Garcia, M., Yun, H.-B., Masri, S. F., and Li, P. (2015).
Nonlinear data-driven computational models for response prediction and change detec-
tion. Structural Control and Health Monitoring, 22(2):273{288.
[41] Derkevorkian, A., Masri, S. F., Fujino, Y., and Siringoringo, D. M. (2014). De-
velopment and validation of nonlinear computational models of dispersed structures
under strong earthquake excitation. Earthquake Engineering & Structural Dynamics,
43(7):1089{1105.
[42] Detchev, I., Habib, A., and El- Badry, M. (2013). Dynamic beam deformation
measurements with o-the-shelf digital cameras. Journal of Applied Geodesy, 7(3):147{
157.
[43] Doebling, S. W., Farrar, C. R., Prime, M. B., et al. (1998). A summary review of
vibration-based damage identication methods. Shock and vibration digest, 30(2):91{105.
[44] Doebling, S. W., Farrar, C. R., Prime, M. B., and Shevitz, D. W. (1996). Damage
identication and health monitoring of structural and mechanical systems from changes
in their vibration characteristics: a literature review.
[45] Dutta, T. (2012). Evaluation of the Kinect sensor for 3-D kinematic measurement in
the workplace. Applied Ergonomics, 43(4):645{649.
[46] El-laithy, R. A., Huang, J., and Yeh, M. (2012). Study on the use of Microsoft Kinect
for robotics applications. pages 1280{1288.
[47] Elvin, N., Elvin, A., and Choi, D. (2003). A self-powered damage detection sensor.
The Journal of Strain Analysis for Engineering Design, 38(2):115{124.
192
[48] Fabian, J., Young, T., Jones, J. C. P., and Clayton, G. M. (2014). Integrating
the microsoft kinect with simulink: Real-time object tracking example. IEEE/ASME
Transactions on Mechatronics, 19(1):249{257.
[49] Farinholt, K. M., Miller, N., Sifuentes, W., MacDonald, J., Park, G., and Farrar,
C. R. (2010). Energy harvesting and wireless energy transmission for embedded shm
sensor nodes. Structural Health Monitoring, 9(3):269{280.
[50] Farrar, C. R. and Doebling, S. W. (1997). An overview of modal-based damage
identication methods. Proceedings of DAMAS conference, pages 269{278.
[51] Farrar, C. R. and Worden, K. (2007). An introduction to structural health monitoring.
Philosophical Transactions of the Royal Society of London A: Mathematical, Physical
and Engineering Sciences, 365(1851):303{315.
[52] Feng, D. and Feng, M. Q. (2016). Vision-based multipoint displacement measurement
for structural health monitoring. Structural Control and Health Monitoring, 23(5):876{
890. STC-15-0104.R2.
[53] Fiedler, D. and M uller, H. (2013). Impact of thermal and environmental conditions
on the kinect sensor. In Advances in depth image analysis and applications, volume 7854
LNCS of Lecture Notes in Computer Science, pages 21{31, Tsukuba, Japan. Springer.
[54] Figueiredo, E., Park, G., Figueiras, J., Farrar, C., and Worden, K. (2009). Structural
health monitoring algorithm comparisons using standard data sets. Technical report,
Los Alamos National Laboratory (LANL), Los Alamos, NM (United States).
[55] F.K., C. (2017). Proceedings of the 11th International Workshop on Structural Health
Monitoring. DEStech Publications, Inc.
[56] Friswell, M. and Mottershead, J. E. (2013). Finite element model updating in structural
dynamics, volume 38. Springer Science & Business Media.
[57] Fukuda, Y., Feng, M. Q., and Shinozuka, M. (2010). Cost-eective vision-based
system for monitoring dynamic response of civil engineering structures. Structural
Control and Health Monitoring, 17(8):918{936.
[58] Garrido-Jurado, S., Mu~ noz-Salinas, R., Madrid-Cuevas, F. J., and Mar n-Jim enez,
M. J. (2014). Automatic generation and detection of highly reliable ducial markers
under occlusion. Pattern Recognition, 47(6):2280{2292.
[59] Geiger, A., Moosmann, F., Car, O., and Schuster, B. (2012). Automatic camera and
range sensor calibration using a single shot. In 2012 IEEE International Conference on
Robotics and Automation, pages 3936{3943. IEEE.
[60] Giurgiutiu, V., Lin, B., Santoni-Bottai, G., and Cuc, A. (2011). Space application of
piezoelectric wafer active sensors for structural health monitoring. Journal of Intelligent
Material Systems and Structures, 22(12):1359{1370.
193
[61] Gkatzogias, K. I. and Kappos, A. J. (2016). Semi-active control systems in bridge engi-
neering: A review of the current state of practice. Structural Engineering International,
26(4):290{300.
[62] Gordon, S. J. and Lichti, D. D. (2007). Modeling terrestrial laser scanner data
for precise structural deformation measurement. Journal of Surveying Engineering,
133(2):72{80.
[63] Guyomar, D., Richard, C., Badel, A., Lefeuvre, E., and Lallart, M. (2009). Energy
harvesting using non-linear techniques. In Energy Harvesting Technologies, pages
209{266. Springer.
[64] H. Sohn, C.R. Farrar, F. H. and Shunk, D. (2003). A review of structural health
monitoring literature. Los Alamos National Laboratory Report.
[65] H. Sohn, C.R. Farrar, F. H. and Shunk, D. (2007). Nonlinear system identication
for damage detection. Los Alamos National Laboratory Report.
[66] Hasni, H., Alavi, A. H., Jiao, P., and Lajnef, N. (2017a). Detection of fatigue cracking
in steel bridge girders: a support vector machine approach. Archives of Civil and
Mechanical Engineering, 17(3):609{622.
[67] Hasni, H., Alavi, A. H., Lajnef, N., Abdelbarr, M., Masri, S. F., and Chakrabartty, S.
(2017b). Self-powered piezo-
oating-gate sensors for health monitoring of steel plates.
Engineering Structures, 148:584{601.
[68] Helfrick, M. N., Niezrecki, C., Avitabile, P., and Schmidt, T. (2011). 3D digital
image correlation methods for full-eld vibration measurement. Mechanical Systems
and Signal Processing, 25(3):917{927.
[69] Hernandez-Garcia, M., Masri, S., Ghanem, R., Figueiredo, E., and Farrar, C. (2010a).
An experimental investigation of change detection in uncertain chain-like systems.
Journal of Sound and Vibration, 329(12):2395 { 409.
[70] Hernandez-Garcia, M. R. (2014). Analytical and experimental studies in modeling
and monitoting of uncertain nonlinear systems using data-driven reduced-order models.
PhD thesis, University of Southern California.
[71] Hernandez-Garcia, M. R., Masri, S. F., Ghanem, R., Figueiredo, E., and Farrar, C. R.
(2010b). A structural decomposition approach for detecting, locating, and quantifying
nonlinearities in chain-like systems. Structural Control and Health Monitoring, 17(7):761{
777.
[72] Homan, M., Varcholik, P., and LaViola, J. J. (2010). Breaking the status quo:
Improving 3d gesture recognition with spatially convenient input devices. In Virtual
Reality Conference (VR), 2010 IEEE, pages 59{66. IEEE.
194
[73] Hou, X., Yang, X., and Huang, Q. (2005). Using inclinometers to measure bridge
de
ection. Journal of Bridge Engineering, 10(5):564{569.
[74] Huang, C., Lajnef, N., and Chakrabartty, S. (2010). Calibration and characterization
of self-powered
oating-gate usage monitor with single electron per second operational
limit. IEEE Transactions on Circuits and Systems I: Regular Papers, 57(3):556{567.
[75] Im, S. B., Hurlebaus, S., and Kang, Y. J. (2011). Summary Review of GPS Technology
for Structural Health Monitoring. Journal of Structural Engineering, (October):357.
[76] James, G., Carne, T. G., and Lauer, J. P. (1995). The natural excitation technique
(next) for modal parameter extraction from operating structures. Modal Analysis-the
International Journal of Analytical and Experimental Modal Analysis, 10(4):260.
[77] Jenkins, C. H. (2001). Gossamer Spacecraft: Membrane And In
atable Structures
Technology For Space Applications. American Institute of Aeronautics and Astronautics,
Reston ,VA.
[78] Jeon, H., Bang, Y., and Myung, H. (2011). A paired visual servoing system for 6-DOF
displacement measurement of structures. Smart Materials and Structures, 20(4):1{16.
[79] Jeon, H., Myeong, W., Shin, J. U., Park, J. W., Jung, H. J., and Myung, H. (2014).
Experimental validation of visually servoed paired structured light system (ViSP)
for structural displacement monitoring. IEEE/ASME Transactions on Mechatronics,
19(5):1603{1611.
[80] Jiang, R., J auregui, D. V., and White, K. R. (2008). Close-range photogrammetry
applications in bridge measurement: Literature review. Measurement, 41(8):823{834.
[81] Juang, J.-N. (1997). System realization using information matrix. Journal of Guidance,
Control, and Dynamics, 20(3):492{500.
[82] Juang, J.-N. and Pappa, R. S. (1985). An eigensystem realization algorithm for modal
parameter identication and model reduction. Journal of Guidance, 8(5):620{627.
[83] Kadambi, A., Bhandari, A., and Raskar, R. (2014). 3D Depth Cameras in Vision:
Benets and Limitations of the Hardware. In Springer, pages 3{26.
[84] Kadambi, A., Taamazyan, V., Shi, B., and Raskar, R. (2015). Polarized 3D: High-
Quality Depth Sensing with Polarization Cues. In International Conference on Computer
Vision (ICCV), Santiago Chile.
[85] Kao, C.-Y. and Loh, C.-H. (2013). Monitoring of long-term static deformation data of
Fei-Tsui arch dam using articial neural network-based approaches. Structural Control
and Health Monitoring, 20(3):282{303.
[86] Kerschen, G., Worden, K., Vakakis, A. F., and Golinval, J.-C. (2006). Past, present
and future of nonlinear system identication in structural dynamics. Mechanical systems
and signal processing, 20(3):505{592.
195
[87] Khoshelham, K. and Elberink, S. O. (2012). Accuracy and resolution of kinect depth
data for indoor mapping applications. Sensors, 12(2):1437{1454.
[88] Kijewski-Correa, T. and Kochly, M. (2007). Monitoring the wind-induced response of
tall buildings: GPS performance and the issue of multipath eects. Journal of Wind
Engineering and Industrial Aerodynamics, 95(9-11):1176{1198.
[89] Kim, J., Kim, K., and Sohn, H. (2014). Autonomous dynamic displacement estimation
from data fusion of acceleration and intermittent displacement measurements. Mechanical
Systems and Signal Processing, 42:194 { 205.
[90] Kim, J.-B., Lee, E.-T., Rahmatalla, S., and Eun, H.-C. (2013). Non-baseline dam-
age detection based on the deviation of displacement mode shape data. Journal of
Nondestructive Evaluation, 32(1):14 { 24.
[91] Kim, K. and Kim, J. (2015). Dynamic displacement measurement of a vibratory object
using a terrestrial laser scanner. Measurement Science and Technology, 26(4):045002.
[92] Kohler, M. D., Massari, A., Heaton, T. H., Kanamori, H., Hauksson, E., Guy,
R., Clayton, R. W., Bunn, J., and Chandy, K. (2016). Downtown los angeles 52-
story high-rise and free-eld response to an oil renery explosion. Earthquake Spectra,
32(3):1793{1820.
[93] Korhonen, I. and Lankinen, R. (2014). Energy harvester for a wireless sensor in a
boiler environment. Measurement, 58:241{248.
[94] Kulesh, V. P. (2015). Measurements of deformation of the passenger plane wing in

ight by the videogrammetry method. TsAGI Science Journal, 46(2).
[95] Kurita, M., Koike, S., Nakakita, K., and Masui, K. (2013). In-
ight wing deformation
measurement. In 51st AIAA Aerospace Sciences Meeting including the New Horizons
Forum and Aerospace Exposition, page 967.
[96] Lajnef, N., Chatti, K., Chakrabartty, S., Rhimi, M., and Sarkar, P. (2013). Smart
pavement monitoring system. Technical report.
[97] Lajnef, N., Elvin, N. G., and Chakrabartty, S. (2008). A piezo-powered
oating-
gate sensor array for long-term fatigue monitoring in biomechanical implants. IEEE
transactions on biomedical circuits and systems, 2(3):164{172.
[98] Lajnef, N., Rhimi, M., Chatti, K., Mhamdi, L., and Faridazar, F. (2011). Toward an
integrated smart sensing system and data interpretation techniques for pavement fatigue
monitoring. Computer-Aided Civil and Infrastructure Engineering, 26(7):513{523.
[99] Lee, J.-H., Ho, H.-N., Shinozuka, M., and Lee, J.-J. (2012). An advanced vision-based
system for real-time displacement measurement of high-rise buildings. Smart Materials
and Structures, 21(12):125019.
196
[100] Lee, J. J. and Shinozuka, M. (2006). A vision-based system for remote sensing of
bridge displacement. NDT and E International, 39(5):425{431.
[101] Lichter, M. and Dubowsky, S. (2005). Shape, Motion, and Parameter Estimation of
Large Flexible Space Structures using Range Images. Proceedings of the 2005 IEEE
International Conference on Robotics and Automation.
[102] Lin, B. and Giurgiutiu, V. (2012). Power and energy transduction analysis of
piezoelectric wafer-active sensors for structural health monitoring. Structural Health
Monitoring, 11(1):109{121.
[103] Lin, P.-C., Komsuoglu, H., and Koditschek, D. E. (2006). Sensor data fusion for
body state estimation in a hexapod robot with dynamical gaits. IEEE Transactions on
Robotics, 22(5):932{943.
[104] Liu, Y. and Nayak, S. (2012). Structural health monitoring: State of the art and
perspectives. JOM, 64(7):789{792.
[105] Lovse, J. W., Teskey, W. F., Lachapelle, G., and Cannon, M. E. (1995). Dynamic
Deformation Monitoring of Tall Structure Using GPS Technology. Journal of Surveying
Engineering, 121(1):35{40.
[106] Lynch, J. P. (2006). A Summary Review of Wireless Sensors and Sensor Networks
for Structural Health Monitoring. The Shock and Vibration Digest, 38(2):91{128.
[107] Lynch, J. P. (2007). An overview of wireless structural health monitoring for civil
structures. Philosophical Transactions of the Royal Society of London A: Mathematical,
Physical and Engineering Sciences, 365(1851):345{372.
[108] Mallick, T., Das, P. P., and Majumdar, A. K. (2014). Characterizations of Noise in
Kinect Depth Images: A Review. IEEE Sensors Journal, 14(6):1731{1740.
[109] Marsi, S., Bekey, G., Sassi, H., and Caughey, T. (1982). Non-parametric identication
of a class of non-linear multidegree dynamic systems. Earthquake Engineering and
Structural Dynamics, 10(1):1 { 30.
[110] Marsi, S. and Caughey, T. (1979). A nonparametric identication technique for
nonlinear dynamic problems. Transactions of the ASME. Journal of Applied Mechanics,
46:433 { 447.
[111] Masri, S. (1978). Response of a multidegree-of-freedom system to nonstationary
random excitation. ASME J. Appl. Mech, 45(3):649{656.
[112] Masri, S., Carey, J., Caughey, T., Smyth, A., and Chassiakos, A. (2004). Identi-
cation of the state equation in complex non-linear systems. International Journal of
Non-Linear Mechanics, 39(7):1111 { 27.
197
[113] Masri, S., Carey, J., Caughey, T., Smyth, A., and Chassiakos, A. (2005). A general
data-based approach for developing reduced-order models of nonlinear mdof systems.
Nonlinear Dynamics, 39(1-2):95{112.
[114] Masri, S., Ghanem, R., Govindan, R., and Nayeri, R. (2008). A decentralized
procedure for structural health monitoring of uncertain nonlinear systems provided with
dense active sensor arrays. Smart Materials and Structures, 17(4):045024.
[115] Masri, S., Miller, R., Saud, A., and Caughey, T. (1987a). Identication of nonlinear
vibrating structures. i. formulation. Transactions of the ASME. Journal of Applied
Mechanics, 54(4):918 { 22.
[116] Masri, S., Miller, R., Saud, A., and Caughey, T. (1987b). Identication of nonlinear
vibrating structures: Part ii - applications. Journal of Applied Mechanics, Transactions
ASME, 54(4):923 { 929.
[117] Masri, S., Tasbihgoo, F., P Carey, J., W Smyth, A., and G Chassiakos, A. (2006a).
Data-based model-free representation of complex hysteretic mdof systems. Structural
Control and Health Monitoring, 13(1):365{387.
[118] Masri, S. F., Ghanem, R., Arrate, F., and Carey, J. (2006b). Stochastic nonpara-
metric models of uncertain hysteretic oscillators. AIAA journal, 44(10):2319.
[119] Masri, S. F., Smyth, A., Chassiakos, A., Caughey, T., and Hunter, N. (2000).
Application of neural networks for detection of changes in nonlinear systems. Journal
of Engineering Mechanics, 126(7):666{676.
[120] Massari, A., Kohler, M., Clayton, R., Guy, R., Heaton, T., Bunn, J., Chandy, K., and
Demetri, D. (2016). Dense building instrumentation application for city-wide structural
health monitoring. 16th World Conference on Earthquake, 16WCEE 2017, pages 1{12.
[121] Massari, A. T. (2017). Achieving Higher Fidelity Building Response through Emerging
Technologies and Analytical Techniques. PhD thesis, California Institute of Technology.
[122] Matsuda, N., Cossairt, O., and Gupta, M. (2015). MC3D: Motion Contrast 3D
Scanning. In 2015 IEEE International Conference on Computational Photography
(ICCP), pages 1{10. IEEE.
[123] McIlwraith, D., Pansiot, J., and Yang, G.-Z. (2010). Wearable and ambient sensor
fusion for the characterisation of human motion. In Intelligent Robots and Systems
(IROS), 2010 IEEE/RSJ International Conference on, pages 5505{5510. IEEE.
[124] Meng, M., Fallavollita, P., Blum, T., Eck, U., Sandor, C., Weidert, S., Waschke, J.,
and Navab, N. (2013). Kinect for interactive AR anatomy learning. Proceedings of the
2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages
277{278.
198
[125] Moss, R., Yuan, P., Bai, X., Quesada, E., Sudharsanan, R., Stann, B. L., Dammann,
J. F., Giza, M. M., and Lawler, W. B. (2012). Low-cost compact mems scanning ladar
system for robotic applications. pages 837903{837903.
[126] Nayeri, R. D., Masri, S. F., Ghanem, R. G., and Nigbor, R. L. (2008). A novel
approach for the structural identication and monitoring of a full-scale 17-story build-
ing based on ambient vibration measurements. Smart Materials and Structures,
17(2):025006.
[127] Olaszek, P. (1999). Investigation of the dynamic characteristic of bridge structures
using a computer vision method. Measurement, 25(3):227{236.
[128] Olivares, A., Olivares, G., Gorriz, J., and Ramirez, J. (2009). High-eciency low-cost
accelerometer-aided gyroscope calibration. Proceedings of the International Conference
on Test and Measurement, 2009. ICTM'09, 1:354{360.
[129] Olsen, M. J., Kuester, F., Chang, B. J., and Hutchinson, T. C. (2009). Terrestrial
laser scanning-based structural damage assessment. Journal of Computing in Civil
Engineering, 24(3):264{272.
[130] OuYang, D. and Feng, H.-Y. (2005). On the normal vector estimation for point
cloud data from smooth surfaces. Computer-Aided Design, 37(10):1071 { 1079.
[131] Painter, C. and Shkel, A. (2003). Detection of orientation and predicted performance
of a mems absolute angle measuring gyroscope. Proceedings of the 4th International
Workshop on Structural Health Monitoring, 1:1011{1018.
[132] Pan, B., Qian, K., Xie, H., and Asundi, A. (2009). Two-dimensional digital image
correlation for in-plane displacement and strain measurement: a review. Measurement
Science and Technology, 20(6):062001.
[133] Park, H. S., Lee, H. M., Adeli, H., and Lee, I. (2007). A new approach for health mon-
itoring of structures: Terrestrial laser scanning. Computer-Aided Civil and Infrastructure
Engineering, 22(1):19{30.
[134] Park, J.-W., Lee, J.-J., Jung, H.-J., and Myung, H. (2010). Vision-based displacement
measurement method for high-rise building structures using partitioning approach. NDT
& E International, 43(7):642 { 647.
[135] Park, J. W., Sim, S. H., Jung, H. J., and Billie, S. (2013). Development of a wireless
displacement measurement system using acceleration responses. Sensors (Switzerland),
13(7):8377{8392.
[136] Park, S. (2010). Adaptive control of a vibratory angle measuring gyroscope. Sensors,
10(9):8478{8490.
[137] Park, S., Horowitz, R., and Tan, C.-W. (2008). Dynamics and control of a MEMS
angle measuring gyroscope. Sensors and Actuators A: Physical, 144(1):56{63.
199
[138] Park, S., Park, H., Kim, J., and Adeli, H. (2015). 3D displacement measurement
model for health monitoring of structures using a motion capture system. Measurement,
59:352{362.
[139] Patsadu, O., Nukoolkit, C., and Watanapa, B. (2012). Human gesture recognition
using kinect camera. Proceedings of the 2012 Ninth International Joint Conference on
Computer Science and Software Engineering (JCSSE), pages 28{32.
[140] Petrie, G. and Toth, C. (2008). Terrestrial Laser Scanners. In Topographic Laser
Ranging and Scanning. CRC Press.
[141] Qi, X., Lichti, D., El-Badry, M., Chow, J., and Ang, K. (2014). Vertical dynamic de-

ection measurement in concrete beams with the microsoft kinect. Sensors (Switzerland),
14(2):3293{3307.
[142] Rajaram, S., Vanniamparambil, P. A., Khan, F., Bolhassani, M., Koutras, A., Bartoli,
I., Moon, F., Hamid, A., Benson Shing, P., Tyson, J., and Kontsos, A. (2016). Full-eld
deformation measurements during seismic loading of masonry buildings. Structural
Control and Health Monitoring. STC-16-0037.R1.
[143] Ren, Z., Yuan, J., Meng, J., and Zhang, Z. (2013). Robust part-based hand gesture
recognition using kinect sensor. IEEE transactions on multimedia, 15(5):1110{1120.
[144] Rezaei, D. and Taheri, F. (2010). A novel application of a laser Doppler vibrometer
in a health monitoring system. Journal of Mechanics of Materials and Structures,
5(2):289{304.
[145] Rhimi, M., Lajnef, N., Chatti, K., and Faridazar, F. (2012). A self-powered sensing
system for continuous fatigue monitoring of in-service pavements. International Journal
of Pavement Research and Technology, 5(5):303 { 310.
[146] Ribeiro, D., Cal cada, R., Ferreira, J., and Martins, T. (2014). Non-contact measure-
ment of the dynamic displacement of railway bridges using an advanced video-based
system. Engineering Structures, 75:164{180.
[147] Rutherford, A. C., Park, G., and Farrar, C. R. (2007). Non-linear feature identica-
tions based on self-sensing impedance measurements for structural health assessment.
Mechanical Systems and Signal Processing, 21(1):322{333.
[148] Sarbolandi, H., Le
och, D., and Kolb, A. (2015). Kinect range sensing: Structured-
light versus Time-of-Flight Kinect. Computer Vision and Image Understanding, 139:1{
20.
[149] Scaioni, M., Barazzetti, L., Giussani, A., Previtali, M., Roncoroni, F., and Alba, M.
(2014). Photogrammetric techniques for monitoring tunnel deformation. Earth Science
Informatics, 7(2):83{95.
200
[150] Schall, G., Wagner, D., Reitmayr, G., Taichmann, E., Wieser, M., Schmalstieg, D.,
and Hofmann-Wellenhof, B. (2009). Global pose estimation using multi-sensor fusion
for outdoor augmented reality. In Mixed and Augmented Reality, 2009. ISMAR 2009.
8th IEEE International Symposium on, pages 153{162. IEEE.
[151] Simoen, E., Roeck, G. D., and Lombaert, G. (2015). Dealing with uncertainty in
model updating for damage assessment: A review. Mechanical Systems and Signal
Processing, 56:123 { 149.
[152] Smisek, J., Jancosek, M., and Pajdla, T. (2013). 3D with Kinect. In Consumer
Depth Cameras for Computer Vision, chapter 1, page 167. Springer.
[153] Smyth, A., Masri, S., Caughey, T., and Hunter, N. (2000). Surveillance of mechanical
systems on the basis of vibration signature analysis. Journal of Applied Mechanics,
67(3):540{551.
[154] Smyth, A. W. and Masri, S. F. (2004). The robustness of an ecient probabilistic data-
based tool for simulating the non-stationary response of non-linear systems. International
Journal of Non-Linear Mechanics, 39(9):1453{1461.
[155] Song, G., Wang, C., and Wang, B. (2017). Structural health monitoring (shm) of
civil structures.
[156] Soon, H., Farrar, C. R., Hemez, F. M., Shunk, D. D., Stinemates, D. W., and Nadler,
B. R. (2003). A review of structural health monitoring literature. Los Alamos National
Laboratory Report LA-13976-MS.
[157] Sparks, D., Zarabadi, S., Johnson, J., Jiang, Q., Chia, M., Larsen, O., Higdon, W.,
and Castillo-Borelley, P. (1997). A cmos integrated surface micromachined angular
rate sensor: its automotive applications. In Solid State Sensors and Actuators, 1997.
TRANSDUCERS'97 Chicago., 1997 International Conference on, volume 2, pages
851{854. IEEE.
[158] Staiger, R. (2003). Terrestrial laser scanning-technology, systems and applications.
FIG Regional Conference, Marrakech Morocco, pages 1{10.
[159] Stan cin, S. and Toma zi c, S. (2011). Angle estimation of simultaneous orthogonal
rotations from 3D gyroscope measurements. Sensors, 11(9):8536{8549.
[160] Stone, E. E. and Skubic, M. (2011). Evaluation of an inexpensive depth camera for
passive in-home fall risk assessment. 2011 5th International Conference on Pervasive
Computing Technologies for Healthcare (PervasiveHealth) and Workshops, pages 71{77.
[161] Stowers, J., Hayes, M., and Bainbridge-Smith, A. (2011). Altitude control of
a quadrotor helicopter using depth map from Microsoft Kinect sensor. 2011 IEEE
International Conference on Mechatronics, pages 358{362.
201
[162] Tahavori, F., Alnowami, M., Jones, J., Elangovan, P., Donovan, E., and Wells, K.
(2013). Assessment of Microsoft Kinect technology (Kinect for Xbox and Kinect for
windows) for patient monitoring during external beam radiotherapy. IEEE Nuclear
Science Symposium Conference Record.
[163] Takewaki, I., Nakamura, M., Nakamura, M., and Yoshitomi, S. (2012). System
identication for structural health monitoring. WIT Press.
[164] Tamura, Y., Matsui, M., Pagnini, L. C., Ishibashi, R., and Yoshida, a. (2002).
Measurement of wind-induced response of buildings using RTK-GPS. Journal of Wind
Engineering and Industrial Aerodynamics, 90(12-15):1783{1793.
[165] Thomas, J. J., Kosnik, D., Kotowsky, M. P., Marron, D., and Schofer, J. L. (2009).
Structural health monitoring of civil infrastructure. Technical report.
[166] Tortini, R., Bonali, F. L., Corazzato, C., Carn, S. a., and Tibaldi, A. (2014). An
innovative application of the kinect in earth sciences: Quantifying deformation in
analogue modelling of volcanoes. Terra Nova, 26(4):273{281.
[167] Tun cel, O., Altun, K., and Barshan, B. (2009). Classifying human leg motions with
uniaxial piezoelectric gyroscopes. Sensors, 9(11):8508{8546.
[168] Van der Auweraer, H. and Peeters, B. (2003). International research projects on
structural health monitoring: an overview. Structural Health Monitoring, 2(4):341{358.
[169] Vera, L., Gimeno, J., Coma, I., and Fern andez, M. (2011). Augmented mirror:
Interactive augmented reality system based on Kinect. Proceedings of the 13th IFIP
TC 13 International Conference on Human-Computer Interaction-INTERACT 2011,
pages 483{486.
[170] Wahbeh, a. M., Carey, J. P., and Masri, S. F. (2003). A vision-based approach for
the direct measurement of displacements in vibrating systems. Smart Materials and
Structures, 12(5):785{794.
[171] Wang, J. and Gao, Y. (2010). Land vehicle dynamics-aided inertial navigation. IEEE
Transactions on Aerospace and Electronic Systems, 46(4):1638{1653.
[172] Watters, D. G., Jayaweera, P., Bahr, A. J., Huestis, D. L., Priyantha, N., Meline,
R., Reis, R., and Parks, D. (2003). Smart pebble: Wireless sensors for structural health
monitoring of bridge decks. In Proc. SPIE, volume 5057, pages 20{28.
[173] Whelan, M. J., Gangone, M. V., Janoyan, K. D., and Jha, R. (2009). Real-time
wireless vibration monitoring for operational modal analysis of an integral abutment
highway bridge. Engineering Structures, 31(10):2224{2235.
[174] Worden, K., Farrar, C. R., Haywood, J., and Todd, M. (2008). A review of nonlinear
dynamics applications to structural health monitoring. Structural Control and Health
Monitoring, 15(4):540{567.
202
[175] Wu, L.-J., Casciati, F., and Casciati, S. (2014). Dynamic testing of a laboratory
model via vision-based sensing. Engineering Structures, 60:113{125.
[176] Xia, J. and Siochi, R. (2012). A Prototype of a Real-Time Respiratory Motion
Monitoring System Using Microsoft Kinect Sensor. Medical Physics, 39(6):3689.
[177] Yang, Y., Dorn, C., Mancini, T., Talken, Z., Kenyon, G., Farrar, C., and Mascare~ nas,
D. (2017). Blind identication of full-eld vibration modes from video measurements
with phase-based video motion magnication. Mechanical Systems and Signal Processing,
85:567{590.
[178] Yi, T.-H., Li, H.-N., and Gu, M. (2013). Recent research and applications of gps-
based monitoring technology for high-rise structures. Structural Control and Health
Monitoring, 20(5):649{670.
[179] Yu, Y., Kang, W.-H., Zhang, C., Wang, J., and Ou, J. (2015). A stochastic analysis
framework for a steel frame structure using wireless sensor system measurements.
Measurement, 69:202{209.
[180] Yun, C.-B. and Min, J. (2011). Smart sensing, monitoring, and damage detection
for civil infrastructures. KSCE Journal of Civil Engineering, 15(1):1{14.
[181] Yun, G. J., Lee, S.-G., Carletta, J., and Nagayama, T. (2011). Decentralized
damage identication using wavelet signal analysis embedded on wireless smart sensors.
Engineering Structures, 33(7):2162{2172.
[182] Yun, H.-B. and Masri, S. F. (2008). Stochastic change detection in uncertain
nonlinear systems using reduced-order models: system identication. Smart Materials
and Structures, 17(1):015040.
[183] Zhang, Z. (2000). A
exible new technique for camera calibration. IEEE Transactions
on Pattern Analysis and Machine Intelligence, 22(11):1330{1334.
[184] Zhang, Z. (2004). Camera Calibration. In Medioni, G. and Kang, S. B., editors,
Emerging Topics in Computer Vision, chapter 2, pages 4{43. Prentice Hall Professional
Technical Reference.
[185] Zhao, L., Yan, L., Cheng, J., and Wang, X. (2008). The research of inertial navigation
system based on submarine space motion. In Computational Intelligence and Industrial
Application, 2008. PACIIA'08. Pacic-Asia Workshop on, volume 1, pages 751{755.
IEEE.
203

Abstract (if available)

Abstract The recent developments in sensor-based technologies and computational tools provide a potential opportunity to overcome challenges that are widely encountered in the field of condition assessment and health monitoring of structural systems. The main challenges can be categorized based on sensors types, measurements types and data analysis methodologies. The first part of this dissertation focuses on the calibration and application of newly developed class of sensors capable of resolving some problems related to sensor and measurement problems. ❧ The quantification of evolving 2D/3D displacement fields associated with a continuous structural system is an important challenging task in health monitoring. The inexpensive RGB-D cameras are very promising technology to quantify 3D displacement fields as they are non-contact displacement sensors, with full-field measurements, and 3D data acquisition. Microsoft Kinect is a representative of this class of sensors but originally designed for gaming purposes. This study investigated the accuracy and performance of the Microsoft Kinect sensor as a representative RGB-D sensor in measuring dynamic multi-component deformations by performing experimental and analytical studies. The calibration process was divided into two phases: (1) sensor-calibration and (2) system application. In the first phase, the laboratory experiments were performed to determine the performance envelope for dynamic measurements such as amplitude and frequency bounds, linearity of measurements, distortion, effect of noise. In the second phase, the Kinect was deployed to acquire dynamic displacement field measurements, including 3D transnational motion with simultaneous rotational/torsional motion components from flexible structure. The calibration results showed that if the observed displacement field generated discrete (pixel) sensor measurements with sufficient resolution (observed displacements more than 10 mm) beyond the sensor noise floor, then the subject sensors could provide reasonable accuracy for transnational motion (about 5%) when the frequency range of the evolving field is within about 10 Hz. However, the expected error for torsional measurements was around 6% for static motion and 10% for dynamic rotation for measurements greater than 5°. ❧ Another major challenge for long-term monitoring and condition assessment is efficiently storing the acquired data and providing the health monitoring system including sensors and data acquisition with external power requirements. A newly developed self-powered wireless sensor uses piezoelectric transducers for the self-powering of wireless sensors by harvesting energy from the mechanical loading experienced by the structure. An experimental study was performed on a cantilever aluminum beam subjected to bending to calibrate the proposed technology the output of the piezo-floating-gate sensor in a fatigue damage environment. Different piezoelectric transducers were mounted on the beam for both powering the sensor and monitoring the damage progression. Furthermore, conventional strain gages were mounted in the vicinity of the piezoelectric transducers to calibrate their output. The change of charge on the floating-gates of the sensor due to electron injection were considered as damage indicator parameters. The experimental results show that the performance of the proposed sensor is satisfactory for detecting damage progression in beams. ❧ The second part of this dissertation focused on implementing promising and robust data-driven methodologies to build high-fidelity, reduced-order models for different structural systems. The developed reduced-order models were used for condition assessment and modeling of critical components. Different data sets were obtained from a high-fidelity 3D finite element model developed based on experiential data acquired from a 52-story high-rise building and re-configurable test structure designed, built, and tested at University of Southern California (USC) were employed to evaluate the robustness of the developed data-based reduced-order models for detecting, locating, quantifying, and classifying structural changes in probabilistic way.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

An analytical and experimental study of evolving 3D deformation fields using vision-based approaches

PDF

Vision-based studies for structural health monitoring and condition assesment

PDF

Vision-based and data-driven analytical and experimental studies into condition assessment and change detection of evolving civil, mechanical and aerospace infrastructures

PDF

Analytical and experimental studies in system identification and modeling for structural control and health monitoring

PDF

Studies into data-driven approaches for nonlinear system identification, condition assessment, and health monitoring

PDF

Studies into vibration-signature-based methods for system identification, damage detection and health monitoring of civil infrastructures

PDF

Analytical and experimental studies of modeling and monitoring uncertain nonlinear systems

PDF

Analytical and experimental studies in modeling and monitoring of uncertain nonlinear systems using data-driven reduced‐order models

PDF

Stochastic and multiscale models for urban and natural ecology

PDF

Damage detection using substructure identification

PDF

Analytical and experimental studies in the development of reduced-order computational models for nonlinear systems

PDF

Dynamic latent structured data analytics

PDF

Stochastic data assimilation with application to multi-phase flow and health monitoring problems

PDF

Zero-power sensing and processing with piezoelectric resonators

PDF

In-situ quality assessment of scan data for as-built models using building-specific geometric features

PDF

Smart buildings: synergy in structural control, structural health monitoring and environmental systems

PDF

Magnetic induction-based wireless body area network and its application toward human motion tracking

PDF

Dynamic graph analytics for cyber systems security applications

PDF

Studies into computational intelligence approaches for the identification of complex nonlinear systems

PDF

Latent space dynamics for interpretation, monitoring, and prediction in industrial systems

Asset Metadata

Creator Abdelbarr, Mohamed Hassan (author)

Core Title Experimental and analytical studies of infrastructure systems condition assessment using different sensing modality

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Civil Engineering

Publication Date 02/07/2018

Defense Date 01/18/2018

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag condition assessment,inexpensive sensors,Infrastructure,long-term monitoring,OAI-PMH Harvest,Structural health monitoring,vision-based sensors

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Masri, Sami F. (committee chair)

Creator Email abdelbar@usc.edu,abdelbarr87@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c40-470004

Unique identifier UC11265722

Identifier etd-AbdelbarrM-5997.pdf (filename),usctheses-c40-470004 (legacy record id)

Legacy Identifier etd-AbdelbarrM-5997.pdf

Dmrecord 470004

Document Type Dissertation

Rights Abdelbarr, Mohamed Hassan

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA